Audio watermark extracting method, audio processing method, audio watermark adding method and related products
By using deep learning models and frequency-domain-time domain conversion techniques, combined with candidate audio watermark analysis, the problem of low accuracy in watermark extraction from audio data was solved, achieving efficient and accurate audio watermark extraction and addition.
Patent Information
- Application Number
- CN202411028932.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-26
AI Technical Summary
Existing technologies struggle to effectively extract watermarks from audio data, and traditional methods are prone to damaging audio data, resulting in low extraction accuracy.
A watermark extraction model trained with a deep learning model is used to extract watermark encoding information from the spectral information carried in the center frequency range by performing frequency domain transformation and time domain transformation on the target audio data. Combined with the occurrence frequency and probability prediction of candidate audio watermarks, the extraction accuracy is improved.
It improves the accuracy and anti-interference ability of audio watermark extraction, reduces the impact of watermark interference processing on extraction, and realizes rapid addition and efficient extraction of audio watermarks.
Smart Images

Figure CN119028366A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to a method for extracting an audio watermark, an audio processing method, a method for adding an audio watermark, and related products. Background Art
[0002] In the Internet age, audio data can be quickly disseminated and obtained, but it also breeds copyright infringements such as illegal dissemination, illegal copying, and illegal tampering of audio data. Therefore, in order to protect the copyright of audio data, audio watermarking technology has emerged. Audio watermarking technology involves extracting watermarks from audio data, and then judging whether the audio data is obtained through copyright infringement based on the extracted watermark. Therefore, how to extract watermarks from audio data is of great significance. Summary of the invention
[0003] The present application provides a method for extracting audio watermarks, an audio processing method, a method for adding audio watermarks and related products to extract audio watermarks from audio data, wherein the related products include an apparatus for extracting audio watermarks, an audio processing apparatus, a method for adding audio watermarks, an electronic device, a computer-readable storage medium and a computer program product.
[0004] In a first aspect, a method for extracting an audio watermark is provided, the method comprising:
[0005] Acquire target audio data, where the target audio data is obtained by performing time domain conversion on target spectrum information carrying watermark coding information, where spectrum information within a center frequency range of the target spectrum information carries the watermark coding information;
[0006] The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, and the watermark encoding information is an encoding result of the target audio watermark.
[0007] In combination with any embodiment of the present application, the watermark extraction model is obtained by iteratively updating the difference between a training watermark and a reference watermark, the training watermark is the watermark extracted by the watermark extraction model from the training audio data, and the reference watermark is the watermark carried in the training audio data.
[0008] In combination with any embodiment of the present application, before inputting the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, the method further includes:
[0009] Intercepting n audio frames to be extracted from the target audio data, where n is an integer greater than 1;
[0010] The step of inputting the target audio data into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model comprises:
[0011] Inputting the n audio frames to be extracted into the watermark extraction model respectively to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively;
[0012] The target audio watermark is obtained according to the n candidate audio watermarks.
[0013] In combination with any implementation manner of the present application, obtaining the target audio watermark according to the n candidate audio watermarks includes:
[0014] Determine the number of times each audio watermark appears in the n candidate audio watermarks to obtain a target number;
[0015] The target audio watermark is obtained according to the target number and the n candidate audio watermarks.
[0016] In combination with any implementation manner of the present application, obtaining the target audio watermark according to the target number of times and the n candidate audio watermarks includes:
[0017] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the audio watermark that appears most frequently among the n candidate audio watermarks is used as the target audio watermark.
[0018] In combination with any implementation manner of the present application, obtaining the target audio watermark according to the target number of times and the n candidate audio watermarks includes:
[0019] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, determining the value of the target position in the n candidate audio watermarks to obtain n candidate values;
[0020] Determine the value that appears most frequently among the n candidate values as the target value of the target position in the target audio watermark;
[0021] The target audio watermark is obtained according to the target value.
[0022] In combination with any implementation manner of the present application, the step of intercepting n audio frames to be extracted from the target audio data includes:
[0023] The target audio data is segmented according to a preset step length to obtain n audio frames to be extracted, wherein the preset step length is greater than 0 and less than the length of the watermark encoding information.
[0024] In combination with any embodiment of the present application, the n audio frames to be extracted include a first audio frame, the n candidate audio watermarks include a first audio watermark, and the first audio watermark is an audio watermark obtained after the first audio frame is input into the watermark extraction model;
[0025] The step of inputting the n audio frames to be extracted into the watermark extraction model to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively includes:
[0026] Input the first audio frame into the audio frame division module of the watermark extraction model to obtain m segments of audio frames to be confirmed, where the audio frames to be confirmed are part of the first audio frame;
[0027] Inputting the m audio frames to be confirmed into the probability prediction module of the watermark extraction model respectively, predicting the probability that each audio frame to be confirmed contains a watermark respectively, and obtaining m prediction probabilities;
[0028] Inputting the m prediction probabilities into the candidate frame determination module of the watermark extraction model, determining that the audio frames to be confirmed corresponding to the largest s of the m prediction probabilities are s candidate audio frames, where s is less than m;
[0029] The s candidate audio frames are input into the audio watermark extraction module of the watermark extraction model to obtain the first audio watermark.
[0030] In combination with any embodiment of the present application, before inputting the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, the method further includes:
[0031] Performing frequency domain conversion on the target audio data to obtain the target frequency spectrum information;
[0032] The step of inputting the target audio data into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model comprises:
[0033] Inputting the target spectrum information into the watermark extraction model;
[0034] Using the watermark extraction model, extracting spectrum information within the center frequency range from the target spectrum information to obtain target spectrum information;
[0035] The watermark extraction model is used to extract the watermark encoding information in the target spectrum information to obtain the target audio watermark.
[0036] In combination with any implementation manner of the present application, the center frequency range is determined according to the resonance peak frequency.
[0037] In a second aspect, a method for extracting an audio watermark is provided, the method comprising:
[0038] Acquire target audio data carrying a watermark, wherein the watermark in the target audio data is obtained by adding the watermark to the original target audio through a first watermark adding method, wherein the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method;
[0039] Extracting a watermark from the target audio data by a second watermark extraction method to obtain a target audio watermark carried by the target audio data, wherein the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method;
[0040] Wherein, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method;
[0041] For the same audio data, the accuracy of extracting the watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting the watermark from the audio data by the first watermark extraction method. In combination with any embodiment of the present application, the first watermark adding method adjusts the spectrum information of the audio data based on the watermark encoding information to add the watermark to the audio data.
[0042] In combination with any embodiment of the present application, the step of extracting a watermark from the target audio data by using a second watermark extraction method to obtain a target audio watermark carried by the target audio data includes:
[0043] The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, where the watermark extraction model is a deep learning model capable of extracting watermarks from audio data.
[0044] In combination with any implementation manner of the present application, the target audio data is audio data of an audio on-demand platform.
[0045] In a third aspect, an audio processing method is provided, the method comprising:
[0046] Get the audio data to be processed and the watermark to be added;
[0047] Encoding the watermark to be added to obtain watermark encoding information;
[0048] Performing frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed;
[0049] Based on the watermark coding information, adjusting the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information;
[0050] Performing a time domain transformation on the target spectrum information to obtain target audio data carrying a watermark;
[0051] The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, and the watermark encoding information is an encoding result of the target audio watermark.
[0052] In a fourth aspect, a method for adding an audio watermark is provided, the method comprising:
[0053] Get the audio data to be processed and the watermark to be added;
[0054] Encoding the watermark to be added to obtain watermark encoding information;
[0055] Performing frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed;
[0056] Based on the watermark coding information, adjusting the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information;
[0057] The target frequency spectrum information is transformed in the time domain to obtain target audio data carrying a watermark.
[0058] In combination with any embodiment of the present application, before adjusting the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark encoding information to obtain the target spectrum information, the method also includes: determining the center frequency range based on the resonance peak frequency of the spectrum information to be processed.
[0059] In combination with any implementation manner of the present application, the watermark encoding information is a binary sequence, and the number of bits of the binary sequence is x;
[0060] The step of adjusting the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain target spectrum information includes:
[0061] Determine x groups of amplitudes from the amplitudes within the central frequency range of the spectrum information to be processed;
[0062] Determine x amplitude adjustment amounts according to the x-bit value in the binary sequence, wherein the amplitude adjustment amounts correspond to the values in the binary sequence one by one, the same values correspond to the same amplitude adjustment amounts, and different values correspond to different amplitude adjustment amounts;
[0063] The x groups of amplitudes in the spectrum information to be processed are adjusted by using the x amplitude adjustment amounts to obtain the target spectrum information.
[0064] In combination with any implementation manner of the present application, when the amplitude of the audio data is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the audio data is within a preset range.
[0065] In combination with any implementation manner of the present application, the step of performing frequency domain transformation on the audio data to be processed to obtain spectrum information to be processed includes:
[0066] Intercepting a segment of audio from the audio data to be processed as the audio data to be added;
[0067] Performing frequency domain transformation on the audio data to be added to obtain the frequency spectrum information to be processed;
[0068] The step of performing a time domain transformation on the target spectrum information to obtain target audio data carrying a watermark includes:
[0069] Performing a time domain transformation on the target spectrum information to obtain watermarked audio data carrying a watermark;
[0070] The watermarked audio data is used to replace the audio data to be added in the audio data to be processed to obtain the target audio data.
[0071] In combination with any implementation manner of the present application, the audio data to be processed is audio data released by a target object through a target platform;
[0072] Get the watermark to be added, including:
[0073] Obtain a target identifier of the target object on the target platform;
[0074] The watermark to be added is generated according to the target identifier.
[0075] In combination with any implementation manner of the present application, the step of generating the watermark to be added according to the target identifier includes:
[0076] Generate watermark information to be processed according to the target identifier;
[0077] The watermark to be added is obtained by merging the watermark information to be processed and the preset identifier.
[0078] In a fifth aspect, a device for extracting an audio watermark is provided, wherein the device for extracting an audio watermark comprises:
[0079] an acquisition unit, configured to acquire target audio data, wherein the target audio data is obtained by performing time domain conversion on target spectrum information carrying watermark coding information, wherein spectrum information within a center frequency range of the target spectrum information carries the watermark coding information;
[0080] The extraction unit is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
[0081] In combination with any embodiment of the present application, the watermark extraction model is obtained by iteratively updating the difference between a training watermark and a reference watermark, the training watermark is the watermark extracted by the watermark extraction model from the training audio data, and the reference watermark is the watermark carried in the training audio data.
[0082] In combination with any implementation manner of the present application, the device for extracting an audio watermark further includes:
[0083] An interception unit, used for intercepting n audio frames to be extracted from the target audio data, where n is an integer greater than 1;
[0084] The extraction unit is specifically used for:
[0085] Inputting the n audio frames to be extracted into the watermark extraction model respectively to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively;
[0086] The target audio watermark is obtained according to the n candidate audio watermarks.
[0087] In combination with any embodiment of the present application, the extraction unit is specifically used for:
[0088] Determine the number of times each audio watermark appears in the n candidate audio watermarks to obtain a target number of times;
[0089] The target audio watermark is obtained according to the target number and the n candidate audio watermarks.
[0090] In combination with any embodiment of the present application, the extraction unit is specifically used for:
[0091] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the audio watermark that appears most frequently among the n candidate audio watermarks is used as the target audio watermark.
[0092] In combination with any embodiment of the present application, the extraction unit is specifically used for:
[0093] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, determining the value of the target position in the n candidate audio watermarks to obtain n candidate values;
[0094] Determine the value that appears most frequently among the n candidate values as the target value of the target position in the target audio watermark;
[0095] The target audio watermark is obtained according to the target value.
[0096] In combination with any embodiment of the present application, the interception unit is specifically used for:
[0097] The target audio data is segmented according to a preset step length to obtain n audio frames to be extracted, where the preset step length is greater than 0 and less than the length of the watermark encoding information.
[0098] In combination with any embodiment of the present application, the n audio frames to be extracted include a first audio frame, the n candidate audio watermarks include a first audio watermark, and the first audio watermark is an audio watermark obtained after the first audio frame is input into the watermark extraction model;
[0099] The extraction unit is specifically used for:
[0100] Input the first audio frame into the audio frame division module of the watermark extraction model to obtain m segments of audio frames to be confirmed, where the audio frames to be confirmed are part of the first audio frame;
[0101] Inputting the m audio frames to be confirmed into the probability prediction module of the watermark extraction model respectively, predicting the probability that each audio frame to be confirmed contains a watermark respectively, and obtaining m prediction probabilities;
[0102] Inputting the m prediction probabilities into the candidate frame determination module of the watermark extraction model, determining that the to-be-confirmed audio frames corresponding to the largest s of the m prediction probabilities are s candidate audio frames, where s is less than m;
[0103] The s candidate audio frames are input into the audio watermark extraction module of the watermark extraction model to obtain the first audio watermark.
[0104] In combination with any implementation manner of the present application, the device for extracting an audio watermark further includes:
[0105] A conversion unit, used for performing frequency domain conversion on the target audio data to obtain the target frequency spectrum information;
[0106] The extraction unit is specifically used for:
[0107] Inputting the target spectrum information into the watermark extraction model;
[0108] Using the watermark extraction model, extracting spectrum information within the center frequency range from the target spectrum information to obtain target spectrum information;
[0109] The watermark extraction model is used to extract the watermark encoding information in the target spectrum information to obtain the target audio watermark.
[0110] In combination with any implementation manner of the present application, the center frequency range is determined according to the resonance peak frequency.
[0111] In a sixth aspect, a device for extracting an audio watermark is provided, the device for extracting an audio watermark comprising:
[0112] an acquisition unit, configured to acquire target audio data carrying a watermark, wherein the watermark in the target audio data is obtained by adding a first watermark adding method to the original target audio, wherein the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method;
[0113] an extraction unit, configured to extract a watermark from the target audio data by using a second watermark extraction method to obtain a target audio watermark carried by the target audio data, wherein the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method;
[0114] Wherein, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method;
[0115] For the same audio data, the accuracy of extracting the watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting the watermark from the audio data by the first watermark extraction method.
[0116] In combination with any embodiment of the present application, the first watermark adding method adjusts the frequency spectrum information of the audio data based on the watermark encoding information to add the watermark to the audio data.
[0117] In combination with any embodiment of the present application, the extraction unit is specifically used for:
[0118] The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, where the watermark extraction model is a deep learning model capable of extracting watermarks from audio data.
[0119] In combination with any implementation manner of the present application, the target audio data is audio data of an audio on-demand platform.
[0120] In a seventh aspect, an audio processing device is provided, the audio processing device comprising:
[0121] An acquisition unit, used for acquiring audio data to be processed and watermark to be added;
[0122] An encoding unit, used for encoding the watermark to be added to obtain watermark encoding information;
[0123] A transform unit, used for performing frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed;
[0124] An adjustment unit, configured to adjust the spectrum information within a center frequency range in the spectrum information to be processed based on the watermark coding information to obtain target spectrum information;
[0125] The transform unit is used to perform a time domain transform on the target spectrum information to obtain target audio data carrying a watermark;
[0126] The extraction unit is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
[0127] In an eighth aspect, a device for adding an audio watermark is provided, wherein the device for adding an audio watermark comprises:
[0128] An acquisition unit, used for acquiring audio data to be processed and watermark to be added;
[0129] An encoding unit, used for encoding the watermark to be added to obtain watermark encoding information;
[0130] A transform unit, used for performing frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed;
[0131] An adjustment unit, configured to adjust the spectrum information within a center frequency range in the spectrum information to be processed based on the watermark coding information to obtain target spectrum information;
[0132] The transformation unit is used to perform time domain transformation on the target spectrum information to obtain target audio data carrying a watermark.
[0133] In combination with any embodiment of the present application, the apparatus for adding an audio watermark further includes: a determining unit, configured to determine the center frequency range according to the resonance peak frequency of the frequency spectrum information to be processed.
[0134] In combination with any implementation manner of the present application, the watermark encoding information is a binary sequence, and the number of bits of the binary sequence is x;
[0135] The adjustment unit is specifically used for:
[0136] Determine x groups of amplitudes from the amplitudes within the central frequency range of the spectrum information to be processed;
[0137] Determine x amplitude adjustment amounts according to the x-bit value in the binary sequence, wherein the amplitude adjustment amounts correspond to the values in the binary sequence one by one, the same values correspond to the same amplitude adjustment amounts, and different values correspond to different amplitude adjustment amounts;
[0138] The x groups of amplitudes in the spectrum information to be processed are adjusted by using the x amplitude adjustment amounts to obtain the target spectrum information.
[0139] In combination with any implementation manner of the present application, when the amplitude of the audio data is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the audio data is within a preset range.
[0140] In combination with any implementation manner of the present application, the transformation unit is specifically used to:
[0141] Intercepting a segment of audio from the audio data to be processed as the audio data to be added;
[0142] Performing frequency domain transformation on the audio data to be added to obtain the frequency spectrum information to be processed;
[0143] The transformation unit is specifically used for:
[0144] Performing a time domain transformation on the target spectrum information to obtain watermarked audio data carrying a watermark;
[0145] The watermarked audio data is used to replace the audio data to be added in the audio data to be processed to obtain the target audio data.
[0146] In combination with any implementation manner of the present application, the audio data to be processed is audio data released by a target object through a target platform;
[0147] The acquisition unit is specifically used for:
[0148] Obtain a target identifier of the target object on the target platform;
[0149] The watermark to be added is generated according to the target identifier.
[0150] In combination with any implementation manner of the present application, the acquisition unit is specifically used to:
[0151] Generate watermark information to be processed according to the target identifier;
[0152] The watermark to be added is obtained by merging the watermark information to be processed and the preset identifier.
[0153] In a ninth aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the first aspect and any embodiment thereof, or the electronic device executes the second aspect and any embodiment thereof, or the electronic device executes the third aspect, or the electronic device executes the fourth aspect and any embodiment thereof.
[0154] In the tenth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the first aspect and any embodiment thereof, or the electronic device executes the second aspect and any embodiment thereof, or the electronic device executes the third aspect, or the electronic device executes the fourth aspect and any embodiment thereof.
[0155] In the eleventh aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the first aspect and any embodiment thereof, or the processor is caused to execute the second aspect and any embodiment thereof, or the processor is caused to execute the third aspect, or the processor is caused to execute the fourth aspect and any embodiment thereof.
[0156] In the twelfth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the above-mentioned first aspect and any embodiment thereof, or the computer is caused to execute the above-mentioned second aspect and any embodiment thereof, or the computer is caused to execute the above-mentioned third aspect, or the computer is caused to execute the above-mentioned fourth aspect and any embodiment thereof.
[0157] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.
[0158] In the present application, since the spectrum information within the center frequency range of the target spectrum information carries the watermark encoding information, the center frequency range is the frequency range of the speech in the spectrum information, and the frequency of the watermark in the target audio data is within the frequency range of the speech in the target audio data. At this time, if the watermark in the target audio data is removed by filtering the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness. In this way, after acquiring the target audio data, the extraction device inputs the target audio data into the watermark extraction model obtained through pre-training, and obtains the target audio watermark output by the watermark extraction model, which can improve the accuracy of the target audio watermark.
[0159] Moreover, the target audio data is obtained by time-domain conversion of the target spectrum information carrying the watermark coding information, which means that the watermark in the target audio data is obtained by adjusting the spectrum information of the audio data. Therefore, the calculation amount of the target audio data obtained by adding the watermark to the audio data is small and the speed is fast. The watermark extraction model is obtained through pre-training. During the pre-training process, the watermark extraction model can learn how to extract the watermark from the audio data, wherein the audio data includes the audio data that has not been processed by the watermark interference and the audio data that has been processed by the watermark interference. The watermark interference processing refers to the processing that can remove the watermark in the audio data. For example, the watermark in the audio data can be removed by filtering the audio data, and the watermark interference processing includes filtering. Therefore, by extracting the watermark from the audio data through the watermark extraction model, the influence of the watermark interference processing on the accuracy of the extracted watermark can be reduced, and the accuracy of the extracted watermark can be improved. Therefore, by extracting the target audio watermark from the target audio data through the watermark extraction model, the accuracy of the target audio watermark can be improved. In this way, the watermark can be quickly added to the audio data, and the accuracy of extracting the watermark from the audio data can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0160] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.
[0161] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0162] Figure 1 A flowchart of a method for extracting an audio watermark provided in an embodiment of the present application;
[0163] Figure 2A schematic diagram of a process for obtaining a target audio watermark based on n candidate audio watermarks provided in an embodiment of the present application;
[0164] Figure 3 A flowchart of another method for extracting an audio watermark provided in an embodiment of the present application;
[0165] Figure 4 A flowchart of an audio processing method is provided for an embodiment of the present application;
[0166] Figure 5 A flowchart of a method for adding an audio watermark provided in an embodiment of the present application;
[0167] Figure 6 A flowchart of another audio processing method provided in an embodiment of the present application;
[0168] Figure 7 A schematic diagram of the structure of a device for extracting an audio watermark provided in an embodiment of the present application;
[0169] Figure 8 A schematic diagram of the structure of a device for extracting an audio watermark provided in an embodiment of the present application;
[0170] Fig. 9 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;
[0171] Fig.10 A schematic diagram of the structure of a device for adding an audio watermark provided in an embodiment of the present application;
[0172] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0173] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0174] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0175] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0176] The embodiments of the present application provide two different methods for extracting audio watermarks, an audio processing method, and a method for adding audio watermarks, wherein the method for extracting audio watermarks is used to extract audio watermarks from audio data, the audio processing method is used to add watermarks to audio data and extract watermarks from the watermarked audio data, and the method for adding audio watermarks is used to add watermarks to audio data. A method for extracting audio watermarks will be described below, and the executor of the method is a device for extracting audio watermarks (hereinafter referred to as the extraction device), wherein the extraction device can be any electronic device that can execute the technical solution disclosed in the method embodiments of the present application. Optionally, the extraction device can be one of the following: a computer, a server.
[0177] It should be understood that the method embodiment of the present application can also be implemented by a processor executing a computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A flowchart of a method for extracting an audio watermark provided in an embodiment of the present application.
[0178] 101. Obtain target audio data.
[0179] In the embodiment of the present application, the target audio data is obtained by performing time domain conversion on the target spectrum information carrying the watermark coding information, wherein the spectrum information within the center frequency range of the target spectrum information carries the watermark coding information.
[0180] The above-mentioned spectrum information is the frequency domain information of the audio. By performing a time domain transformation on the frequency domain information, the frequency domain information can be converted into the time domain information, wherein the time domain transformation is used to convert the frequency domain information into the time domain information. For example, the time domain transformation is an inverse Fourier transform. By performing an inverse Fourier transform on the spectrum information of the audio, the spectrum information of the audio can be converted into the time domain information to obtain the time domain information of the audio. Optionally, the spectrum information of the audio is obtained by performing a frequency domain transformation on the time domain information of the audio, wherein the frequency domain transformation is used to convert the time domain information into the frequency domain information. For example, the frequency domain transformation is a Fourier transform. By performing a Fourier transform on the time domain information of the audio, the time domain information of the audio can be converted into the frequency domain information to obtain the spectrum information of the audio. For another example, the frequency domain transformation is a short-time Fourier transform (STFT). By performing an STFT on the time domain information of the audio, the time domain information of the audio can be converted into the frequency domain information to obtain the spectrum information of the audio.
[0181] The watermark encoding information is obtained by encoding the watermark, and the watermark encoding information carries the watermark information. In one possible implementation, a binary sequence is obtained by encoding the watermark as the watermark encoding information. In another possible implementation, an octal sequence is obtained by encoding the watermark. The octal sequence is converted into a binary sequence to obtain the watermark encoding information.
[0182] The above-mentioned center frequency range is the frequency range of speech in the spectrum information. The spectrum information within the center frequency range in the target spectrum information carries watermark coding information, indicating that the frequency of the watermark coding information in the target spectrum information is within the frequency range of speech in the spectrum information. In this way, when the target audio data is obtained by time domain transformation of the target spectrum information, and the watermark coding information is the coding result of the watermark in the target audio data, the frequency of the watermark in the target audio data is within the frequency range of speech in the target audio data. At this time, if the target audio data is filtered to remove the watermark in the target audio data, then the watermark whose frequency is within the frequency range of speech needs to be removed by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness.
[0183] In the current technology, the method of adding watermarks to audio data includes a method of adding watermarks to audio data by deep learning technology and a method of adding watermarks to audio data by technology other than deep learning technology. For the convenience of expression, the method of adding watermarks to audio data by deep learning technology is referred to as the deep learning adding method, and the method of adding watermarks to audio data by technology other than deep learning technology is referred to as the traditional adding method. Adding watermarks to audio data by deep learning adding method means adding watermarks to audio data by deep learning model. Due to the large number of parameters of deep learning model, adding watermarks to audio data by deep learning model requires large amount of calculation and slow speed, that is, the deep learning adding method requires large amount of calculation and slow speed.
[0184] Adding a watermark to audio data through a traditional adding method is to adjust the frequency spectrum information of the audio data based on the watermark, and add the watermark to the audio data. Optionally, the traditional adding method includes: determining an amplitude adjustment amount based on the watermark, and then adjusting the amplitude of the frequency spectrum information of the audio data based on the amplitude adjustment amount, and finally converting the adjusted frequency spectrum information into the time domain to obtain the audio data with the watermark. Since the amplitude adjustment amount is obtained based on the watermark, the amplitude adjustment amount carries the information of the watermark, and then the amplitude of the frequency spectrum information of the audio data is adjusted based on the amplitude adjustment amount, so that the watermark information can be added to the frequency spectrum information of the audio data. Since adding a watermark to audio data through a traditional adding method is computationally simpler than adding a watermark to audio data through a deep learning adding method, adding a watermark to audio data through a traditional adding method has a small amount of computation and a fast speed.
[0185] In the embodiment of the present application, the target audio data is obtained by performing time domain conversion on the target spectrum information carrying the watermark coding information, which means that the watermark in the target audio data is obtained by adjusting the spectrum information of the audio data. Therefore, the target audio data is obtained by adding the watermark to the audio data by the traditional adding method, which means that the calculation amount of obtaining the target audio data by adding the watermark to the audio data is small and the speed is fast.
[0186] Optionally, the center frequency range is determined based on the formant frequency. The formant frequency is the region where the energy of speech is concentrated in the spectrum information. For example, if the energy of speech in the spectrum information is concentrated between 100 Hz and 8000 Hz, then the formant frequency is between 100 Hz and 8000 Hz.
[0187] In a possible implementation, the absolute value of the difference between the center frequency of the center frequency range and the center frequency of the formant frequency is less than or equal to the frequency threshold, and the boundary of the center frequency range is within the formant frequency. For example, if the formant frequency is 100 Hz to 8000 Hz, then the center frequency of the formant frequency is 4050 Hz. If the frequency threshold is 5 Hz, then the center frequency of the center frequency range is 4045 Hz to 4050 Hz. The absolute value of the difference between the center frequency of the center frequency range and the center frequency of the formant frequency is less than or equal to the frequency threshold, indicating that the center frequency of the center frequency range is not much different from the center frequency of the formant frequency. At this time, the coupling degree between the center frequency range and the formant frequency is high, and the smaller the frequency threshold, the higher the coupling degree. The higher the coupling degree, the higher the overlap between the center frequency range and the formant frequency, and further the higher the overlap between the frequency range of the watermark coding information and the frequency range of the speech, which also means that the overlap between the frequency range of the watermark in the target audio data and the frequency range of the speech is high. At this time, if the watermark in the target audio data is filtered, the distortion of the speech is also higher. That is to say, the higher the coupling degree, the stronger the anti-interference ability and the higher the robustness of the watermark in the target audio data.
[0188] In another possible implementation, the maximum value of the amplitude within the center frequency range is the same as the maximum value of the amplitude within the resonance peak frequency, and the minimum value of the amplitude within the center frequency range, the maximum value of the amplitude within the resonance peak frequency, and the minimum value of the amplitude within the resonance peak frequency satisfy the following formula: (maximum value of the resonance peak frequency-minimum value of the amplitude within the center frequency range) / (minimum value of the amplitude within the center frequency range-minimum value of the amplitude within the resonance peak frequency)=preset ratio. Since the maximum value of the amplitude within the center frequency range is the same as the maximum value of the amplitude within the resonance peak frequency, when (maximum value of the resonance peak frequency-minimum value of the amplitude within the center frequency range) / (minimum value of the amplitude within the center frequency range-minimum value of the amplitude within the resonance peak frequency)=preset ratio, the amplitude within the center frequency range is the maximum amplitude within the resonance peak frequency (1-preset ratio). For example, if the preset ratio is 80%, then (maximum value of the resonance peak frequency-minimum value of the amplitude within the center frequency range) / (minimum value of the amplitude within the center frequency range-minimum value of the amplitude within the resonance peak frequency)=80%. At this time, the amplitude within the center frequency range is the maximum 20% amplitude within the resonance peak frequency. Because the larger the amplitude of the audio data, the more difficult it is to remove the audio data, so when the amplitude within the center frequency range is the maximum (1-preset ratio) amplitude within the resonance peak frequency, the information within the center frequency range is difficult to remove, and the difficulty is positively correlated with the preset ratio. In this way, when the spectrum information within the center frequency range of the target spectrum information carries watermark coding information, the amplitude of the watermark coding information is a larger amplitude within the speech frequency range, and thus the watermark coding information is difficult to remove. Therefore, when the target audio data is obtained by time domain conversion of the target spectrum information carrying the watermark coding information, the robustness of the watermark in the target audio data can be improved.
[0189] In another possible implementation, a frequency range with a preset bandwidth is selected from the formant frequency as the center frequency range. For example, if the preset bandwidth is 800 Hz, then a frequency range with a bandwidth of 800 Hz can be selected from the formant frequency. In this implementation, the center frequency range is the frequency range within the formant frequency, and the center frequency range is a subset of the formant frequency. Because the formant frequency is the frequency range of the speech in the spectrum information, the center frequency range and the frequency range of the speech in the spectrum information have an intersection. Therefore, the spectrum information within the center frequency range in the target spectrum information carries the watermark coding information, indicating that the frequency range of the watermark coding information and the frequency range of the speech in the spectrum information have an intersection. In this way, when the target audio data is obtained by time domain conversion of the target spectrum information carrying the watermark coding information, the frequency range of the watermark in the target audio data and the frequency range of the speech in the target audio data have an intersection. At this time, if the watermark in the target audio data is filtered, the higher the distortion of the speech, the stronger the anti-interference ability and the higher the robustness of the watermark in the corresponding target audio data.
[0190] Based on the above three possible implementation methods, it can be seen that when the center frequency range is determined according to the resonance peak frequency, there is an intersection between the frequency range of the watermark coding information and the frequency range of the speech. In this way, when the target audio data is obtained by time domain conversion of the target spectrum information carrying the watermark coding information, there is an intersection between the frequency range of the watermark in the target audio data and the frequency range of the speech in the target audio data. At this time, if the watermark in the target audio data is filtered, the distortion of the speech will be higher. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness.
[0191] 102. Input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model.
[0192] In an embodiment of the present application, the watermark encoding information is the encoding result of the target audio watermark. The watermark extraction model is obtained through pre-training, and the pre-training enables the watermark extraction model to have the ability to extract watermarks from audio data. The watermark extraction model is a deep learning model. In one possible implementation method, pre-training means that the watermark extraction model is iteratively updated based on the difference between the training watermark and the reference watermark, the training watermark is the watermark extracted by the watermark extraction model from the training audio data, and the reference watermark is the watermark carried in the training audio data. Since the reference watermark is the actual watermark in the training audio data, the reference watermark can be used as the true value (groundtruth, GT) in the training process of the watermark extraction model. Therefore, in the case where the training watermark is the watermark extracted by the watermark extraction model from the training audio data, the difference between the training watermark and the reference watermark can characterize the error of the watermark extracted by the watermark extraction model from the training audio data. Therefore, by iteratively updating the watermark extraction model based on the difference between the training watermark and the reference watermark, the error of the watermark extracted by the watermark extraction model from the training audio data can be reduced, and the accuracy of the watermark extracted by the watermark extraction model from the training audio data can be improved. Therefore, the watermark extraction model has the ability to extract watermarks from audio data.
[0193] Optionally, the process of iteratively updating the watermark extraction model based on the difference between the training watermark and the reference watermark includes: determining the loss of the watermark extraction model based on the difference between the training watermark and the reference watermark, wherein the difference is positively correlated with the loss. Determining the partial derivative of the loss of the watermark extraction model with respect to the parameters in the watermark extraction model as the gradient of the watermark extraction model. Updating the parameters of the watermark extraction model in the opposite direction of the gradient of the watermark extraction model to reduce the loss of the watermark extraction model. When the loss of the watermark extraction model is less than or equal to a preset value, the training of the watermark extraction model is completed.
[0194] Since the watermark extraction model has the ability to extract watermarks from audio data, after the target audio data is input into the watermark extraction model, the watermark extraction model can extract the watermark in the target audio data to obtain the target audio watermark, wherein, since the watermark in the target audio data is obtained by performing time domain conversion on the watermark encoding information, the watermark encoding information is the encoding result of the target audio watermark.
[0195] Optionally, the watermark encoding information includes encoding information of a target identifier, wherein the position of the target identifier in the watermark is a preset position. The extraction device inputs the target audio data, the target identifier and the preset position together into the watermark extraction model, so that after the watermark extraction model determines the target identifier from the target audio data, it extracts the target audio watermark from the target audio data based on the target identifier and the preset position, thereby improving the accuracy of the target audio watermark. For example, the target identifier is XHS, and the preset position is the first three digits, that is, the first three digits in the watermark are the target identifier. Then, after the watermark extraction model determines XHS from the target audio data, it determines that the character sequence after XHS is the content of the target audio watermark.
[0196] In the current technology, the method of extracting watermarks from audio data includes a method of extracting watermarks from audio data by deep learning technology and a method of extracting watermarks from audio data by technology other than deep learning technology. For the convenience of expression, the method of extracting watermarks from audio data by deep learning technology is referred to as a deep learning extraction method, and the method of extracting watermarks from audio data by technology other than deep learning technology is referred to as a traditional extraction method. Extracting watermarks from audio data by traditional extraction methods includes extracting watermarks from audio data according to the inverse operation of traditional adding methods. For example, the traditional adding method is to add the watermark to the least significant bit by adjusting the least significant bit of the audio data. The inverse operation of the traditional adding method is to extract the watermark from the least significant bit of the audio data. Then the traditional extraction method includes extracting the watermark from the least significant bit of the audio data. However, watermark interference processing of audio data is easy to destroy the watermark in the audio data. Watermark interference processing refers to processing that can remove the watermark in the audio data. For example, by filtering the audio data, the watermark in the audio data can be removed. The watermark interference processing includes filtering. For another example, denoising the audio data may lead to the destruction of the watermark in the audio data. However, the traditional extraction method extracts the watermark information existing in the audio data. Therefore, the accuracy of extracting the watermark from the audio data by the traditional extraction method is low.
[0197] Extracting watermarks from audio data by deep learning extraction methods is to extract watermarks from audio data by deep learning models. Since the ability of deep learning models to extract watermarks from audio data is acquired through training, when the training process includes extracting watermarks from audio data that has been processed with watermark interference by deep learning models, the deep learning models have the ability to extract watermarks from audio data that has been processed with watermark interference. Therefore, extracting watermarks from audio data by deep learning extraction methods can reduce the impact of watermark interference processing on the accuracy of the extracted watermark, thereby improving the accuracy of the extracted watermark. For example, when watermark interference processing is performed on audio data, resulting in the loss of watermark information in the audio data, the deep learning model can predict the lost watermark information based on the watermark information in the audio data, thereby improving the information of the watermark in the audio data, thereby improving the accuracy of the extracted watermark. Since the watermark extraction model is a deep learning model, the accuracy of the target audio watermark can be improved by extracting the target audio watermark from the target audio data by the watermark extraction model.
[0198] Optionally, the extraction device further performs the following steps before executing step 102: performing frequency domain conversion on the target audio data to obtain target spectrum information. Since the target audio data is obtained by performing time domain conversion on the target spectrum information, the target spectrum information can be obtained by performing frequency domain conversion on the target audio data. After obtaining the target spectrum information, the extraction device performs the following steps in the process of executing step 102: inputting the target spectrum information into a watermark extraction model. Using the watermark extraction model, the spectrum information within the center frequency range is intercepted from the target spectrum information to obtain the target spectrum information. Using the watermark extraction model, the watermark encoding information in the target spectrum information is extracted to obtain the target audio watermark.
[0199] Since the spectrum information within the center frequency range of the target spectrum information carries the watermark coding information, after the target spectrum information is input into the watermark extraction model, the watermark extraction model can be used to intercept the spectrum information within the center frequency range from the target spectrum information to obtain the target spectrum information containing the watermark coding information. The watermark extraction model is then used to extract the watermark coding information from the target spectrum information to obtain the target audio watermark of the target audio data. This can reduce the interference of the spectrum information other than the target spectrum information in the target spectrum information on the watermark extraction model to extract the watermark in the target audio data, thereby improving the accuracy of the target audio watermark of the target audio data extracted by the watermark extraction model.
[0200] In the embodiment of the present application, since the spectrum information within the center frequency range of the target spectrum information carries the watermark encoding information, the center frequency range is the frequency range of the speech in the spectrum information, and the frequency of the watermark in the target audio data is within the frequency range of the speech in the target audio data. At this time, if the watermark in the target audio data is removed by filtering the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness. In this way, after acquiring the target audio data, the extraction device inputs the target audio data into the watermark extraction model obtained through pre-training, and obtains the target audio watermark output by the watermark extraction model, which can improve the accuracy of the target audio watermark.
[0201] Moreover, the target audio data is obtained by time-domain conversion of the target spectrum information carrying the watermark coding information, which means that the watermark in the target audio data is obtained by adjusting the spectrum information of the audio data. Therefore, the calculation amount of the target audio data obtained by adding the watermark to the audio data is small and the speed is fast. The watermark extraction model is obtained through pre-training. During the pre-training process, the watermark extraction model can learn how to extract the watermark from the audio data, wherein the audio data includes the audio data that has not been processed by the watermark interference and the audio data that has been processed by the watermark interference. The watermark interference processing refers to the processing that can remove the watermark in the audio data. For example, the watermark in the audio data can be removed by filtering the audio data, and the watermark interference processing includes filtering. Therefore, by extracting the watermark from the audio data through the watermark extraction model, the influence of the watermark interference processing on the accuracy of the extracted watermark can be reduced, and the accuracy of the extracted watermark can be improved. Therefore, by extracting the target audio watermark from the target audio data through the watermark extraction model, the accuracy of the target audio watermark can be improved. In this way, the watermark can be quickly added to the audio data, and the accuracy of extracting the watermark from the audio data can be improved.
[0202] In a possible scenario, the target audio data is the audio data of the audio on-demand platform. In order to protect the copyright of the audio data, the audio on-demand platform can obtain the target spectrum information carrying the watermark coding information based on the audio data before publishing the audio data to the audio on-demand platform, and then obtain the target audio data by performing time domain conversion on the target spectrum information carrying the watermark coding information. After obtaining the target audio data, the target audio data can be published to the audio on-demand platform. In this way, the amount of calculation for adding watermarks to the audio data can be reduced, and the speed of adding watermarks to the audio data can be increased, thereby reducing the delay of the audio on-demand platform publishing the audio data. Specifically, after the user uploads the audio data to be published to the server of the audio on-demand platform, the server needs to obtain the target audio data by adding a watermark to the audio data to be published, and then publish the target audio data to the audio on-demand platform. The time from the user uploading the audio data to be published to the target audio data being published to the audio on-demand platform is the delay of the audio on-demand platform publishing the audio data, and the delay includes the time consumed by adding a watermark to the audio data to be published to obtain the target audio data. When the target audio data is obtained by time-domain conversion of the target spectrum information carrying the watermark coding information, it means that the target audio data is obtained by adding watermarks by the traditional adding method, so the speed of adding watermarks can be increased, and the time consumed to obtain the target audio data can be reduced, thereby reducing the delay. When it is necessary to extract the watermark from the target audio data of the audio on-demand platform, the target audio data can be input into the pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, thereby improving the accuracy of the target audio watermark.
[0203] In general, the audio on-demand platform uses a method of obtaining target audio data by performing time domain conversion on the target spectrum information carrying watermark encoding information, and extracting the target audio watermark from the target audio data through a watermark extraction model. In this way, the audio on-demand platform can not only increase the speed of adding watermarks to audio data to obtain target audio data, thereby achieving a low latency effect, but also improve the accuracy of extracting target audio watermarks from target audio data.
[0204] As an optional implementation, before executing step 102, the extraction device further executes the following steps:
[0205] 2001. Extract n audio frames to be extracted from the target audio data.
[0206] In the embodiment of the present application, n is an integer greater than 1. In one possible implementation, the extraction device obtains the length of the target audio data. The target audio data is divided into n segments according to the length of the target audio data to obtain n audio frames to be extracted. Optionally, any two audio frames to be extracted in the n audio frames to be extracted do not overlap.
[0207] In another possible implementation, the length of the audio frame to be extracted is greater than the length of the watermark coding information, where the length of the watermark coding information is the duration of the watermark coding information in the audio data. For example, if the duration of the watermark coding information in the audio data is 2 seconds and the length of the watermark coding information is 2 seconds, then the length of the audio frame to be extracted is greater than 2 seconds. By making the length of the audio frame to be extracted greater than the reference length, the probability that a frame of the audio frame to be extracted includes a complete watermark coding information can be increased, thereby improving the efficiency of extracting the target watermark from the audio frame to be extracted.
[0208] Optionally, the length of the audio frame to be extracted is greater than the length of the watermark encoding information and less than twice the length of the watermark encoding information. Since the number of complete watermark encoding information in a frame of audio frame to be extracted exceeds 1, the watermark extraction model can extract at least two watermarks from a frame of audio frame to be extracted. When a frame of audio frame to be extracted corresponds to a candidate audio watermark, the watermark extraction model extracts at least two watermarks from a frame of audio frame to be extracted, which will interfere with the determination of the candidate audio watermark of the audio frame to be extracted, thereby causing errors in the candidate audio watermark output by the watermark extraction model. When the length of the audio frame to be extracted is greater than the length of the watermark encoding information and less than twice the length of the watermark encoding information, it can be avoided that a frame of audio frame to be extracted contains at most 1 complete watermark encoding information, thereby improving the accuracy of the candidate audio watermark.
[0209] In another possible implementation, the extraction device divides the target audio data according to a preset step size to obtain n frames of audio frames to be extracted, wherein the preset step size is greater than 0 and less than the length of the audio frame to be extracted. In this implementation, n is greater than 1. The extraction device divides the target audio data once every preset step size to obtain a frame of audio frame to be extracted. For example, the length of the audio frame to be extracted is 3 seconds, and the preset step size is 1 second. The extraction device can divide the 0th to 3rd seconds of the target audio data into a frame of audio frame to be extracted, and divide the 4th to 7th seconds into a frame of audio frame to be extracted. When the preset step size is greater than 0, the extraction device divides the target audio data according to the preset step size to obtain n frames of audio frames to be extracted, which can improve the efficiency of the division, and further improve the efficiency of obtaining n frames of audio frames to be extracted.
[0210] After intercepting n audio frames to be extracted, the extraction device performs the following steps during the execution of step 102:
[0211] 2002. Input the n audio frames to be extracted into the watermark extraction model respectively to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively.
[0212] In the embodiment of the present application, the candidate audio watermarks correspond to the audio frames to be extracted one by one. The extraction device extracts the watermark from an audio frame to be extracted to obtain a candidate audio watermark.
[0213] 2003. Obtain the target audio watermark based on the n candidate audio watermarks.
[0214] Since the candidate audio watermarks extracted from the audio frame to be extracted may have errors, obtaining the target audio watermark based on n candidate audio watermarks can smooth the error of a single candidate audio watermark, thereby improving the accuracy of the target audio watermark.
[0215] In a possible implementation, n is an integer greater than 1. The number of occurrences of each audio watermark in the n candidate audio watermarks is determined to obtain a target number of occurrences. The target audio watermark is obtained based on the target number of occurrences and the n candidate audio watermarks.
[0216] Since the candidate audio watermarks extracted from different audio frames to be extracted may be the same or different, the same audio watermark may exist in the n candidate audio watermarks, so the number of occurrences of each audio watermark in the n candidate audio watermarks can be any positive integer less than or equal to n, that is, the target number is any positive integer less than or equal to n. For example, n is 3, and the n candidate audio watermarks include candidate audio watermark a, candidate audio watermark b, and candidate audio watermark c, where candidate audio watermark a is: XHS1234567890987654321, candidate audio watermark b is: XHS1234567890987654320, and candidate audio watermark c is: XHS1234567890987654321. Since both candidate audio watermark a and candidate audio watermark c are XHS1234567890987654321, at this time, the number of occurrences of the audio watermark XHS1234567890987654321 is 2, that is, the target number of XHS1234567890987654321 is 2. Since among the n candidate audio watermarks, only candidate audio watermark b is XHS1234567890987654320, the number of occurrences of the audio watermark XHS1234567890987654320 is 1, that is, the target number of XHS1234567890987654320 is 1.
[0217] Since the more times an audio watermark appears, the higher the confidence that the audio watermark is the target audio watermark, the extraction device obtains the target audio watermark based on the target number of times and the n candidate audio watermarks, which can improve the confidence of the target audio watermark. In a possible implementation, the extraction device uses the audio watermark with the largest target number of times among the n candidate audio watermarks as the target audio watermark, thereby determining the target audio watermark from the n candidate audio watermarks through the principle of minority obeys majority, thereby improving the accuracy of the target audio watermark.
[0218] In another possible implementation, n is an integer greater than 1. When the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the extraction device uses the audio watermark that appears most frequently among the n candidate audio watermarks as the target audio watermark. The difference between the maximum value of the target number of times and the second largest value of the target number of times is large, indicating that the confidence that the audio watermark with the largest target number of times among the n candidate audio watermarks is the target audio watermark is high. In the embodiment of the present application, the extraction device uses the number threshold as a basis to determine whether the difference between the maximum value of the target number of times and the second largest value of the target number of times is large or small. Specifically, the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, indicating that the difference between the maximum value of the target number of times and the second largest value of the target number of times is large, and vice versa, the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, indicating that the difference between the maximum value of the target number of times and the second largest value of the target number of times is small. Therefore, when the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the extraction device uses the audio watermark that appears most frequently among the n candidate audio watermarks as the target audio watermark, which can improve the accuracy of the target audio watermark.
[0219] In another possible implementation, n is an integer greater than 1. When the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, the value of the target position in the n candidate audio watermarks is determined to obtain n candidate values. The value with the largest number of occurrences among the n candidate values is determined as the target value of the target position in the target audio watermark. The target audio watermark is obtained according to the target value.
[0220] As mentioned above, the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, indicating that the difference between the maximum value of the target number of times and the second largest value of the target number of times is small, which also means that the difference between the confidence of the candidate audio watermark with the largest target number of times as the target audio watermark and the confidence of the candidate audio watermark with the second largest target number of times as the target audio watermark is small. At this time, taking the candidate audio watermark with the largest target number of times or the candidate audio watermark with the second largest target number of times as the target audio watermark is likely to lead to low accuracy of the target audio watermark. In addition, the confidence of the audio watermarks other than the audio watermark with the largest target number of times and the audio watermark with the second largest target number of times among the n candidate audio watermarks as the target audio watermark is lower than the confidence of taking the candidate audio watermark with the largest target number of times as the target audio watermark. Therefore, the extraction device determines the value of the target position among the n candidate audio watermarks to obtain n candidate values, and then determines the value with the most occurrences among the n candidate values as the target value of the target position in the target audio watermark, wherein the target position is any position in the target audio watermark.
[0221] In the embodiment of the present application, the watermark is a sequence of characters, and the position in the watermark is the position of the character in the sequence. For example, the target audio watermark is XHS1234567890987654321. Since X is the first character in the sequence, the position of X in the target audio watermark is the first position. Since H is the second character in the sequence, the position of H in the target audio watermark is the second position.
[0222] Since the more times the candidate value appears, the higher the confidence of the target value in the target audio watermark as the candidate value, the extraction device determines the value with the most appearances among the n candidate values as the target value of the target position in the target audio watermark, which can improve the confidence of the target value. For example, n is 3, and the n candidate audio watermarks include candidate audio watermark a, candidate audio watermark b, and candidate audio watermark c, where candidate audio watermark a is: XHS1234567890987654321, candidate audio watermark b is: XHS1234567890987654320, and candidate audio watermark c is: XHS1234567890987654321. If the target position is the last digit in the watermark, then the candidate value in candidate audio watermark a is 1, the candidate value in candidate audio watermark b is 0, and the candidate value in candidate audio watermark a is 1. At this time, candidate value 1 appears the most times, so the target value is determined to be 1, that is, the last digit in the target audio watermark is 1.
[0223] After determining the target value, the extraction device may determine the target audio watermark based on the target value. Optionally, the extraction device determines each value in the target audio watermark in a manner of determining the target value, thereby obtaining the target audio watermark.
[0224] In this implementation, when the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, the value of the target position in the n candidate audio watermarks is determined to obtain n candidate values. The value with the largest number of occurrences among the n candidate values is determined as the target value of the target position in the target audio watermark, thereby improving the accuracy of the target value. Then, the target audio watermark is obtained according to the target value, which can improve the accuracy of the target audio watermark.
[0225] Optionally, an audio frame to be extracted is intercepted from the target audio data and the audio frame to be extracted is input into the watermark extraction model to obtain the target audio watermark.
[0226] See also Figure 2 , Figure 2 The present invention provides a flow chart of obtaining a target audio watermark based on n candidate audio watermarks. Figure 2 As shown, the n candidate audio watermarks include: 123456789, 123456789, 123456789, 123456788, and 123456786, among which there are 3 candidate audio watermarks of 123456789, 1 candidate audio watermark of 123456788, and 1 candidate audio watermark of 123456786. The number of occurrences of 123456789 in the n candidate audio watermarks is 3, that is, the target number of occurrences of 123456789 is 3. The number of occurrences of 123456788 in the n candidate audio watermarks is 1, that is, the target number of occurrences of 123456788 is 1. The number of occurrences of 123456786 in the n candidate audio watermarks is 1, that is, the target number of occurrences of 123456786 is 1. It is determined whether the maximum value of the target number of times - the second largest value of the target number of times is greater than the number threshold. If the judgment result is yes, the target audio watermark is determined to be the audio watermark that appears the most times among the n candidate audio watermarks, that is, the target audio watermark is 123456789. If the judgment result is no, a bit-by-bit vote is performed. Specifically, a bit-by-bit vote is performed based on the n candidate audio watermarks (that is, 3 123456789, 1 123456788, and 1 123456786). Figure 2 In the figure, n candidate audio watermarks are represented by 123456789, 123456788, and 123456786. Specifically, for each bit in the target audio watermark, the value that appears most frequently in the same position of the n candidate audio watermarks is taken, and finally the target audio watermark is determined to be 123456789. For the implementation method of taking the value that appears most frequently in the same position of the n candidate audio watermarks for each bit in the target audio watermark, refer to determining the value of the target position in n candidate audio watermarks, obtaining n candidate values, and determining the value that appears most frequently in the n candidate values as the target value of the target position in the target audio watermark.
[0227] As an optional implementation, the n audio frames to be extracted include a first audio frame, the n candidate audio watermarks include the first audio watermark, and the first audio watermark is an audio watermark obtained after the first audio frame is input into a watermark extraction model. The extraction device performs the following steps during the execution of step 2002:
[0228] 3001. Input the first audio frame into the audio frame segmentation module of the watermark extraction model to obtain m audio frames to be confirmed.
[0229] In the embodiment of the present application, the audio frame to be confirmed is a part of the first audio frame. The audio frame division module is a module in the watermark extraction model, and the audio frame division module is used to divide the audio frame. After the watermark extraction model obtains the first audio frame, it is necessary to first determine the location of the watermark from the first audio frame, and then extract the watermark from the location of the watermark. Therefore, the first audio frame is input into the audio frame division module of the watermark extraction model to divide the first audio frame into m segments of audio frames to be confirmed, so as to facilitate the subsequent determination of the audio frame containing the watermark from the m segments of audio frames to be confirmed, and to extract the watermark from the audio frame containing the watermark.
[0230] In a possible implementation, the audio frame division module determines the absolute value of the difference between the information of each sampling point in the first audio frame and the information of the adjacent sampling point as the reference difference of each sampling point, wherein the information includes amplitude and phase, and adjacent means adjacent in time stamp. For example, the first audio frame includes sampling point a, sampling point b, and sampling point c, wherein sampling point a is the sampling point with the smallest time stamp in the first audio frame, sampling point b is the sampling point with a time stamp adjacent to sampling point a in the first audio frame, sampling point c is the sampling point with a time stamp adjacent to sampling point b in the first audio frame, and the time stamp of sampling point c is larger than the time stamp of sampling point b. If the information is amplitude, then the reference difference of sampling point a is the absolute value of the difference between the amplitude of sampling point a and the amplitude of sampling point b. Since the sampling points adjacent to sampling point b include sampling point a and sampling point c, the reference difference of sampling point b is obtained by determining the average of the two absolute values after respectively determining the absolute value of the difference between the amplitude of sampling point b and two adjacent sampling points. Specifically, the absolute value of the difference between the amplitude of sampling point b and the amplitude of sampling point a is determined to obtain an absolute value 1, the absolute value of the difference between the amplitude of sampling point b and the amplitude of sampling point c is determined to obtain an absolute value 2, and an average value of the absolute value 1 and the absolute value 2 is determined as a reference difference of sampling point b. The reference difference of sampling point c is the absolute value of the difference between the amplitude of sampling point c and the amplitude of sampling point b.
[0231] In the target audio data, the time interval between the timestamps of adjacent sampling points is small, so the difference in information of adjacent sampling points is small. The watermark is added to the target spectrum information by adjusting the target spectrum information. Therefore, when the target audio data is obtained by time domain transformation of the target spectrum information, the information of the sampling point containing the watermark in the target audio data will change, which will lead to a large difference between the information of the sampling point containing the watermark and the information of the adjacent sampling points. Therefore, after obtaining the reference difference of each sampling point, the audio frame division module determines the watermark confidence of each sampling point based on the difference of each sampling point, wherein the reference difference is positively correlated with the watermark confidence.
[0232] The audio frame division module then determines the m sampling points with the highest watermark confidence from all the sampling points of the first audio frame as m seed sampling points. Then, for each of the m seed sampling points, the seed sampling point and the sampling points adjacent to the seed sampling point are determined as m segments of candidate audio frames. For each of the m segments of candidate audio frames, when the average value of the watermark confidence of the sampling points in the candidate audio frame is greater than the confidence threshold, the sampling points whose timestamps are adjacent to the candidate audio frame are added to the candidate audio frame. When the average value of the watermark confidence of the sampling points in the candidate audio frame is less than or equal to the confidence threshold, the candidate audio frame is determined to be an audio frame to be confirmed. In this way, m segments of audio frames to be confirmed can be obtained based on the m seed sampling points.
[0233] A watermark is usually added to one or more sampling points in the audio data, and the timestamps of the sampling points containing the same watermark are usually adjacent. Therefore, the candidate audio frame can be expanded outward with the seed sampling point as the center. If the average value of the watermark confidence of the sampling points in the candidate audio frame is large, it means that the sampling points with timestamps adjacent to the candidate audio frame have a high confidence that they contain the watermark, and then the sampling points with timestamps adjacent to the candidate audio frame can be further expanded outward to be added to the candidate audio frame. On the contrary, if the average value of the watermark confidence of the sampling points in the candidate audio frame is small, it means that the sampling points with timestamps adjacent to the candidate audio frame have a low confidence that they contain the watermark, and then the candidate audio frame can be determined as the audio frame to be confirmed.
[0234] The audio frame division module uses the confidence threshold as a basis to determine whether the average value of the watermark confidence of the sampling points in the candidate audio frame is large or small. Specifically, if the average value of the watermark confidence of the sampling points in the candidate audio frame is greater than the confidence threshold, it means that the average value of the watermark confidence of the sampling points in the candidate audio frame is large. Conversely, if the average value of the watermark confidence of the sampling points in the candidate audio frame is less than or equal to the confidence threshold, it means that the average value of the watermark confidence of the sampling points in the candidate audio frame is small. Therefore, when the average value of the watermark confidence of the sampling points in the candidate audio frame is less than or equal to the confidence threshold, the audio frame division module determines that the candidate audio frame is the audio frame to be confirmed, which can increase the probability that the audio frame to be confirmed contains a watermark.
[0235] In another possible implementation, the audio frame division module determines the starting position of the sliding time window and the ending position of the sliding time window from the first audio frame. The sliding time window is controlled to slide from the starting position to the ending position according to a preset sliding step. According to the audio frames in the sliding time window, m segments of audio frames to be confirmed are obtained. Optionally, when the sliding time window is at the starting position, the sliding time window includes the sampling point with the smallest timestamp in the first audio frame. When the sliding time window is at the ending position, the sliding time window includes the sampling point with the largest timestamp in the first audio frame. Optionally, the preset sliding step is i sampling points, wherein the smaller i is, the more sampling points the sliding time window can cover during the sliding process, thereby increasing the probability that the audio frame in the sliding time window contains a watermark.
[0236] 3002. Input the m audio frames to be confirmed into the probability prediction module of the watermark extraction model respectively, predict the probability that each audio frame to be confirmed contains a watermark, and obtain m prediction probabilities.
[0237] In the embodiment of the present application, the probability prediction module is used to predict the probability of a watermark in an audio frame. A segment of an audio frame to be confirmed is input into the probability prediction module, and the probability of the segment of the audio frame to be confirmed containing a watermark can be predicted to obtain the predicted probability of the segment of the audio frame to be confirmed, that is, the predicted probability is the probability of the audio frame to be confirmed containing a watermark, and the predicted probability corresponds to the audio frame to be confirmed.
[0238] Optionally, the probability prediction module is trained by the following training process: inputting a training audio frame into the probability prediction module. The probability prediction module predicts the probability that the training audio frame contains a watermark to obtain a training probability. Based on the difference between the training probability and the label of the training audio frame, a probability prediction loss is obtained, wherein the label represents whether the training audio frame contains a watermark. Based on the probability prediction loss, the parameters of the probability prediction module are updated until the probability prediction loss converges.
[0239] 3003. Input the m prediction probabilities into the candidate frame determination module of the watermark extraction model, and determine that the to-be-confirmed audio frames corresponding to the largest s of the m prediction probabilities are s candidate audio frames.
[0240] In an embodiment of the present application, the candidate frame determination module is used to determine s candidate audio frames that are most likely to contain a watermark from m audio frames to be confirmed, where s is less than m. The greater the predicted probability of the audio frame to be confirmed, the greater the probability that the audio frame to be confirmed contains a watermark. Therefore, the candidate frame determination module determines the audio frames to be confirmed corresponding to the largest s of the m predicted probabilities as s candidate audio frames. The candidate frame determination module can filter out the audio frames to be confirmed that have a low probability of containing a watermark by determining s candidate audio frames from m audio frames to be confirmed.
[0241] 3004. Input the s candidate audio frames into the audio watermark extraction module of the watermark extraction model to obtain the first audio watermark.
[0242] In the embodiment of the present application, the audio watermark extraction module is used to extract the watermark from the audio frame. Since the s candidate audio frames are audio frames with a high probability of containing the watermark in the first audio frame, the s candidate audio frames are input into the audio watermark extraction module, so that the audio watermark extraction module extracts the watermark from the s candidate audio frames as the first audio watermark.
[0243] In this embodiment, the extraction device inputs the first audio frame into the audio frame division module of the watermark extraction model, so that the audio frame division module can obtain m segments of audio frames to be confirmed by dividing the first audio frame. Then, the m segments of audio frames to be confirmed are respectively input into the probability prediction module of the watermark extraction model, and the probability of each audio frame to be confirmed containing a watermark can be predicted respectively to obtain m prediction probabilities. The m prediction probabilities are then input into the candidate frame determination module of the watermark extraction model, and s candidate audio frames with the largest prediction probabilities can be determined from the m segments of audio frames to be confirmed, thereby determining the s candidate audio frames most likely to contain a watermark from the first audio frame. Finally, the s candidate audio frames are input into the audio watermark extraction module of the watermark extraction model, and the first audio watermark in the first audio frame can be obtained by extracting the audio watermark from the s candidate audio frames.
[0244] It should be understood that, during the execution of step 2002, the extraction device may extract the audio watermarks in each audio frame to be extracted in the same manner as extracting the first audio watermark in the first audio frame, to obtain n candidate audio watermarks.
[0245] The present application embodiment also provides another method for extracting audio watermarks. The following will describe another method for extracting audio watermarks. The execution subject of the method is an extraction device. Figure 3 , Figure 3A flowchart of another method for extracting an audio watermark provided in an embodiment of the present application.
[0246] 301. Obtain target audio data carrying a watermark.
[0247] In the embodiment of the present application, the watermark in the target audio data is obtained by adding the watermark to the original target audio through a first watermark adding method. The first watermark adding method can be any watermark adding method.
[0248] In a possible implementation, the watermark adding method is to adjust the amplitude of the spectrum information of the audio data based on the watermark coding information, and perform time domain transformation on the adjusted spectrum information. For example, the watermark coding information is a binary sequence, and the number of bits of the binary sequence is x, where x is an integer greater than 1. The amplitude of the spectrum information of the audio data is divided into x groups to obtain x groups of amplitudes, where the difference between the average values of the amplitudes in any two groups of amplitudes is less than or equal to the amplitude threshold. In the binary sequence, the value of the bit is 0 or 1, and a first amplitude adjustment amount corresponding to the bit with a value of 0 is determined, and a second amplitude adjustment amount corresponding to the bit with a value of 1 is determined, where the first amplitude adjustment amount is different from the second amplitude adjustment amount. When the value of the bit in the binary sequence is 0, the amplitude in a group of amplitudes is adjusted based on the first amplitude adjustment amount, and when the value of the bit in the binary sequence is 1, the amplitude in a group of amplitudes is adjusted based on the second amplitude adjustment amount, and the bit in the binary sequence corresponds to a group of amplitudes one by one. In this way, the information of the binary sequence can be added to the spectrum information of the audio data by adjusting the spectrum information of the audio data, thereby achieving the addition of the watermark to the spectrum information of the audio data.
[0249] In another possible implementation, the watermark adding method is to add the watermark to the least significant bit by adjusting the least significant bit in the audio data. In yet another possible implementation, the watermark adding method is to add the watermark to the audio data based on a watermark adding model, wherein the watermark adding model is a deep learning model capable of adding a watermark to the audio data.
[0250] In an embodiment of the present application, the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method, that is, the first watermark extraction method is a watermark extraction method corresponding to the first watermark adding method, wherein the watermark extraction method is used to extract watermarks from audio data.
[0251] In one possible implementation, the first watermark extraction method is the reverse processing of the first watermark adding method. For example, the first watermark adding method is to add the watermark to the audio data by adjusting the amplitude of the spectral information of the audio data. Then the reverse processing of the first watermark adding scheme is to extract the watermark from the audio data based on the amplitude of the spectral information of the audio data.
[0252] Optionally, the first watermark adding method is as follows: the watermark encoding information of the watermark is a binary sequence, and the number of bits of the binary sequence is x. The amplitude of the frequency spectrum information of the audio data is divided into x groups of amplitude groups, and the difference between the average values of the amplitudes in any two groups of amplitudes is less than or equal to the amplitude threshold. According to the x-bit numerical value in the binary sequence, x amplitude adjustment amounts are determined, and the amplitude adjustment amounts correspond to the numerical values in the binary sequence one by one. The amplitude adjustment amounts corresponding to the same numerical values are the same, and the amplitude adjustment amounts corresponding to different numerical values are different. Using the x amplitude adjustment amounts, the x groups of amplitudes in the frequency spectrum information of the audio data are adjusted to add the watermark to the audio data.
[0253] At this time, the first watermark extraction method is as follows: after determining x groups of amplitudes from the spectrum information of the audio data, the bits corresponding to the group of amplitudes are determined based on the average value of the amplitudes in a group of amplitudes. Specifically, before adjusting the amplitudes in the group of amplitudes, the difference between the average values of the amplitudes in any two groups of amplitudes is less than or equal to the amplitude threshold, indicating that the difference in the average values of the amplitudes in any two groups of amplitudes is not large. Because the amplitude adjustment amounts corresponding to bits of different sizes are different, the average values of the amplitudes of different groups of amplitudes corresponding to bits of different sizes are relatively different after the amplitudes are adjusted. Therefore, the value of the bits corresponding to the group of amplitudes can be determined based on the average value of the amplitudes in a group of amplitudes. For example, if the first adjustment amount is 0 and the second adjustment amount is 0.3, then the average value of the amplitudes in a group of amplitudes adjusted based on the first adjustment amount is smaller than the average value of the amplitudes in a group of amplitudes adjusted based on the first adjustment amount. Therefore, the value corresponding to the group of amplitudes with a smaller average value in the x groups of amplitudes is determined to be 0, and the value corresponding to the group of amplitudes with a larger average value in the x groups of amplitudes is determined to be 1. After determining the values corresponding to each group of amplitudes, the binary sequence can be determined based on the correspondence between each group of amplitudes and the values of the bits of the binary sequence. For example, x is 2, which means that the binary sequence has 2 bits. The group of amplitudes corresponding to the value of the first bit in the binary sequence is the first group of amplitudes in the x groups of amplitudes, and the group of amplitudes corresponding to the value of the second bit in the binary sequence is the second group of amplitudes in the x groups of amplitudes. If the value corresponding to the first group of amplitudes is 0 and the value corresponding to the second group of amplitudes is 1, then the binary sequence is 01, that is, the audio watermark extracted based on the watermark extraction method is 01.
[0254] In another possible implementation, the method adopted by the first watermark extraction method is corresponding to the method adopted by the first watermark extraction method. For example, the first watermark adding method is to add the watermark to the audio data by adjusting the amplitude of the spectrum information of the audio data, and the method adopted by the first watermark adding method is to adjust the amplitude of the spectrum information of the audio data. The method corresponding to adjusting the amplitude of the spectrum information of the audio data is to extract the watermark from the amplitude of the spectrum information of the audio data, so the first watermark extraction method is to extract the watermark from the amplitude of the spectrum information of the audio data. For another example, the method adopted by the first watermark adding method is to add the watermark to the audio data by adjusting the least significant bit of the audio data. The method corresponding to adjusting the least significant bit of the audio data is to extract the watermark from the least significant bit of the audio data, so the first watermark extraction method is to extract the watermark from the least significant bit of the audio data. For another example, the method adopted by the first watermark adding method is to add the watermark to the audio data by deep learning technology. The method corresponding to adding the watermark by deep learning technology is to extract the watermark by deep learning technology, so the first watermark extraction method is to extract the watermark from the audio data by deep learning technology, that is, the first watermark extraction method is a deep learning extraction method.
[0255] In another possible implementation, when the first watermark adding method is a traditional adding method, the first watermark extraction method is a traditional extraction method. When the first watermark adding method is a deep learning adding method, the first watermark extraction method is a deep learning extraction method.
[0256] 302. Extract a watermark from the target audio data using a second watermark extraction method to obtain a target audio watermark carried by the target audio data.
[0257] In the embodiment of the present application, the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method, that is, the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method.
[0258] In one possible implementation, the second watermark extraction method is the reverse processing of the second watermark adding method. For example, the second watermark adding method is to add the watermark to the audio data by adjusting the amplitude of the spectral information of the audio data. Then the reverse processing of the second watermark adding scheme is to extract the watermark from the audio data based on the amplitude of the spectral information of the audio data.
[0259] In another possible implementation, the method adopted by the second watermark extraction method is corresponding to the method adopted by the second watermark extraction method. For example, the second watermark adding method is to add the watermark to the audio data by adjusting the amplitude of the spectrum information of the audio data, and the method adopted by the second watermark adding method is to adjust the amplitude of the spectrum information of the audio data. The method corresponding to adjusting the amplitude of the spectrum information of the audio data is to extract the watermark from the amplitude of the spectrum information of the audio data, so the second watermark extraction method is to extract the watermark from the amplitude of the spectrum information of the audio data. For another example, the method adopted by the second watermark adding method is to add the watermark to the audio data by adjusting the least significant bit of the audio data. The method corresponding to adjusting the least significant bit of the audio data is to extract the watermark from the least significant bit of the audio data, so the second watermark extraction method is to extract the watermark from the least significant bit of the audio data. For another example, the method adopted by the second watermark adding method is to add the watermark to the audio data by deep learning technology. The method corresponding to adding the watermark by deep learning technology is to extract the watermark by deep learning technology, so the second watermark extraction method is to extract the watermark from the audio data by deep learning technology, that is, the second watermark extraction method is a deep learning extraction method.
[0260] In another possible implementation, when the second watermark adding method is a traditional adding method, the second watermark extraction method is a traditional extraction method. When the second watermark adding method is a deep learning adding method, the second watermark extraction method is a deep learning extraction method.
[0261] In the embodiment of the present application, the second watermark adding method is a watermark adding method different from the first watermark adding method. For the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method. Moreover, for the same audio data, the accuracy of extracting a watermark from the audio data by the second watermark extraction method corresponding to the second watermark adding method is higher than the accuracy of extracting a watermark from the audio data by the first watermark extraction method corresponding to the first watermark adding method.
[0262] In a possible implementation, the extraction device performs the following steps during the execution of step 302: inputting the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model. Since the watermark extraction model is a deep learning model, the second watermark extraction method is a deep learning extraction method.
[0263] Optionally, the target audio data is input into a pre-trained watermark extraction model, and the implementation process of obtaining the target audio watermark output by the watermark extraction model can refer to the implementation process of step 102.
[0264] Optionally, the first watermark adding method is to adjust the frequency spectrum information of the audio data based on the watermark encoding information to add the watermark to the audio data. As described in step 101, adding the watermark to the audio data by the traditional adding method is to adjust the frequency spectrum information of the audio data based on the watermark, so the first watermark adding method is a traditional adding method.
[0265] As described in step 101, adding a watermark to audio data by a traditional adding method is simpler than adding a watermark to audio data by a deep learning adding method, so the calculation amount of adding a watermark to audio data by a traditional adding method is small and fast. Therefore, in the case where the first watermark adding method is a traditional adding method and the second watermark adding method is a deep learning adding method, for the same watermark and original audio data, the calculation amount of adding a watermark to the original audio data by the first watermark adding method is less than the calculation amount of adding a watermark to the original audio data by the second watermark adding method. As described in step 102, extracting a watermark from audio data by a deep learning extraction method has a higher accuracy than extracting a watermark from audio data by a traditional extraction method. Therefore, in the case where the second watermark extraction method is a traditional extraction method and the second watermark extraction method is a deep learning extraction method, for the same audio data, the accuracy of extracting a watermark from audio data by the second watermark extraction method is higher than the accuracy of extracting a watermark from audio data by the first watermark extraction method.
[0266] In the embodiment of the present application, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method. For the same audio data, the accuracy of extracting a watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting a watermark from the audio data by the first watermark extraction method. Therefore, in the case where the watermark in the target audio data is obtained by adding the first watermark adding method to the original target audio, after acquiring the target audio data, the extraction device extracts the watermark from the target audio data by the second watermark extraction method to obtain the target audio watermark carried by the target audio data, thereby realizing the addition of a watermark by a watermark adding method with a small amount of calculation, and the extraction of a watermark by a watermark extraction method with a high accuracy. This can both increase the speed of adding watermarks and increase the accuracy of watermark extraction.
[0267] In a possible scenario, the target audio data is the audio data of the audio on-demand platform. In order to protect the copyright of the audio data, the audio on-demand platform can add a watermark to the audio data by a first watermark adding method before publishing the audio data to the audio on-demand platform. When it is necessary to extract the watermark in the audio data of the audio on-demand platform, the watermark can be extracted by a second watermark extraction method. In this way, the amount of calculation for adding a watermark to the audio data can be reduced, and the speed of adding a watermark to the audio data can be increased, thereby reducing the delay of the audio on-demand platform publishing the audio data. Specifically, after the user uploads the audio data to be published to the server of the audio on-demand platform, the server needs to obtain the target audio data by adding a watermark to the audio data to be published, and then publish the target audio data to the audio on-demand platform. The time from the user uploading the audio data to be published to the target audio data being published to the audio on-demand platform is the delay of the audio on-demand platform publishing the audio data, and the delay includes the time consumed by adding a watermark to the audio data to obtain the target audio data. Since the target audio data is obtained by adding a watermark to the audio data by the first watermark adding method, the speed of adding watermark can be improved, so the target audio data is obtained by adding a watermark to the audio data by the first watermark adding method, which can reduce the delay. Moreover, the watermark is extracted from the target audio data by the second watermark extraction method to obtain the target audio watermark carried by the target audio data, which can also improve the accuracy of the target audio watermark.
[0268] The embodiment of the present application also provides an audio processing method, which will be described below. The execution subject of the method is an audio processing device, wherein the audio processing device can be any electronic device that can execute the technical solution disclosed in the embodiment of the method of the present application. Optionally, the audio processing device may include a device for adding an audio watermark and a device for extracting an audio watermark. For example, the audio processing method can be implemented together by a device for adding an audio watermark and a device for extracting an audio watermark, wherein the device for adding an audio watermark is used to execute the step of adding the watermark to be added to the audio data to obtain the target audio data, and the device for extracting the audio watermark is used to execute the step of extracting the watermark from the audio data. At this time, the execution subject of the audio processing method is the device for adding an audio watermark and the device for extracting the audio watermark, that is, the audio processing device includes a device for adding an audio watermark and a device for extracting the audio watermark.
[0269] See also Figure 4 , Figure 4 A flowchart of an audio processing method provided in an embodiment of the present application.
[0270] 401. Obtain audio data to be processed and watermark to be added.
[0271] In the embodiment of the present application, the audio data to be processed may be any piece of audio data. For example, the audio data to be processed is a segment of speech, another example, the audio data to be processed is a segment of audio data in a video, and another example, the audio data to be processed is a song.
[0272] In an implementation of obtaining audio data to be processed, the audio processing device receives the audio data to be processed input by the user through an input component. The input component includes at least one of the following: a keyboard, a mouse, a touch screen, a touch pad, and an audio data input device.
[0273] In another implementation of obtaining the audio data to be processed, the audio processing device receives the audio data to be processed sent by a terminal. The terminal may be any of the following: a mobile phone, a computer, a tablet computer, or a server.
[0274] In the embodiment of the present application, the watermark to be added is a digital watermark. The watermark to be added may include any information, for example, the watermark to be added is a string of numbers, and for example, the watermark to be added is a combination of letters and numbers. In one implementation of obtaining the watermark to be added, the audio processing device receives the watermark to be added input by the user through the input component. In another implementation of obtaining the watermark to be added, the audio processing device receives the watermark to be added sent by the terminal.
[0275] It should be understood that in the embodiment of the present application, the step of the audio processing device obtaining the audio data to be processed and the step of obtaining the watermark to be added can be performed separately or simultaneously, and the present application does not limit this.
[0276] 402. Encode the watermark to be added to obtain watermark encoding information.
[0277] The audio processing device can convert the format of the watermark to be added into a data format that can be processed by the computer device by encoding the watermark to be added. In a possible implementation, the audio processing device can convert the watermark to be added into a binary sequence by encoding the watermark to be added. In this case, the watermark encoding information is a binary sequence.
[0278] 403. Perform frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed.
[0279] The audio processing device can convert the audio data to be processed into time domain information by performing frequency domain transformation on the audio data to be processed, thereby obtaining frequency spectrum information to be processed.
[0280] 404. Based on the watermark coding information, adjust the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information.
[0281] In the embodiment of the present application, the adjusted spectrum information includes one or more of the following: amplitude, phase. The audio processing device determines the adjustment amount of the spectrum information within the center frequency range based on the watermark coding information, and then adjusts the spectrum information within the center frequency range based on the adjustment amount of the spectrum information to obtain the target spectrum information. Since the adjustment amount of the spectrum information is obtained based on the watermark coding information, the adjustment amount of the spectrum information carries the watermark coding information, and then the spectrum information within the center frequency range in the spectrum information to be processed is adjusted based on the adjustment amount of the spectrum information, and the watermark coding information can be added to the spectrum information within the center frequency range.
[0282] In a possible implementation, the adjusted spectrum information includes amplitude. The audio processing device determines the amplitude adjustment amount within the center frequency range based on the watermark coding information, and then adjusts the amplitude within the center frequency range based on the amplitude adjustment amount to obtain the target spectrum information. Since the amplitude adjustment amount is obtained based on the watermark coding information, the amplitude adjustment amount carries the watermark coding information, and then the spectrum information within the center frequency range in the spectrum information to be processed is adjusted based on the amplitude adjustment amount, so that the watermark coding information can be added to the spectrum information within the center frequency range.
[0283] In the embodiment of the present application, the center frequency range is the frequency range of the speech in the spectrum information. The spectrum information within the center frequency range in the spectrum information to be processed is the spectrum information of the speech in the spectrum information to be processed. Based on the watermark coding information, the spectrum information within the center frequency range in the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark coding information can be added to the spectrum information within the center frequency range to obtain the target spectrum information. At this time, the spectrum information within the center frequency range in the target spectrum information carries the watermark coding information, which means that the frequency of the watermark coding information in the target spectrum information is within the frequency range of the speech in the spectrum information.
[0284] 405. Perform time domain transformation on the target spectrum information to obtain target audio data carrying a watermark.
[0285] The audio processing device can convert the target spectrum information into time domain information by performing a time domain transformation on the target spectrum information, and obtain the target audio data carrying the watermark, wherein the watermark coding information is the coding result of the watermark in the target audio data. Since the spectrum information within the center frequency range of the target spectrum information carries the watermark coding information, when the target audio data is obtained by performing a time domain transformation on the target spectrum information, and the watermark coding information is the coding result of the watermark in the target audio data, the frequency of the watermark in the target audio data is within the frequency range of the speech in the target audio data. At this time, if the target audio data is filtered to remove the watermark in the target audio data, then the watermark whose frequency is within the frequency range of the speech needs to be removed by filtering, which will cause the speech to be removed as well, thereby causing the speech to be distorted. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness.
[0286] 406. Input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model.
[0287] In the embodiment of the present application, the watermark encoding information is the encoding result of the target audio watermark. The watermark extraction model is obtained through pre-training, and the pre-training enables the watermark extraction model to have the ability to extract watermarks from audio data. The watermark extraction model is a deep learning model. The implementation process of this step can refer to the implementation process of step 102, which will not be repeated here.
[0288] In an embodiment of the present application, after obtaining the audio data to be processed and the watermark to be added, the watermark to be added is encoded to obtain watermark encoding information. The audio data to be processed is transformed in the frequency domain to obtain the spectrum information to be processed. Based on the watermark encoding information, the spectrum information within the center frequency range in the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark encoding information can be added to the spectrum information within the center frequency range. The target spectrum information is transformed in the time domain to obtain the target audio data carrying the watermark. In the target audio data, the frequency of the watermark is within the center frequency range. If the target audio data is filtered to remove the watermark in the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness. Furthermore, as described in step 101, adding a watermark to audio data by a traditional adding method is based on adjusting the frequency spectrum information of the audio data by the watermark. Therefore, the audio processing device obtains the target audio data by adding the watermark to be added to the audio data to be processed by the traditional adding method, thereby reducing the amount of calculation for adding the watermark to be added to the audio data to be processed.
[0289] After obtaining the target audio data, the audio processing device inputs the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model. That is, the audio processing device can extract the target audio watermark from the target audio data through a deep learning extraction method, thereby improving the accuracy of the target audio watermark.
[0290] In a possible implementation scenario, the target audio data is the audio data of the audio on-demand platform. In order to protect the copyright of the audio data, the audio on-demand platform needs to add a watermark to the audio data to be processed to obtain the target audio data before publishing the audio data to the audio on-demand platform. After obtaining the target audio data, the target audio data can be published to the audio on-demand platform. The watermark to be added is encoded to obtain watermark encoding information. The audio data to be processed is transformed in the frequency domain to obtain the spectrum information to be processed. Based on the watermark encoding information, the spectrum information within the center frequency range in the spectrum information to be processed is adjusted to obtain the target spectrum information. The target spectrum information is transformed in the time domain to obtain the target audio data carrying the watermark. The amount of calculation for adding a watermark to the audio data can be reduced, and the speed of adding a watermark to the audio data to be processed can be increased, thereby reducing the delay of the audio on-demand platform to publish the audio data. When it is necessary to extract the watermark from the target audio data of the audio on-demand platform, the target audio data can be input into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, thereby improving the accuracy of the target audio watermark.
[0291] The embodiment of the present application also provides a method for adding an audio watermark. The method for adding an audio watermark is described below. The execution subject of the method is a device for adding an audio watermark (hereinafter referred to as an adding device). The adding device can be any electronic device that can execute the technical solution disclosed in the embodiment of the method of the present application. Optionally, the adding device can be one of the following: a computer, a server. Please refer to Figure 5 , Figure 5 A flowchart of a method for adding an audio watermark provided in an embodiment of the present application.
[0292] 501. Obtain audio data to be processed and watermark to be added.
[0293] In the embodiment of the present application, the audio data to be processed may be any piece of audio data. For example, the audio data to be processed is a segment of speech, another example, the audio data to be processed is a segment of audio data in a video, and another example, the audio data to be processed is a song.
[0294] In an implementation of obtaining the audio data to be processed, the adding device receives the audio data to be processed input by the user through an input component. The input component includes at least one of the following: a keyboard, a mouse, a touch screen, a touch pad, and an audio data input device.
[0295] In another implementation of obtaining the audio data to be processed, the adding device receives the audio data to be processed sent by the terminal. The terminal may be any of the following: a mobile phone, a computer, a tablet computer, or a server.
[0296] In the embodiment of the present application, the watermark to be added is a digital watermark. The watermark to be added may include any information, for example, the watermark to be added is a string of numbers, and for example, the watermark to be added is a combination of letters and numbers. In one implementation of obtaining the watermark to be added, the adding device receives the watermark to be added input by the user through the input component. In another implementation of obtaining the watermark to be added, the adding device receives the watermark to be added sent by the terminal.
[0297] It should be understood that in the embodiment of the present application, the step of the adding device acquiring the audio data to be processed and the step of acquiring the watermark to be added can be performed separately or simultaneously, and the present application does not limit this.
[0298] 502. Encode the watermark to be added to obtain watermark encoding information.
[0299] The adding device can convert the format of the watermark to be added into a data format that can be processed by the computer device by encoding the watermark to be added. In a possible implementation, the adding device converts the watermark to be added into a binary sequence by encoding the watermark to be added. In this case, the watermark encoding information is a binary sequence.
[0300] 503. Perform frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed.
[0301] The adding device can convert the audio data to be processed into time domain information by performing frequency domain transformation on the audio data to be processed, thereby obtaining the frequency spectrum information to be processed.
[0302] 504. Based on the watermark coding information, adjust the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information.
[0303] In an embodiment of the present application, the adjusted spectrum information includes one or more of the following: amplitude, phase. The center frequency range is the frequency range of speech in the spectrum information. The spectrum information within the center frequency range in the spectrum information to be processed is the spectrum information of speech in the spectrum information to be processed. Based on the watermark coding information, the spectrum information within the center frequency range in the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark coding information can be added to the spectrum information within the center frequency range to obtain the target spectrum information. At this time, the spectrum information within the center frequency range in the target spectrum information carries the watermark coding information, which means that the frequency of the watermark coding information in the target spectrum information is within the frequency range of speech in the spectrum information.
[0304] In one possible implementation, the adding device adjusts the amplitude within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain the target spectrum information. In another possible implementation, the adding device adjusts the phase within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain the target spectrum information. In yet another possible implementation, the adding device adjusts the amplitude and phase within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain the target spectrum information.
[0305] Optionally, the adding device determines the adjustment amount of the spectrum information within the center frequency range based on the watermark coding information, and then adjusts the spectrum information within the center frequency range based on the adjustment amount of the spectrum information to obtain the target spectrum information. Since the adjustment amount of the spectrum information is obtained based on the watermark coding information, the adjustment amount of the spectrum information carries the watermark coding information, and then the spectrum information within the center frequency range in the spectrum information to be processed is adjusted based on the adjustment amount of the spectrum information, the watermark coding information can be added to the spectrum information within the center frequency range.
[0306] Optionally, before executing step 504, the adding device further executes the following steps: determining the center frequency range according to the formant frequency of the spectrum information to be processed. The formant frequency is the area where the energy of the speech is concentrated in the spectrum information. For example, if the energy of the speech in the spectrum information is concentrated between 100 Hz and 8000 Hz, then the formant frequency is between 100 Hz and 8000 Hz. In a possible implementation, the adding device uses the formant frequency as the center frequency range.
[0307] In another possible implementation, the absolute value of the difference between the center frequency of the center frequency range and the center frequency of the formant frequency is less than or equal to the frequency threshold, and the boundary of the center frequency range is within the formant frequency. For example, if the formant frequency is 100 Hz to 8000 Hz, then the center frequency of the formant frequency is 4050 Hz. If the frequency threshold is 5 Hz, then the center frequency of the center frequency range is 4045 Hz to 4050 Hz. The absolute value of the difference between the center frequency of the center frequency range and the center frequency of the formant frequency is less than or equal to the frequency threshold, indicating that the center frequency of the center frequency range is not much different from the center frequency of the formant frequency. At this time, the coupling degree between the center frequency range and the formant frequency is high, and the smaller the frequency threshold, the higher the coupling degree. The higher the coupling degree, the higher the overlap between the center frequency range and the formant frequency, and further the higher the overlap between the frequency range of the watermark coding information and the frequency range of the speech, which also means that the overlap between the frequency range of the watermark in the target audio data and the frequency range of the speech is high. At this time, if the watermark in the target audio data is filtered, the distortion of the speech is also higher. That is to say, the higher the coupling degree, the stronger the anti-interference ability and the higher the robustness of the watermark in the target audio data.
[0308] 505. Perform time domain transformation on the target spectrum information to obtain target audio data carrying a watermark.
[0309] After adding the watermark coding information to the spectrum information within the center frequency range to obtain the target spectrum information, the adding device can convert the target spectrum information into time domain information by performing a time domain transform on the target spectrum information to obtain target audio data carrying the watermark, wherein the encoding result of the watermark in the target audio data is the watermark coding information.
[0310] In an embodiment of the present application, after acquiring the audio data to be processed and the watermark to be added, the adding device encodes the watermark to be added to obtain watermark coding information. The audio data to be processed is transformed in the frequency domain to obtain the spectrum information to be processed. Based on the watermark coding information, the spectrum information within the center frequency range of the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark coding information can be added to the spectrum information within the center frequency range. The target spectrum information is transformed in the time domain to obtain the target audio data carrying the watermark. In the target audio data, the frequency of the watermark is within the center frequency range. If the target audio data is filtered to remove the watermark in the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness.
[0311] As an optional implementation, the watermark encoding information is a binary sequence, wherein the number of bits of the binary sequence is x, that is, the binary sequence has a total of x bits. In this implementation, the adding device performs the following steps during the execution of step 504:
[0312] 6001. Determine x groups of amplitudes from the amplitudes within the central frequency range of the above-mentioned spectrum information to be processed.
[0313] In the embodiment of the present application, x is an integer greater than 1. Optionally, for x groups of amplitudes, the difference between the average values of the amplitudes in any two groups of amplitudes is less than or equal to the amplitude threshold, wherein the average value of the amplitudes is less than or equal to the amplitude threshold, indicating that the difference in the average values of the amplitudes is small, and the average value of the amplitudes is less than or equal to the amplitude threshold, indicating that the difference in the average values of the amplitudes is large.
[0314] In one possible implementation, the adding device randomly groups the amplitudes within the center frequency range of the spectrum information to be processed to obtain x groups of amplitudes. In another possible implementation, the adding device divides the center frequency range into x sub-ranges. The x groups of amplitudes are determined based on the amplitudes within the x sub-ranges, wherein the sub-ranges correspond to the amplitudes one by one.
[0315] 6002. Determine x amplitude adjustment amounts according to the x-bit value in the above binary sequence.
[0316] In the embodiment of the present application, the amplitude adjustment amount corresponds to the numerical value in the binary sequence one by one, the amplitude adjustment amount corresponding to the same numerical value is the same, and the amplitude adjustment amount corresponding to different numerical values is different. Specifically, in the binary sequence, the value of the bit is 0 or 1, that is, the numerical value in the binary sequence is 0 or 1. Therefore, the amplitude adjustment amount corresponding to the numerical value 0 is the same, the amplitude adjustment amount corresponding to the numerical value 1 is the same, and the amplitude adjustment amount corresponding to the numerical value 0 is different from the amplitude adjustment amount corresponding to the numerical value 1. For example, the binary sequence is 001, the amplitude adjustment amount corresponding to the numerical value 0 is -0.1, and the amplitude adjustment amount corresponding to the numerical value 1 is 0.3. Since the numerical value of the first bit and the numerical value of the second bit in the binary sequence are both 0, and the numerical value of the third bit in the binary sequence is 1, the amplitude adjustment amount corresponding to the first bit in the binary sequence is -0.1, the amplitude adjustment amount corresponding to the second bit in the binary sequence is -0.1, and the amplitude adjustment amount corresponding to the third bit in the binary sequence is 0.3.
[0317] The specific value of the amplitude adjustment amount corresponding to the numerical value 0 and the specific value of the amplitude adjustment amount corresponding to the numerical value 1 can be set according to actual needs, and this application does not limit this.
[0318] Optionally, the larger the amplitude of the audio data, the larger the frequency offset of the audio data, and the larger the dispersion of the spectrum of the corresponding audio data, and the larger the dispersion of the spectrum of the audio data, the larger the bandwidth of the audio data. Therefore, there is a positive correlation between the amplitude of the audio data and the bandwidth of the audio data. In an embodiment of the present application, the adding device adjusts the x groups of amplitudes of the to-be-processed spectrum information based on x amplitude adjustment amounts, and adds the watermark to be added to the to-be-processed spectrum information to obtain the target spectrum information. Therefore, in the target spectrum information, the amplitude of the watermark to be added is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount. Because the watermark to be added in the target spectrum information is audio data, the larger the amplitude is, the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, so there is a positive correlation between the bandwidth of the watermark to be added and the difference. Therefore, the bandwidth of the watermark to be added can be determined according to the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount.
[0319] On the one hand, since the bandwidth of the watermark to be added is the frequency range of the watermark to be added, and the larger the frequency range of the watermark to be added, the more difficult it is to remove the watermark to be added. Therefore, the larger the bandwidth of the watermark to be added, the higher the robustness of the watermark to be added. On the other hand, the amplitude of the watermark to be added is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, and the amplitude of the watermark to be added is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, which is positively correlated with the bandwidth of the watermark to be added. Therefore, the bandwidth of the watermark to be added is positively correlated with the amplitude of the watermark to be added. Since the larger the amplitude of the watermark to be added, the greater the energy of the watermark to be added, and accordingly, the watermark to be added as audio data is easier for people to hear, so the larger the bandwidth of the watermark to be added, the easier it is for people to hear the watermark to be added.
[0320] Based on the above two aspects, in order to improve the robustness of the watermark to be added and reduce the probability that the watermark to be added is heard by people, the adding device needs to control the amplitude of the watermark to be added within a reasonable range. Optionally, when the amplitude of the audio data is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the audio data is within a preset range, wherein the preset range is the above reasonable range. When the bandwidth of the audio data is within the preset range, the robustness of the audio data can be improved and the probability that the audio data can be heard by people can be reduced. Since the amplitude of the watermark to be added is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the watermark to be added is also within the preset range, which can improve the robustness of the watermark to be added and reduce the probability that the watermark to be added as audio data can be heard by people. Optionally, the preset range is 0 to 500 Hz. Optionally, the bandwidth of the watermark to be added is 200 Hz. Since there is a positive correlation between the amplitude of the watermark to be added and the bandwidth of the watermark to be added, the bandwidth of the watermark to be added can be adjusted by adjusting the amplitude of the watermark to be added. And because the amplitude of the watermark to be added is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the watermark to be added can be adjusted to 200Hz by adjusting the amplitude of the watermark to be added to the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount.
[0321] 6003. Use the x amplitude adjustment amounts to adjust the x groups of amplitudes in the to-be-processed spectrum information to obtain the target spectrum information.
[0322] In the embodiment of the present application, an amplitude adjustment amount corresponds to a group of amplitudes. The adding device adjusts the amplitudes in a group of amplitudes corresponding to it by using the amplitude adjustment amount, and adds the information of the numerical value corresponding to the amplitude adjustment amount in the binary sequence to the group of amplitudes to obtain the target spectrum information. For example, the binary sequence is 001, the amplitude adjustment amount corresponding to the numerical value 0 is -0.1, and the amplitude adjustment amount corresponding to the numerical value 1 is 0.3. Since the binary sequence has a total of 3 digits, x is 3, and there are a total of 3 amplitude adjustment amounts and 3 groups of amplitudes. If the 3 amplitude adjustment amounts are called amplitude adjustment amount a, amplitude adjustment amount b, and amplitude adjustment amount c, wherein the amplitude adjustment amount a is the amplitude adjustment amount corresponding to the first numerical value in the binary sequence, the amplitude adjustment amount b is the amplitude adjustment amount corresponding to the second numerical value in the binary sequence, and the amplitude adjustment amount c is the amplitude adjustment amount corresponding to the third numerical value in the binary sequence. At this time, the amplitude adjustment amount a and the amplitude adjustment amount b are both -0.1, and the amplitude adjustment amount c is 0.3. Among the three groups of amplitudes, the first group of amplitudes corresponds to the amplitude adjustment amount a, the second group of amplitudes corresponds to the amplitude adjustment amount b, and the third group of amplitudes corresponds to the amplitude adjustment amount c. Then the adding device adjusts the amplitudes in the first group of amplitudes using the amplitude adjustment amount a, adjusts the amplitudes in the first group of amplitudes using the amplitude adjustment amount b, and adjusts the amplitudes in the first group of amplitudes using the amplitude adjustment amount c. Specifically, since the amplitude adjustment amount a and the amplitude adjustment amount b are both -0.1, the adding device reduces all amplitudes in the first group of amplitudes and all amplitudes in the second group of amplitudes by 0.1. Since the amplitude adjustment amount c is 0.3, the adding device increases all amplitudes in the third group of amplitudes by 0.3.
[0323] In this embodiment, the watermark encoding information is a binary sequence, and the number of bits in the binary sequence is x. The adding device determines x groups of amplitudes from the amplitudes within the center frequency range of the spectrum information to be processed. According to the x-bit numerical value in the binary sequence, x amplitude adjustment amounts are determined, and the amplitude adjustment amounts correspond to the numerical values in the binary sequence one by one. The same numerical values correspond to the same amplitude adjustment amounts, and different numerical values correspond to different amplitude adjustment amounts. Thus, the x-bit numerical values in the binary sequence can be expressed by x amplitude adjustment amounts. Using the x amplitude adjustment amounts, the x groups of amplitudes in the spectrum information to be processed are adjusted to obtain the target spectrum information, thereby achieving the goal of adding the x-bit numerical value of the binary sequence to the spectrum information to be processed to obtain the target spectrum information.
[0324] As an optional implementation manner, the adding device performs the following steps during the process of executing step 503:
[0325] 7001. Extract a segment of audio from the above-mentioned audio data to be processed as audio data to be added.
[0326] The adding device can intercept the audio data to be added from the audio data to be processed in any manner. For example, the adding device can intercept a 2-second audio segment from the audio data to be processed as the audio data to be added.
[0327] 7002. Perform frequency domain transformation on the audio data to be added to obtain the frequency spectrum information to be processed.
[0328] The adding device converts the audio data to be added into the frequency domain by performing frequency domain transformation on the audio data to be added, thereby obtaining the frequency spectrum information to be processed.
[0329] After the watermark coding information to be watermarked is added to the spectrum information to be processed to obtain the target spectrum information by executing step 504, the adding device performs the following steps in the process of executing step 505:
[0330] 7003. Perform a time domain transformation on the target spectrum information to obtain watermarked audio data carrying the watermark.
[0331] Since the spectrum information within the center frequency range of the target spectrum information carries watermark encoding information, the adding device can obtain watermarked audio data carrying the watermark by converting the target spectrum information into the time domain through time domain transformation of the target spectrum information.
[0332] 7004. Use the watermarked audio data to replace the audio data to be added in the audio data to be processed to obtain the target audio data.
[0333] Under the condition that the audio data to be processed except the audio data to be added is kept unchanged, the audio data to be added in the audio data to be processed is replaced with the watermark audio data to obtain the target audio data, so as to obtain the target audio data by adding the watermark to be added to the audio data to be added in the audio data to be processed.
[0334] Since, based on this implementation, the adding device can add the watermark to be added to the audio data to be added of the audio data to be processed to obtain the target audio data, and the audio data to be added can be any audio segment in the audio data to be processed, therefore, based on this implementation, the watermark to be added can be added to any position in the audio data to be processed to obtain the target audio data. Since the position of the watermark in the target audio data can be any position, it is more difficult to remove the watermark in the target audio data by removing part of the audio of the target audio data, thereby improving the robustness of the watermark in the target audio data. For example, if the watermark usually appears from the 2nd to the 4th second of the audio data, then the watermark in the target audio data can be removed by removing the audio from the 2nd to the 4th second of the target audio data. If the watermark can appear at any position of the target audio data, then it is difficult to remove the watermark by removing the audio in the target audio data.
[0335] As an optional implementation, steps 7001 to 7004 are implementations of adding the watermark to be added to the audio data to be processed. Based on this implementation and steps 6001 to 6003, the adding device can add the watermark to be added to at least two positions in the audio data to be processed in an interval manner, so that the target audio data obtained after adding includes at least two watermarks to be added, and the watermarks to be added exist at intervals. In this way, the robustness of the watermark to be added in the target audio data can be improved.
[0336] Specifically, the adding device performs the following steps during the execution of step 503: intercepting at least two segments of audio data to be added from the audio data to be processed, wherein the length of the audio data to be added is the same as the length of the watermark coding information, and the time interval between the two segments of audio data to be added with adjacent timestamps is a preset time length. The length of the watermark coding information is the duration of the watermark coding information in the audio data. For example, the duration of the watermark coding information in the audio data is 2 seconds, the length of the watermark coding information is 2 seconds, and the length of the audio data to be added is 2 seconds. If the preset time length is 3 seconds, the first segment of audio data to be added is from the 0th second to the 2nd second in the audio data to be processed, then the second segment of audio data to be added is from the 5th second to the 7th second in the audio data to be processed, and the time interval between the first segment of audio data to be added and the second segment of audio data to be added is 3 seconds. Performing frequency domain transformation on each of the at least two segments of audio data to be added, respectively, to obtain at least two spectral information to be processed, wherein the spectral information to be processed corresponds one-to-one to the audio data to be added.
[0337] In step 504, the following steps are performed: for each of the at least two spectral information to be processed, x groups of amplitudes are determined from the amplitudes within the center frequency range, and each spectral information to be processed corresponds to the x groups of amplitudes. For the x groups of amplitudes corresponding to each spectral information to be processed, x amplitude adjustment amounts are determined according to the x-bit values in the binary sequence, and each spectral information to be processed corresponds to the x amplitude adjustment amounts. Using the x amplitude adjustment amounts, the x groups of amplitudes in the spectral information to be processed corresponding to the x amplitude adjustment amounts are adjusted to obtain at least two added spectral information.
[0338] In step 505, the following steps are performed: performing time domain transformation on at least two added spectral information to obtain at least two watermarked audio data. Using at least two watermarked audio data, replace at least two audio data to be added in the audio data to be processed to obtain target audio data.
[0339] In this embodiment, the adding device can add the watermark to be added to each segment of the audio data to be added in the audio data to be processed. Since the audio data to be added exists at intervals in the audio data to be processed, adding the watermark to be added to each segment of the audio data to be added in the audio data to be processed can achieve adding the watermark to be added to at least two different positions of the audio data to be processed in an interval adding manner to obtain the target audio data, thereby improving the robustness of the watermark to be added in the target audio data.
[0340] As an optional implementation, the audio data to be processed is the audio data released by the target object through the target platform. For example, the target platform is an audio on-demand platform, and the audio data to be processed is the audio data released by the target object on the audio on-demand platform. For another example, the target platform is a video on-demand platform, and the audio data to be processed is the audio data released by the target object on the video on-demand platform. In this implementation, the adding device obtains the watermark to be added by performing the following steps:
[0341] 8001. Obtain the target identifier of the target object on the target platform.
[0342] In the embodiment of the present application, the target identifier is the identifier of the target object on the target platform. Optionally, the target identifier is the name of the target object on the target platform, or the target identifier is the identity document (ID) of the target object on the target platform.
[0343] 8002. Generate the watermark to be added according to the target identifier.
[0344] The adding device generates the watermark to be added by executing step 8002, and can generate an audio watermark matching the target identifier. In one possible implementation, the adding device obtains the watermark to be added by encoding the target identifier. In another possible implementation, the adding device encrypts the target identifier to obtain the watermark to be added.
[0345] Optionally, the adding device generates watermark information to be processed according to the target identifier, and obtains the watermark to be added by merging the watermark information to be processed and the preset identifier. The position of the preset identifier in the watermark to be added can be any position. Optionally, the preset identifier is in the first r bits of the watermark to be added, where r is the number of bits of the preset identifier. For example, the preset identifier has a total of 3 digits, and the first 3 bits of the watermark to be added are the preset identifier. Optionally, the watermark to be added is a sequence represented by binary. When the preset identifier is added to the watermark to be added, it is beneficial to subsequently determine the watermark to be added from the audio data based on the preset identifier.
[0346] In this embodiment, the audio data to be processed is the audio released by the target object through the target platform. After obtaining the target identifier of the target object on the target platform, the adding device can generate a watermark to be added that matches the target identifier according to the target identifier. In this way, the target audio is obtained by adding the watermark to be added to the audio data to be processed, so that the target audio has a watermark that matches the target identifier. In this way, it can be achieved that a watermark matching the identifier of the object is generated for the audio data uploaded by the object, and the watermark matching the identifier of the object can be added to the audio data uploaded by the object.
[0347] See also Figure 6 , Figure 6 1 is a flow chart of another audio processing method provided in an embodiment of the present application, the audio processing method is used to add a watermark to be added to the audio data to be processed to obtain target audio data, and extract the target audio watermark from the target audio data. Figure 6 As shown, first, the watermark to be added is obtained by merging the watermark information to be processed and the preset identifier, wherein the watermark information to be processed is obtained according to the target identifier. Then the watermark to be added is added to the audio data to be processed to obtain the target audio data. The specific implementation process of this step can be seen from steps 501 to 505. The target audio data is transcoded, compressed, and low-passed to obtain the audio data to be decoded, wherein low-pass refers to low-pass filtering. For example, the audio data to be processed is the audio data uploaded by the target object to the target platform, and the target audio data is the audio data obtained after the target platform adds the watermark to be added to the audio data to be processed. The audio data to be decoded is published to the target platform. After obtaining the audio data to be decoded from the target platform, n frames of audio frames to be extracted are intercepted from the audio data to be decoded. The n frames of audio frames to be extracted are input into the watermark extraction model to obtain n candidate audio watermarks output by the watermark extraction model. Optionally, when the watermark to be added includes a preset identifier, and the position of the preset identifier in the watermark to be added is the preset position, n frames of audio frames to be extracted, the preset identifier and the preset position are input into the watermark extraction model together, so that after the watermark extraction model determines the preset identifier from the audio frame to be extracted, it extracts the candidate audio watermark from the audio frame to be extracted based on the preset identifier and the preset position, thereby improving the accuracy of the candidate audio watermark. For example, the preset identifier is XHS, and the preset position is the first three digits. Then, after the watermark extraction model determines XHS from the audio frame to be extracted, it determines that the character sequence after XHS is the content of the candidate audio watermark, and then the candidate audio watermark can be obtained based on XHS and the character sequence after XHS. After obtaining n candidate audio watermarks, the target audio watermark can be obtained based on the n candidate audio watermarks. The specific implementation process of this step can be seen in step 2003.
[0348] Since steps 501 to 505 are implemented by adjusting the frequency spectrum information of the audio data based on the watermark, in this audio processing method, adding the watermark to the audio data to be processed to obtain the target audio data is also implemented by adjusting the frequency spectrum information of the audio data based on the watermark. That is to say, this audio processing method adds the watermark to be added to the audio data to be processed by the traditional adding method to obtain the target audio data. Therefore, the amount of calculation for adding the watermark to be added to the audio data to be processed can be reduced. Moreover, since steps 501 to 505 are to obtain the target audio data by adding the watermark to be added to the frequency spectrum information within the center frequency range, the robustness of the watermark in the target audio data is high. Therefore, it is difficult to remove the watermark in the target audio data by low-passing the target audio data, and the watermark in the audio data to be decoded after the low-pass can be made more complete, thereby improving the accuracy of the audio watermark extracted from the audio data to be decoded. After obtaining the audio data to be decoded of the target platform, the target audio watermark is extracted from the audio data to be decoded based on the watermark extraction model. The target audio watermark can be extracted from the audio data to be decoded through a deep learning extraction method, thereby improving the accuracy of the target audio watermark.
[0349] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0350] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.
[0351] See also Figure 7 , Figure 7 A schematic diagram of the structure of a device for extracting an audio watermark provided in an embodiment of the present application, wherein the device 1 for extracting an audio watermark comprises: an acquisition unit 11, an extraction unit 12, and optionally, the device 1 for extracting an audio watermark further comprises: an interception unit 13, a conversion unit 14, specifically:
[0352] An acquisition unit 11 is used to acquire target audio data, wherein the target audio data is obtained by performing time domain conversion on target spectrum information carrying watermark coding information, wherein spectrum information within a center frequency range of the target spectrum information carries the watermark coding information;
[0353] The extraction unit 12 is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
[0354] In combination with any embodiment of the present application, the watermark extraction model is obtained by iteratively updating the difference between a training watermark and a reference watermark, the training watermark is the watermark extracted by the watermark extraction model from the training audio data, and the reference watermark is the watermark carried in the training audio data.
[0355] In combination with any embodiment of the present application, the device 1 for extracting an audio watermark further includes:
[0356] An interception unit 13, configured to intercept n audio frames to be extracted from the target audio data, where n is an integer greater than 1;
[0357] The extraction unit 12 is specifically used for:
[0358] Inputting the n audio frames to be extracted into the watermark extraction model respectively to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively;
[0359] The target audio watermark is obtained according to the n candidate audio watermarks.
[0360] In combination with any embodiment of the present application, the extraction unit 12 is specifically used for:
[0361] Determine the number of times each audio watermark appears in the n candidate audio watermarks to obtain a target number;
[0362] The target audio watermark is obtained according to the target number and the n candidate audio watermarks.
[0363] In combination with any embodiment of the present application, the extraction unit 12 is specifically used for:
[0364] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the audio watermark that appears most frequently among the n candidate audio watermarks is used as the target audio watermark.
[0365] In combination with any embodiment of the present application, the extraction unit 12 is specifically used for:
[0366] When the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, determining the value of the target position in the n candidate audio watermarks to obtain n candidate values;
[0367] Determine the value that appears most frequently among the n candidate values as the target value of the target position in the target audio watermark;
[0368] The target audio watermark is obtained according to the target value.
[0369] In combination with any embodiment of the present application, the interception unit 13 is specifically used for:
[0370] The target audio data is segmented according to a preset step length to obtain n audio frames to be extracted, wherein the preset step length is greater than 0 and less than the length of the watermark encoding information.
[0371] In combination with any embodiment of the present application, the n audio frames to be extracted include a first audio frame, the n candidate audio watermarks include a first audio watermark, and the first audio watermark is an audio watermark obtained after the first audio frame is input into the watermark extraction model;
[0372] The extraction unit 12 is specifically used for:
[0373] Input the first audio frame into the audio frame division module of the watermark extraction model to obtain m segments of audio frames to be confirmed, where the audio frames to be confirmed are part of the first audio frame;
[0374] Inputting the m audio frames to be confirmed into the probability prediction module of the watermark extraction model respectively, predicting the probability that each audio frame to be confirmed contains a watermark respectively, and obtaining m prediction probabilities;
[0375] Inputting the m prediction probabilities into the candidate frame determination module of the watermark extraction model, determining that the audio frames to be confirmed corresponding to the largest s of the m prediction probabilities are s candidate audio frames, where s is less than m;
[0376] The s candidate audio frames are input into the audio watermark extraction module of the watermark extraction model to obtain the first audio watermark.
[0377] In combination with any embodiment of the present application, the device 1 for extracting an audio watermark further includes:
[0378] The conversion unit 14 is used to perform frequency domain conversion on the target audio data to obtain the target spectrum information;
[0379] The extraction unit 12 is specifically used for:
[0380] Inputting the target spectrum information into the watermark extraction model;
[0381] Using the watermark extraction model, extracting spectrum information within the center frequency range from the target spectrum information to obtain target spectrum information;
[0382] The watermark extraction model is used to extract the watermark encoding information in the target spectrum information to obtain the target audio watermark.
[0383] In combination with any implementation manner of the present application, the center frequency range is determined according to the resonance peak frequency.
[0384] In the embodiment of the present application, since the spectrum information within the center frequency range of the target spectrum information carries the watermark encoding information, the center frequency range is the frequency range of the speech in the spectrum information, and the frequency of the watermark in the target audio data is within the frequency range of the speech in the target audio data. At this time, if the watermark in the target audio data is removed by filtering the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness. In this way, after acquiring the target audio data, the device for extracting the audio watermark inputs the target audio data into a watermark extraction model obtained through pre-training, and obtains the target audio watermark output by the watermark extraction model, which can improve the accuracy of the target audio watermark.
[0385] Moreover, the target audio data is obtained by time-domain conversion of the target spectrum information carrying the watermark coding information, which means that the watermark in the target audio data is obtained by adjusting the spectrum information of the audio data. Therefore, the calculation amount of the target audio data obtained by adding the watermark to the audio data is small and the speed is fast. The watermark extraction model is obtained through pre-training. During the pre-training process, the watermark extraction model can learn how to extract the watermark from the audio data, wherein the audio data includes the audio data that has not been processed by the watermark interference and the audio data that has been processed by the watermark interference. The watermark interference processing refers to the processing that can remove the watermark in the audio data. For example, the watermark in the audio data can be removed by filtering the audio data, and the watermark interference processing includes filtering. Therefore, by extracting the watermark from the audio data through the watermark extraction model, the influence of the watermark interference processing on the accuracy of the extracted watermark can be reduced, and the accuracy of the extracted watermark can be improved. Therefore, by extracting the target audio watermark from the target audio data through the watermark extraction model, the accuracy of the target audio watermark can be improved. In this way, the watermark can be quickly added to the audio data, and the accuracy of extracting the watermark from the audio data can be improved.
[0386] See also Figure 8 , Figure 8 The present invention provides a schematic diagram of a device for extracting an audio watermark according to an embodiment of the present invention. The device 2 for extracting an audio watermark includes: an acquisition unit 21 and an extraction unit 22. Specifically:
[0387] An acquisition unit 21 is used to acquire target audio data carrying a watermark, wherein the watermark in the target audio data is added to the original target audio by a first watermark adding method, wherein the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method;
[0388] An extraction unit 22 is used to extract a watermark from the target audio data by a second watermark extraction method to obtain a target audio watermark carried by the target audio data, wherein the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method;
[0389] Wherein, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method;
[0390] For the same audio data, the accuracy of extracting the watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting the watermark from the audio data by the first watermark extraction method.
[0391] In combination with any embodiment of the present application, the first watermark adding method adjusts the frequency spectrum information of the audio data based on the watermark encoding information to add the watermark to the audio data.
[0392] In combination with any embodiment of the present application, the extraction unit 22 is specifically used for:
[0393] The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, where the watermark extraction model is a deep learning model capable of extracting watermarks from audio data.
[0394] In combination with any implementation manner of the present application, the target audio data is audio data of an audio on-demand platform.
[0395] In the embodiment of the present application, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method. For the same audio data, the accuracy of extracting a watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting a watermark from the audio data by the first watermark extraction method. Therefore, in the case where the watermark in the target audio data is obtained by adding the first watermark adding method to the original target audio, after acquiring the target audio data, the device for extracting the audio watermark extracts the watermark from the target audio data by the second watermark extraction method to obtain the target audio data, thereby realizing the addition of a watermark by a watermark adding method with a small amount of calculation, and the extraction of a watermark by a watermark extraction method with a high accuracy. This can both increase the speed of adding watermarks and increase the accuracy of watermark extraction.
[0396] See also Fig. 9 , Fig. 9This is a structural diagram of an audio processing device provided in an embodiment of the present application. The audio processing device 3 includes: an acquisition unit 31, an encoding unit 32, a transformation unit 33, an adjustment unit 34, and an extraction unit 35. Specifically:
[0397] An acquisition unit 31, used for acquiring audio data to be processed and watermark to be added;
[0398] The encoding unit 32 is used to encode the watermark to be added to obtain watermark encoding information;
[0399] The transform unit 33 is used to perform frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed;
[0400] An adjusting unit 34 is used to adjust the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain target spectrum information;
[0401] The transform unit 33 is used to perform a time domain transform on the target spectrum information to obtain target audio data carrying a watermark;
[0402] The extraction unit 35 is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
[0403] In an embodiment of the present application, after obtaining the audio data to be processed and the watermark to be added, the watermark to be added is encoded to obtain watermark encoding information. The audio data to be processed is transformed in the frequency domain to obtain the spectrum information to be processed. Based on the watermark encoding information, the spectrum information within the center frequency range in the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark encoding information can be added to the spectrum information within the center frequency range. The target spectrum information is transformed in the time domain to obtain the target audio data carrying the watermark. In the target audio data, the frequency of the watermark is within the center frequency range. If the target audio data is filtered to remove the watermark in the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness. Furthermore, as described in step 101, adding a watermark to audio data by a conventional adding method is based on adjusting the frequency spectrum information of the audio data by the watermark, so the audio processing device obtains the target audio data by adding the watermark to be added to the audio data to be processed by the conventional adding method, thereby reducing the amount of calculation for adding the watermark to be added to the audio data to be processed. After obtaining the target audio data, the audio processing device inputs the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, that is, the audio processing device can extract the target audio watermark from the target audio data by a deep learning extraction method, thereby improving the accuracy of the target audio watermark.
[0404] See also Fig.10 , Fig.10 A schematic diagram of the structure of a device for adding an audio watermark provided in an embodiment of the present application, wherein the device 4 for adding an audio watermark comprises: an acquisition unit 41, an encoding unit 42, a transformation unit 43, and an adjustment unit 44. Optionally, the device 4 for adding an audio watermark further comprises a determination unit 45. Specifically:
[0405] An acquisition unit 41 is used to acquire the audio data to be processed and the watermark to be added;
[0406] The encoding unit 42 is used to encode the watermark to be added to obtain watermark encoding information;
[0407] The transform unit 43 is used to perform frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed;
[0408] An adjusting unit 44 is used to adjust the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain target spectrum information;
[0409] The transform unit 43 is used to perform time domain transform on the target spectrum information to obtain target audio data carrying a watermark.
[0410] In combination with any embodiment of the present application, the apparatus 4 for adding an audio watermark further includes: a determining unit 45, configured to determine the center frequency range according to the resonance peak frequency of the frequency spectrum information to be processed.
[0411] In combination with any implementation manner of the present application, the watermark encoding information is a binary sequence, and the number of bits of the binary sequence is x;
[0412] The adjustment unit 44 is specifically used for:
[0413] Determine x groups of amplitudes from the amplitudes within the central frequency range of the spectrum information to be processed;
[0414] Determine x amplitude adjustment amounts according to the x-bit value in the binary sequence, wherein the amplitude adjustment amounts correspond to the values in the binary sequence one by one, the same values correspond to the same amplitude adjustment amounts, and different values correspond to different amplitude adjustment amounts;
[0415] The x groups of amplitudes in the spectrum information to be processed are adjusted by using the x amplitude adjustment amounts to obtain the target spectrum information.
[0416] In combination with any implementation manner of the present application, when the amplitude of the audio data is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the audio data is within a preset range.
[0417] In combination with any implementation manner of the present application, the transformation unit 43 is specifically configured to:
[0418] Intercepting a segment of audio from the audio data to be processed as the audio data to be added;
[0419] Performing frequency domain transformation on the audio data to be added to obtain the frequency spectrum information to be processed;
[0420] The transformation unit 43 is specifically used for:
[0421] Performing a time domain transformation on the target spectrum information to obtain watermarked audio data carrying a watermark;
[0422] The watermarked audio data is used to replace the audio data to be added in the audio data to be processed to obtain the target audio data.
[0423] In combination with any implementation manner of the present application, the audio data to be processed is audio data released by a target object through a target platform;
[0424] The acquisition unit 41 is specifically used for:
[0425] Obtain a target identifier of the target object on the target platform;
[0426] The watermark to be added is generated according to the target identifier.
[0427] In combination with any implementation manner of the present application, the acquisition unit 41 is specifically used to:
[0428] Generate watermark information to be processed according to the target identifier;
[0429] The watermark to be added is obtained by merging the watermark information to be processed and the preset identifier.
[0430] In an embodiment of the present application, after obtaining the audio data to be processed and the watermark to be added, the device for adding an audio watermark encodes the watermark to be added to obtain watermark coding information. The audio data to be processed is transformed in the frequency domain to obtain the spectrum information to be processed. Based on the watermark coding information, the spectrum information within the center frequency range of the spectrum information to be processed is adjusted to obtain the target spectrum information, and the watermark coding information can be added to the spectrum information within the center frequency range. The target spectrum information is transformed in the time domain to obtain the target audio data carrying the watermark. In the target audio data, the frequency of the watermark is within the center frequency range. If the target audio data is filtered to remove the watermark in the target audio data, then it is necessary to remove the watermark whose frequency is within the frequency range of the speech by filtering, which will cause the speech to be removed as well, resulting in distortion of the speech. Therefore, the watermark in the target audio data has strong anti-interference ability and high robustness.
[0431] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.
[0432] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 5 includes a processor 51 and a memory 52. Optionally, the electronic device 5 also includes an input device 53 and an output device 54. The processor 51, the memory 52, the input device 53 and the output device 54 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.
[0433] The processor 51 may include one or more processors, for example, one or more central processing units (CPUs). When the processor is a CPU, the CPU may be a single-core CPU or a multi-core CPU. Optionally, the processor 51 may be a processor group consisting of multiple CPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.
[0434] The memory 52 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.
[0435] The input device 53 is used to input data and / or signals, and the output device 54 is used to output data and / or signals. The input device 53 and the output device 54 can be independent devices or an integrated device.
[0436] It can be understood that in the embodiment of the present application, the memory 52 can be used not only to store relevant instructions, but also to store relevant data. For example, the memory 52 can be used to store the audio to be processed and the audio watermark to be added obtained through the input device 53, or the memory 52 can also be used to store the target audio obtained by the processor 51, etc. The embodiment of the present application does not limit the specific data stored in the memory.
[0437] Understandably, Fig.11 Only a simplified design of an electronic device is shown. In practical applications, the electronic device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.
[0438] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0439] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.
[0440] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0441] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0442] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or one or more units may be integrated into one unit.
[0443] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).
[0444] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
Claims
1. A method for extracting an audio watermark, characterized in that: The method comprises: Acquire target audio data, where the target audio data is obtained by performing time domain conversion on target spectrum information carrying watermark coding information, where spectrum information within a center frequency range of the target spectrum information carries the watermark coding information; The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, and the watermark encoding information is an encoding result of the target audio watermark.
2. The method according to claim 1, characterized in that The watermark extraction model is obtained by iteratively updating the difference between a training watermark and a reference watermark. The training watermark is a watermark extracted from training audio data by the watermark extraction model, and the reference watermark is a watermark carried in the training audio data.
3. The method according to claim 1 or 2, characterized in that: Before inputting the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, the method further includes: Intercepting n audio frames to be extracted from the target audio data, where n is an integer greater than 1; The step of inputting the target audio data into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model comprises: Inputting the n audio frames to be extracted into the watermark extraction model respectively to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively; The target audio watermark is obtained according to the n candidate audio watermarks.
4. The method according to claim 3, characterized in that The step of obtaining the target audio watermark according to the n candidate audio watermarks comprises: Determine the number of times each audio watermark appears in the n candidate audio watermarks to obtain a target number of times; The target audio watermark is obtained according to the target number and the n candidate audio watermarks.
5. The method according to claim 4, characterized in that The step of obtaining the target audio watermark according to the target number of times and the n candidate audio watermarks includes: When the difference between the maximum value of the target number of times and the second largest value of the target number of times is greater than the number threshold, the audio watermark that appears most frequently among the n candidate audio watermarks is used as the target audio watermark.
6. The method according to claim 4, characterized in that The step of obtaining the target audio watermark according to the target number of times and the n candidate audio watermarks includes: When the difference between the maximum value of the target number of times and the second largest value of the target number of times is less than or equal to the number threshold, determining the value of the target position in the n candidate audio watermarks to obtain n candidate values; Determine the value that appears most frequently among the n candidate values as the target value of the target position in the target audio watermark; The target audio watermark is obtained according to the target value.
7. The method according to claim 3, characterized in that The step of intercepting n frames of audio frames to be extracted from the target audio data comprises: The target audio data is segmented according to a preset step length to obtain n audio frames to be extracted, where the preset step length is greater than 0 and less than the length of the watermark encoding information.
8. The method according to claim 3, characterized in that The n audio frames to be extracted include a first audio frame, the n candidate audio watermarks include a first audio watermark, and the first audio watermark is an audio watermark obtained after the first audio frame is input into the watermark extraction model; The step of inputting the n audio frames to be extracted into the watermark extraction model to obtain n candidate audio watermarks corresponding to the n audio frames to be extracted respectively includes: Input the first audio frame into the audio frame division module of the watermark extraction model to obtain m segments of audio frames to be confirmed, where the audio frames to be confirmed are part of the first audio frame; Inputting the m audio frames to be confirmed into the probability prediction module of the watermark extraction model respectively, predicting the probability that each audio frame to be confirmed contains a watermark respectively, and obtaining m prediction probabilities; Inputting the m prediction probabilities into the candidate frame determination module of the watermark extraction model, determining that the to-be-confirmed audio frames corresponding to the largest s of the m prediction probabilities are s candidate audio frames, where s is less than m; The s candidate audio frames are input into the audio watermark extraction module of the watermark extraction model to obtain the first audio watermark.
9. The method according to claim 1 or 2, characterized in that: Before inputting the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, the method further includes: Performing frequency domain conversion on the target audio data to obtain the target frequency spectrum information; The step of inputting the target audio data into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model comprises: Inputting the target spectrum information into the watermark extraction model; Using the watermark extraction model, extracting spectrum information within the center frequency range from the target spectrum information to obtain target spectrum information; The watermark extraction model is used to extract the watermark encoding information in the target spectrum information to obtain the target audio watermark.
10. The method according to claim 1 or 2, characterized in that: The center frequency range is determined based on the resonance peak frequency.
11. A method for extracting an audio watermark, characterized in that: The method comprises: Acquire target audio data carrying a watermark, wherein the watermark in the target audio data is obtained by adding the watermark to the original target audio through a first watermark adding method, wherein the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method; Extracting a watermark from the target audio data by a second watermark extraction method to obtain a target audio watermark carried by the target audio data, wherein the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method; Wherein, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method; For the same audio data, the accuracy of extracting the watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting the watermark from the audio data by the first watermark extraction method.
12. The method according to claim 11, characterized in that The first watermark adding method adjusts the frequency spectrum information of the audio data based on the watermark encoding information to add the watermark to the audio data.
13. The method according to claim 11 or 12, characterized in that: The step of extracting a watermark from the target audio data by using a second watermark extraction method to obtain a target audio watermark carried by the target audio data includes: The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, where the watermark extraction model is a deep learning model capable of extracting watermarks from audio data.
14. The method according to claim 11 or 12, characterized in that: The target audio data is audio data of an audio-on-demand platform.
15. An audio processing method, characterized in that: The method comprises: Get the audio data to be processed and the watermark to be added; Encoding the watermark to be added to obtain watermark encoding information; Performing frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed; Based on the watermark coding information, adjusting the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information; Performing a time domain transformation on the target spectrum information to obtain target audio data carrying a watermark; The target audio data is input into a pre-trained watermark extraction model to obtain a target audio watermark output by the watermark extraction model, and the watermark encoding information is an encoding result of the target audio watermark.
16. A method for adding an audio watermark, characterized in that: The method comprises: Get the audio data to be processed and the watermark to be added; Encoding the watermark to be added to obtain watermark encoding information; Performing frequency domain transformation on the audio data to be processed to obtain frequency spectrum information to be processed; Based on the watermark coding information, adjusting the spectrum information within the center frequency range of the spectrum information to be processed to obtain target spectrum information; The target frequency spectrum information is transformed in the time domain to obtain target audio data carrying a watermark.
17. The method according to claim 16, characterized in that Before adjusting the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain the target spectrum information, the method further includes: determining the center frequency range according to the resonance peak frequency of the spectrum information to be processed.
18. The method according to claim 16 or 17, characterized in that The watermark encoding information is a binary sequence, and the number of bits of the binary sequence is x; The step of adjusting the spectrum information within the center frequency range of the spectrum information to be processed based on the watermark coding information to obtain target spectrum information includes: Determine x groups of amplitudes from the amplitudes within the central frequency range of the spectrum information to be processed; Determine x amplitude adjustment amounts according to the x-bit value in the binary sequence, wherein the amplitude adjustment amounts correspond to the values in the binary sequence one by one, the same values correspond to the same amplitude adjustment amounts, and different values correspond to different amplitude adjustment amounts; The x groups of amplitudes in the spectrum information to be processed are adjusted by using the x amplitude adjustment amounts to obtain the target spectrum information.
19. The method according to claim 18, characterized in that When the amplitude of the audio data is the difference between the maximum value of the amplitude adjustment amount and the minimum value of the amplitude adjustment amount, the bandwidth of the audio data is within a preset range.
20. The method according to claim 16 or 17, characterized in that The performing frequency domain transformation on the audio data to be processed to obtain spectrum information to be processed includes: Intercepting a segment of audio from the audio data to be processed as the audio data to be added; Performing frequency domain transformation on the audio data to be added to obtain the frequency spectrum information to be processed; The step of performing a time domain transformation on the target spectrum information to obtain target audio data carrying a watermark includes: Performing a time domain transformation on the target spectrum information to obtain watermarked audio data carrying a watermark; The watermarked audio data is used to replace the audio data to be added in the audio data to be processed to obtain the target audio data.
21. The method according to claim 16 or 17, characterized in that The audio data to be processed is audio data released by the target object through the target platform; Get the watermark to be added, including: Obtain a target identifier of the target object on the target platform; The watermark to be added is generated according to the target identifier.
22. The method according to claim 21, characterized in that The step of generating the watermark to be added according to the target identifier includes: Generate watermark information to be processed according to the target identifier; The watermark to be added is obtained by merging the watermark information to be processed and the preset identifier.
23. A device for extracting audio watermark, characterized in that: The device for extracting audio watermark comprises: an acquisition unit, configured to acquire target audio data, wherein the target audio data is obtained by performing time domain conversion on target spectrum information carrying watermark coding information, wherein spectrum information within a center frequency range of the target spectrum information carries the watermark coding information; The extraction unit is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
24. A device for extracting audio watermark, characterized in that: The device for extracting audio watermark comprises: an acquisition unit, configured to acquire target audio data carrying a watermark, wherein the watermark in the target audio data is obtained by adding a first watermark adding method to the original target audio, wherein the first watermark adding method is a watermark adding method corresponding to the first watermark extraction method; an extraction unit, configured to extract a watermark from the target audio data by using a second watermark extraction method to obtain a target audio watermark carried by the target audio data, wherein the second watermark extraction method is a watermark extraction method corresponding to the second watermark adding method; Wherein, for the same watermark and original audio data, the amount of calculation for adding a watermark to the original audio data by the first watermark adding method is less than the amount of calculation for adding a watermark to the original audio data by the second watermark adding method; For the same audio data, the accuracy of extracting the watermark from the audio data by the second watermark extraction method is higher than the accuracy of extracting the watermark from the audio data by the first watermark extraction method.
25. An audio processing device, characterized in that: The audio processing device comprises: An acquisition unit, used for acquiring audio data to be processed and watermark to be added; An encoding unit, used for encoding the watermark to be added to obtain watermark encoding information; A transform unit, used for performing frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed; An adjustment unit, configured to adjust the spectrum information within a center frequency range in the spectrum information to be processed based on the watermark coding information to obtain target spectrum information; The transform unit is used to perform time domain transform on the target spectrum information to obtain target audio data carrying a watermark; The extraction unit is used to input the target audio data into a pre-trained watermark extraction model to obtain the target audio watermark output by the watermark extraction model, and the watermark encoding information is the encoding result of the target audio watermark.
26. A device for adding an audio watermark, characterized in that: The device for adding an audio watermark comprises: An acquisition unit, used for acquiring audio data to be processed and watermark to be added; An encoding unit, used for encoding the watermark to be added to obtain watermark encoding information; A transform unit, used for performing frequency domain transform on the audio data to be processed to obtain frequency spectrum information to be processed; An adjustment unit, configured to adjust the spectrum information within a center frequency range in the spectrum information to be processed based on the watermark coding information to obtain target spectrum information; The transformation unit is used to perform time domain transformation on the target spectrum information to obtain target audio data carrying a watermark.
27. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program code, wherein the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as claimed in any one of claims 1 to 10, or the electronic device executes the method as claimed in any one of claims 11 to 14, or the electronic device executes the method as claimed in claim 15, or the electronic device executes the method as claimed in any one of claims 16 to 22.
28. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method described in any one of claims 1 to 10, or the processor is caused to execute the method described in any one of claims 11 to 14, or the processor is caused to execute the method described in any one of claim 15, or the processor is caused to execute the method described in any one of claims 16 to 22.
29. A computer program product, characterized in that The computer program product includes a computer program or instructions; when the computer program or instructions are run on a computer, the computer is enabled to execute the method described in any one of claims 1 to 10, or the computer is enabled to execute the method described in any one of claims 11 to 14, or the computer is enabled to execute the method described in claim 15, or the computer is enabled to execute the method described in any one of claims 16 to 22.
Citation Information
Cited By
Audio identification method and system and storage medium
CN120299464A
High-precision dynamic watermarking method and device for streaming audio
CN120472915A
