Method and device for implementing audio watermark

Abstract

This application discloses a method and apparatus for implementing audio watermarking. On the one hand, a relatively short frequency domain transform ensures high real-time audio processing, enabling low-complexity, low-latency watermark embedding. Watermark embedding is achieved through symbol sequence modulation, offsetting the effects of carrier distortion. Furthermore, watermark extraction through correlation significantly improves decoding robustness. The method for implementing audio watermarking provided by this application meets anti-copying requirements.

Description

Technical Field

[0001] The present application relates to, but is not limited to, audio and video processing technology, and in particular to a method and device for implementing audio watermarking. Background Art

[0002] With the rapid development of the internet, access to audio and video content has become increasingly convenient. However, if online audio content is illegally captured using recording equipment, the copyright of this content cannot be effectively protected. This is especially true in audio and video conferencing scenarios where confidentiality is a concern, as recordings of the meeting could lead to leaks.

[0003] In this context, digital watermarking technology that can effectively protect copyrights has emerged. Figure 1 As shown in the figure, digital audio watermarking technology is to embed information with specific meaning and easy to extract (such as Figure 1 The process of embedding watermarks (additional data in the audio data). Depending on the application, the embedded information can include copyright identifiers, work serial numbers, text (such as artist and song names), or even a small image or audio clip. Watermarks are tightly integrated with the original audio data and hidden within it. They are typically inaudible and resistant to common audio signal processing and certain malicious attacks by pirates.

[0004] In recent years, digital watermarking technology has achieved certain results in the field of audio and video copyright protection. In particular, due to the low complexity and low cost of audio watermarks compared to video watermarks, audio watermarks have been widely used in audio and video copyright protection and piracy tracing scenarios.

[0005] In related technologies, the production of pirated or leaked content usually involves external dubbing. Therefore, it is particularly important to make the audio watermark have anti-dubbing properties. Summary of the Invention

[0006] The present application provides a method and apparatus for implementing audio watermarking, which can meet the requirements of anti-copying.

[0007] An embodiment of the present invention provides a method for implementing audio watermarking, comprising:

[0008] Divide the audio signal to be processed into frames and perform frequency domain transformation processing, where each frame includes less than a preset number of sampling points;

[0009] Select m regions from the divided audio frame as embedding regions, where the m regions correspond to vectors ci formed by m frequency domain transformations, where m is an integer greater than 1;

[0010] According to the watermark information modulated by the symbol sequence of length m and the m orthogonal vectors generated by the key, the vectors formed by the m frequency domain transformations are superimposed to obtain the audio signal containing the watermark.

[0011] In an exemplary embodiment, before selecting m segments of regions as embedding regions, the method further includes: analyzing the divided audio frames and selecting frequency bands with weak perception.

[0012] In an exemplary embodiment, selecting a frequency band with weak perception includes:

[0013] The frequency domain masking effect of the human ear is used to analyze the divided audio frames, and the frequency bands below the frequency domain masking threshold are selected as the frequency bands with weak perception.

[0014] In an exemplary embodiment, the method further includes: determining an embedding strength β of the embedding region according to characteristics of the embedding region.

[0015] In an exemplary embodiment, determining the embedding strength β of the embedding region according to the characteristics of the embedding region includes:

[0016] The frequency domain masking threshold of each frame of audio after the framing is calculated respectively, and the embedding strength β is set to the maximum value of the calculated frequency domain masking thresholds.

[0017] In an exemplary embodiment, the method further includes: generating the m orthogonal vectors wi by using a key;

[0018] The watermark information is modulated by a symbol sequence with a prime number length of m to obtain the modulated watermark information.

[0019] In an exemplary embodiment, the step of performing superposition processing on the vectors formed by the m frequency domain transformations to obtain the audio signal containing the watermark includes:

[0020] Perform watermark embedding on the vector ci formed by the continuous m frequency domain transformations:

[0021] Using the obtained ci' to replace the vector ci formed by the frequency domain transformation;

[0022] Performing inverse frequency domain transformation on the frequency domain transformation coefficients of each frame to obtain the audio signal containing the watermark.

[0023] In an exemplary embodiment, the preset number is 256.

[0024] An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute any of the above methods for implementing audio watermarking.

[0025] An embodiment of the present application further provides a device for implementing audio watermarking, comprising a memory and a processor, wherein the memory stores the following instructions executable by the processor: for executing the steps of any of the above-mentioned methods for implementing audio watermarking.

[0026] The present embodiment also provides another method for implementing audio watermarking, including:

[0027] Divide the audio to be detected into frames, where each frame includes less than a preset number of sampling points;

[0028] Locate the non-silent audio segments based on the frame average energy and a preset energy threshold, and apply different offsets to the located non-silent audio segments for frame frequency domain transformation;

[0029] Select m segments from the audio frame after framing as the embedding area, where the m segments correspond to the vector ci formed by m frequency domain transformations; calculate the correlation value of adjacent frames using a symbol sequence of length m;

[0030] According to the offset corresponding to the maximum value among the calculated correlation values ​​of adjacent frames, the offset corresponding to the maximum value is applied to perform frame frequency domain transformation processing on all audio;

[0031] The correlation values ​​of adjacent frames are calculated and summed for the vector ci composed of m consecutive frequency domain transform coefficients, and the watermark information is determined based on the calculated sum.

[0032] In an exemplary embodiment, locating a non-silent audio segment according to the frame average energy and a preset energy threshold includes:

[0033] Counting the square sum of sample values ​​of the framed audio frame of a preset length L, and calculating the average value to obtain the frame average energy;

[0034] The calculated frame average energy is compared with the energy threshold, and the audio frame whose energy is not less than the energy threshold is determined as the non-silent frame.

[0035] In an exemplary embodiment, applying different offsets to the located non-silent audio segments to perform frame-by-frame frequency domain transformation processing includes:

[0036] Applying N offset samples to perform frequency domain transform processing on the L non-silent frames respectively to obtain corresponding frequency domain transform coefficient vectors;

[0037] Calculate the correlation value between each frequency domain transform coefficient vector and the frequency domain transform coefficient vector of the adjacent segment respectively, and obtain the correlation values ​​of P consecutive frames;

[0038] For each offset, calculate the average of the absolute values ​​of P correlation values;

[0039] Take the maximum value among the N average values. If the maximum value is greater than or equal to the preset threshold T2, the corresponding offset value is used as the correct offset value; if the maximum value is less than the preset threshold T2, determine whether to continue trying different audio segments based on the real-time detection requirements. For scenarios with high real-time detection requirements, end this process; for scenarios with low real-time requirements, continue to try to apply different offsets to different audio segments to perform the frame frequency domain transformation processing until the audio ends or the offset value corresponding to the average value being greater than or equal to the threshold T2 is found.

[0040] In an exemplary embodiment, calculating correlation values ​​of adjacent frames using a preset symbol sequence of length m includes:

[0041] The correlation value of the frequency domain transform coefficient vector between the coefficient vector ci of a certain frame and the coefficient vector cj of the m / 2 frames adjacent to the frame is Corr=<ci,cj> , where the operator <> represents the sum of the dot products;

[0042] The number of adjacent frames to a certain frame is m / 2 frames, and j=i+m / 2.

[0043] In an exemplary embodiment, determining the watermark information according to the calculated sum value includes:

[0044] The sum is a positive value, and the bit determining the watermark information is 1;

[0045] The sum is a positive value, and the bit of the determined watermark information is 0.

[0046] In an exemplary embodiment, the preset number is 256.

[0047] An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute any one of the above methods for implementing an audio watermark.

[0048] An embodiment of the present application further provides a device for implementing audio watermarking, including a memory and a processor, wherein the memory stores the following instructions that can be executed by the processor: used to execute the steps of another method for implementing audio watermarking described in any one of the above.

[0049] The embodiments of the present application, on the one hand, through a relatively short frequency domain transform (i.e., each frame after framing includes less than a preset number of sampling points), the computational complexity is relatively controllable, which can ensure the real-time performance of the computation, thereby ensuring high real-time performance of audio processing and achieving low-complexity, low-latency watermark embedding; watermark embedding is achieved through symbol sequence modulation, offsetting the effects of carrier distortion; on the other hand, watermark extraction through correlation greatly improves the robustness of decoding. The method for implementing audio watermarking provided by this application meets the requirements of anti-copying.

[0050] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0052] Figure 1 This is a diagram of the architecture of a digital audio watermarking system in related technology;

[0053] Figure 2 A flowchart of a method for implementing audio watermarking in an embodiment of the present application;

[0054] Figure 3 This is a flowchart of an application embodiment of a method for implementing audio watermarking in the present application;

[0055] Figure 4 This is a flowchart of another method for implementing audio watermarking in an embodiment of the present application;

[0056] Figure 5 This is a flow chart of another embodiment of the method for implementing audio watermarking in the present application. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of this application can be combined with each other in any way.

[0058] In a typical configuration of the present application, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0059] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0060] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0061] The steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Also, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be performed in an order different from that shown here.

[0062] The inventors of this application have discovered that audio watermarking solutions in the related art are not robust enough in dubbing scenarios. This is particularly true for low-end devices with low-quality external audio playback. Due to the severe audio distortion and noise introduced by external audio playback, the audio watermark often cannot be correctly extracted. Furthermore, the dubbing-resistant audio watermarking algorithms in the related art are very complex and require a large amount of computation, which cannot meet the requirements of real-time audio watermark embedding and extraction. This greatly limits their application scenarios and makes them difficult to deploy on mobile devices.

[0063] In the embodiment of the present application, high real-time performance of audio processing is ensured by frequency domain transformation processing with a relatively short length, thereby achieving low-complexity and low-latency watermark embedding; watermark embedding is achieved by symbol sequence modulation, which can offset the influence of carrier distortion and greatly improve the robustness of decoding through correlation extraction.

[0064] Figure 2 FIG. 1 is a flow chart of a method for implementing audio watermarking in an embodiment of the present application. Figure 2 Shown, including:

[0065] Step 200: Divide the audio signal to be processed into frames and perform frequency domain transformation processing, where each frame includes less than a preset number of sampling points.

[0066] In an exemplary embodiment, the frequency domain transformation may include, but is not limited to, discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete wavelet transform (DWT), and the like.

[0067] In an exemplary embodiment, the preset number may include, for example, approximately 256 sampling points, 512 sampling points, 1024 sampling points, or sampling points that are powers of 2. In one embodiment, the preset number is 256.

[0068] In the embodiment of the present application, the number of sampling points in each audio frame is limited. By providing a shorter frequency domain transformation, the high real-time performance of audio processing is guaranteed, which provides a guarantee for low-complexity and low-latency watermark embedding.

[0069] Step 201: Selecting a preset digital m-segment region from the divided audio frame as an embedding region, where the m-segment region corresponds to a vector ci formed by m frequency domain transformations.

[0070] In an exemplary embodiment, before step 201, the following steps may also be included:

[0071] Analyze the divided audio frames and select the frequency bands with weak perception.

[0072] In an exemplary embodiment, analyzing the divided audio frames and selecting the frequency band with weak perception may include:

[0073] The frequency domain masking effect of the human ear is used to analyze the divided audio frames, and the frequency bands below the frequency domain masking threshold are selected as the frequency bands with weak perception.

[0074] The phenomenon in which a weaker sound (the masked sound) is affected by a stronger sound (the masking sound) is called the masking effect. A strong pure tone can mask weaker pure tones emitted simultaneously nearby, a characteristic known as frequency-domain masking.

[0075] In an exemplary embodiment, the m segments of the embedded area belong to a frequency band with weak perception, and the length of the frequency domain transform coefficient is a preset length L.

[0076] In an exemplary embodiment, the minimum value of m is 4, and may also be a multiple of 4.

[0077] In an exemplary embodiment, the m regions used as embedding regions may be continuous frequency bands, or frequency bands obtained according to a preset interval or pattern, for example, one region is selected as the embedded region for every two regions.

[0078] In an exemplary embodiment, the value of the preset length L can be set to a value between 16 and 32 to avoid L being too long and affecting the hearing quality. It should be noted that the value of L is only an example and is not intended to limit the scope of protection of this application.

[0079] In an exemplary embodiment, step 201 may further include:

[0080] Determine the embedding strength β of the embedded region.

[0081] In an exemplary embodiment, the embedding strength β of the embedding region may be a preset default value.

[0082] In one implementation example, it also includes:

[0083] The embedding strength β of the embedding area is adjusted according to the characteristics of the embedding area. In this way, the embedding strength of the embedding area is adaptively adjusted, thereby ensuring the maximum embedding strength under the condition of imperceptibility and achieving stronger watermark robustness.

[0084] In an implementation example, adjusting the embedding strength β of the embedding region according to the characteristics of the embedding region may include:

[0085] The frequency domain masking threshold is calculated for each frame of audio after framing, and the embedding strength β is set to the maximum value of the calculated frequency domain masking thresholds to achieve the optimal watermark embedding effect.

[0086] Step 202: Based on the watermark information modulated by the symbol sequence of length m and the m orthogonal vectors generated by the key, the m vectors formed by the frequency domain transformation are superimposed to obtain an audio signal containing the watermark.

[0087] In an exemplary embodiment, before step 202, the following step may be further included: generating m orthogonal vectors using a key, which may include:

[0088] According to the preset key K, m normalized orthogonal vectors wi are generated using, for example, the Gram-Schmidt orthogonalization method, thereby generating controllable noise.

[0089] In an exemplary embodiment, before step 202, the following step may be further included: modulating the watermark information by a symbol sequence of length m to obtain the modulated watermark information, which may include:

[0090] Use symbol sequences of length m, such as SP(+1) and SN(-1), to modulate the watermark information and convert it into a bit stream. For example, when m=4, you can preset SP=[+1,+1,-1,+1] and SN=[-1,-1,+1,-1] to satisfy<SP,SP> =4,<SP,SN> =-4.

[0091] In an exemplary embodiment, performing superposition processing on m vectors formed by frequency domain transformations to obtain an audio signal containing a watermark may include:

[0092] To implement the embedding operation, the projection component of the vector ci on the preset normalized noise wi is first calculated, and then the above projection component is replaced by the preset noise with adaptive intensity after modulation; in order to distinguish whether the embedded bit is 0 or 1, the m vectors are modulated using the preset symbol sequences SN and SP respectively. According to the following relationship, the vector ci composed of m frequency domain transformations is watermarked:

[0093] When 1 is embedded, ci'=ci-<ci,wi> wi+SPi*β*|ci|*wi; where, i=1~m;

[0094] When 0 is embedded, ci'=ci-<ci,wi> wi+SNi*β*|ci|*wi;

[0095] Use the obtained ci' to replace the vector ci formed by frequency domain transformation;

[0096] The frequency domain transform coefficients of each frame are inversely transformed to obtain an audio signal containing a watermark.

[0097] In an exemplary embodiment, the inverse frequency domain transform may include, but is not limited to, inverse discrete Fourier transform (IDFT), inverse discrete cosine transform (IDCT), inverse discrete wavelet transform (IDWT), and the like.

[0098] The method for implementing audio watermarking provided in this application ensures high real-time performance of audio processing through shorter frequency domain transformation, and realizes low-complexity and low-latency watermark embedding; watermark embedding is realized through symbol sequence modulation, which offsets the influence of carrier distortion.

[0099] The present application also provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned Figure 2 A method for implementing audio watermarking as shown in any one of the items.

[0100] The present application further provides a device for implementing audio watermarking, comprising a memory and a processor, wherein the memory stores the following instructions that can be executed by the processor: Figure 2 The steps of any one of the methods for implementing audio watermarking are shown.

[0101] The following example uses audio content A, which is a paid audio played online, and it is desired to embed a user ID into it for pirated source tracing. In this embodiment, DCT is used as the frequency domain transformation method, such as Figure 3 As shown, watermark embedding is implemented for audio content A, including:

[0102] Steps 300 to 301: Divide the audio content A on the server into frames, where the number of sampling points, or samples, in each frame is N. In this embodiment, N=256. Perform DCT processing on the audio frames obtained after framing.

[0103] Step 302: Analyze the audio frame, select the embedding area and determine the embedding strength.

[0104] In this embodiment, a frequency band with weak perception can be selected according to the frequency domain masking threshold of the audio frame; and m continuous regions are selected from the frequency band with weak perception as embedding regions, where the m regions correspond to vectors ci composed of m DCTs.

[0105] In this embodiment, the embedding strength β of the embedding region may be determined according to the characteristics of the embedding region.

[0106] Steps 3011 to 3013: Generate watermark information according to the user ID; modulate the watermark information through a symbol sequence of length m to obtain modulated watermark information.

[0107] It should be noted that steps 3011 to 3013 only need to be completed before step 303 .

[0108] Step 303: Based on the modulated watermark information and the m normalized orthogonal vectors generated by the key, the vectors formed by the m DCTs are superimposed, that is, watermark embedding is achieved through correlated orthogonal processing.

[0109] In this embodiment, watermark embedding can be performed on a vector ci consisting of m consecutive DCTs according to the following relationship:

[0110] When 1 is embedded, ci'=ci-<ci,wi> wi+SPi*β*|ci|*wi; where, i=1~m;

[0111] When 0 is embedded, ci'=ci-<ci,wi> wi+SNi*β*|ci|*wi;

[0112] Use the obtained ci' to replace the vector ci formed by DCT;

[0113] Steps 304 to 305: Perform IDCT transformation on the DCT coefficients of each frame after orthogonal processing to obtain audio content containing user ID watermark information.

[0114] Figure 4 FIG. 1 is a flow chart of another method for implementing audio watermarking in an embodiment of the present application. Figure 4 Shown, including:

[0115] Step 400: Divide the audio to be detected into frames, where each frame includes less than a preset number of sampling points.

[0116] Step 401: locating non-silent audio segments according to the average energy of the framed audio frames and a preset energy threshold, and applying different offsets to the located non-silent audio segments for frame frequency domain transformation.

[0117] In an exemplary embodiment, the frequency domain transformation may include but is not limited to: DFT, DCT, DWT, etc.

[0118] In an exemplary embodiment, the preset number may include, for example, approximately 256 sampling points, 512 sampling points, 1024 sampling points, or sampling points that are powers of 2. In one embodiment, the preset number is 256.

[0119] In an exemplary embodiment, locating a non-silent audio segment based on the average frame energy of the divided audio frame and a preset energy threshold may include:

[0120] The square sum of the sample values ​​of an audio frame with a preset length L is statistically calculated, and the average value is calculated to obtain the frame average energy; the calculated frame average energy is compared with the energy threshold T1, and audio frames less than the energy threshold T1 are determined as silent frames, and audio frames not less than the energy threshold T1 are determined as non-silent frames.

[0121] In an exemplary embodiment, applying different offsets to the located non-silent audio segments to perform frame-by-frame frequency domain transformation processing may include:

[0122] Apply N offset samples (for example, N can be 16) to perform frequency domain transformation on the non-silent frames (for example, L frames), and obtain corresponding frequency domain transformation coefficient vectors, which are L*0 / N, L*1 / N, ..., L*(N-1) / N respectively;

[0123] Calculate the correlation value Corr between each frequency domain transform coefficient vector and the frequency domain transform coefficient vector of the adjacent segment;

[0124] For each offset, obtain the correlation values ​​of P consecutive frames through the above operation, and calculate the average of the absolute values ​​of the P correlation values;

[0125] Take the maximum value among the N average values. If the maximum value is greater than or equal to the preset threshold T2, it is determined that the segment contains a watermark, and the corresponding offset value is used as the correct offset value; if the maximum value is less than the preset threshold T2, it is determined that the segment does not contain a watermark, and whether to further try different audio segments is determined based on the real-time detection requirements. For scenarios with high real-time detection requirements, no further attempts are made and the watermark extraction ends here; for scenarios with low real-time requirements, continue to try to apply different offsets to different audio segments until the audio ends or an offset value corresponding to the above average value being greater than or equal to the threshold T2 is found.

[0126] This step uses different offset values ​​to try to find the position of the embedded watermark, finds the correct offset, and thus obtains the correct position of the embedded watermark.

[0127] Step 402: m regions are selected from the divided audio frame as embedding regions, where the m regions correspond to vectors ci formed by m frequency domain transformations; and correlation values ​​of adjacent frames are calculated using a symbol sequence of length m.

[0128] In an exemplary embodiment, the m regions used as embedding regions may be continuous frequency bands, or frequency bands obtained according to a preset interval or pattern, for example, one region is selected as the embedded region for every two regions.

[0129] In an exemplary embodiment, the value of the preset length L can be set to a value between 16 and 32 to avoid L being too long and affecting the hearing quality. It should be noted that the value of L is only an example and is not intended to limit the scope of protection of this application.

[0130] In an exemplary embodiment, calculating the correlation values ​​of adjacent frames using a preset symbol sequence of length m in step 402 may include:

[0131] Suppose the coefficient vector of a certain frame is ci, and the coefficient vector cj of the m / 2 frame adjacent to the frame is selected (j=i+m / 2), then the correlation value of the frequency domain transform coefficient vector between the coefficient vector of a certain frame ci and the coefficient vector cj of the m / 2 frame adjacent to the frame is Corr=<ci,cj> , where the operator <> represents the calculation of the sum of dot products.

[0132] Step 403: Based on the offset corresponding to the maximum value among the calculated correlation values ​​of adjacent frames, apply the offset corresponding to the maximum value to perform frame-by-frame frequency domain transformation processing on all audios.

[0133] Here, the offset corresponding to the maximum value among the correlation values ​​of adjacent frames is the optimal offset.

[0134] Step 404: Calculate the correlation values ​​of adjacent frames for the m segments of frequency domain transform coefficients constituting the vector ci and sum them up, and determine the watermark information according to the calculated sum.

[0135] In an exemplary embodiment, if the sum is greater than a preset threshold T, then the bit of the watermark information is determined to be 1; if the sum is less than the preset threshold -T, then the bit of the watermark information is determined to be 0; in other cases, it is determined that the segment does not contain a watermark.

[0136] The watermark extraction method in this embodiment of the application is based on the fact that adjacent segments of watermarked audio often experience similar distortion after being ripped or otherwise attacked, and the signals of adjacent segments are also similar. Therefore, this application extracts the watermark by calculating the correlation between adjacent segments and summing them. This preserves the correlation of the watermark signal and greatly improves the robustness of decoding.

[0137] The present application also provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned Figure 4 A method for implementing audio watermarking as shown in any one of the items.

[0138] The present application further provides a device for implementing audio watermarking, comprising a memory and a processor, wherein the memory stores the following instructions that can be executed by the processor: Figure 4 The steps of any one of the methods for implementing audio watermarking are shown.

[0139] The following takes the audio file recovered from pirated channels as an example. In this embodiment, DCT is used as the frequency domain transformation method. Figure 5 As shown, the extraction of the watermark embedded in the audio includes:

[0140] Steps 500 to 501: Divide the audio to be detected into frames. In this embodiment, the number of samples per frame is N=256.

[0141] Steps 501 to 502: Rapidly locate potential watermarked frequency bands by frame division: locate non-silent audio segments by comparing the frame average energy with a threshold, and perform frame-by-frame DCT processing on these located audio segments using different offsets.

[0142] Steps 503 to 504: Select m segments from the audio frame after framing as embedding areas, where the m segments correspond to a vector ci composed of m DCTs; use a symbol sequence of length m to calculate the correlation values ​​of adjacent frames; find the maximum value from the calculated correlation values, and the offset corresponding to the maximum value among the correlation values ​​of adjacent frames is the optimal offset. Apply the optimal offset to perform framed DCT processing on all audio.

[0143] Steps 505 to 506: Calculate the correlation values ​​of adjacent frames for the vector ci of the continuous m segments of DCT coefficients and sum them up. If the sum is greater than the preset threshold T, then the bit of the watermark information is determined to be 1; if the sum is less than the preset threshold -T, then the bit of the watermark information is determined to be 0.

[0144] The technical solution for implementing audio watermarks provided by the embodiment of the present application can be applied to scenarios such as audio and video copyright detection. For example: by using the audio watermark extraction method provided by the embodiment of the present application, the copyright of the audio and video can be detected by detecting whether the played audio and video contains a watermark embedded by the audio watermark embedding method provided by the embodiment of the present application. Furthermore, when playing watermarked audio and video, it is also possible to choose whether to synchronously display on the interface whether the audio and video contains genuine watermark information, or to prompt that the audio and video does not have watermark information and may be pirated, so that users can have a more comprehensive understanding of the status of the audio and video they are using.

[0145] Although the embodiments disclosed in this application are as described above, the contents described are merely embodiments adopted to facilitate understanding of this application and are not intended to limit this application. Any person skilled in the art to which this application belongs may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application. However, the scope of patent protection of this application shall still be based on the scope defined by the attached claims.

Claims

1. A method for implementing audio watermarking, comprising: Divide the audio signal to be processed into frames and perform frequency domain transformation processing, where each frame includes less than a preset number of sampling points; Select m regions from the divided audio frame as embedding regions, where the m regions correspond to vectors ci formed by m frequency domain transformations, where m is an integer greater than 1; According to the watermark information modulated by the symbol sequence of length m and the m orthogonal vectors generated by the key, the vectors formed by the m frequency domain transformations are superimposed to obtain the audio signal containing the watermark.

2. The method according to claim 1, before selecting m segments of regions as embedding regions, further comprising: The divided audio frames are analyzed to select frequency bands with weak perception.

3. The method according to claim 2, wherein: The selecting of a frequency band with weak perception includes: The frequency domain masking effect of the human ear is used to analyze the divided audio frames, and the frequency bands below the frequency domain masking threshold are selected as the frequency bands with weak perception.

4. The method according to claim 1, further comprising: The embedding strength β of the embedding region is determined according to the characteristics of the embedding region.

5. The method according to claim 4, wherein The determining the embedding strength β of the embedding region according to the characteristics of the embedding region includes: The frequency domain masking threshold of each frame of audio after the framing is calculated respectively, and the embedding strength β is set to the maximum value of the calculated frequency domain masking thresholds.

6. The method according to claim 5, further comprising: Generate the m orthogonal vectors wi using a key; The watermark information is modulated by the symbol sequence with a length of m to obtain the modulated watermark information.

7. The method according to claim 6, wherein: The step of superimposing the m vectors formed by the frequency domain transformations to obtain the audio signal containing the watermark includes: Perform watermark embedding on the vector ci formed by the continuous m frequency domain transformations: Using the obtained ci' to replace the vector ci formed by the frequency domain transformation; Performing inverse frequency domain transformation on the frequency domain transformation coefficients of each frame to obtain the audio signal containing the watermark.

8. The method according to any one of claims 1 to 7, wherein: The preset number is 256.

9. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method for implementing audio watermarking according to any one of claims 1 to 8.

10. A device for implementing audio watermarking, comprising a memory and a processor, wherein: The memory stores the following instructions that can be executed by the processor: instructions for executing the steps of the method for implementing audio watermarking as described in any one of claims 1 to 8.

11. A method for implementing audio watermarking, comprising: Divide the audio to be detected into frames, where each frame includes less than a preset number of sampling points; Locate the non-silent audio segments based on the frame average energy and a preset energy threshold, and apply different offsets to the located non-silent audio segments for frame frequency domain transformation processing; Select m segments from the audio frame after framing as the embedding area, where the m segments correspond to the vector ci formed by m frequency domain transformations; calculate the correlation value of adjacent frames using a symbol sequence of length m; According to the offset corresponding to the maximum value among the calculated correlation values ​​of adjacent frames, the offset corresponding to the maximum value is applied to perform frame frequency domain transformation processing on all audio; The correlation values ​​of adjacent frames are calculated and summed for the vector ci composed of m consecutive frequency domain transform coefficients, and the watermark information is determined based on the calculated sum.

12. The method according to claim 11, wherein The method of locating a non-silent audio segment according to the frame average energy and a preset energy threshold comprises: Counting the square sum of sample values ​​of the framed audio frame of a preset length L, and calculating the average value to obtain the frame average energy; The calculated frame average energy is compared with the energy threshold, and the audio frames whose energy is not less than the energy threshold are determined as the non-silent audio segments.

13. The method according to claim 11, wherein The applying different offsets to the located non-silent audio segments to perform frame frequency domain transformation processing includes: Applying N offset samples to perform frequency domain transform processing on the L non-silent audio segments respectively to obtain corresponding frequency domain transform coefficient vectors; Calculate the correlation value between each frequency domain transform coefficient vector and the frequency domain transform coefficient vector of the adjacent segment respectively, and obtain the correlation values ​​of P consecutive frames; For each offset, calculate the average of the absolute values ​​of P correlation values; Take the maximum value among the N average values. If the maximum value is greater than or equal to the preset threshold T2, the corresponding offset value is used as the correct offset value; if the maximum value is less than the preset threshold T2, determine whether to continue trying different audio segments based on the real-time detection requirements. For scenarios with high real-time detection requirements, end this process; for scenarios with low real-time requirements, continue to try to apply different offsets to different audio segments to perform the frame frequency domain transformation processing until the audio ends or the offset value corresponding to the average value being greater than or equal to the threshold T2 is found.

14. The method according to claim 11, wherein The method of calculating correlation values ​​of adjacent frames using a symbol sequence having a length of m includes: The correlation value of the frequency domain transform coefficient vector between the coefficient vector ci of a certain frame and the coefficient vector cj of the m / 2 frames adjacent to the frame is Corr=<ci,cj> , where the operator <> represents the sum of the dot products; The number of adjacent frames to a certain frame is m / 2 frames, and j=i+m / 2.

15. The method according to claim 11, wherein The determining of watermark information according to the calculated sum value includes: The sum is a positive value, and the bit determining the watermark information is 1; If the sum is a negative value, the bit of the determined watermark information is 0.

16. The method according to any one of claims 11 to 15, wherein: The preset number is 256.

17. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method for implementing audio watermarking according to any one of claims 11 to 16.

18. A device for implementing audio watermarking, comprising a memory and a processor, wherein: The memory stores the following instructions that can be executed by the processor: used to execute the steps of the method for implementing audio watermarking as described in any one of claims 11 to 16.

Images