Encoding method and decoding method

By using text-to-image conversion and DCT/SVD algorithm to embed watermarks in audio, the problems of watermark robustness and unaudibility in audio are solved, and the effective protection of audio intellectual property rights is achieved.

CN119028362APending Publication Date: 2024-11-26HTC CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311395521.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-26
Filing Date
2023-10-25
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the audio field, how to develop an unaudible and robust digital watermark in audio to effectively protect intellectual property rights and identify creators or owners, especially the problem of the watermark being damaged after the audio is compressed .

Method used

By converting text watermarks into image watermarks and using discrete cosine transform (DCT) and singular value decomposition (SVD) algorithms, the image watermarks are embedded into audio, converted from the time domain to the frequency domain, and embedded in the frequency domain to Improve robustness and convert it back to the time domain to produce watermarked audio while using masking effects to increase inaudibility.

Benefits of technology

It achieves the robustness and inaudibility of watermarks in audio, ensures the effectiveness of watermarks during audio compression and distribution, can effectively identify the creators or owners of audio, and improves the protection of intellectual property rights.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028362A_ABST
    Figure CN119028362A_ABST
Patent Text Reader

Abstract

The invention provides an encoding method and a decoding method. The encoding method is used for embedding a watermark into audio. And obtaining a text watermark and an original audio. And converting the text watermark into an image watermark. Raw audio is converted from a time domain to a frequency domain to produce pre-processed audio. An image watermark is embedded into the pre-processed audio to produce an encoded audio. The encoded audio is converted from a frequency domain to a time domain to produce watermarked audio.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an encoding method; specifically, the present disclosure relates to an encoding method and a decoding method. Background Art

[0002] Due to the advancement of technology, information spreads quite quickly, so how to protect intellectual property rights has become an important issue now. Signatures, watermarks or trademarks are often used to help identify the ownership of intellectual property rights. However, in the field of audio, it is relatively difficult to prove that a sound effect or a piece of audio is created by someone or belongs to someone. In order to solve the above problems, digital watermarks hidden in audio can be used as evidence to find out the creator or owner of a sound effect or a piece of audio. Summary of the invention

[0003] The present disclosure relates to an encoding method and a decoding method to improve the robustness and inaudibility of a watermark in audio.

[0004] In the present disclosure, a coding method for embedding a watermark into audio is provided. The coding method includes: obtaining a text watermark and an original audio; converting the text watermark into an image watermark; converting the original audio from a time domain to a frequency domain to generate a preprocessed audio; embedding the image watermark into the preprocessed audio to generate an encoded audio; and converting the encoded audio from the frequency domain to the time domain to generate a watermarked audio.

[0005] In the present disclosure, a decoding method for verifying a watermark in audio is provided. The decoding method includes: obtaining a text watermark and an audio with the watermark; converting the text watermark into an image watermark; converting the audio with the watermark from the time domain to the frequency domain to generate a target audio; extracting an extracted image from the target audio; and comparing the extracted image with the image watermark to generate a verification result.

[0006] Based on the above, according to the encoding method and the decoding method, the robustness and inaudibility of the watermark in the audio are improved.

[0007] In order to make the above contents easier to understand, several embodiments attached with the drawings will be described in detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure includes accompanying drawings to provide a further understanding of the present disclosure, and the accompanying drawings are incorporated in and constitute a part of this specification. The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure.

[0009] Figure 1A is a schematic diagram of a coding scenario according to an embodiment of the present disclosure.

[0010] Figure 1B is a schematic diagram of a decoding scenario according to an embodiment of the present disclosure.

[0011] Figure 2 is a schematic diagram of a coding scenario according to an embodiment of the present disclosure.

[0012] Figure 3 is a schematic flowchart of an encoding scenario according to an embodiment of the present disclosure.

[0013] Figure 4 is a schematic flowchart of a decoding scenario according to an embodiment of the present disclosure.

[0014] Figure 5 is a schematic flowchart of an encoding method according to an embodiment of the present disclosure.

[0015] Figure 6 is a schematic flowchart of a decoding method according to an embodiment of the present disclosure.

[0016] [Explanation of Symbols]

[0017] 100A, 200, 300: Coding scenarios

[0018] 100B, 400: Decoding scenario

[0019] 202, 302, 402: Text to Image Converter

[0020] 204: Encoder / Encoding Module

[0021] 304, 404: Select channel module

[0022] 306, 406: Hybrid modules

[0023] 308, 408: Discrete Cosine Transform (DCT) module

[0024] 310: Encoding module

[0025] 310-1, 410-1: Quantization Index Modulation (QIM) module

[0026] 310-2, 410-2: Singular Value Decomposition (SVD) module

[0027] 312: Inverse Discrete Cosine Transform (IDCT) Module

[0028] 314: Separation module

[0029] 316: Crossfader

[0030] 410: Decoding module

[0031] 412: Image Extractor

[0032] 414: Normalized Cross-Correlation Module

[0033] A_O: Original audio

[0034] A_W: Audio with watermark

[0035] DEC: Decoder

[0036] E_I: Extracted Image

[0037] ENC: Encoder

[0038] RST: Verification result

[0039] S510, S520, S530, S540, S550, S610, S620, S630, S640, S650: Steps

[0040] W: Watermark

[0041] W_A: Audio watermark

[0042] W_I: Image watermark

[0043] W_T: Text watermark DETAILED DESCRIPTION

[0044] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the description to refer to the same or like components.

[0045] Throughout the specification and claims of the present disclosure, certain terms are used to refer to specific components. It should be understood by those skilled in the art that electronic device manufacturers may refer to the same component by different names. It is not intended herein to distinguish between components that have the same function but different names. In the following description and claims, words such as "comprise" and "include" are open-ended terms and should be interpreted as "including but not limited to...".

[0046] The term "coupling (coupling) (or connection (connection))" used throughout the entire specification of this application (including the attached claims) may refer to any direct or indirect connection mode. For example, if the text states that a first device is coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices to be connected or some connection modes. The terms "first (first)", "second (second)" and similar terms mentioned throughout the entire specification of this application (including the attached claims) are only used to name discrete elements or to distinguish different embodiments or scopes. Therefore, the terms should not be regarded as limiting the upper or lower limits of the number of elements and should not be used to limit the order of arrangement of elements. In addition, the elements / components / steps using the same reference numbers in the drawings and embodiments are used as much as possible to represent the same or similar parts. In different embodiments, the same reference numbers or the same terms can be used to refer to the relevant descriptions of the elements / components / steps.

[0047] It should be noted that in the following embodiments, without departing from the spirit of the present disclosure, the technical features of several different embodiments may be replaced, recombined and mixed to complete other embodiments. As long as the features of each embodiment do not violate the spirit of the present disclosure or conflict with each other, the embodiments may be mixed and used together at will.

[0048] Intellectual property is very important in all fields. However, in the field of audio, it is relatively difficult to prove that a sound effect or a piece of audio is created by someone or belongs to someone. To solve the above problems, digital watermarks hidden in the audio can be used as evidence to find out the creator or owner of a sound effect or a piece of audio.

[0049] When the watermark is not properly embedded in the audio, the audio may sound noisy due to the inconsistency between the watermark and the audio. In addition, after the audio is compressed, the watermark may be damaged, making the watermark unrecognizable. That is, how to develop an inaudible and robust watermark in audio has become an urgent problem to be solved.

[0050] Figure 1A is a schematic diagram of a coding scenario according to an embodiment of the present disclosure. Figure 1B is a schematic diagram of a decoding scenario according to an embodiment of the present disclosure. Figure 1A and Figure 1B , the encoding scenario 100A illustrates how the watermark W can be embedded into the original audio A_O, and the decoding scenario 100B illustrates how the watermark W can be extracted from the watermarked audio A_W.

[0051] exist Figure 1AIn the illustrated encoding scenario 100A, the watermark W can be embedded into the original audio A_O using the encoder ENC. After the original audio A_O is watermarked, the watermarked audio A_W can be generated.

[0052] In one embodiment, the watermark W may include, for example, an identification number of the original audio A_O, an identification number of a user, a trademark, an artist's name, a creator's name, an owner's name, or other identifiable information. However, the present disclosure is not limited thereto. That is, the watermarked audio A_W may carry information for identifying the creator or owner of the original audio A_O.

[0053] exist Figure 1B In the illustrated decoding scenario 100B, the watermark W can be extracted from the watermarked audio A_W using the decoder DEC. Based on the watermark W extracted from the watermarked audio A_W, the creator or owner of the original audio A_O can be found.

[0054] In one embodiment, encoder ENC and / or decoder DEC can be implemented as multiple program codes.Program code is stored in a memory and is executed by a controller or a processor. Alternatively, in an embodiment, each of the functions of encoder ENC and / or decoder DEC can be implemented as one or more circuits.The present disclosure does not limit the use of software or hardware to implement the function of encoder ENC and / or decoder DEC.

[0055] In this way, it is possible to hide a digital watermark in the audio, which helps to find out the creator or owner of the audio.

[0056] Figure 2 is a schematic diagram of a coding scenario according to an embodiment of the present disclosure. Figure 1A to Figure 2 , encoding scenario 200 illustrates how a text watermark W_T may be embedded into audio. Encoding scenario 200 may include a text-to-image converter 202 and an encoder 204.

[0057] In one embodiment, the text-to-image converter 202 may be configured to convert the text watermark W_T into an image watermark W_I. For example, a text-to-image database may be stored in the memory of the text-to-image converter 202, and the text-to-image database may indicate the relationship between the pattern corresponding to the letters A to Z and the pattern corresponding to the numbers 0 to 9. However, the present disclosure is not limited thereto. Therefore, the text watermark W_T may be converted from a text format into an image format (i.e., an image watermark W_I) based on the text-to-image database.

[0058] In one embodiment, the encoder 204 may be configured to convert the image watermark W_I into an audio watermark W_A. For example, the encoder 204 may be configured to convert a spatial distribution into a temporal distribution in the time domain or a frequency distribution in the frequency domain. Thus, the image watermark W_I may be converted from an image format into an audio format (i.e., an audio watermark W_A).

[0059] In one embodiment, text to image converter 202 and / or encoder 204 can be implemented as multiple program codes. The program code is stored in a memory and is executed by a controller or a processor. Alternatively, in an embodiment, each of the functions of text to image converter 202 and / or encoder 204 can be implemented as one or more circuits. The present disclosure does not limit the use of software or hardware to implement the function of text to image converter 202 and / or encoder 204.

[0060] In this way, the digital watermark, text watermark W_T in the audio can be converted from text format to audio format (ie, audio watermark W_A). In this way, the audio watermark W_A can be prepared to be embedded in the audio.

[0061] Figure 3 is a schematic flow chart of a coding scenario according to an embodiment of the present disclosure. Figures 1A to 3 , encoding scenario 300 illustrates how a digital watermark may be embedded into audio. Encoding scenario 300 may include multiple modules to implement different functions of encoding. Although the functions of encoding scenario 300 are illustrated and described as being implemented by different modules for ease of explanation, it should be noted that the functions of encoding scenario 300 may be implemented by a single module. That is, the multiple modules in encoding scenario 300 may be integrated together as a single module.

[0062] In one embodiment, the multiple modules or a single module may be implemented as multiple program codes. The program code is stored in a memory and executed by a controller or a processor. Alternatively, in an embodiment, each of the functions of the multiple modules or a single module may be implemented as one or more circuits. The present disclosure does not limit the use of software or hardware to implement the functions of the multiple modules or a single module.

[0063] First, a text watermark W_T and an original audio A_O may be obtained. For example, the text watermark W_T and the original audio A_O may be provided by a user. The user may be the creator or owner of the original audio A_O, but the present disclosure is not limited thereto.

[0064] The text-to-image converter 302 may convert the text watermark W_T into an image watermark W_I. In addition, the encoding module 310 may convert the image watermark W_I into an audio watermark W_A. The details of the text-to-image converter 302 and the encoding module 310 may be referred to in Figure 2 The description of the text-to-image converter 202 and the encoding module 204 in the accompanying drawings provides sufficient teachings, suggestions and implementation examples, and the details are not repeated here one by one.

[0065] The original audio A_O may include a first channel and a second channel. For example, the original audio A_O may be stereo audio, and the first channel and the second channel may be the left channel and the right channel of the stereo audio, respectively. However, the present disclosure is not limited thereto. The channel selection module 304 may compare the first channel energy of the first channel with the second channel energy of the second channel. The channel selection module 304 may be configured to select a channel with a larger energy for embedding an audio watermark W_A. For example, in response to the first channel energy being greater than the second channel energy, the channel selection module 304 may be configured to determine the first channel for embedding the audio watermark W_A into the first channel to generate the watermarked audio A_W. On the other hand, in response to the first channel energy being not greater than the second channel energy, the channel selection module 304 may be configured to determine the second channel for embedding the audio watermark W_A into the second channel to generate the watermarked audio A_W.

[0066] In addition, the first channel and the second channel of the original audio A_O may include a plurality of first frames and a plurality of second frames, respectively. The mixing module 306 may mix the first channel and the second channel of the original audio A_O together. Then, the discrete cosine transform (DCT) module 308 may convert the mixed channel of the first channel and the second channel from the time domain to the frequency domain to generate preprocessed audio. That is, the original audio A_O may be converted from the time domain to the frequency domain based on the DCT algorithm to generate preprocessed audio. The preprocessed audio may be input into the encoding module 310 for embedding the audio watermark W_A into the preprocessed audio to generate encoded audio.

[0067] In one embodiment, the encoding module 310 may be configured to detect the frame with the maximum energy for determining the encoding timing. For example, the encoding module 310 may be configured to detect multiple first frame energies of the multiple first frames and multiple second frame energies of the multiple second frames. In addition, the encoding module 310 may be configured to determine the frame with the maximum energy among the multiple first frames and the multiple second frames as the maximum energy frame. The timing of the maximum energy frame can be determined as the encoding timing. In other words, the encoding timing can be determined based on the maximum energy frame. According to the encoding timing, the audio watermark W_A can be embedded into the preprocessed audio. In other words, the audio watermark W_A can be embedded at a specific timing where the preprocessed audio has the maximum energy, thereby increasing the inaudibility of the digital watermark due to the masking effect.

[0068] In one embodiment, assuming that the channel selection module 304 selects the first channel for embedding the audio watermark W_A, and the maximum energy frame also belongs to the first channel, the encoding module 310 can be configured to embed the audio watermark W_A into the first channel at the encoding time according to the maximum energy frame.

[0069] It should be noted that, although the maximum energy frame also belongs to the second channel, the encoding module 310 can still be configured to embed the audio watermark W_A into the first channel at the encoding time according to the maximum energy frame. That is, although the maximum energy may appear in the second channel, the audio watermark W_A can still be embedded in the first channel, because the maximum energy of the second channel will mask the energy of the audio watermark W_A of the first channel.

[0070] After embedding the audio watermark W_A into the preprocessed audio, an encoded audio may be generated. An inverse discrete cosine transform (IDCT) module 312 may be configured to convert the encoded audio from the frequency domain back to the time domain to generate the watermarked audio A_W. That is, the encoded audio may be converted from the frequency domain to the time domain based on the IDCT algorithm to generate the watermarked audio A_W. In addition, after the encoded audio is converted from the frequency domain to the time domain, a split module 314 may separate the first channel from the second channel to generate the watermarked audio A_W. In addition, a cross-fader 316 may be configured to implement a smoothing process to reduce the impact of the audio watermark W_A when it is heard. In this way, the watermarked audio A_W may be generated while improving the inaudibility of the audio watermark W_A.

[0071] It should be noted that the encoding module 310 can be configured to use different algorithms to embed the digital watermark into the audio. In one embodiment, the encoding module 310 may include a quantization index modulation (QIM) module 310-1 and a singular value decomposition (SVD) module 310-2, but the present disclosure is not limited thereto.

[0072] The QIM module 310-1 may be configured to implement three steps for embedding the audio watermark W_A into the preprocessed audio. The three steps may include: checking frame energy, setting strength, and embedding. In the step of checking frame energy, the QIM module 310-1 may be configured to check the frame energy of each frame of the preprocessed audio. For example, when the channel selection module 304 selects the first channel for embedding the audio watermark W_A, the QIM module 310-1 may be configured to check the frame energy of each frame of the first channel. In the step of setting strength, the QIM module 310-1 may be configured to determine the encoding energy of the audio watermark W_A (i.e., the image watermark W_I in audio format) based on the frame energy. For example, the encoding energy may be less than the frame energy, thereby increasing the inaudibility of the digital watermark due to the masking effect. In the embedding step, the QIM module 310-1 may be configured to embed the audio watermark W_A (ie, the image watermark W_I in audio format) into the preprocessed audio according to the encoding energy based on the QIM algorithm to generate the encoded audio.

[0073] Specifically, the QIM module 310-1 may be configured to quantize the preprocessed audio by rounding the value of the preprocessed audio to a finite number of levels. In addition, the QIM module 310-1 may be configured to associate each bit of the audio watermark W_A with a quantization level. For example, the first bit of the audio watermark W_A may be associated with a first quantization level, the second bit of the audio watermark W_A may be associated with a second quantization level, and so on. In addition, the QIM module 310-1 may be configured to modify the preprocessed audio according to the audio watermark W_A by changing the quantization level of each bit of the preprocessed audio based on the quantization level of each bit of the audio watermark W_A. In this way, the audio watermark W_A may be embedded into the preprocessed audio based on the QIM algorithm to produce encoded audio.

[0074] Similarly, the SVD module 310-2 can be configured to implement three steps for embedding the audio watermark W_A into the pre-processed audio. The three steps may include: checking frame energy, setting strength, and embedding. The details of the step of checking frame energy and the step of setting strength can refer to the description of the QIM module 310-1 to obtain sufficient teachings, suggestions, and implementation examples, and the details will not be repeated one by one here.

[0075] In the embedding step, the SVD module 310-2 may be configured to embed the audio watermark W_A (ie, the image watermark W_I in audio format) into the preprocessed audio according to the encoding energy based on the SVD algorithm to generate the encoded audio.

[0076] Specifically, the SVD module 310-2 may be configured to perform singular value decomposition on the preprocessed audio to obtain singular values ​​of the preprocessed audio. For example, singular value decomposition may be performed by transforming the preprocessed audio into three matrices, and one of the three matrices having eigenvalues ​​on the diagonal may be an eigenvalue matrix. In addition, the SVD module 310-2 may be configured to associate each bit of the audio watermark W_A with a singular value. For example, the first bit of the audio watermark W_A may be associated with a first singular value, the second bit of the audio watermark W_A may be associated with a second singular value, and so on. In addition, the SVD module 310-2 may be configured to modify the preprocessed audio according to the audio watermark W_A by changing each eigenvalue of the preprocessed audio based on each singular value of each bit of the audio watermark W_A. In this way, the audio watermark W_A may be embedded in the preprocessed audio based on the SVD algorithm to generate encoded audio.

[0077] In one embodiment, the audio watermark W_A (i.e., the image watermark W_I in audio format) can be configured to be embedded into a first frame of preprocessed audio based on a QIM algorithm to generate a first embedded frame of encoded audio, and embedded into a second frame of preprocessed audio based on an SVD algorithm to generate a second embedded frame of encoded audio. That is, the digital watermark is not embedded into the audio using just one algorithm. The digital watermark is embedded into the audio using both the QIM algorithm and the SVD algorithm. In other words, the digital watermark is embedded into the audio according to two different algorithms, thereby improving the robustness of the digital watermark in the watermarked audio A_W.

[0078] In one embodiment, the encoding module 310 may be configured to repeatedly and alternately embed a digital watermark into the preprocessed audio based on the QIM algorithm and the SVD algorithm. For example, the first frame of the preprocessed audio may be embedded with a digital watermark based on the QIM algorithm. In addition, the second frame of the preprocessed audio after the first frame may be embedded with a digital watermark based on the SVD algorithm. In addition, the third frame of the preprocessed audio after the second frame may be embedded with a digital watermark based on the QIM algorithm. In addition, the fourth frame of the preprocessed audio after the third frame may be embedded with a digital watermark based on the SVD algorithm. For ease of explanation, the first frame, the third frame, etc. may be referred to as or may belong to multiple first frames, and the second frame, the fourth frame, etc. may be referred to as or may belong to multiple second frames. That is, the audio watermark W_A (i.e., the image watermark W_I in audio format) can be configured to be embedded into the plurality of first frames of the preprocessed audio based on the QIM algorithm to generate a plurality of first embedded frames of the encoded audio, and to be embedded into the plurality of second frames of the preprocessed audio based on the SVD algorithm to generate a plurality of second embedded frames of the encoded audio. In this way, the digital watermark in the watermarked audio A_W can be repeatedly and alternately set, thereby improving the robustness of the watermarked audio A_W.

[0079] Figure 4 is a schematic flow chart of a decoding scenario according to an embodiment of the present disclosure. Figures 1A to 4 , decoding scenario 400 illustrates how a digital watermark may be extracted from audio. Similar to encoding scenario 300, decoding scenario 400 may include multiple modules to implement different functions of decoding. Although the functions of decoding scenario 400 are illustrated and described as being implemented by different modules for ease of explanation, it should be noted that the functions of decoding scenario 400 may be implemented by a single module. That is, the multiple modules in decoding scenario 400 may be integrated together as a single module.

[0080] First, a text watermark W_T and a watermarked audio A_W may be obtained. For example, the text watermark W_T and the watermarked audio A_W may be provided by a user. The user may be the creator or owner of the original audio A_O, but the present disclosure is not limited thereto.

[0081] The text-to-image converter 402 may convert the text watermark W_T into an image watermark W_I. In addition, the image watermark W_I may be input into a normalized cross correlation module 414. The details of the text-to-image converter 402 may be referred to in Figure 2 The description of the text-to-image converter 202 in FIG. 1 provides sufficient teachings, suggestions and implementation examples, and the details are not repeated here one by one.

[0082] The watermarked audio A_W may include a first channel and a second channel. For example, the watermarked audio A_W may be stereo audio, and the first channel and the second channel may be the left channel and the right channel of the stereo audio, respectively. However, the present disclosure is not limited thereto. The channel selection module 404 may compare the first channel energy of the first channel with the second channel energy of the second channel. The channel selection module 404 may be configured to select a channel with a larger energy for extracting a target audio watermark. For example, in response to the first channel energy of the first channel of the watermarked audio A_W being greater than the second channel energy of the second channel of the watermarked audio A_W, the channel selection module 304 may be configured to determine the first channel for extracting the target audio watermark from the watermarked audio A_W. On the other hand, in response to the first channel energy being not greater than the second channel energy, the channel selection module 404 may be configured to determine the second channel for extracting the target audio watermark from the watermarked audio A_W. For example, the unselected channel may be a silent channel or a relatively quiet channel.

[0083] In addition, the first channel and the second channel of the watermarked audio A_W may include a plurality of first frames and a plurality of second frames, respectively. The mixing module 406 may mix the first channel and the second channel of the watermarked audio A_W together. Then, the discrete cosine transform (DCT) module 408 may convert the mixed channel of the first channel and the second channel from the time domain to the frequency domain to generate the target audio. That is, the watermarked audio A_W may be converted from the time domain to the frequency domain based on the DCT algorithm to generate the target audio. The target audio may be input into the decoding module 410 for extracting the target audio watermark from the watermarked audio A_W.

[0084] In one embodiment, the decoding module 410 may be configured to detect the frame with the maximum energy for determining the decoding timing. For example, the decoding module 410 may be configured to detect the multiple first frame energies of the multiple first frames and the multiple second frame energies of the multiple second frames. In addition, the decoding module 410 may be configured to determine the frame with the maximum energy among the multiple first frames and the multiple second frames as the maximum energy frame. The timing of the maximum energy frame can be determined as the decoding timing. In other words, the decoding timing can be determined based on the maximum energy frame. Based on the decoding timing, the target audio watermark can be extracted from the target audio. In other words, the target audio watermark can be extracted at a specific timing where the target audio has the maximum energy, because the audio watermark W_A is designed to be embedded using a masking effect.

[0085] In one embodiment, assuming that the channel selection module 404 selects the first channel for extracting the target audio watermark, and the maximum energy frame also belongs to the first channel, the decoding module 410 may be configured to extract the target audio watermark from the first channel at the decoding time according to the maximum energy frame. In another embodiment, assuming that the channel selection module 404 selects the first channel for extracting the target audio watermark, the decoding module 410 may be configured to extract the target audio watermark from the mixed channel of the first channel and the second channel mixed by the mixing module 406. That is, the decoding module 410 may extract the target audio watermark from the mixed channel instead of the first channel. However, the present disclosure is not limited to this.

[0086] It should be noted that, although the maximum energy frame also belongs to the second channel, the decoding module 410 can still be configured to extract the target audio watermark from the first channel at the decoding time according to the maximum energy frame. That is, although the maximum energy may appear in the second channel, the audio watermark W_A can still be embedded in the first channel, because the maximum energy of the second channel will mask the energy of the audio watermark W_A of the first channel.

[0087] After extracting the target audio watermark from the target audio, a decoded audio may be generated.The image extractor 412 may be configured to extract an image E_I from the target audio watermark by converting the target audio watermark from an audio format to an image format.

[0088] The normalized cross-correlation module 414 may be configured to compare the extracted image E_I with the image watermark W_I to determine the similarity between the extracted image E_I and the image watermark W_I. Based on the similarity, the normalized cross-correlation module 414 may be configured to output a verification result RST. For example, in response to the similarity being greater than a predetermined threshold, the verification result RST may indicate that the extracted image E_I is similar to the image watermark W_I. On the other hand, in response to the similarity being not greater than a predetermined threshold, the verification result RST may indicate that the extracted image E_I is not similar to the image watermark W_I. In this way, unauthorized distribution of audio may be discovered, thereby improving the protection of intellectual property rights of the audio.

[0089] It should be noted that the decoding module 410 can be configured to extract a digital watermark from the audio using different algorithms. In one embodiment, the decoding module 410 may include a quantization index modulation (QIM) module 410-1 and a singular value decomposition (SVD) module 410-2, but the present disclosure is not limited thereto. For example, the QIM module 410-1 can be configured to extract an extracted image E_I from a first frame of the target audio based on a quantization index modulation algorithm to generate a first extracted image. In addition, the SVD module 410-2 can be configured to extract an extracted image E_I from a second frame of the target audio based on a quantization index modulation algorithm to generate a second extracted image.

[0090] Similar to the encoding module 310, the QIM module 410-1 and the SVD module 410-2 can be respectively configured to implement three steps for extracting the target audio watermark from the target audio. The three steps may include: checking frame energy, setting strength, and extracting. The details of the step of checking frame energy and the step of setting strength can refer to the description of the QIM module 310-1 to obtain sufficient teachings, suggestions and implementation examples, and the details are not repeated one by one here. After the step of setting strength, the decoding energy can be determined according to the target audio.

[0091] In the extraction step, the decoding module 410 (e.g., the QIM module 410-1 or the SVD module 410-2) can be configured to extract the target audio watermark (i.e., the extracted image E_I in the audio format) from the target audio according to the decoding energy based on the QIM algorithm or the SVD algorithm to generate the decoded audio. In this way, the target audio watermark can be extracted.

[0092] Figure 5 is a schematic flow chart of an encoding method according to an embodiment of the present disclosure. The encoding method for embedding a watermark into audio may include steps S510, S520, S530, S540, and S550.

[0093] In step S510, a text watermark W_T and an original audio A_O may be obtained. In step S520, the text watermark W_T may be converted into an image watermark W_I. In step S530, the original audio A_O may be converted from the time domain to the frequency domain to generate a preprocessed audio. In step S540, the image watermark W_I may be embedded into the preprocessed audio to generate an encoded audio. In step S550, the encoded audio may be converted from the frequency domain to the time domain to generate a watermarked audio A_W.

[0094] In addition, the implementation details of the encoding method can be referred to Figures 1A to 3 The present invention provides sufficient instructions, suggestions and implementation examples, and the details will not be repeated here one by one.

[0095] In this way, it is possible to hide a digital watermark in the audio, which helps to find out the creator or owner of the audio.

[0096] Figure 6 is a schematic flow chart of a decoding method according to an embodiment of the present disclosure. The decoding method for verifying a watermark in an audio may include steps S610, S620, S630, S640, and S650.

[0097] In step S610, a text watermark W_T and an original audio A_O may be obtained. In step S620, the text watermark W_T may be converted into an image watermark W_I. In step S630, the watermarked audio may be converted from the time domain to the frequency domain to generate a target audio. In step S640, an extracted image E_I may be extracted from the target audio. In step S650, the extracted image E_I may be compared with the image watermark W_I to generate a verification result RST.

[0098] In addition, the implementation details of the decoding method can be referred to Figures 1A to 4 The present invention provides sufficient instructions, suggestions and implementation examples, and the details will not be repeated here one by one.

[0099] In this way, unauthorized distribution of audio can be discovered, thereby improving the protection of the intellectual property rights of the audio.

[0100] In summary, according to the encoding method and the decoding method, it is possible to hide a digital watermark in audio, which helps to find the creator or owner of the audio. In addition, the digital watermark can be embedded according to the energy of the audio, thereby improving the inaudibility of the digital watermark due to the masking effect. In addition, different algorithms can be used to embed the digital watermark in the audio, thereby improving the robustness of the digital watermark in the audio.

[0101] It will be apparent to those skilled in the art that various modifications and variations may be made to the disclosed embodiments without departing from the scope or spirit of the present disclosure. In summary, the present disclosure is intended to cover various modifications and variations as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A coding method for embedding a watermark into audio, characterized in that include: Get text watermark and original audio; Converting the text watermark into an image watermark; Converting the raw audio from the time domain to the frequency domain to produce preprocessed audio; embedding the image watermark into the preprocessed audio to produce encoded audio; as well as The encoded audio is converted from the frequency domain to the time domain to produce watermarked audio.

2. The encoding method according to claim 1, characterized in that Also includes: Based on a text-to-image database, the text watermark is converted into the image watermark.

3. The encoding method according to claim 1, characterized in that Also includes: Based on a discrete cosine transform algorithm, convert the original audio from the time domain to the frequency domain to generate the preprocessed audio; as well as The encoded audio is converted from the frequency domain to the time domain based on an inverse discrete cosine transform algorithm to generate the watermarked audio.

4. The encoding method according to claim 1, characterized in that Also includes: checking a frame energy of each frame of the preprocessed audio; Determining a coding energy of the image watermark based on the frame energy; as well as The image watermark is embedded into the preprocessed audio according to the encoding energy to generate the encoded audio.

5. The encoding method according to claim 1, characterized in that: Also includes: The image watermark is embedded into the preprocessed audio based on a quantization index modulation algorithm to generate the encoded audio.

6. The encoding method according to claim 1, characterized in that: Also includes: The image watermark is embedded into the preprocessed audio based on a singular value decomposition algorithm to generate the encoded audio.

7. The encoding method according to claim 1, characterized in that: Also includes: Embedding the image watermark into a first frame of the preprocessed audio based on a quantization index modulation algorithm to generate a first embedded frame of the encoded audio; as well as The image watermark is embedded into a second frame of the preprocessed audio based on a singular value decomposition algorithm to generate a second embedded frame of the encoded audio.

8. The encoding method according to claim 1, characterized in that: Also includes: Embedding the image watermark into a plurality of first frames of the preprocessed audio based on a quantization index modulation algorithm to generate a plurality of first embedded frames of the encoded audio; as well as embedding the image watermark into a plurality of second frames of the preprocessed audio based on a singular value decomposition algorithm to generate a plurality of second embedded frames of the encoded audio, The plurality of first frames and the plurality of second frames are alternately arranged in the preprocessed audio.

9. The encoding method according to claim 1, characterized in that: The original audio includes a first channel and a second channel, and the encoding method further includes: comparing a first channel energy of the first channel with a second channel energy of the second channel; In response to the first channel energy being greater than the second channel energy, embedding the image watermark into the first channel to produce the watermarked audio; and In response to the first channel energy being not greater than the second channel energy, embedding the image watermark into the second channel to produce the watermarked audio.

10. The encoding method according to claim 9, characterized in that: The first channel includes a plurality of first frames and the second channel includes a plurality of second frames, and the encoding method further includes: mixing the first channel with the second channel to produce the preprocessed audio; detecting a plurality of first frame energies of the plurality of first frames and a plurality of second frame energies of the plurality of second frames; determining a maximum energy frame having maximum energy among the plurality of first frames and the plurality of second frames; Determining a coding timing according to the timing of the maximum energy frame; embedding the image watermark into the preprocessed audio according to the encoding timing to generate the encoded audio; and After converting the encoded audio from the frequency domain to the time domain, the first channel is separated from the second channel to produce the watermarked audio.

11. A decoding method for verifying a watermark in audio, characterized in that: include: Get text watermarks and watermarked audio; Converting the text watermark into an image watermark; Converting the watermarked audio from the time domain to the frequency domain to generate a target audio; extracting an image from the target audio; as well as The extracted image is compared with the image watermark to generate a verification result.