A method and device for processing audio watermark

Abstract

The present invention discloses a method and device for processing audio watermarks, wherein the method includes obtaining a first audio stream after collecting an audio signal and performing a first encoding; detecting the frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; and after performing a second encoding on data of different frame types, using different watermark embedding rules to respectively embed watermarks in the inactive frame and the active frame. By using different methods to embed watermarks in the inactive frame and the active frame, respectively, a larger data embedding capacity can be obtained, which overcomes the problem in the related art that the embedding capacity cannot be well balanced with imperceptibility and robustness.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to a method and device for processing an audio watermark. Background Art

[0002] In the digital age, with the widespread dissemination and use of audio content, the importance of audio watermarking technology has become increasingly prominent. As one of the key technologies in the field of digital rights management (DRM) and information security, audio watermark embedding technology allows information to be secretly embedded in audio signals to achieve purposes such as copyright identification, content authentication, and tampering detection. In recent years, with the enhancement of computing power and the advancement of algorithms, the field of audio watermarking has achieved significant development, but it is also facing ever-changing challenges and demands.

[0003] Existing research focuses on improving the imperceptibility of watermarks, enhancing robustness, and increasing embedding capacity. Imperceptibility ensures that the embedding of watermarks will not have a significant impact on the audio quality, maintaining the clarity and aesthetic value of the original audio. Robustness means that the watermark can resist various destructive processing, such as signal compression, noise addition, digital-to-analog conversion, etc., to ensure that the watermark information can be reliably extracted in a wide range of application scenarios. Embedding capacity involves the amount of information that can be hidden in the audio signal, which directly affects the application scope and efficiency of watermarking technology.

[0004] Despite the progress, audio watermarking technology still faces some inherent challenges. For example, there is often a trade-off between high watermark capacity and imperceptibility and robustness. In addition, real-world application scenarios require watermarking technology to be able to adapt to changing environments and attacks, including resisting malicious tampering, adapting to different transmission media and formats, etc. Summary of the invention

[0005] The present application provides an audio watermark processing method and device to solve the problems existing in the related art.

[0006] In a first aspect, the present invention provides an audio watermark processing method, comprising obtaining a first audio stream after collecting an audio signal and performing a first encoding; detecting a frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; and after performing a second encoding on data of different frame types, using different watermark embedding rules to respectively embed watermarks in the inactive frames and the active frames.

[0007] Optionally, the method further includes: acquiring and decoding the audio embedded with the watermark; performing frame type detection on the decoded audio stream; and extracting the watermark information using different extraction rules for different frame types.

[0008] Optionally, detecting the frame type in the first audio stream includes detecting inactive frames based on the autocorrelation coefficient of the FIR filter, wherein the calculation of the autocorrelation coefficient includes: adding a Hamming window to the frame sequence to calculate a new subframe sequence; calculating the autocorrelation coefficient for the new subframe sequence; adjusting each autocorrelation coefficient; calculating the residual energy of the frame based on the adjusted autocorrelation coefficient; comparing the residual energy with a preset threshold, and determining the frame type based on the comparison result.

[0009] Optionally, adding a Hamming window to the frame sequence to calculate a new subframe sequence includes: adding a Hamming window to the frame sequence to calculate a new subframe sequence includes: , where n=0,...179;

[0010] Calculating the autocorrelation coefficient for the new subframe sequence includes:

[0011] , where n=0,...,10 is the number of autocorrelation coefficients.

[0012] Optionally, adjusting each autocorrelation coefficient includes: adding and adjusting the first coefficient White noise: ; Adjust the other coefficients using a binomial window: .

[0013] Optionally, when embedding the inactive frame, the method includes: performing frame segmentation on the data of the inactive frame type after the second encoding after DC component filtering to obtain multiple sub-frames; performing preset processing on the multiple sub-frames to obtain audio parameters; and filtering out target audio parameters from the audio parameters to embed into the inactive frame.

[0014] Optionally, the multiple subframes are subjected to preset processing to obtain audio parameters, including: determining an open-loop fundamental pitch period based on every two subframes; constructing a harmonic noise shaping filter based on the open-loop fundamental pitch period, and then constructing a closed-loop fundamental pitch predictor based on the harmonic noise shaping filter; and determining audio parameters based on the open-loop fundamental pitch period and the closed-loop fundamental pitch predictor.

[0015] Optionally, filtering out target audio parameters from the audio parameters to embed into the inactive frame includes:

[0016] The difference in signal-to-noise ratio is determined based on the audio signal and the audio signal after the watermark is embedded, and a signal-to-noise ratio distance is obtained; and the audio parameters are classified into levels based on the signal-to-noise ratio distance, wherein different levels represent imperceptible levels of the watermark.

[0017] In a second aspect, the present invention provides an audio watermark processing device, comprising: an encoding unit, configured to obtain a first audio stream after collecting an audio signal and performing a first encoding; a frame type detection unit, configured to detect the frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; a watermark embedding unit, configured to perform a second encoding on data of different frame types, and then use different watermark embedding rules to respectively embed watermarks into the inactive frames and the active frames.

[0018] In a third aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0019] In a fourth aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided in the first aspect when executing the program.

[0020] The present invention discloses a method and device for processing audio watermarks, wherein the method includes obtaining a first audio stream after collecting an audio signal and performing a first encoding; detecting the frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; and after performing a second encoding on data of different frame types, using different watermark embedding rules to respectively embed watermarks in the inactive frame and the active frame. By embedding watermarks for different types of frames, a larger data embedding capacity can be obtained, overcoming the problem in the related art that the embedding capacity cannot be well balanced with imperceptibility and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A flow chart of a method for processing audio watermarks for this application;

[0023] Figure 2 This is a schematic diagram of the application flow of the audio watermark processing method of the present application including the watermark embedding and watermark extraction processes;

[0024] Figure 3 A method corresponding to the present application Figure 1 Schematic diagram of an electronic device. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0026] The following is combined with Figure 1 The following is an exemplary description of the method for processing audio watermarks. The method includes the following steps:

[0027] Step 101: After collecting an audio signal and performing a first encoding, a first audio stream is obtained.

[0028] In this embodiment, reference Figure 2 The steganographic technology in the inactive frame of the VoIP stream encoded by the source codec is illustrated. The watermark can be embedded in the audio signal to be sent at the sender end. When embedding, the audio signal to be sent is collected and encoded. The encoding method can be PCM format encoding, through which it can be encoded into an audio stream in PCM format.

[0029] Step 102: Detect the frame type in the first audio stream, where the frame type includes an active frame and an inactive frame.

[0030] In this embodiment, reference Figure 2 , the VAD algorithm can be used to detect inactive speech in the stream. If the current frame is inactive speech, the frame is marked as an inactive frame, which can be represented by a unique identifier, such as S; otherwise, it is marked as an active frame, represented by a unique identifier, such as A. After detection, the audio stream can be divided into a series of frames.

[0031] Step 103: After performing the second encoding on the data of different frame types, different watermark embedding rules are used to respectively embed watermarks on the inactive frames and the active frames.

[0032] In this embodiment, reference Figure 2 ,According to the frame type, two different watermarking algorithms are used to embed the secret information.,For example, Algorithm 1 is used to embed information in inactive frames, and Algorithm 2 is used for watermarking in active frames.

[0033] As an optional implementation method of this embodiment, detecting the frame type in the first audio stream includes detecting inactive frames based on the autocorrelation coefficient of the FIR filter, wherein the calculation of the autocorrelation coefficient includes: adding a Hamming window to the frame sequence to calculate a new subframe sequence; calculating the autocorrelation coefficient for the new subframe sequence; adjusting each autocorrelation coefficient; calculating the residual energy of the frame based on the adjusted autocorrelation coefficient; comparing the residual energy with a preset threshold, and determining the frame type based on the comparison result.

[0034] As an optional implementation of this embodiment, adding a Hamming window to a frame sequence to calculate a new subframe sequence includes: adding a Hamming window to a frame sequence to calculate a new subframe sequence includes: , where n=0,...179; calculating the autocorrelation coefficient for the new subframe sequence includes:

[0035] , where n=0,...,10 is the number of autocorrelation coefficients.

[0036] In the above optional implementation, the Hangover algorithm is usually used for voice activity detection in the speech coding process. In order to synchronize the embedding and extraction in watermarking, it is very important to keep the VAD results consistent between the sender and the receiver, because inconsistent VAD results will cause errors in the extraction process. Some factors, such as packet loss, watermarks, etc., may affect the VAD results. Therefore, an improved VAD algorithm, called the residual energy method, is proposed below to detect inactive audio frames considering packet loss. The residual energy method uses the autocorrelation coefficient that is not affected by the codec state to detect inactive speech in speech.

[0037] The coefficients of the FIR filter are calculated as follows:

[0038] ,in Represents the autocorrelation coefficient of audio frame i, and calculates frame i

[0039]

[0040] When a frame contains 4 subframes and each subframe has 11 autocorrelation coefficients, all the autocorrelation coefficients of this frame can be expressed as .

[0041] In order to calculate the coefficients of the first subframe, it is necessary to obtain data from three consecutive subframes. th, th, The three subframes will form a sequence of 180 samples. .when =0, This subframe belongs to the previous frame. If the predecessor of the current frame is lost, the calculation of the autocorrelation coefficient of the current frame will be wrong. This is because the Hangover algorithm has memory loss or delayed packets that will cause error propagation.

[0042] In order to solve this problem, this embodiment proposes an improved stateless algorithm to calculate the autocorrelation coefficient : First, define a new subframe sequence , which is calculated by adding a Hamming window to the frame sequence and is expressed as:

[0043] , n=0,...179;

[0044] In this case, the calculation method of the autocorrelation coefficient can be expressed as

[0045] ,

[0046] where n = 0,…,10, which is the number of autocorrelation coefficients.

[0047] As an optional implementation of this embodiment, adjusting each autocorrelation coefficient includes: adding the adjustment of the first coefficient White noise: ; Adjust the other coefficients using a binomial window: .

[0048] In this optional implementation, a white noise is added to adjust the first coefficient , the calculation method is: . The binomial window is then used to adjust the other coefficients using the following equations:

[0049] This indicates that the 180 samples required for the calculation are located between two consecutive frames. Due to the short-term stationarity of speech, the samples in the previous frame can be replaced by Therefore, even if the previous frame is lost, it will not affect the calculation result of the autocorrelation coefficient of the current frame.

[0050] Further, once all the autocorrelation coefficients of the four subframes of a frame are obtained, the residual energy of the frame can be calculated by the Yule-Walker equation:

[0051]

[0052] in represents the adaptive coefficient, is the LPC (linear predictive coding) coefficient. The calculation method of the Yule-Walker equation is as follows:

[0053] initialization:

[0054] Calculate k:

[0055]

[0056] calculate :

[0057]

[0058] Calculate E:

[0059]

[0060] Among them hour, Repeat the second step above to calculate k until The final remaining energy is .

[0061] A new active speech detection method is then used, where the threshold is compared to the residual energy of the frame instead of the energy of the frame:

[0062]

[0063] As an optional implementation method of this embodiment, when embedding the inactive frame, the method includes: performing frame segmentation on the data of the inactive frame type after the second encoding after DC component filtering to obtain multiple sub-frames; performing preset processing on the multiple sub-frames to obtain audio parameters; and filtering out target audio parameters from the audio parameters to embed into the inactive frame.

[0064] In this optional implementation, each frame is filtered through a high-pass filter to remove the DC component and then divided into four subframes of 60 samples each. A tenth-order linear predictive coding (LPC) filter is calculated using the unprocessed input signal of each subframe, and the last subframe is quantized using a predictive split vector quantizer. For every two subframes (120 samples), the weighted speech signal is used to calculate the open-loop pitch period. A harmonic noise shaping filter is then constructed using the previously calculated open-loop pitch period, and a closed-loop pitch predictor is constructed based on the impulse response created by the noise shaping filter. Finally, both the pitch period and the difference value are transmitted to the decoder and approximate the non-periodic component of the excitation. After completing these operations, all speech parameters such as LPC, pulse symbol (Pamp) and pulse position (Ppos) are obtained.

[0065] As an optional implementation method of this embodiment, filtering out target audio parameters from the audio parameters to embed into the inactive frame includes: determining a signal-to-noise ratio difference based on the audio signal and the audio signal after the watermark is embedded to obtain a signal-to-noise ratio distance; and classifying the audio parameters based on the signal-to-noise ratio distance, wherein different levels represent imperceptible levels of the watermark.

[0066] In this optional implementation, it is determined which speech parameters of the inactive frames are suitable for data embedding. All speech parameters are classified into three imperceptible levels of watermarking based on the distance of signal-to-noise ratio (DSNR), which is defined as the difference in signal-to-noise ratio (SNR) between the original speech and the watermarked speech. Different parameters of the inactive frames have a very different imperceptible level of watermarking. Therefore, different parameters and various parameter bits can be selected to embed data according to the needs of the actual application.

[0067] This embodiment is aimed at the watermark algorithm under different speech parameters of inactive frames. Performance evaluation shows that embedding data in different speech parameters leads to different degrees of hiding.

[0068] The embedding process of this embodiment includes speech activity detection: the speech in PCM format is divided into frames, and each frame is passed to the VAD detector using the above-mentioned residual energy algorithm. Active frames are marked with "A" and inactive frames are marked with "S". All frames are encoded by the codec, regardless of the category of the frame, to obtain a low-bit rate audio stream. Information is embedded in the frame, and the inactive frame is embedded using the preferred solution of this embodiment, and then encapsulated and sent. The inactive frames and active frames with hidden information are encapsulated in VoIP data packets and transmitted over the Internet.

[0069] The high capacity watermarking algorithm is used to embed data in inactive frames of an audio stream encoded by a source codec. Under the same imperceptibility, the watermark in the inactive audio frame achieves a larger data embedding capacity than that in the active audio frame.

[0070] As an optional implementation of this embodiment, the method further includes acquiring and decoding the audio embedded with the watermark; performing frame type detection on the decoded audio stream; and extracting the watermark information using different extraction rules for different frame types.

[0071] In this optional implementation, after receiving the audio embedded with the watermark, the receiver extracts the secret information from it. Exemplarily, the receiver receives and decapsulates the VoIP data packet; decodes and performs active frame detection; decodes the decapsulated data into an audio stream in PCM format, and uses an improved VAD algorithm to detect active frames and inactive frames. The secret information is extracted, and the method of the present invention is used for inactive frames.

[0072] The above is a method provided for one or more embodiments of the present application. Based on the same idea, the present application also provides a corresponding audio watermark processing device, including an encoding unit, configured to obtain a first audio stream after collecting an audio signal and performing a first encoding; a frame type detection unit, configured to detect the frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; a watermark embedding unit, configured to perform a second encoding on data of different frame types, and then use different watermark embedding rules to respectively embed watermarks in the inactive frames and the active frames.

[0073] Optionally, the device also includes an extraction unit configured as a decoder to obtain and decode the audio embedded with the watermark; a detection unit configured to perform frame type detection on the decoded audio stream; and the extraction unit uses different extraction rules for different frame types to extract watermark information.

[0074] Optionally, detecting the frame type in the first audio stream includes detecting inactive frames based on the autocorrelation coefficient of the FIR filter, wherein the calculation of the autocorrelation coefficient includes: adding a Hamming window to the frame sequence to calculate a new subframe sequence; calculating the autocorrelation coefficient for the new subframe sequence; adjusting each autocorrelation coefficient; calculating the residual energy of the frame based on the adjusted autocorrelation coefficient; comparing the residual energy with a preset threshold, and determining the frame type based on the comparison result.

[0075] As an optional implementation of this embodiment, adding a Hamming window to a frame sequence to calculate a new subframe sequence includes: adding a Hamming window to a frame sequence to calculate a new subframe sequence includes: , where n=0,...179;

[0076] Calculating the autocorrelation coefficient for the new subframe sequence includes: , where n=0,...,10 is the number of autocorrelation coefficients.

[0077] As an optional implementation of this embodiment, adjusting each autocorrelation coefficient includes: adding the adjustment of the first coefficient White noise: ; Adjust the other coefficients using a binomial window: .

[0078] As an optional implementation method of this embodiment, when embedding the inactive frame, the method includes performing frame segmentation on the data of the inactive frame type after the second encoding after DC component filtering to obtain multiple sub-frames; performing preset processing on the multiple sub-frames to obtain audio parameters; and filtering out target audio parameters from the audio parameters to embed into the inactive frame.

[0079] As an optional implementation method of this embodiment, the multiple subframes are preset processed to obtain audio parameters, including: determining an open-loop fundamental pitch period based on every two subframes; constructing a harmonic noise shaping filter based on the open-loop fundamental pitch period, and then constructing a closed-loop fundamental pitch predictor based on the harmonic noise shaping filter; and determining audio parameters based on the open-loop fundamental pitch period and the closed-loop fundamental pitch predictor.

[0080] As an optional implementation method of this embodiment, filtering out target audio parameters from the audio parameters to embed into the inactive frame includes: determining a signal-to-noise ratio difference based on the audio signal and the audio signal after the watermark is embedded to obtain a signal-to-noise ratio distance; and classifying the audio parameters based on the signal-to-noise ratio distance, wherein different levels represent imperceptible levels of the watermark.

[0081] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503;

[0082] The processor 501 and the memory 502 communicate with each other via the bus 503 ; the processor 501 is used to call program instructions in the memory 502 to execute the methods provided by the above-mentioned method embodiments.

[0083] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above method embodiments.

[0084] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various storage media that can store program codes.

[0085] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0087] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for processing an audio watermark, characterized in that: include: After collecting the audio signal and performing first encoding, a first audio stream is obtained; Detecting a frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; After performing the second encoding on the data of different frame types, adopting different watermark embedding rules to respectively embed watermarks on the inactive frame and the active frame; The method for embedding the inactive frame includes: performing frame segmentation on the data of the inactive frame type after the second encoding after DC component filtering to obtain multiple subframes; performing preset processing on the multiple subframes to obtain audio parameters; and selecting target audio parameters from the audio parameters to embed into the inactive frame; The multiple subframes are processed in advance to obtain audio parameters, including: determining an open-loop pitch period based on every two subframes; constructing a harmonic noise shaping filter based on the open-loop pitch period, and then constructing a closed-loop pitch predictor based on the harmonic noise shaping filter; determining audio parameters based on the open-loop pitch period and the closed-loop pitch predictor, wherein screening out target audio parameters from the audio parameters to embed into inactive frames includes: determining a signal-to-noise ratio difference based on the audio signal and the audio signal after the watermark is embedded to obtain a signal-to-noise ratio distance; and classifying the audio parameters based on the signal-to-noise ratio distance, wherein different levels represent imperceptible levels of the watermark.

2. The method for processing audio watermark according to claim 1, characterized in that: The method further comprises: Get the audio embedded with watermark and decode it; Perform frame type detection on the decoded audio stream; Different extraction rules are used to extract watermark information for different frame types.

3. The method for processing audio watermark according to claim 1, characterized in that: Detecting the frame type in the first audio stream includes detecting an inactive frame based on an autocorrelation coefficient of a FIR filter, wherein the autocorrelation coefficient, when calculated, includes: A Hamming window is added to the frame sequence to calculate the new subframe sequence; Calculating an autocorrelation coefficient for the new subframe sequence; Adjust each autocorrelation coefficient; calculating residual energy of the frame based on the adjusted autocorrelation coefficient; The remaining energy is compared with a preset threshold, and the frame type is determined based on the comparison result.

4. The method for processing audio watermark according to claim 3, characterized in that: A Hamming window is added to the frame sequence to calculate the new sub-frame sequence including: A Hamming window is added to the frame sequence to calculate the new subframe sequence including: ThreeSubFrmH i (n) = ThreeSubFrm i (n)×HammiWindow(n), where n=0,...,179; Calculating the autocorrelation coefficient for the new subframe sequence includes: Where n=0,...,10 is the number of autocorrelation coefficients.

5. The method for processing audio watermark according to claim 4, characterized in that: Adjustments to the various autocorrelation coefficients include: Add the first coefficient R i White noise of [0]: Adjusting other coefficients with binomial window: R i [n] = R i [n]×Binormial[n], n≠0.

6. An audio watermark processing device, comprising: The encoding unit is configured to obtain a first audio stream after collecting the audio signal and performing a first encoding; a frame type detection unit, configured to detect a frame type in the first audio stream, wherein the frame type includes an active frame and an inactive frame; a watermark embedding unit configured to, after performing a second encoding on data of different frame types, respectively embed watermarks on the inactive frame and the active frame using different watermark embedding rules; The method for embedding the inactive frame includes: performing frame segmentation on the data of the inactive frame type after the second encoding after DC component filtering to obtain multiple subframes; performing preset processing on the multiple subframes to obtain audio parameters; and selecting target audio parameters from the audio parameters to embed into the inactive frame; The multiple subframes are processed in advance to obtain audio parameters, including: determining an open-loop pitch period based on every two subframes; constructing a harmonic noise shaping filter based on the open-loop pitch period, and then constructing a closed-loop pitch predictor based on the harmonic noise shaping filter; determining audio parameters based on the open-loop pitch period and the closed-loop pitch predictor, wherein screening out target audio parameters from the audio parameters to embed into inactive frames includes: determining a signal-to-noise ratio difference based on the audio signal and the audio signal after the watermark is embedded to obtain a signal-to-noise ratio distance; and classifying the audio parameters based on the signal-to-noise ratio distance, wherein different levels represent imperceptible levels of the watermark.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images