An audio watermark adding, analyzing method, device and medium

Abstract

Embodiments of the present application disclose an audio watermark adding method, comprising: a playing terminal acquiring a first audio in real time; the playing terminal embedding an audio watermark in the first audio, the audio watermark being associated with the playing terminal; and the playing terminal playing the first audio with the embedded audio watermark. Embodiments of the present application also provide an audio watermark analysis method and device and a medium, in a scenario of playing an audio in real time, a playing terminal adds an audio watermark in an audio stream in real time, so that a later device can determine the playing terminal according to the audio watermark when analyzing the watermark, and tracing is facilitated after the first audio is transcribed.

Description

Technical Field

[0001] This application relates to the field of electronics, and more particularly to an audio watermarking method, device, and medium. Background Technology

[0002] Watermarking is a commonly used image processing method. By marking important images with watermarks, the source of the image can be traced during its dissemination. This deters those who might secretly take and distribute images, thus serving as a deterrent and enabling accountability.

[0003] Currently, similar needs exist in the audio field. For example, during remote audio and video conferencing, there are instances of meeting content being leaked, leading to the disclosure of confidential business or personal information and causing adverse effects. The main method is to use devices such as mobile phones or voice recorders to secretly film or record at a remote terminal, and then transmit the secretly recorded audio and video files to other people, ultimately spreading them on the Internet and causing negative impacts.

[0004] Existing audio watermarking methods mainly rely on offline processing of the audio to obtain the watermark. However, in the context of live conferencing, these methods cannot effectively process real-time online audio streams.

[0005] Therefore, the aforementioned problems in the existing technology still need to be improved. Summary of the Invention

[0006] This application provides an audio watermarking method, device, and medium to solve the problem of real-time audio watermarking.

[0007] In view of the above, the first aspect of the present application provides an audio watermarking method, including: a playback terminal acquiring a first audio in real time; the playback terminal embedding an audio watermark in the first audio, the audio watermark being associated with the playback terminal; and the playback terminal playing the first audio with the embedded audio watermark.

[0008] In this embodiment, the playback terminal acquires the first audio in real time; the playback terminal embeds an audio watermark in the first audio, and the audio watermark is associated with the playback terminal; the playback terminal plays the first audio with the embedded audio watermark. Thus, in a scenario of real-time audio playback, the playback terminal adds an audio watermark to the audio stream in real time, enabling subsequent devices to identify the playback terminal based on the audio watermark when parsing it, facilitating source tracing after the first audio is transcribed.

[0009] Optionally, the playback terminal embedding an audio watermark in the first audio includes: the playback terminal determining a first target frame in the first audio that meets a first preset condition; the playback terminal determining a second target frame after the first target frame that meets a second preset condition, the first target frame being used to mark the second target frame; and the playback terminal embedding the audio watermark in the second target frame.

[0010] In this embodiment, during real-time processing of the first audio, the playback terminal determines a first target frame based on a first preset condition, then determines a second target frame based on a second preset condition, and finally embeds the audio watermark in the second target frame. This method enables the playback terminal to accurately locate the appropriate position for embedding the audio watermark in the first audio during real-time processing.

[0011] Optionally, the playback terminal determines a first target frame that satisfies a first preset condition in the first audio, including: when the sampling rate of the first audio is greater than or equal to a first threshold, the playback terminal determines an audio frame whose maximum low-frequency value is within a first interval as the first target frame; or, when the sampling rate of the first audio is less than the first threshold, the playback terminal determines an audio frame containing a first characteristic sound as the first target frame.

[0012] In this embodiment, when the sampling rate of the first audio is greater than or equal to the first threshold, the playback terminal determines the audio frame whose maximum low-frequency value is within the first interval as the first target frame; when the sampling rate of the first audio is less than the first threshold, the playback terminal determines the first target frame by determining the first characteristic sound, without needing to embed a synchronization frame marker. This ensures that, regardless of whether the sampling rate of the first audio is greater than or less than the first threshold, the first target frame with the watermark embedding position can be found in the first audio.

[0013] Optionally, when the sampling rate of the first audio is greater than or equal to the first threshold, after the playback terminal takes the audio frame in which the maximum value of the low frequency part is within the first interval as the first target frame, the playback terminal further includes adding a synchronization frame marker to the first target frame.

[0014] In this embodiment, when the sampling rate of the first audio is greater than or equal to the first threshold, a synchronization frame marker is added to the first target frame so that the subsequent parsing terminal can quickly locate the first target frame based on the synchronization frame marker during parsing.

[0015] Optionally, the playback terminal adds a synchronization frame marker to the first target frame, including: the playback terminal acquiring a first sampling point, the first sampling point being a sampling point in the intermediate frequency part; the playback terminal increasing the energy value of the first sampling point so that the ratio of the energy value of the first sampling point to the energy value of the low frequency part is greater than or equal to a second threshold.

[0016] In this embodiment, the playback terminal determines in real time whether the target frame in the first audio meets the first preset condition as the first target frame or the second target frame, and then increases the energy value of the first sampling point in the mid-frequency part of the first target frame or the second target frame, so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the preset ratio, thereby realizing the addition of the synchronization frame marker.

[0017] Optionally, when the sampling rate of the first audio is less than the first threshold, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame, including: when the first characteristic sound is detected and the duration of the first characteristic sound is greater than or equal to a preset time, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame.

[0018] In this embodiment, the first characteristic sound can be human voice, or when a specific statement is detected in the human voice state, the target frame where the specific statement is located is determined as the first target frame, thereby ensuring that the subsequent watermark embedding can be embedded into the target frame that records the voice information.

[0019] Optionally, the playback terminal determines a second target frame that meets the second preset condition after the first target frame, including: the playback terminal determines a target frame whose intermediate frequency energy value is greater than or equal to a third threshold and less than a fourth threshold as the second target frame.

[0020] In this embodiment, the second target frame is the target frame located after the first target frame. Since the second target frame is added to the first audio in real time, it cannot be guaranteed that every frame after the first target frame in the first audio is suitable as the second target frame. Therefore, it is necessary to judge the conditions of the second target frame. Only when the target frame meets the second preset condition will the target frame be used as the second target frame.

[0021] Optionally, the playback terminal embeds the audio watermark in the third target frame, including: the playback terminal obtaining a first sequence corresponding to the audio watermark, the first sequence including at least one element; the playback terminal obtaining at least one second sampling point from the third target frame; the playback terminal embedding at least one element in the first sequence into the at least one second sampling point respectively, wherein one element in the first sequence corresponds to one second sampling point.

[0022] In this embodiment, to embed an audio watermark in real time into the first audio file, the playback terminal searches for a second target frame that meets preset conditions after the first target frame according to the time sequence. During the embedding process, the energy ratio of the energy values ​​of the sampling points in the second target frame in different time and / or different frequency domains is changed to achieve real-time embedding of the audio watermark. The embedded watermark has strong anti-interference capabilities during audio transcription and can be propagated through digital channels or air channels.

[0023] Optionally, the playback terminal adds at least one element from the first sequence to the at least one second sampling point, including: the playback terminal adjusting the energy ratio of the energy values ​​of the second sampling point in different time domains and / or different frequency domains, wherein the energy ratio of one of the second sampling points is associated with an element in the first sequence.

[0024] In this embodiment, optionally, the playback terminal adjusts the energy ratio of the energy values ​​of the second sampling point in different time domains and / or different frequency domains in the following ways: increasing the energy value of the first half of the first sub-sampling point so that the ratio of the energy values ​​of the first half to the second half of the first sub-sampling point is greater than or equal to a fifth threshold, and recording the first sub-sampling point as 1, wherein the first sub-sampling point is one of the at least one second sampling point; increasing the energy value of the second half of the second sub-sampling point so that the ratio of the energy values ​​of the second half to the first half of the second sampling point is greater than or equal to the fifth threshold, and recording the second sub-sampling point as 0.

[0025] Subsequently, after adding at least one element from the first sequence to the at least one second sampling point, the playback terminal further includes:

[0026] When the ratio of high-energy portion to low-energy portion in the second sampling point is less than the fifth threshold, the energy value of the high-energy portion in the second sampling point is increased.

[0027] Optionally, after obtaining the first sequence corresponding to the audio watermark, the playback terminal further includes:

[0028] The playback terminal adds a check bit to the first sequence of data, which is used to verify the transmission integrity of the first sequence of data.

[0029] A second aspect of this application provides an audio watermark parsing method, comprising: a parsing terminal acquiring a first audio, wherein the first audio contains an embedded audio watermark, the audio watermark being associated with a playback terminal, the playback terminal being used to embed the audio watermark into the first audio in real time; the parsing terminal parsing the audio watermark from the first audio; and the parsing terminal determining the playback terminal based on the audio watermark.

[0030] This application provides an audio watermark parsing method, comprising: a parsing terminal acquiring a first audio file, wherein the first audio file contains an audio watermark, the audio watermark being associated with a playback terminal, and the playback terminal being used to embed the audio watermark into the first audio file in real time; the parsing terminal parsing the audio watermark from the first audio file; and the parsing terminal determining the playback terminal based on the audio watermark. Thus, the parsing terminal can determine the playback terminal to which the audio watermark is added to the first audio file based on the audio watermark.

[0031] Optionally, before the parsing terminal parses the audio watermark from the first audio, the parsing terminal further includes: determining a first target frame in the first audio that meets a first preset condition; determining a second target frame after the first target frame that meets a second preset condition; and the parsing terminal parses the audio watermark from the first audio, including: the parsing terminal parses the audio watermark from the second target frame.

[0032] In this embodiment, since the playback terminal adds the audio watermark to the first audio in real time, it cannot guarantee that every frame in the first audio meets the conditions for watermark embedding. Therefore, in the real-time audio watermark embedding scheme, it cannot be embedded according to preset rules like offline watermark embedding. Instead, the first target frame and the second target frame need to be determined according to the first preset conditions and the second preset conditions, respectively. Therefore, when parsing, the parsing terminal also needs to parse the first target frame and the second target frame according to the same conditions.

[0033] Optionally, when the sampling rate of the first audio is less than a first threshold, the parsing terminal determines a first target frame in the first audio that satisfies a first preset condition, including: the parsing terminal determines from the first audio that it contains a first characteristic sound.

[0034] The target frame whose duration of the first characteristic sound is greater than or equal to a preset time is designated as the first target frame.

[0035] In this embodiment, when the sampling rate of the first audio is greater than or equal to the first threshold, the parsing terminal determines the audio frame whose maximum low-frequency value is within the first interval as the first target frame; when the sampling rate of the first audio is less than the first threshold, the parsing terminal determines the first target frame by determining the first characteristic sound, without needing to parse the synchronization frame marker. This ensures that, regardless of whether the sampling rate of the first audio is greater than or less than the first threshold, the parsing terminal can find the first target frame with the watermark embedding position.

[0036] Optionally, when the sampling rate of the first audio is greater than or equal to a first threshold, the parsing terminal determines a first target frame in the first audio that satisfies a first preset condition; this includes: the parsing terminal acquiring a first ratio of the energy values ​​of the mid-frequency part to the low-frequency part of the first audio frame by frame; when the parsing terminal acquires an initial target frame in which the first ratio is greater than or equal to a second threshold, it slides backwards from the initial target frame to detect the first audio in a sliding window manner to acquire a second ratio of the energy values ​​of the mid-frequency part to the low-frequency part within each sliding window; the parsing terminal identifies the frame containing the sliding window with the largest second ratio as the first target frame.

[0037] In this embodiment, the parsing terminal obtains a first ratio of the energy values ​​of the mid-frequency and low-frequency sample points in each frame. When a first sample point with a first ratio greater than or equal to a second threshold is found, the target frame containing the first sample point is determined as the initial target frame. However, since a frame has 2048 sample points, and the first sample point is only a portion of them, there may be an offset between the actual target frame containing the first sample point and the initial target frame containing the current first sample point after finding a first sample point that meets the first ratio. To solve this problem, it is necessary to start from the initial target frame and move backward in a sliding window manner, detecting the first audio after the initial target frame frame by frame to obtain a second ratio of the energy values ​​of the mid-frequency and low-frequency components within each sliding window. The frame containing the sliding window with the largest second ratio is then selected as the first target frame, thereby preventing parsing deviation.

[0038] Optionally, the first target frame includes a synchronization frame marker. The parsing terminal obtains the frame containing the sliding window with the largest second ratio as the first target frame, including: the parsing terminal obtaining the first sampling point with the highest intermediate frequency energy value from the sliding window with the largest second ratio; the parsing terminal obtaining a third sampling point at a preset length before the first sampling point; the parsing terminal determining that the portion where the ratio of the energy value of the first sampling point to the energy value of the third sampling point is greater than a seventh threshold is the synchronization frame marker; and the parsing terminal determining the frame containing the sliding window with the largest second ratio as the first target frame based on the synchronization frame marker.

[0039] In this embodiment, the method for detecting the synchronization frame marker is determined by detecting a first ratio of the energy values ​​of the mid-frequency and low-frequency components of the first audio. However, in actual operation, the original content of the first audio (i.e., non-watermarked content) may also contain a situation where the ratio of the energy values ​​of the mid-frequency and low-frequency components is greater than the first ratio. This can lead to false detection of the synchronization frame marker. To address this, the parsing terminal determines the synchronization frame marker by the ratio of the first sampling point to a third sampling point of a preset length prior to the first sampling point, thereby preventing the aforementioned false detection.

[0040] Optionally, the parsing terminal determines a second target frame that meets the second preset condition after the first target frame, including: the parsing terminal moves backward frame by frame starting from the first target frame, and obtains candidate target frames whose energy in the intermediate frequency part of each frame is greater than or equal to a third threshold and less than a fourth threshold; the parsing terminal obtains from the candidate target frames a target frame whose energy ratio of different time domain and / or different frequency domain energy values ​​is greater than or equal to a fifth threshold as the second target frame.

[0041] In this embodiment, after the parsing terminal detects the first target frame, it uses the first target frame as the positioning frame to continue searching for the second target frame located after the first target frame. The second target frame has an embedded audio watermark and satisfies a second preset condition. The parsing terminal can quickly find the second target frame after the first target frame based on the second preset condition.

[0042] Optionally, the parsing terminal parses the audio watermark from the second target frame, including: the parsing terminal obtaining a second sampling point from the second target frame, the second sampling point being a sampling point in the second target frame where the energy ratio is greater than or equal to the fifth threshold; the parsing terminal obtaining energy ratios of different time domain and / or different frequency domain energy values ​​from the second sampling point; and the parsing terminal obtaining a first element associated with the energy ratio, the first element being an element in a first sequence recorded by the audio watermark.

[0043] In this embodiment, optionally, if the first sequence includes a check bit, then after generating the first sequence from the numbers of at least one second target frame in the parsing order, the method further includes: determining whether the first sequence is complete based on the check bit; if so, converting the first sequence into a decimal sequence; if not, ignoring the first sequence.

[0044] Optionally, before the parsing terminal parses the audio watermark from the second target frame, the method further includes: adjusting the length value of the first length according to the duration of the second target frame, wherein the longer the duration of the second target frame, the larger the length of the first length. The parsing terminal removes the energy values ​​of the first length from the beginning and end of the second target frame respectively.

[0045] Optionally, the method includes multiple watermark detection cycles, wherein each watermark detection cycle resolves one audio watermark, and the method further includes: determining the audio watermark with the highest repetition rate from the audio watermarks resolved by the multiple watermark detection cycles as the watermark of the first audio.

[0046] In this embodiment, the first audio includes multiple watermark parsing cycles. Each cycle includes a first target frame and a second target frame, and the same audio watermark is embedded in each watermark detection cycle. During actual parsing, the parsing terminal may encounter parsing errors, resulting in not all watermark detection cycles obtaining the same sequence of audio watermarks. When a parsing error occurs, the incorrect audio watermark obtained is always random and non-repeating. Therefore, the audio watermark with the highest repetition rate among the multiple watermark detection cycles can be determined as the correct audio watermark. This multi-cycle decision-making method accurately parses the correct watermark embedded in the first audio, further preventing misparsing by the parsing terminal.

[0047] A third aspect of this application provides a playback terminal, including:

[0048] The acquisition unit is used to acquire the first audio in real time;

[0049] An execution unit is configured to embed an audio watermark in the first audio obtained by the acquisition unit, the audio watermark being associated with the playback terminal;

[0050] A playback unit is used to play the first audio file containing the audio watermark embedded by the execution unit.

[0051] Optionally, the execution unit is also used for:

[0052] In the first audio, determine the first target frame that meets the first preset condition;

[0053] A second target frame that meets the second preset condition is determined after the first target frame, and the first target frame is used to mark the second target frame;

[0054] The audio watermark is embedded in the second target frame.

[0055] Optionally, the execution unit is also used for:

[0056] When the sampling rate of the first audio is greater than or equal to the first threshold, the audio frame whose maximum low-frequency value is within the first interval is determined as the first target frame; or,

[0057] When the sampling rate of the first audio is less than the first threshold, the audio frame containing the first characteristic sound is determined as the first target frame.

[0058] Optionally, when the sampling rate of the first audio is greater than or equal to the first threshold, the execution unit is further configured to:

[0059] Add a synchronization frame marker to the first target frame.

[0060] Optionally, the execution unit is also used for:

[0061] Obtain the first sampling point, which is the sampling point of the intermediate frequency part;

[0062] Increase the energy value of the first sampling point so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the second threshold.

[0063] Optionally, when the sampling rate of the first audio is less than the first threshold, the execution unit is further configured to:

[0064] When the first characteristic sound is detected and the duration of the first characteristic sound is greater than or equal to a preset time, the audio frame containing the first characteristic sound is determined as the first target frame.

[0065] Optionally, the execution unit is also used for:

[0066] The target frame whose intermediate frequency energy value is greater than or equal to the third threshold and less than the fourth threshold is identified as the second target frame.

[0067] Optionally, the execution unit is also used for:

[0068] Obtain the first sequence corresponding to the audio watermark, wherein the first sequence includes at least one element;

[0069] Obtain at least one second sampling point from the third target frame;

[0070] At least one element of the first sequence is embedded into the at least one second sampling point, wherein one element of the first sequence corresponds to one second sampling point.

[0071] Optionally, the execution unit is also used for:

[0072] Adjust the energy ratio of the energy values ​​of the second sampling point in different time domains and / or different frequency domains, wherein the energy ratio of one of the second sampling points is associated with an element of the first sequence.

[0073] The beneficial effects of the third aspect of the embodiments of this application can be found in the first aspect described above, and will not be repeated here.

[0074] A fourth aspect of this application provides a parsing terminal, including:

[0075] The acquisition unit is used to acquire a first audio, which contains an audio watermark. The audio watermark is associated with a playback terminal, and the playback terminal is used to embed the audio watermark into the first audio in real time.

[0076] A parsing unit is used to parse the audio watermark from the first audio obtained by the acquisition unit;

[0077] An execution unit is used to determine the playback terminal based on the audio watermark parsed by the parsing unit.

[0078] Optionally, this parsing unit is also used for:

[0079] Identify the first target frame in the first audio that satisfies the first preset condition;

[0080] A second target frame that meets the second preset condition is determined after the first target frame;

[0081] The audio watermark is parsed from the second target frame.

[0082] Optionally, when the sampling rate of the first audio is less than a first threshold, the parsing unit is further configured to:

[0083] The first target frame is determined from the first audio that contains a first characteristic sound and the duration of the first characteristic sound is greater than or equal to a preset time.

[0084] Optionally, when the sampling rate of the first audio is greater than or equal to the first threshold, the parsing unit is further configured to:

[0085] The first ratio of the energy values ​​of the mid-frequency part to the low-frequency part of the first audio is obtained frame by frame;

[0086] When an initial target frame with the first ratio greater than or equal to the second threshold is obtained, the first audio is detected by sliding a window backward from the initial target frame to obtain the second ratio of the energy values ​​of the mid-frequency part and the low-frequency part in each sliding window.

[0087] The frame containing the sliding window with the largest second ratio is taken as the first target frame.

[0088] Optionally, the first target frame includes a synchronization frame marker, and the parsing unit is further used for:

[0089] The first sampling point with the highest energy value in the intermediate frequency part is obtained from the sliding window with the largest second ratio.

[0090] Obtain the third sampling point at a preset distance from the first sampling point;

[0091] The portion of the ratio of the energy value of the first sampling point to the energy value of the third sampling point that is greater than the seventh threshold is identified as the synchronization frame marker;

[0092] The frame containing the sliding window with the largest second ratio is determined based on the synchronization frame marker as the first target frame.

[0093] Optionally, this parsing unit is also used for:

[0094] Starting from the first target frame, move backward frame by frame and obtain candidate target frames whose energy in the intermediate frequency part of each frame is greater than or equal to the third threshold and less than the fourth threshold.

[0095] The second target frame is the target frame whose energy ratio of different time domain and / or different frequency domain energy values ​​obtained from the candidate target frames is greater than or equal to the fifth threshold.

[0096] Optionally, this parsing unit is also used for:

[0097] A second sampling point is obtained from the second target frame, wherein the second sampling point is a sampling point in the second target frame where the energy ratio is greater than or equal to the fifth threshold;

[0098] Obtain the energy ratio of different time-domain and / or different frequency-domain portions of the energy values ​​at the second sampling point;

[0099] Obtain the first element associated with the energy ratio, which is an element in the first sequence recorded by the audio watermark.

[0100] Optionally, the first audio includes multiple watermark detection cycles, wherein each watermark detection cycle resolves one audio watermark, and the resolution unit is further used for:

[0101] The audio watermark with the highest repetition rate is determined from the multiple watermark detection cycles and used as the watermark for the first audio.

[0102] The beneficial effects of the fourth aspect of the embodiments of this application can be found in the second aspect above, and will not be repeated here.

[0103] A fifth aspect of this application provides an electronic device, comprising: an interactive device, an input / output (I / O) interface, a processor, and a memory, wherein the memory stores program instructions; the interactive device is used to acquire operation instructions input by a user; and the processor is used to execute the program instructions stored in the memory to perform the method described in any of the optional implementations of the first or second aspect described above.

[0104] A sixth aspect of this application provides a computer-readable storage medium including instructions that, when executed on a computer device, cause the computer device to perform the method as described in any of the optional implementations of the first or second aspect described above. Attached Figure Description

[0105] Figure 1 This is a schematic diagram illustrating a use case of the audio watermarking method provided in the embodiments of this application;

[0106] Figure 2A schematic diagram of one embodiment of the audio watermarking method provided in this application;

[0107] Figure 3 This is a schematic diagram of another embodiment of the audio watermarking method provided in this application.

[0108] Figure 4 This is a schematic diagram of another embodiment of the audio watermarking method provided in this application.

[0109] Figure 5a This is a schematic diagram of another embodiment of the audio watermarking method provided in this application.

[0110] Figure 5b This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0111] Figure 5c This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0112] Figure 5d This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0113] Figure 5e This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0114] Figure 5f This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0115] Figure 5g This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0116] Figure 5h This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0117] Figure 5i This is a schematic diagram illustrating another implementation of the audio watermarking method provided in the embodiments of this application;

[0118] Figure 6 This is a schematic diagram of an embodiment of the audio watermark parsing method provided in this application.

[0119] Figure 7 This is a schematic diagram of another embodiment of the audio watermark parsing method provided in the embodiments of this application;

[0120] Figure 8This is a schematic diagram of another embodiment of the audio watermark parsing method provided in the embodiments of this application;

[0121] Figure 9a This is a schematic diagram illustrating another implementation of the audio watermark parsing method provided in the embodiments of this application;

[0122] Figure 9b This is a schematic diagram illustrating another implementation of the audio watermark parsing method provided in the embodiments of this application;

[0123] Figure 10 This is a schematic diagram of another embodiment of the audio watermark parsing method provided in the embodiments of this application;

[0124] Figure 11 This is a schematic diagram of another embodiment of the audio watermark parsing method provided in the embodiments of this application;

[0125] Figure 12a This is a schematic diagram illustrating another implementation of the audio watermark parsing method provided in the embodiments of this application;

[0126] Figure 12b This is a schematic diagram illustrating another implementation of the audio watermark parsing method provided in the embodiments of this application;

[0127] Figure 13 This is a schematic diagram illustrating one usage scenario of an embodiment of this application;

[0128] Figure 14 This is a schematic diagram illustrating one usage scenario of an embodiment of this application;

[0129] Figure 15 A schematic diagram of an electronic device provided in an embodiment of this application;

[0130] Figure 16 A schematic diagram of the playback terminal provided in an embodiment of this application;

[0131] Figure 17 This is a schematic diagram of the parsing terminal provided in an embodiment of this application. Detailed Implementation

[0132] This invention provides an audio watermarking method, device, and medium that can solve the problem of real-time audio watermarking.

[0133] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0134] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0135] Watermarking is a commonly used image processing method. By marking important images with watermarks, the source of the image can be traced during its dissemination. This deters those who might secretly take and distribute images, thus serving as a deterrent and enabling accountability.

[0136] Currently, similar needs exist in the audio field. For example, during remote audio and video conferencing, there are instances of meeting content being leaked, leading to the disclosure of confidential business or personal information and causing adverse effects. The main method is to use devices such as mobile phones or voice recorders to secretly film or record at a remote terminal, and then transmit the secretly recorded audio and video files to other people, ultimately spreading them on the Internet and causing negative impacts.

[0137] Currently, the main method for adding audio watermarks is to process the audio offline in post-production. However, in the context of live conferencing, this method cannot effectively process online real-time audio streams.

[0138] For example, the application scenarios of the methods provided in the embodiments of this application will be described.

[0139] Please see Figure 1 ,like Figure 1As shown, the usage scenario of this application embodiment can be applied to a conference scenario, which includes meeting room A101, meeting room B102, meeting room C103, and a media center 104 that coordinates audio between the three meeting rooms. Meeting rooms A101, B102, and C103 can be remote meeting rooms located in different locations. In specific operation, for example, a representative in meeting room A101 speaks, the recording device in meeting room A101 captures the representative's speech, and then the communication device in meeting room A101 sends the real-time audio stream to the media center 104. The media center 104 then sends the audio stream to meeting rooms B102 and C103 respectively. The communication devices in meeting rooms B102 and C103, upon receiving the audio stream, play the real-time audio from meeting room A101 through external speakers. This achieves remote audio conferencing between meeting rooms A101, B102, and C103.

[0140] During the aforementioned process, attendees in venues A, B, and C may have secretly recorded the audio being played and leaked it. Therefore, when tracing the source and holding those responsible accountable, it is necessary to know which venue's audience member specifically recorded the audio.

[0141] To address the aforementioned issues, this application provides an audio watermarking method that can solve the problem of audio source tracing by adding watermarks to real-time playing audio.

[0142] The audio watermarking method provided in this application embodiment is described in detail below with reference to the accompanying drawings. It should be noted that the above application scenarios are merely examples and do not constitute a limitation on the application scenarios of this application. The method provided in this application embodiment can also be applied to other scenarios, and this application embodiment does not limit such applications.

[0143] Please see Figure 2 ,like Figure 2 As shown, an embodiment of the audio watermarking method provided in this application includes the following steps.

[0144] 201. Obtain audio watermark from the playback terminal.

[0145] In this embodiment, the playback terminal obtains the audio watermark associated with the playback terminal. During the subsequent watermark parsing process, the parsing terminal can know through the audio watermark that the audio watermark is embedded in the audio by the playback terminal.

[0146] Optionally, the audio watermark can be pre-stored locally on the playback terminal, or it can be sent to the playback terminal by other devices, such as a service management center (SMC) sending its own audio watermark to multiple different playback terminals. For ease of explanation, this embodiment uses the number 14 as an example for the audio watermark. After the current playback terminal receives the audio watermark "14" sent by the SMC, it stores the audio watermark locally. This number 14 is the audio watermark associated with the current playback terminal.

[0147] 202. The playback terminal acquires the first audio in real time.

[0148] In this embodiment, the first audio can be obtained by the playback terminal from an external source; for example, the playback terminal can be... Figure 1 The playback terminal in venue B can also be Figure 1 The playback terminal in venue C acquires the first audio in real time from the media center. Optionally, in other application scenarios, the first audio can also be acquired in real time from the playback terminal's own memory; this embodiment of the application does not limit this.

[0149] 203. The playback terminal embeds an audio watermark in the first audio.

[0150] In this embodiment, the audio watermark "14" of the playback terminal is associated with the playback terminal, thus uniquely identifying the playback terminal through this audio watermark. During operation, the playback terminal embeds the audio watermark into the first audio in real time while playing the first audio.

[0151] Optionally, embodiments of this application further provide a specific working method for embedding an audio watermark in a first audio file. For ease of understanding, a detailed description is provided below in conjunction with the accompanying drawings.

[0152] Please see Figure 3 ,like Figure 3 As shown, the method for embedding an audio watermark in the first audio file provided in this application includes the following steps.

[0153] 301. The playback terminal determines the first target frame that meets the first preset condition in the first audio.

[0154] In this embodiment, the first target frame serves as a marker. During the subsequent parsing of the audio watermark, the parsing terminal can know that there is an audio watermark after the first target frame based on the first target frame, thereby enabling rapid location of the audio watermark.

[0155] It should be noted that since the first target frame is added to the first audio in real time, it cannot be guaranteed that every frame in the first audio is suitable as the first target frame. Therefore, it is necessary to judge the conditions of the first target frame. Only when the target frame meets the first preset conditions will the target frame be used as the first target frame.

[0156] Optionally, a synchronization frame condition detector can be set in the playback terminal. The synchronization frame condition detector can be a physical device set in the playback terminal or an operating logic stored in the playback terminal. The synchronization frame condition detector runs the method described in step 301 above, thereby determining the first target frame that meets the first preset condition.

[0157] It should be further explained that, as mentioned above, the first target frame satisfies the first preset condition. However, in actual operation, the first preset condition varies depending on the sampling rate. Specifically, when the sampling rate of the first audio is greater than or equal to the first threshold, the playback terminal determines the audio frame whose maximum low-frequency value is within the first interval as the first target frame. When the sampling rate of the first audio is less than the first threshold, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame. For ease of understanding, the two cases of the first preset condition will be explained in detail below.

[0158] 1. When the sampling rate of the first audio is greater than or equal to the first threshold, the playback terminal determines the audio frame whose maximum value of the low frequency part is within the first interval as the first target frame.

[0159] In this embodiment, when the sampling rate of the first audio is greater than or equal to a first threshold, the playback terminal identifies the first target frame by adding a synchronization frame marker to it. Since adding the synchronization frame marker changes the energy value of the mid-frequency portion of the first target frame, this requires that the low-frequency energy of the original audio of the first target frame cannot be too high or too low. In practical applications, the low-frequency energy of the original audio is often too low, resulting in a relatively low mid-frequency capability, which leads to the problem that the parsing terminal cannot resolve the synchronization frame marker after recording. Conversely, if the low-frequency capability of the original audio is too high, resulting in an excessively high mid-frequency energy, popping sounds may occur. Therefore, audio frames whose maximum low-frequency value is within a first interval are determined as the first target frame.

[0160] The specific manner of the first interval can be (Tlow, Thigh). When the playback terminal determines the first target frame, it takes the maximum value Value of the ability of the low-frequency part of the first audio, and determines whether this Value meets the formula: Tlow < Value < Thigh. If the condition is satisfied, the current target frame is determined as the first target frame. Here, Tlow is the lower limit of the energy value of the first interval, and Thigh is the upper limit of the energy value of the first interval. The specific values of Tlow and Thigh can be set by those skilled in the art according to actual needs. For example, different threshold ranges are set for different audio recording scenarios such as conference room scenarios, auditorium scenarios, and open office areas, especially the threshold for too low energy, to ensure that in different scenarios, the synchronization frame marker can obtain good embedding strength in the first target frame.

[0161] Further, after the playback terminal determines the first target frame in the above manner, it is necessary to add a synchronization frame marker to the target frame so that the subsequent parsing terminal can determine the first target frame according to this synchronization frame marker. The addition of the synchronization frame marker specifically includes the following steps.

[0162] 1. The playback terminal obtains the first sampling point from the first target frame.

[0163] In this embodiment, the first sampling point is the sampling point of the intermediate frequency part of the first target frame.

[0164] 2. The playback terminal increases the energy value of the first sampling point so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the second threshold.

[0165] In this embodiment, the first sampling point is the sampling point of the intermediate frequency part. After the energy value of the first sampling point is increased by the playback terminal, there is a significant difference in the ratio of the energy value between the first sampling point and other sampling points of the low-frequency part. Among them, the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the second threshold. Thus, when the subsequent parsing terminal obtains the first target frame, it can parse this synchronization frame marker according to the ratio of the energy values of the sampling points of the intermediate frequency part to the sampling points of the low-frequency part.

[0166] For easy understanding, based on the idea described in the above steps 1-2, a more specific implementation manner of adding the synchronization frame marker is provided below.

[0167] 1). Perform a fast Fourier transform (FFT) on the audio signal of the first target frame, take its low-frequency part, and obtain the maximum value max1 of the energy. Calculate the reference coefficient for modifying the energy value of the synchronization signal

[0168] In this embodiment, as Figure 4As shown, after performing a Fast Fourier Transform on the audio signal of the first target frame, line graphs of the audio signal of the first target frame in the frequency and time domains are obtained. Figure 4 This allows us to determine the maximum energy value in the low-frequency range.

[0169] 2) Take the 8 energy values ​​E(i)1, 0≤i≤8, from the mid-frequency part, calculate the maximum value max′1 of the 8 points, and then modify E(i)′1.

[0170] In this embodiment, as Figure 4 As shown, after modification, the energy value of the second sampling point 401 in the intermediate frequency section is significantly improved. This is because the modified formula for the intermediate frequency energy value E(i)' includes the maximum energy value of the low-frequency section, max_E1, resulting in a significant improvement in the energy value of the first sampling point in the intermediate frequency section relative to the low-frequency section. It should be noted that the above use of eight energy values ​​is merely a preferred example; those skilled in the art can select different numbers of energy values ​​according to actual needs, and this application does not limit this selection.

[0171] 3) Take a point in the mid-frequency range that is symmetrical to the above 8 energy points and perform the same energy value boosting operation.

[0172] In this embodiment, since the energy distribution after Fourier transform is symmetrical, eight points are taken in the part symmetrical to these eight points and the same embedding operation is performed.

[0173] 4) Perform Inverse Fast Fourier Transform (IFFT) on the frequency domain signal.

[0174] In this embodiment, after the fast Fourier transform, the energy value of the intermediate frequency part of the first target frame is modified in the above manner. Then, the fast inverse Fourier transform is performed on the first target frame to obtain the time domain signal with the embedded synchronization frame mark. This time domain signal is the first target frame with the embedded synchronization frame mark.

[0175] In this embodiment, the energy value of the first sampling point in the intermediate frequency (IF) portion of the first target frame is increased through steps 1)-4) as described above, so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency portion exceeds a preset range. This ensures that during subsequent parsing, when the parsing terminal obtains that the ratio of the energy value of a sampling point in the IF portion of a target frame to the energy value of the low-frequency portion is greater than a preset value, it can determine that the target frame is the first target frame with the added synchronization frame marker. This achieves the addition of the synchronization frame marker to the first target frame.

[0176] In this embodiment, the playback terminal determines in real time the target frame in the first audio that meets the first preset condition as the first target frame. Then, it increases the energy value of the first sampling point in the mid-frequency portion of the first target frame so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency portion is greater than or equal to a preset ratio, thereby achieving the addition of a synchronization frame marker. Thus, when the sampling rate of the first audio is greater than or equal to the first threshold, the playback terminal determines the audio frame whose maximum low-frequency value is within the first interval as the first target frame.

[0177] It should be noted that when the sampling rate of the first audio is less than the first threshold, the energy value of the mid-frequency part is improved by the method of synchronous frame marking. For audio with a sampling rate less than the preset value, the embedding of synchronous frame marking will affect the human ear's listening experience of the first audio. Therefore, the method of synchronous frame can no longer be used to locate the first target frame. In order to solve this problem, the embodiments of this application provide a second solution.

[0178] Second, when the sampling rate of the first audio is less than the first threshold, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame.

[0179] The playback terminal determines the audio frame containing the first characteristic sound as the first target frame. The specific method is as follows.

[0180] When a first characteristic sound is detected and the duration of the first characteristic sound is greater than or equal to a preset time, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame.

[0181] In this embodiment, the first characteristic sound can be detected by a sound detection method. The method for detecting the characteristic sound can be any method in the prior art, and this application embodiment does not limit this method. Specifically, the first characteristic sound can be human voice. For example, when the time without human voice is detected exceeds a preset time, the moment when human voice is detected again is taken as the first target frame. This approach is advantageous because it is applicable to meeting scenarios. In meeting scenarios, people communicate via voice. To prevent the voice from being transcribed, an audio watermark needs to be added to the audio recording containing the voice. Using the aforementioned human voice detection method, when the time without human voice is detected exceeds a preset time (e.g., 1.5 seconds), the moment when human voice is detected again is taken as the first target frame, thereby ensuring that subsequent watermark embedding can be embedded into the audio recording the voice information.

[0182] Optionally, the first characteristic sound can be further refined. For example, in the case of human voice, when a specific statement is detected, the target frame containing that specific statement is determined as the first target frame.

[0183] In this embodiment, when the sampling rate of the first audio is less than the first threshold, the playback terminal determines the first target frame by identifying the first characteristic sound. This allows for the confirmation of the watermark embedding start position (i.e., the first target frame) through the characteristic sound, eliminating the need to embed a synchronization frame marker. This ensures that, regardless of whether the sampling rate of the first audio is greater than or less than the first threshold, the first target frame marking the watermark embedding position can always be found in the first audio.

[0184] Optionally, the two methods in Case 1 and Case 2 can be implemented by a synchronization frame embedder in the playback terminal. The synchronization frame embedder can be a physical device set in the playback terminal or running logic stored in the playback terminal. This application embodiment does not limit this.

[0185] Using the above method, the playback terminal determines the first target frame in the first audio. At this point, the playback terminal needs to perform subsequent steps to embed an audio watermark after the first target frame.

[0186] 302. The playback terminal determines the second target frame that meets the second preset condition after the first target frame.

[0187] In this embodiment, the second target frame is the target frame located after the first target frame. Since the second target frame is determined in real time, it cannot be guaranteed that every frame after the first target frame in the first audio is suitable as the second target frame. Therefore, it is necessary to judge the conditions of the second target frame. Only when the target frame meets the second preset condition will the target frame be used as the second target frame.

[0188] It should be noted that in actual operation, the determination of the first and second target frames in the first audio is periodic. Firstly, because the playback terminal needs to add audio watermarks to the first audio in real time, it can only process the first audio in chronological order of playback. During processing, the playback terminal first determines the first target frame according to a first preset condition; then, it determines the second target frame that meets the second preset condition after the first target frame. At this point, the first and second target frames constitute one audio watermarking cycle. In the next audio watermarking cycle, the playback terminal again first determines the first target frame according to the first preset condition, and then determines the second target frame that meets the second preset condition after the first target frame. The result of this processing is that the first audio includes multiple first target frames, wherein every two adjacent first target frames in the first audio contain one second target frame.

[0189] Therefore, as an optional technical solution, the playback terminal can determine the first target frame and then the second target frame during the process of adding a watermark to the first audio. The result will still be that every two adjacent first target frames in the first audio contain a second target frame. Therefore, those skilled in the art can determine the order of determining the first and second target frames according to actual needs, and this application does not limit this. For ease of understanding, this application only describes the second target frame as occurring after the first target frame.

[0190] Furthermore, the specific implementation of the second preset condition for determining the second target frame can be as follows:

[0191] The playback terminal determines the target frame whose intermediate frequency energy value is greater than or equal to the third threshold and less than the fourth threshold as the second target frame.

[0192] In this embodiment, since the watermark is embedded in the mid-frequency region of the second target frame, if the energy of the mid-frequency region is too low, it may lead to misinterpretation or failure to detect the watermark after embedding; if the energy of the mid-frequency region is too high, it may cause popping sounds after embedding the watermark. Furthermore, since continuous watermark embedding may cause mutual interference, when determining the second target frame, it is also possible to ensure that there is a sufficient interval between the second target frames of multiple cycles. Furthermore, the interval between the third target frames needs to be greater than or equal to a sixth threshold. The specific size of the sixth threshold can be set by those skilled in the art according to the actual situation, and this embodiment does not limit it.

[0193] It should be noted that the energy value of the intermediate frequency portion of the second target frame is greater than or equal to the third threshold and less than the fourth threshold. The third threshold is less than the fourth threshold. The specific values ​​of the third threshold and the fourth threshold can be determined by those skilled in the art according to actual needs. This application does not limit this aspect.

[0194] 303. The playback terminal embeds an audio watermark in the second target frame.

[0195] In this embodiment, the second target frame is a target frame located after the first target frame. During the multiple watermarking cycles of the first audio, the second target frame is located between two first target frames. The audio watermark is embedded in the second target frame so that during the subsequent watermark parsing process, the parsing terminal can determine the location of the second target frame based on the first target frame, thereby finding the audio watermark.

[0196] Furthermore, this application embodiment provides a specific implementation method for embedding an audio watermark in the second target frame by a playback terminal. For ease of understanding, the following is combined with the appendix. Figure 5a This situation will be explained in detail, such as Figure 5aAs shown, the steps for embedding an audio watermark include:

[0197] 501. The playback terminal obtains the first sequence corresponding to the audio watermark.

[0198] In this embodiment, optionally, the audio watermark can be presented in the form of a sequence. For example, if the playback terminal is the playback terminal where venue B is located, as in the example in step 201 above, the identifier of the playback terminal is "14", then the first sequence corresponding to the audio watermark is "14". Subsequent steps need to embed the sequence 14 as an audio watermark into the first audio, thereby marking the playback terminal associated with it in the first audio.

[0199] 502. The playback terminal obtains at least one second sampling point from the second target frame.

[0200] In this embodiment, a target frame includes 2048 sampling points. The playback terminal obtains at least one second sampling point from the second target frame. The second sampling point is used to embed the elements in the first sequence in the subsequent working process.

[0201] 503. The playback terminal embeds at least one element of the first sequence into at least one second sampling point.

[0202] In this embodiment, the playback terminal embeds elements from the first sequence into the second sampling point, wherein one element from the first sequence is embedded into one second sampling point. This ensures that the second sampling point records the content of the first sequence. Subsequently, the parsing terminal can parse the content recorded in the audio watermark by reading the first sequence recorded in the second sampling point.

[0203] It should be noted that the step of embedding at least one element in the first sequence into at least one second sampling point can be achieved by the playback terminal in the following manner.

[0204] The playback terminal adjusts the energy ratio of the second sampling point in different time domains and / or different frequency domains.

[0205] In this embodiment, the magnitude of the energy ratio is associated with the numbers in the first sequence. Different energy ratios can correspond to different numbers, thus recording different numbers in the first sequence through different energy ratios. Therefore, by changing the energy ratio, the content of the first sequence is recorded at the second sampling point, achieving the embedding of the audio watermark.

[0206] For example, if the playback terminal needs to embed the audio watermark as the number "14", when embedding the audio watermark into the first audio file, the audio watermark needs to be converted into binary. This conversion of the number "14" into binary yields the first sequence: "1110". This first sequence is the content to be embedded as the audio watermark in the first audio file. The first sequence contains four elements: 1110. These four elements are embedded into the four second sampling points in the second target frame, thus achieving the embedding of the audio watermark.

[0207] In the specific working process, the embedding of each element in the first sequence into the second sampling point is achieved by changing the energy ratio of the second sampling point. The following three different schemes can be used to change the energy ratio in the second sampling point: 1. Changing the ratio of energy values ​​in different time domain components; 2. Changing the ratio of energy values ​​in different frequency domain components; 3. Changing the ratio of energy values ​​in both different time domains and different frequency domains simultaneously. For ease of understanding, these three methods are explained in detail below with reference to the accompanying diagram.

[0208] 1. Change the ratio of energy values ​​in different time domains.

[0209] In this embodiment, the method of achieving time-domain embedding by changing the ratio of energy values ​​in different time-domain parts includes the following steps.

[0210] 1. Reduce the energy value of the first half of the second sub-sampling point so that the ratio of the energy value of the second half of the second sampling point to the energy value of the first half is greater than or equal to the fifth threshold.

[0211] In this embodiment, as Figure 5b As shown, in the second sampling point, the energy value of the first half 5061 of the second sampling point is reduced according to the time domain order, so that the energy value of the second half 5062 of the second sampling point is significantly higher than the energy value of the first half 5061. At this time, the waveform of this energy distribution can be preset to the digital 0.

[0212] Specifically, this can be achieved in the following ways.

[0213] like Figure 5c As shown, Figure 5c The waveform of the original frame at the second sampling point is shown below. Figure 5c The original frame shown is obtained after undergoing the first Discrete Cosine Transform (DCT). Figure 5d The waveform diagram shown, then from Figure 5d The waveform diagram shown is used to select the intermediate frequency part and perform a second DCT transformation to obtain the following result: Figure 5e The waveform shown is then processed according to the following formula 1. Figure 5e The energy of the waveform shown is processed.

[0214]

[0215] In Formula 1 above, j represents the time interval in the time domain, and P(j) is... Figure 5e The total energy of the waveform within time period j, where λ is a preset coefficient, and the specific value of λ can be adjusted according to actual needs, and mid represents... Figure 5e The midpoint of the waveform in the time domain. From the above formula, we know that the range of j is from 1 to the midpoint, i.e. Figure 5e The first half of the sample is processed by dividing the total energy value P(j) of the first half by the coefficient λ, thereby reducing the energy value of the first half of the second sampling point, resulting in the following: Figure 5b The waveform diagram shown.

[0216] 2. Reduce the energy value of the second half of the second sub-sampling point so that the ratio of the energy value of the first half to the energy value of the second sampling point is greater than or equal to the fifth threshold.

[0217] In this embodiment, as Figure 5f As shown, in the second sampling point, the energy value of the latter half of the second sampling point 5063 is reduced according to the time domain order, so that the energy value of the first half of the second sampling point 5064 is significantly higher than the energy value of the latter half of the second sampling point 5063. At this time, the waveform of this energy distribution can be preset to digital 1.

[0218] Specifically, this can be achieved in the following ways.

[0219] Based on the above Figure 5e The waveform shown is then processed according to the following formula 2. Figure 5e The energy of the waveform shown is processed.

[0220]

[0221] In Formula 1 above, j represents the time interval in the time domain, and P(j) is... Figure 5e The total energy of the waveform within time period j, where λ is a preset coefficient, and the specific value of λ can be adjusted according to actual needs, and mid represents... Figure 5e The midpoint of the waveform in the time domain, S'-T' represents Figure 5e The upper limit of the time domain. From the above formula, we know that the range of j is from mid+1 to S'-T', that is, from the midpoint of the time domain to the end point of the time domain. Figure 5e The latter half of the sample is obtained by dividing the total energy value P(j) of the latter half by the coefficient λ, thereby reducing the energy value of the latter half of the second sampling point, as shown below. Figure 5f The waveform diagram shown.

[0222] In this embodiment, the watermark is embedded by changing the ratio of energy values ​​in different time-domain portions of the second sampling point. Specifically, the energy ratio of the preceding and following portions of the second sampling point in the time domain is adjusted, and then the different energy ratios of the preceding and following portions are preset to 0 and 1 respectively, thereby achieving binary digital embedding at the second sampling point. Subsequently, this binary sequence can be converted into a decimal sequence as needed, thus realizing the process of adding the first sequence to the second sampling point. When the subsequent parsing terminal parses the audio watermark, it can directly obtain the ratio of energy values ​​in the preceding and following portions of the target frame to achieve the parsing of the audio watermark.

[0223] It should be noted that in the above analysis process, the energy ratio is determined by taking the midpoint of the target frame in the time domain as the boundary, obtaining the ratio of the front and rear parts, and then analyzing it according to the above preset rules. However, in actual work, when copying the first audio, due to echoes or reverberation in the venue, the high-energy part will increase in the time domain. For example, when the audio watermark is 1, the ratio of the energy value of the first half to the energy value of the second half in the time domain is greater than or equal to the fifth preset value, with the midpoint as the boundary. However, due to the presence of echoes or reverberation during the transcription process, the high-energy region crosses the midpoint boundary in the time domain, thereby affecting the size of the energy ratio, causing the analysis terminal to be unable to obtain the watermark based on the energy ratio. To solve the above problem, this application embodiment further provides a method for changing the energy ratio based on the frequency domain to overcome the influence of echoes and reverberation generated during the transcription process on the energy distribution in the time domain.

[0224] 2. Change the ratio of energy values ​​in different frequency domains.

[0225] In this embodiment, a similar approach is used to change the energy ratio of the second sampling point in the frequency domain, thereby achieving watermark embedding. For specific implementation methods, please refer to the above-described method of changing the energy ratio of different time domain parts. The difference is that this implementation method does not use time as the boundary in the time domain, but uses frequency as the boundary in the frequency domain.

[0226] The energy distribution diagram obtained by changing the energy ratio of different frequency domain components is as follows: Figure 5g As shown, it should be noted that the aforementioned Figure 5b to Figure 5f The image shown is a line graph depicting the relationship between energy values ​​and the time domain. Figure 5g The displayed energy distribution maps show the relationship between energy distribution in the time and frequency domains. Figure 5gIn the energy distribution box, the shaded area represents the high-energy portion, and the white area represents the low-energy portion. At the second sampling point, the midpoint of the frequency domain is used as the boundary. Energy distribution 5065, where the ratio of the upper half to the lower half is greater than the fifth threshold, represents the digit 1; energy distribution 5066, where the ratio of the lower half to the upper half is greater than the fifth threshold, represents the digit 0. This achieves audio watermark embedding by changing the energy ratio in the frequency domain, overcoming the influence of echo reverberation generated during transcription on the energy distribution in the time domain.

[0227] Third, simultaneously change the ratio of energy values ​​in different time domains and different frequency domains.

[0228] This embodiment combines the solutions of Scheme 1 and Scheme 2. First, using the method of Scheme 1, the energy ratio of the second sampling point in the time domain is adjusted to achieve the first step of watermark embedding. For example... Figure 5h As shown, the ratio of the energy value of the first half (5067) to the energy value of the second half (5068) of the second sampling point is greater than the fifth threshold, and the embedded audio watermark is the number 1. Furthermore, to prevent the echo reverberation generated during transcription from affecting the temporal distribution of the audio watermark, as shown... Figure 5h As shown, for Figure 5h The latter half (i.e. the part with lower energy) adopts the method of the above scheme two to change the energy ratio of the high-frequency part 50681 to the low-frequency part 50682. The energy distribution in which the ratio of the upper half to the lower half of the second sampling point is greater than the fifth threshold also represents the number 1.

[0229] Similarly, by reversing the ratios of the above energy distributions, the number 0 can be represented.

[0230] The method provided in Scheme 3 adds two watermarks, one large and one small, to the second sampling point. These two watermarks record the same value, with the larger watermark being obtained by changing the energy ratio of the two portions of the second sampling point in the time domain. Figure 5h The watermark is formed by the ratio of the energy value of the first half (5067) to the energy value of the second half (5068) of the second sampling point. The smaller watermark is obtained by changing the energy ratio of high and low frequencies in the low-energy part of the second sampling point. Figure 5h The watermark is formed by the energy ratio of the high-frequency part 50681 to the low-frequency part 50682 in the middle and latter half of 5068.

[0231] Furthermore, for such Figure 5h To prevent the echo reverberation generated during transcription from affecting the energy distribution in the time domain, the low-energy portion of the audio watermark can be further segmented, such as... Figure 5iThe energy distribution map shown represents the number 1. The shaded areas represent high-energy regions, and the white areas represent low-energy regions. Figure 5i As shown, in the time domain, the first half 51 is the high-energy part, and the second half 52 is the low-energy part. When embedding a small watermark in the second half 52, the second half 52 is further divided into two parts in the time domain, which are denoted as the first part 521 and the second part 522, respectively. The first part 521 is the region close to the first half 51 (high-energy part), and the second part 522 is the region far away from the first half 51 (high-energy part). When embedding the small watermark, only the energy ratio of the high-frequency to low-frequency components of the second part 522 is changed. For the first part 521, since it is close to the high-energy part, the energy of the first part 521 may increase due to reverberation generated during transcription. Therefore, the energy ratio of the high and low frequencies of the first part 521 is not changed, thus eliminating the influence of echo reverberation generated during transcription on the energy distribution in the time domain. The small watermark is only embedded in the second part 522, which is far from the high-energy part, by changing the energy ratio of the high and low frequencies. The energy value of the high-frequency part 5221 of the second part 522 is higher than the energy value of the low-frequency part 5222, which also represents the number 1. The two watermarks, large and small, represent the same audio watermark.

[0232] Similarly, Figure 5i By swapping the high-energy and low-energy portions of the image, an audio watermark representing the number 0 can be obtained.

[0233] It should be noted that the above Figure 5i The audio watermark shown is divided into two parts in the time domain for processing the low-energy part (i.e., the second half 52) (the first part 521 and the second part 522). In actual operation, those skilled in the art can divide the low-energy part into more parts in the time domain according to actual needs. This application embodiment does not limit this.

[0234] In this embodiment, in order to embed an audio watermark in the first audio in real time, the playback terminal searches for a second target frame that meets the preset conditions after the first target frame according to the time sequence and embeds the audio watermark. During the embedding of the audio watermark, the energy ratio of the energy values ​​of the sampling points in different time domains and / or different frequency domains in the third target frame is changed to achieve the embedding of the audio watermark, thereby realizing the real-time embedding of the audio watermark. At the same time, the embedded watermark has a strong anti-interference ability during the audio transcription process.

[0235] It should be noted that in the above schemes one to three, the playback terminal achieves binary watermark embedding by changing the energy ratio. In actual operation, those skilled in the art can achieve watermark embedding in other bases, such as decimal or 12-base, by changing the energy ratio according to actual needs. This application embodiment does not limit this.

[0236] It should be noted that, in the first sequence mentioned above, in order to ensure the integrity of the watermark embedded in the first audio during the parsing process, a check bit needs to be added to the first sequence when embedding the audio watermark in the first audio on the playback terminal. This allows the subsequent parsing terminal to determine the integrity of the audio watermark transmission based on the check bit, preventing incomplete audio watermark transmission and parsing failure due to signal loss during the transcription process of the first audio. For ease of understanding, the specific method of adding the check bit is explained in detail below.

[0237] In this embodiment, as described above, the audio watermark obtained by the playback terminal is the number "14". When embedding the audio watermark into the first audio, it is necessary to convert the audio watermark into binary, thereby converting the number "14" into binary to obtain the first sequence: "1110". This first sequence is the content that needs to be embedded into the first audio as an audio watermark. To ensure the integrity of the transmission of the first sequence, a check bit can be added to the first sequence. For example, parity checking can be used. For example, if the first sequence "1110" includes three digits 1, that is, an odd number of digits "1", then a digit 1 is added to the last position of the first sequence to obtain a new first sequence "11101". The 1 in the last position of the first sequence is the check bit. The "1" in the check bit is used to indicate that there are an odd number of digits "1" in the current sequence besides the check bit. Similarly, if the number of "1"s in the first sequence is even, the check bit is "0". This method determines the number of digits in the first sequence. The subsequent parsing terminal can then verify the first sequence based on the check bit to ensure accurate transmission of the first sequence.

[0238] Furthermore, due to the issue of some sampling points being lost during the initial audio recording process, if the first sequence in the audio watermark is too long, it will result in the watermark occupying too many frames. During detection, the loss of sampling points causes a deviation between the start position of each frame and the expected value, leading to inaccurate audio watermark detection. To solve this problem, when the length of the first sequence exceeds a threshold, it is divided into multiple sub-sequences. Each sub-sequence is then combined with a check bit in the manner described above before being reassembled into a larger first sequence. This allows the subsequent parsing terminal to verify the first sequence according to a preset sub-period, ensuring the integrity of the first sequence's transmission.

[0239] In this embodiment, taking the audio watermark as the number "14" as an example, after adding the check bit, the first sequence corresponding to the audio watermark is "11101". The playback terminal acquires the first second target frame and, according to the method described in step 506 above, embeds the first element in the first sequence, "1", into the first second sampling point in the second target frame, thus achieving the embedding of the first digit in the first sequence. This completes the first sub-cycle of the watermark embedding cycle. Then, step 506 is executed repeatedly to acquire the second second sampling point from the second target frame and, in the same way, embed the second element in the first sequence, "1", into the second second sampling point in the second target frame, thus achieving the embedding of the second digit in the first sequence. ... And so on, the playback terminal embeds the five digits in the first sequence into the five second sampling points of the second target frame through a complete watermark embedding cycle consisting of five watermark embedding sub-cycles.

[0240] Optionally, in each of the above-mentioned watermark embedding sub-cycles, in order to ensure that the watermark embedding strength is sufficient in the current sub-cycle, the playback terminal needs to perform an embedding strength detection once after each sub-cycle is completed.

[0241] It should be noted that the specific detection method for watermark embedding strength is as follows: It detects whether the energy ratio of the second sampling point in different time domains and / or different frequency domains in the current sub-cycle is greater than the fifth threshold. Taking the first watermark embedding scheme in step 506 above as an example, assuming the number embedded in the current second sampling point is 1, the playback terminal needs to determine whether the energy ratio of the first half to the second half of the current second sampling point in the time domain is greater than the fifth threshold. If yes, the subsequent steps are executed; if no, the watermark embedding is re-executed. At this time, the value of λ in Formula 1 is increased to further reduce the energy value of the second half of the second sampling point, thereby increasing the energy ratio of the first half to the second half of the second sampling point in the time domain, so that the watermark embedding strength meets the requirements.

[0242] In this embodiment, the playback terminal embeds an audio watermark in the second target frame through the methods described in steps 501 to 503 above. Step 203 is now complete.

[0243] 204. The playback terminal plays the first audio with an embedded audio watermark.

[0244] In this embodiment, the playback terminal plays the first audio with the embedded audio watermark. Since the audio watermark is embedded in the first audio in real time, the audio played by the playback terminal can still carry the audio watermark in the real-time playback scenario. Thus, in the subsequent transcription process, the source of the first audio can be traced based on the audio watermark.

[0245] In summary, the audio watermarking method provided in this application involves the playback terminal acquiring first audio in real time; embedding an audio watermark in the first audio, with the audio watermark associated with the playback terminal; and playing the first audio with the embedded audio watermark. Thus, in scenarios where audio is played in real time, the playback terminal adds an audio watermark to the audio stream in real time, enabling subsequent devices to identify the playback terminal based on the audio watermark during watermark parsing, facilitating source tracing after the first audio is transcribed.

[0246] The first audio file with an added audio watermark using the above-described audio watermarking method, whether ripped via a digital channel or an air channel, will have its audio watermark extracted by the parsing terminal. This allows for tracing the source of the first audio file, and the parsing terminal can use the audio watermark to determine the playback terminal that added the watermark to the first audio file. For ease of understanding, the audio watermark parsing method provided in this application embodiment will be described in detail below with reference to the accompanying drawings.

[0247] Please see Figure 6 ,like Figure 6 As shown, an embodiment of the audio watermark parsing method provided in this application includes the following steps.

[0248] 601. Parse the terminal to obtain the first audio.

[0249] In this embodiment, the first audio is audio with an audio watermark embedded by the playback terminal using the above method. It should be noted that the initial playback source of the first audio is the playback terminal itself; that is, the playback terminal embeds the audio watermark into the first audio. After the playback terminal plays the first audio, it can be directly acquired by the parsing terminal, or it can be transcribed. This transcription can be a transmission transcription via a digital channel or a transcription via an air channel. The method provided in this application can parse both types of transcripts.

[0250] Optionally, after acquiring the first audio, the parsing terminal also needs to convert the format and sampling rate of the first audio.

[0251] In this embodiment, since there are multiple recording devices that might copy the first audio after it is played on the playback device, especially since different brands of recording devices have different audio file formats, with a typical sampling rate of 44.1kHz, it is necessary to first convert the audio file format and change the sampling rate to obtain a format and sampling rate that the parsing terminal can process. Preferably, the parsing terminal can convert the sampling rate of the first audio to 48kHz.

[0252] 602. The parsing terminal parses the audio watermark from the first audio.

[0253] In this embodiment, the parsing terminal can perform real-time parsing or offline parsing of the first audio. This application embodiment does not limit the scope of the parsing. For ease of understanding, this application embodiment mainly describes the offline parsing method, but it does not constitute a limitation on this solution.

[0254] Please see Figure 7 ,like Figure 7 As shown, the audio watermark parsing method provided in this application for parsing audio watermarks from a first audio file includes the following steps.

[0255] 701. The parsing terminal determines the first target frame in the first audio that meets the first preset condition.

[0256] In this embodiment, the first target frame satisfies a first preset condition, therefore the parsing terminal can obtain the first target frame in the first audio according to the first preset condition. It should be noted that the first audio includes multiple first target frames, each corresponding to a watermark parsing cycle. Therefore, whenever the parsing terminal determines a first target frame in the first audio that satisfies the first preset condition, it executes a subsequent parsing step. The first preset condition can be: selecting target frames whose maximum low-frequency value falls within a first interval as the first target frame. For a detailed implementation of the first interval, please refer to the description in step 301 above; it will not be repeated here.

[0257] It should be noted that the first target frame includes marker information. The parsing terminal needs to further determine the first target frame based on the marker information. The implementation of the marker information includes two technical solutions depending on the actual situation of the first audio: 1. When the sampling rate of the first audio is less than a first threshold, the marker information is a feature sound. 2. When the sampling rate of the first audio is greater than or equal to the first threshold, the marker information is a synchronization frame marker. These two cases will be explained in detail below.

[0258] 1. When the sampling rate of the first audio is less than the first threshold, the information is marked as the characteristic sound.

[0259] In this embodiment, when the parsing terminal detects that the sampling rate of the first audio is less than the first threshold, it can determine that the marker information in the first target frame and the second target frame of the first audio is the characteristic sound. At this time, the specific method for the parsing terminal to detect the marker information is as follows:

[0260] The parsing terminal determines the target frames from the first audio that contain the first characteristic sound and whose duration is greater than or equal to a preset time as the first target frames.

[0261] In this embodiment, the first characteristic sound can be detected by a sound detection method. The method for detecting the characteristic sound can be any method in the prior art, and this application embodiment does not limit this method. Specifically, the first characteristic sound can be human voice. For example, when the time during which no human voice is detected exceeds a preset time, the target frame where the human voice is detected again is determined as the first target frame.

[0262] Furthermore, the playback terminal and the parsing terminal can agree on more detailed implementation methods for characteristic sounds. For example, in the state of human voice detection, the target frame containing the specific statement is determined as the first target frame only when the parsing terminal detects a specific statement.

[0263] In this embodiment, when the sampling rate of the first audio is less than the first threshold, the parsing terminal determines the first target frame by identifying the first characteristic sound. This allows for the confirmation of the watermark embedding start position (i.e., the first target frame) through the characteristic sound, eliminating the need to embed a synchronization frame marker in the first audio. This ensures that, regardless of whether the sampling rate of the first audio is greater than or less than the first threshold, the first target frame marking the watermark embedding position can always be found in the first audio.

[0264] Second, when the sampling rate of the first audio is greater than the first threshold, the marking information is a synchronization frame marker.

[0265] In this embodiment, when the parsing terminal determines that the sampling rate of the first audio is greater than or equal to the first threshold, it can determine that the marking information in the first target frame of the first audio is a synchronization frame mark. The method for the parsing terminal to parse the synchronization frame mark specifically includes the following steps.

[0266] 1. The terminal analyzes and obtains the first ratio of the energy values ​​of the first audio mid-frequency part to the low-frequency part frame by frame.

[0267] In this embodiment, since the method for adding synchronization frames by the playback terminal is to increase the energy value of the first sampling point of the intermediate frequency part, thereby increasing the first ratio of the energy values ​​of the intermediate frequency part to the low frequency part, the parsing terminal can determine the synchronization frame marker through this first ratio.

[0268] 2. When the parsing terminal obtains an initial target frame where the first ratio is greater than or equal to the second threshold, it slides the first audio frame by frame from the initial target frame to obtain the second ratio of the energy values ​​of the mid-frequency part and the low-frequency part in each sliding window.

[0269] In this embodiment, the parsing terminal obtains a first ratio of the energy values ​​of the mid-frequency and low-frequency sample points in each frame, frame by frame. When a first sample point with a first ratio greater than or equal to a second threshold is found, the target frame containing the first sample point is determined as the initial target frame. However, since a frame has 2048 sample points, and the first sample point is only a portion of them, after finding a first sample point that meets the first ratio, there may be an offset between the actual target frame containing the first sample point and the initial target frame containing the current first sample point. To solve this problem, it is necessary to start from the initial target frame and move backward in a sliding window manner, detecting the first audio after the initial target frame frame by frame to obtain a second ratio of the energy values ​​of the mid-frequency and low-frequency components within each sliding window.

[0270] 3. The terminal analyzes and obtains the frame containing the sliding window with the largest second ratio as the first target frame.

[0271] In this embodiment, please refer to Figure 8 As shown, the initial target frame 801 is the initial sliding window 801 generated by the parsing terminal. There is an intersection between the initial target frame 801 and the first target frame 802, and the first sampling point 803 is located within this intersection. The parsing terminal needs to completely overlap the sliding window 801 with the first target frame 802 to determine the location of the first target frame. Specifically, the parsing terminal works by detecting the second ratio of the energy values ​​of the intermediate frequency component to the low frequency component within each sliding window 801 using a sliding window method. Since the playback terminal actively boosts the energy of the intermediate frequency component of the first target frame 802, the window where the second ratio reaches its maximum value is the window containing the first target frame. This method achieves the overlap between the sliding window 801 and the first target frame 802. Thus, the sliding window detection method enables the search for the synchronization frame marker and effectively prevents offset problems during the search process, improving the accuracy of subsequent watermark detection.

[0272] It should be noted that the above-mentioned method for detecting synchronization frame markers determines this by detecting the first ratio of the energy values ​​of the mid-frequency and low-frequency components of the first audio. However, in actual operation, the original content of the first audio (i.e., non-watermarked content) may also contain instances where the ratio of the energy values ​​of the mid-frequency and low-frequency components is greater than the first ratio. This can lead to false detection of synchronization frame markers. In fact, the essential difference between the synchronization frame marker and the original content of the first audio is that the energy value of the first sampling point of the mid-frequency component in the synchronization frame marker has a sudden increase relative to the low-frequency component. Audio does not exhibit such a sudden increase in natural recording conditions. Therefore, utilizing this characteristic, when the parsing terminal determines that the target frame containing the sliding window is the first target frame through step 3 above, it can further determine whether the synchronization frame marker in the current target frame is the true target frame marker through the following steps, thereby preventing false detection.

[0273] 4. The parsing terminal obtains the first sampling point with the highest energy value in the intermediate frequency part from the sliding window with the largest second ratio.

[0274] In this embodiment, the first sampling point is the point with the highest energy value in the current window.

[0275] 5. The parsing terminal obtains the third sampling point, which is a preset length before the first sampling point.

[0276] In this embodiment, the third sampling point is located before the first sampling point in the time domain. The preset length between the third sampling point and the first sampling point can be set by those skilled in the art according to actual needs, or it can be determined by the parsing terminal according to parameters such as the sampling rate. This embodiment does not limit this aspect.

[0277] 6. The parsing terminal determines that the portion where the ratio of the energy value of the first sampling point to the energy value of the third sampling point is greater than the seventh threshold is marked as a synchronization frame.

[0278] In this embodiment, please refer to Figure 9a and Figure 9b ,in, Figure 9a Figure 9b shows the first target frame with the synchronization frame mark added, while Figure 9b shows the ordinary target frame without the synchronization frame mark added. Figure 9a and Figure 9b In both images, the energy ratios of the mid-frequency and low-frequency components meet the preset conditions. Therefore, in this case, it is impossible to determine which frame has the synchronization frame marker added based solely on the energy ratio of the mid-frequency and low-frequency components, leading to false detection. To address this, the method described in steps 4 to 6 above is used, assuming that... Figure 9a In the diagram, there are three sampling points between the first sampling point 901 and the third sampling point 902. It can be seen that the energy at the first sampling point increases abruptly relative to the third sampling point.Figure 9b In the waveform without a synchronization frame marker, the energy value changes smoothly. Even when there is a three-sampling-point gap between the first sampling point 903 and the third sampling point 904, the energy value of the first sampling point 903 does not show a significant change relative to the third sampling point 904. Therefore, this method can accurately identify the synchronization frame marker and prevent false detections.

[0279] In this embodiment, by using the aforementioned characteristic sound method and synchronous frame marking method, regardless of the sampling rate of the first audio, the parsing terminal can detect the first target frame determined by the playback terminal in the first audio.

[0280] 702. The parsing terminal determines the second target frame that meets the second preset condition after the first target frame.

[0281] In this embodiment, after the parsing terminal detects the first target frame, it uses the first target frame as the positioning frame to continue searching for the second target frame located after the first target frame. The second target frame has an embedded audio watermark and satisfies a second preset condition. The parsing terminal can quickly find the second target frame after the first target frame based on the second preset condition.

[0282] Optionally, since the first audio is embedded with a multi-period watermark, each period includes a first target frame and a second target frame. Therefore, when parsing offline, the parsing terminal can also obtain the second target frame from the first target frame backward. For ease of understanding, this application embodiment is only described by moving backward from the first target frame to find the second target frame, but it does not constitute a limitation on the solution of this application embodiment.

[0283] Optionally, such as Figure 10 As shown, the parsing terminal can determine the second target frame through the following steps.

[0284] 1001. Starting from the first target frame, the parsing terminal moves backward frame by frame, and obtains candidate target frames whose energy in the intermediate frequency part of each frame is greater than or equal to the third threshold and less than the fourth threshold.

[0285] In this embodiment, since the playback terminal embeds the watermark in the mid-frequency region of the second target frame, if the energy in the mid-frequency region is too low, it will easily lead to mis-parsing and failure to detect the watermark after embedding; if the energy in the mid-frequency region is too high, it will cause popping sounds after embedding the watermark. Therefore, based on this characteristic, the parsing terminal first parses the target frames whose mid-frequency energy is greater than or equal to the third threshold and less than the fourth threshold as candidate target frames where the second target frame may exist.

[0286] 1002. The target frame whose energy ratio of different time domain and / or different frequency domain energy values ​​obtained by the parsing terminal from the candidate target frames is greater than or equal to the fifth threshold is the second target frame.

[0287] In this embodiment, since the playback terminal embeds the audio watermark by changing the energy ratio in the second target frame, the target frame from which the parsing terminal obtains the energy ratio of different time domain and / or different frequency domain energy values ​​from the candidate target frames that is greater than or equal to the fifth threshold is the second target frame, depending on the specific method by which the playback terminal embeds the audio watermark.

[0288] In this embodiment, through the above steps, the parsing terminal determines the second target frame, and then can begin to parse the watermark from the second target frame.

[0289] 703. The parsing terminal parses the audio watermark from the second target frame.

[0290] In this embodiment, the playback terminal embeds the audio watermark into the second target frame. Therefore, when the parsing terminal obtains the second target frame, it can parse the embedded audio watermark from the second target frame.

[0291] Please see Figure 11 ,like Figure 11 As shown, optionally, the parsing terminal parses the audio watermark from the second target frame through the following steps.

[0292] 1101. The parsing terminal obtains the second sampling point from the second target frame.

[0293] In this embodiment, the second sampling point is a sampling point in the second target frame where the energy ratio is greater than or equal to the fifth threshold. The parsing terminal obtains the second sampling point from the second target frame, and the second sampling point records the watermark information embedded by the playback terminal.

[0294] 1102. The parsing terminal obtains the energy ratio of different time domain and / or different frequency domain energy values ​​in the second sampling point.

[0295] In this embodiment, when embedding an audio watermark, the playback terminal can employ three different methods to change the energy ratio in the second sampling point: 1. Changing the ratio of energy values ​​in different time domains; 2. Changing the ratio of energy values ​​in different frequency domains; 3. Simultaneously changing the ratio of energy values ​​in both different time and frequency domains. Therefore, depending on the watermark embedding method, the parsing terminal needs to employ corresponding methods for parsing. For ease of understanding, these three methods are explained in detail below with reference to the accompanying drawings.

[0296] 1. Change the ratio of energy values ​​in different time domains.

[0297] In this embodiment, when the playback terminal embeds the audio watermark, it does so by changing the ratio of energy values ​​in different time domains. Specifically, when the ratio of the energy value of the second half of the second sampling point to the energy value of the first half is greater than the fifth threshold, the number corresponding to the ratio is set to 0; when the ratio of the energy value of the first half of the second sampling point to the energy value of the second half is greater than the fifth threshold, the number corresponding to the ratio is set to 1.

[0298] Based on the above watermark embedding rules, please refer to Figure 12a and Figure 12b The energy distribution maps of the second target frame obtained by the parsing terminal are as follows: Figure 12a and Figure 12b The two scenarios are shown. In the energy distribution map, dark areas represent high-energy regions, and white areas represent low-energy regions. Therefore, by analyzing the energy distribution map, the terminal can intuitively obtain the energy distribution of the second target frame. Specifically, for... Figure 12a As shown in the energy distribution map, the energy value of the latter half 1201 in the second target frame is significantly greater than the energy value of the first half 1202. Furthermore, after calculation by the parsing terminal, when the ratio of the energy value of the latter half 1201 to the energy value of the first half 1202 at the second sampling point exceeds the fifth threshold, the parsing terminal will... Figure 12a The watermark embedded in the second target frame shown is determined to be 0. For Figure 12b As shown in the energy distribution map, the energy value of the first half (1203) of the second target frame is significantly greater than the energy value of the second half (1204). Furthermore, after calculation by the parsing terminal, when the ratio of the energy value of the first half (1203) to the energy value of the second half (1204) of the second sampling point exceeds the fifth threshold, the parsing terminal will... Figure 12b The watermark embedded in the second target frame shown is determined to be 1.

[0299] It should be noted that in the above analysis process, the energy ratio is determined by taking the midpoint of the target frame in the time domain as the boundary, obtaining the ratio of the front and rear parts, and then analyzing it according to the aforementioned preset rules. However, in actual operation, when copying the first audio, the high-energy portion increases in the time domain due to echoes or reverberation in the venue. For example, when the audio watermark is 1, the ratio of the energy value of the first half to the energy value of the second half in the time domain, with the midpoint as the boundary, is greater than or equal to the fifth preset value. However, due to the presence of echoes or reverberation during the transcription process, the high-energy region crosses the midpoint boundary in the time domain, thus affecting the magnitude of the energy ratio and causing the analysis terminal to be unable to obtain the watermark based on the energy ratio. To solve the above problem, the playback terminal further provides a method for changing the energy ratio based on the frequency domain when embedding the watermark, in order to overcome the influence of echoes and reverberation generated during the transcription process on the energy distribution in the time domain.

[0300] 2. Change the ratio of energy values ​​in different frequency domains.

[0301] In this embodiment, the playback terminal uses a similar method to the above when embedding the audio watermark, changing the energy ratio of the second sampling point in the frequency domain to achieve the embedding of the watermark. For specific implementation methods, please refer to the above method of changing the energy ratio of different time domain parts. The difference is that this implementation method does not use time as the boundary in the time domain, but uses frequency as the boundary in the frequency domain.

[0302] Therefore, when the parsing terminal parses the audio watermark, the energy distribution map of the second target frame obtained is as follows: Figure 5g As shown, the second sampling point is demarcated by the midpoint of the frequency domain. In the energy distribution map, the dark area represents the high-energy region, and the white area represents the low-energy region. Thus, through the energy distribution map, the terminal can intuitively obtain the energy distribution of the second target frame. Figure 5g As shown, a ratio of the upper half to the lower half greater than the fifth threshold represents the number 1. A ratio of the lower half to the upper half greater than the fifth threshold represents the number 0. This achieves audio watermark embedding by changing the energy ratio in the frequency domain, overcoming the influence of echo reverberation generated during transcription on the energy distribution in the time domain.

[0303] Third, simultaneously change the ratio of energy values ​​in different time domains and different frequency domains.

[0304] In this embodiment, the playback terminal combines the solutions of Scheme 1 and Scheme 2 when embedding the audio watermark. First, using the method of Scheme 1, the energy ratio of the second sampling point in the time domain is adjusted to achieve the first step of watermark embedding. For example... Figure 5h As shown, the ratio of the energy value of the first half to the energy value of the second half of the second sampling point is greater than the fifth threshold, and the embedded audio watermark is the number 1. Furthermore, to prevent the echo reverberation generated during transcription from affecting the energy distribution of the audio watermark in the time domain, as shown... Figure 5h As shown, for Figure 5h For the latter half (i.e. the part with lower energy), the method described in Scheme 2 above is adopted to change the energy ratio between the high-frequency part and the low-frequency part.

[0305] The method provided in Scheme 3 adds two watermarks, one large and one small, to the second sampling point. These two watermarks record the same value. The large watermark is obtained by changing the energy ratio of the two parts before and after the second sampling point in the time domain, while the small watermark is obtained by changing the energy ratio of high and low frequencies in the low-energy part of the second sampling point.

[0306] Furthermore, for such Figure 5hTo prevent the echo reverberation generated during transcription from affecting the energy distribution in the time domain, the low-energy portion of the audio watermark can be further segmented, such as... Figure 5i As shown, when embedding a small watermark in the low-energy portion of the second sampling point, this low-energy portion is divided into two parts in the time domain, denoted as the first part and the second part, respectively. The first part is the region close to the high-energy portion, and the second part is the region far from the high-energy portion. When embedding the small watermark, only the energy ratio of the high-frequency and low-frequency portions of the second part is changed. For the first part, since it is close to the high-energy portion, the energy of the first part may increase due to reverberation generated during transcription. Therefore, the high-low frequency energy ratio of the first part is not changed, thus eliminating the influence of echo reverberation generated during transcription on the energy distribution in the time domain. The small watermark is only embedded in the second part, which is far from the high-energy portion, by changing the high-low frequency energy ratio.

[0307] It should be noted that the above Figure 5i The audio watermark shown is processed by dividing the low-energy part into two parts in the time domain. In actual work, those skilled in the art can divide the low-energy part into more parts in the time domain according to actual needs. This application embodiment does not limit this.

[0308] For the audio watermark embedded in the playback terminal according to Scheme 3, the parsing terminal first parses the large watermark. The parsing method of the large watermark can be referred to Scheme 1 above. Then, it parses the small watermark in the large watermark. The specific parsing method of the small watermark can be referred to Scheme 2 above. When the numbers parsed from the large watermark and the small watermark are consistent, it is determined that the watermark parsing is correct, and the parsing terminal obtains the watermark embedded in the current second target frame.

[0309] It should be noted that in the above schemes one to three, the playback terminal achieves binary watermark embedding by changing the energy ratio. In actual operation, those skilled in the art can achieve watermark embedding in other bases, such as decimal or 12-base, by changing the energy ratio according to actual needs. This application embodiment does not limit this.

[0310] In this embodiment, the parsing terminal parses the watermark embedded in the first audio by the playback terminal in different ways. With the method provided by this application embodiment, the audio watermark has good anti-interference ability. The audio watermark is not easily lost due to air recording during the transcription process. At the same time, the parsing terminal can accurately parse the audio watermark embedded by the playback terminal, so that the audio watermark embedding scheme has good stability and accuracy.

[0311] It should be noted that, taking the audio watermark as the number "14" as an example, the first sequence corresponding to the audio watermark is "1110". When the parsing terminal parses the audio watermark from the second target frame, following the method described in step 703 above, it first parses the first element in the first sequence of the first second sampling point: "1", thus realizing the parsing of the first digit in the first sequence. This completes the first sub-cycle of the watermark parsing cycle. Then, step 703 above is executed repeatedly to obtain the second second sampling point from the second target frame, and the second element in the first sequence, "1", is parsed in the same way, thus realizing the parsing of the second digit in the first sequence. ... And so on, the playback terminal parses the complete watermark parsing cycle consisting of the four watermark embedding sub-cycles, thus realizing the parsing of the audio watermark.

[0312] Optionally, when the parsing terminal determines the first target frame, deviations may occur due to the starting position offset and the loss of recording sampling points. These deviations cause corresponding deviations in the subsequent second target frame, resulting in the audio watermark embedded in the second target frame being affected by the energy of the next frame. Therefore, the parsing terminal needs to preprocess the second target frame to prevent the influence of energy changes in the next frame. Thus, the energy on both sides of the second target frame is removed. The length of the removal can be determined by those skilled in the art according to actual needs, such as trimming 8 sampling points at the beginning and end, or it can be determined by the parsing terminal according to preset logic, such as adjusting the trimming length based on the length of the detection period. This embodiment of the application does not limit this. The final second target frame has its beginning and end portions reduced by a preset length, retaining only the middle portion, thereby eliminating interference caused by deviations.

[0313] In this embodiment, the parsing terminal extracted the audio watermark from the first audio file using the above method.

[0314] 1103. The parsing terminal verifies the audio watermark.

[0315] In this embodiment, as a preferred implementation, the first sequence embedded in the audio watermark includes a check bit. The parsing terminal verifies the integrity of the first sequence based on this check bit to ensure the accuracy of watermark parsing. For example, as illustrated above, the first sequence is "11101". The playback terminal and the parsing terminal agree to use parity checking, with the last digit in the first sequence being the check bit. When the last digit is 1, it means that the first sequence contains an odd number of digits 1, excluding the check bit. Thus, after parsing the first sequence, the parsing terminal determines whether the number of digits 1 in the parsed first sequence is odd based on the last digit, thereby determining whether the parsing of the first sequence is complete.

[0316] 1104. When the parsing terminal has completed parsing the entire first audio, the parsing terminal determines the audio watermark with the highest repetition rate from the audio watermarks parsed in multiple watermark detection cycles as the watermark of the first audio.

[0317] In this embodiment, the first audio includes multiple sets of first target frames and second target frames. Each first target frame and second target frame constitute a watermark detection period, and the same audio watermark is embedded in each watermark detection period. However, in the actual parsing process, the parsing terminal may encounter some parsing errors, resulting in not all watermark detection periods obtaining the same sequence of audio watermarks. Through experimental observation, when parsing errors occur, the incorrect audio watermarks obtained by misparsing are always random and non-repeating. Therefore, the audio watermark with the highest repetition rate among the audio watermarks parsed in multiple watermark detection periods can be determined as the correct audio watermark. Therefore, by using this multi-period decision-making method, combined with the aforementioned parity check method, the correct watermark embedded in the first audio can be accurately parsed, further preventing misparsing by the parsing terminal.

[0318] Optionally, once the parsing terminal determines that the first sequence of data has been transmitted completely, it can perform a number system conversion on the first sequence as needed, such as converting the first sequence from binary to decimal, ultimately obtaining the number 14. This completes the verification of the audio watermark in the first audio file.

[0319] 603. The parsing terminal determines the playback terminal based on the audio watermark.

[0320] In this embodiment, since the audio watermark is associated with the playback terminal, the playback terminal that added the audio watermark to the current audio segment can be determined based on the audio watermark parsed from the first audio. For example, in a real-time conference scenario, the playback terminals in meeting rooms A, B, and C play the exact same audio content, but each playback terminal embeds a different audio watermark when playing the same audio. The parsing terminal can determine which meeting room's playback terminal played the current first audio by parsing the audio watermark, thus achieving the tracing of the first audio's source.

[0321] It should be noted that the audio watermarking method and audio watermark parsing method provided in this application embodiment can be used in various different application scenarios. For ease of understanding, the application scenarios of the methods provided in this application embodiment are described below with reference to the accompanying drawings.

[0322] I. Remote conferencing scenario.

[0323] The architecture in this embodiment is as follows: Figure 13 As shown, in Figure 1Based on the architecture shown, an additional service management center (SMC) is added. Perform the following steps.

[0324] 1301. SMC sends the audio watermark to the playback terminal.

[0325] In this embodiment, there can be multiple playback terminals. The audio watermark sent by SMC corresponds one-to-one with each playback terminal and is used to uniquely identify each playback terminal.

[0326] 1302. The playback terminal stores the audio watermark locally.

[0327] In this embodiment, after obtaining the audio watermark, each playback terminal stores the audio watermark locally so that it can be embedded into the audio stream when it obtains the real-time audio stream.

[0328] 1303. The media center sends the first audio to the playback terminal.

[0329] In this embodiment, the audio stream of the media center can be generated by the playback terminal of a certain venue and sent to the MCU of the media center. Then, the media center sends the audio stream to the playback terminals of other venues in real time.

[0330] 1304. The playback terminal embeds an audio watermark in the first audio file in real time.

[0331] In this embodiment, the playback terminal embeds the locally stored audio watermark into the first audio in real time using the audio watermarking method provided in any of the above embodiments. For details, please refer to the above description; further elaboration will not be repeated here.

[0332] 1305. The playback terminal plays the first audio.

[0333] In this embodiment, the first audio played by the playback terminal is embedded with an audio watermark. The entire watermark embedding process is carried out in real time and will not affect the live broadcast effect of the first audio. At the same time, the first audio played can be traced back to the playback terminal of the first audio based on the audio watermark.

[0334] 1306. Parse the terminal to obtain the first audio.

[0335] In this embodiment, the parsing terminal can obtain the first audio through a digital channel or through an air channel. The parsing terminal can parse the first audio transcribed in either of these two ways.

[0336] 1307. The parsing terminal parses the audio watermark from the first audio.

[0337] In this embodiment, the parsing terminal parses the audio watermark from the first audio using the audio parsing and adding method provided in any of the above embodiments. For details, please refer to the above description; further elaboration will not be repeated here.

[0338] 1308. The parsing terminal determines the playback terminal based on the audio watermark.

[0339] In this embodiment, since the audio watermark is associated with the playback terminal, the playback terminal that added the audio watermark to the current audio segment can be determined based on the audio watermark parsed from the first audio. This enables the tracing of the source of the first audio.

[0340] In this embodiment, the SMC assigns different venue identifiers (i.e., audio watermarks) to the playback terminals of venues A, B, and C, respectively. These venue identifiers are associated with the playback terminals of each venue and are used to uniquely identify each playback terminal. After the playback terminals of venues A, B, and C obtain the live audio stream from the media center MCU, they execute the audio watermarking method provided in this embodiment, adding their respective venue identifiers as watermarks to the playback audio stream in real time, so that the audio played by the playback terminals of each venue is embedded with the audio watermark. Based on the audio watermark in the transcribed video, the parsing terminal can determine which venue's audience transcribed the audio through the venue identifier recorded in the watermark information of the audio stream. This achieves the tracing of the audio watermark's origin.

[0341] II. Cloud-based video-on-demand scenario.

[0342] In this embodiment, please refer to Figure 14 ,like Figure 14 As shown, users send on-demand information to the cloud server via their terminals, requesting the audio or video content they wish to watch. The cloud service then sends the requested content to the user's terminal in real time based on the user's request. The specific steps are as follows.

[0343] 1401. The user terminal generates on-demand information based on the content selected by the user.

[0344] In this embodiment, the user selects the audio or video content to be played through the interactive interface of the user terminal and generates play information, which is used to record the audio or video content played by the user.

[0345] 1402. The user terminal sends the on-demand information to the cloud server.

[0346] In this embodiment, the user terminal sends the on-demand information to the cloud server so that the cloud server knows the content that the user wants to watch.

[0347] 1403. The cloud server obtains the target content requested by the user based on the on-demand information.

[0348] In this embodiment, the cloud server retrieves the target content requested by the user from the database based on the user's on-demand information.

[0349] 1404. The cloud server generates an audio watermark based on the user's terminal identifier.

[0350] In this embodiment, the user terminal is an audio or video playback terminal, and the audio watermark is associated with the user terminal to uniquely identify it. This enables the acquisition of the audio watermark.

[0351] 1405. The cloud server embeds the audio watermark into the target content.

[0352] In this embodiment, the target content can be audio or video. If it is video, the audio watermark is embedded in the audio content of the video. The specific method by which the cloud server embeds the audio watermark into the target content can be any of the audio watermarking methods provided in this application embodiment, as detailed in the foregoing description. It should be noted that in this case, the watermark embedding step can be performed by the cloud server, or the cloud server can send the audio watermark to the user terminal for the user terminal to perform. This application embodiment does not limit this aspect. Through the method provided in this application embodiment, the cloud server can transmit the target content to the user terminal while embedding the audio watermark in the audio of the target content in real time, thereby improving work efficiency.

[0353] 1406. The cloud server sends the watermark content to the user terminal.

[0354] In this embodiment, the audio or video content in the watermark is the content requested by the user on the user terminal, and an audio watermark has been added to the watermark content.

[0355] 1407. The user terminal plays the watermarked content.

[0356] In this embodiment, since an audio watermark has been added to the content played on the user terminal, if the user of the user terminal then copies the content played on the terminal, the copied content will retain the audio watermark, thus allowing the user terminal that copied the content to be traced back to the user terminal.

[0357] 1408. Parse the terminal to obtain the watermark content.

[0358] In this embodiment, the watermark content can be audio with an embedded watermark, or video with audio. The audio in the video has an embedded watermark. The parsing terminal can obtain the watermark content through a digital channel or through an air channel. The parsing terminal can parse the first audio in the watermark content transcribed by these two methods.

[0359] 1409. The parsing terminal parses the audio watermark from the first audio.

[0360] In this embodiment, the parsing terminal parses the audio watermark from the first audio using the audio parsing and adding method provided in any of the above embodiments. For details, please refer to the above description; further elaboration will not be repeated here.

[0361] 1410. The parsing terminal determines the user terminal based on the audio watermark.

[0362] In this embodiment, since the audio watermark is associated with the user terminal, the user terminal that added the audio watermark to the current audio segment can be identified based on the audio watermark parsed from the first audio segment. This enables the tracing of the source of the first audio.

[0363] In summary, the watermarking method and watermark parsing method provided in this application can be applied to various scenarios with audio watermarking and parsing requirements. The above two methods are just examples and do not constitute a limitation on the application scenarios of this application.

[0364] From a hardware structure perspective, the above method can be implemented by a single physical device, multiple physical devices working together, or a logical functional module within a single physical device. This application does not specifically limit this aspect.

[0365] For example, the above method can be used Figure 15 This is achieved through electronic devices. Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application; the electronic device may be a playback terminal or parsing terminal in the embodiments of this invention, and the electronic device includes at least one processor 1501, a communication line 1502, a memory 1503 and at least one communication interface 1504.

[0366] The processor 1501 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (server IC), or one or more integrated circuits used to control the execution of the program of the present application.

[0367] Communication line 1502 may include a path for transmitting information between the aforementioned components.

[0368] Communication interface 1504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0369] The memory 1503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via communication line 1502. The memory may also be integrated with the processor.

[0370] The memory 1503 stores computer execution instructions for implementing the scheme of this application, and the processor 1501 controls the execution. The processor 1501 executes the computer execution instructions stored in the memory 1503, thereby implementing the billing management method provided in the following embodiments of this application.

[0371] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0372] In a specific implementation, as one example, the processor 1501 may include one or more CPUs, for example... Figure 15 CPU0 and CPU1 in the CPU.

[0373] In a specific implementation, as one example, an electronic device may include multiple processors, for example... Figure 15Processors 1501 and 1505 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0374] In a specific implementation, as one embodiment, the electronic device may further include an output device 1505 and an input device 1506. The output device 1505 communicates with the processor 1501 and can display information in various ways. For example, the output device 1505 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device 1506 communicates with the processor 1501 and can receive user input in various ways. For example, the input device 1506 may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0375] The aforementioned electronic device can be a general-purpose device or a special-purpose device. In specific implementations, the electronic device can be a server, a wireless terminal device, an embedded device, or something else. Figure 15 Devices with similar structures. The embodiments of this application do not limit the type of electronic device.

[0376] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0377] For example, when dividing the functional units using an integrated approach. Figure 16 A schematic diagram of the structure of a playback terminal provided in an embodiment of this application is shown.

[0378] like Figure 16 As shown, the playback terminal provided in this application embodiment includes...

[0379] Acquisition unit 1601 is used to acquire the first audio in real time;

[0380] Execution unit 1602 is configured to embed an audio watermark in the first audio obtained by acquisition unit 1601, wherein the audio watermark is associated with the playback terminal;

[0381] The playback unit 1603 is used to play the first audio that is embedded with the audio watermark by the execution unit 1602.

[0382] Optionally, the execution unit 1602 is further configured to:

[0383] In the first audio, determine the first target frame that meets the first preset condition;

[0384] A second target frame that meets the second preset condition is determined after the first target frame, and the first target frame is used to mark the second target frame;

[0385] The audio watermark is embedded in the second target frame.

[0386] Optionally, the execution unit 1602 is further configured to:

[0387] When the sampling rate of the first audio is greater than or equal to the first threshold, the audio frame whose maximum low-frequency value is within the first interval is determined as the first target frame; or,

[0388] When the sampling rate of the first audio is less than the first threshold, the audio frame containing the first characteristic sound is determined as the first target frame.

[0389] Optionally, when the sampling rate of the first audio is greater than or equal to the first threshold, the execution unit 1602 is further configured to:

[0390] Add a synchronization frame marker to the first target frame.

[0391] Optionally, the execution unit 1602 is further configured to:

[0392] Obtain the first sampling point, which is the sampling point of the intermediate frequency part;

[0393] Increase the energy value of the first sampling point so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the second threshold.

[0394] Optionally, when the sampling rate of the first audio is less than the first threshold, the execution unit 1602 is further configured to:

[0395] When the first characteristic sound is detected and the duration of the first characteristic sound is greater than or equal to a preset time, the audio frame containing the first characteristic sound is determined as the first target frame.

[0396] Optionally, the execution unit 1602 is further configured to:

[0397] The target frame whose intermediate frequency energy value is greater than or equal to the third threshold and less than the fourth threshold is identified as the second target frame.

[0398] Optionally, the execution unit 1602 is further configured to:

[0399] Obtain the first sequence corresponding to the audio watermark, wherein the first sequence includes at least one element;

[0400] Obtain at least one second sampling point from the third target frame;

[0401] At least one element of the first sequence is embedded into the at least one second sampling point, wherein one element of the first sequence corresponds to one second sampling point.

[0402] Optionally, the execution unit 1602 is further configured to:

[0403] Adjust the energy ratio of the energy values ​​of the second sampling point in different time domains and / or different frequency domains, wherein the energy ratio of one of the second sampling points is associated with an element of the first sequence.

[0404] like Figure 17 As shown, the parsing terminal provided in this application embodiment includes...

[0405] The acquisition unit 1701 is used to acquire a first audio, which contains an audio watermark. The audio watermark is associated with a playback terminal, and the playback terminal is used to embed the audio watermark into the first audio in real time.

[0406] The parsing unit 1702 is used to parse the audio watermark from the first audio obtained by the acquisition unit 1701;

[0407] The execution unit 1703 is used to determine the playback terminal based on the audio watermark parsed by the parsing unit 1702.

[0408] Optionally, the parsing unit 1702 is also used for:

[0409] Identify the first target frame in the first audio that satisfies the first preset condition;

[0410] A second target frame that meets the second preset condition is determined after the first target frame;

[0411] The audio watermark is parsed from the second target frame.

[0412] Optionally, when the sampling rate of the first audio is less than a first threshold, the parsing unit 1702 is further configured to:

[0413] The first target frame is determined from the first audio that contains a first characteristic sound and the duration of the first characteristic sound is greater than or equal to a preset time.

[0414] Optionally, when the sampling rate of the first audio is greater than or equal to the first threshold, the parsing unit 1702 is further configured to:

[0415] The first ratio of the energy values ​​of the mid-frequency part to the low-frequency part of the first audio is obtained frame by frame;

[0416] When an initial target frame with the first ratio greater than or equal to the second threshold is obtained, the first audio is detected by sliding a window backward from the initial target frame to obtain the second ratio of the energy values ​​of the mid-frequency part and the low-frequency part in each sliding window.

[0417] The frame containing the sliding window with the largest second ratio is taken as the first target frame.

[0418] Optionally, the first target frame includes a synchronization frame marker, and the parsing unit 1702 is further used for:

[0419] The first sampling point with the highest energy value in the intermediate frequency part is obtained from the sliding window with the largest second ratio.

[0420] Obtain the third sampling point at a preset distance from the first sampling point;

[0421] The portion of the ratio of the energy value of the first sampling point to the energy value of the third sampling point that is greater than the seventh threshold is identified as the synchronization frame marker;

[0422] The frame containing the sliding window with the largest second ratio is determined based on the synchronization frame marker as the first target frame.

[0423] Optionally, the parsing unit 1702 is also used for:

[0424] Starting from the first target frame, move backward frame by frame and obtain candidate target frames whose energy in the intermediate frequency part of each frame is greater than or equal to the third threshold and less than the fourth threshold.

[0425] The second target frame is the target frame whose energy ratio of different time domain and / or different frequency domain energy values ​​is greater than or equal to the fifth threshold.

[0426] Optionally, the parsing unit 1702 is also used for:

[0427] A second sampling point is obtained from the second target frame, wherein the second sampling point is a sampling point in the second target frame where the energy ratio is greater than or equal to the fifth threshold;

[0428] Obtain the energy ratio of different time-domain and / or different frequency-domain portions of the energy values ​​at the second sampling point;

[0429] Obtain the first element associated with the energy ratio, which is an element in the first sequence recorded by the audio watermark.

[0430] Optionally, the first audio includes multiple watermark detection cycles, wherein each watermark detection cycle resolves one audio watermark, and the resolution unit 1702 is further configured to:

[0431] The audio watermark with the highest repetition rate is determined from the multiple watermark detection cycles and used as the watermark for the first audio.

[0432] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0433] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention.

[0434] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0435] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0436] In the several embodiments provided in this application, it should be understood that the disclosed communication methods, relay devices, host base stations, and computer storage media can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0437] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0438] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0439] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0440] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for adding audio watermarks, characterized in that, include: The playback terminal acquires the first audio in real time; The playback terminal embeds an audio watermark in the first audio, and the audio watermark is associated with the playback terminal. The playback terminal plays the first audio file containing the audio watermark; The playback terminal embeds an audio watermark in the first audio, including: The playback terminal determines a first target frame in the first audio that meets a first preset condition. The first target frame that meets the first preset condition is an audio frame in which the maximum value of the low frequency part is within a first interval when the sampling rate of the first audio is greater than or equal to a first threshold, or an audio frame in which the sampling rate of the first audio is less than the first threshold and contains a first characteristic sound. The playback terminal determines a second target frame that meets the second preset condition after the first target frame, and the first target frame is used to mark the second target frame; The playback terminal embeds the audio watermark in the second target frame.

2. The method according to claim 1, characterized in that, The playback terminal determines a first target frame in the first audio that meets a first preset condition, including: When the sampling rate of the first audio is greater than or equal to the first threshold, the playback terminal determines the audio frame whose maximum low-frequency value is within the first interval as the first target frame; or... When the sampling rate of the first audio is less than the first threshold, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame.

3. The method according to claim 1, characterized in that, When the sampling rate of the first audio is greater than or equal to the first threshold, after the playback terminal takes the audio frame whose maximum low-frequency value is within the first interval as the first target frame, it further includes: The playback terminal adds a synchronization frame marker to the first target frame.

4. The method according to claim 1, characterized in that, The playback terminal adds a synchronization frame marker to the first target frame, including: The playback terminal acquires a first sampling point, which is a sampling point of the intermediate frequency part; The playback terminal increases the energy value of the first sampling point so that the ratio of the energy value of the first sampling point to the energy value of the low-frequency part is greater than or equal to the second threshold.

5. The method according to claim 1, characterized in that, When the sampling rate of the first audio is less than the first threshold, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame, including: When the first characteristic sound is detected and the duration of the first characteristic sound is greater than or equal to a preset time, the playback terminal determines the audio frame containing the first characteristic sound as the first target frame.

6. The method according to any one of claims 1 to 5, characterized in that, The playback terminal determines a second target frame that meets a second preset condition after the first target frame, including: The playback terminal determines the target frame whose mid-frequency energy value is greater than or equal to the third threshold and less than the fourth threshold as the second target frame.

7. The method according to any one of claims 1 to 5, characterized in that, The playback terminal embeds the audio watermark in the third target frame, including: The playback terminal obtains a first sequence corresponding to the audio watermark, wherein the first sequence includes at least one element; The playback terminal obtains at least one second sampling point from the third target frame; The playback terminal embeds at least one element from the first sequence into the at least one second sampling point, wherein one element in the first sequence corresponds to one second sampling point.

8. The method according to claim 7, characterized in that, The playback terminal adds at least one element from the first sequence to the at least one second sampling point, including: The playback terminal adjusts the energy ratio of the second sampling point in different time domains and / or different frequency domains, wherein the energy ratio of one of the second sampling points is associated with an element in the first sequence.

9. An audio watermark parsing method, characterized in that, include: The parsing terminal obtains a first audio, which contains an audio watermark. The audio watermark is associated with a playback terminal, which is used to embed the audio watermark into the first audio in real time. The parsing terminal parses the audio watermark from the first audio; The parsing terminal determines the playback terminal based on the audio watermark; Before the parsing terminal parses the audio watermark from the first audio, it also includes: The parsing terminal determines a first target frame in the first audio that meets a first preset condition. The first target frame that meets the first preset condition is an audio frame in which the maximum value of the low frequency part is within a first interval when the sampling rate of the first audio is greater than or equal to a first threshold, or an audio frame in which the sampling rate of the first audio is less than the first threshold and contains a first characteristic sound. The parsing terminal determines a second target frame that meets the second preset condition after the first target frame; The parsing terminal parses the audio watermark from the first audio, including: The parsing terminal parses the audio watermark from the second target frame.

10. The method according to claim 9, characterized in that, When the sampling rate of the first audio is less than a first threshold, the parsing terminal determines a first target frame in the first audio that meets a first preset condition, including: The parsing terminal determines from the first audio a target frame that contains a first characteristic sound and whose duration is greater than or equal to a preset time as the first target frame.

11. The method according to claim 9, characterized in that, When the sampling rate of the first audio is greater than or equal to a first threshold, the parsing terminal determines a first target frame in the first audio that satisfies a first preset condition, including: The parsing terminal obtains the first ratio of the energy values ​​of the mid-frequency part to the low-frequency part of the first audio frame by frame; When the parsing terminal obtains an initial target frame where the first ratio is greater than or equal to the second threshold, it slides backward from the initial target frame to detect the first audio in a sliding window manner, so as to obtain the second ratio of the energy values ​​of the mid-frequency part and the low-frequency part in each sliding window; The parsing terminal identifies the frame containing the sliding window with the largest second ratio as the first target frame.

12. The method according to claim 11, characterized in that, The first target frame includes a synchronization frame marker. The parsing terminal obtains the frame containing the sliding window with the largest second ratio as the first target frame, including: The parsing terminal obtains the first sampling point with the highest energy value in the intermediate frequency part from the sliding window with the largest second ratio. The parsing terminal acquires a third sampling point at a preset length prior to the first sampling point; The parsing terminal determines that the portion of the ratio of the energy value of the first sampling point to the energy value of the third sampling point that is greater than the seventh threshold is the synchronization frame marker; The parsing terminal determines the frame containing the sliding window with the largest second ratio as the first target frame based on the synchronization frame marker.

13. The method according to any one of claims 9 to 12, characterized in that, The parsing terminal determines a second target frame that meets the second preset condition after the first target frame, including: The parsing terminal moves backward frame by frame from the first target frame, and obtains candidate target frames whose energy in the intermediate frequency part of each frame is greater than or equal to the third threshold and less than the fourth threshold. The parsing terminal selects the target frame from the candidate target frames whose energy ratio of different time domain and / or different frequency domain energy values ​​is greater than or equal to the fifth threshold as the second target frame.

14. The method according to claim 13, characterized in that, The parsing terminal parses the audio watermark from the second target frame, including: The parsing terminal obtains a second sampling point from the second target frame. The second sampling point is a sampling point in the second target frame where the energy ratio is greater than or equal to the fifth threshold. The parsing terminal respectively obtains the energy ratio of the energy values ​​of different time domain and / or different frequency domain portions in the second sampling point; The parsing terminal obtains a first element associated with the energy ratio, the first element being an element in a first sequence recorded by the audio watermark.

15. The method according to any one of claims 9 to 12, characterized in that, The method includes multiple watermark detection cycles, wherein each watermark detection cycle parses one audio watermark, and the method further includes: The parsing terminal determines the audio watermark with the highest repetition rate from the audio watermarks parsed in the plurality of watermark detection cycles as the watermark of the first audio.

16. A playback terminal, characterized in that, include: The acquisition unit is used to acquire the first audio in real time; An execution unit is configured to embed an audio watermark in the first audio obtained by the acquisition unit, the audio watermark being associated with the playback terminal; A playback terminal is used to play the first audio file, which is embedded with the audio watermark by the execution unit; The execution unit is specifically used for: In the first audio, a first target frame that meets the first preset condition is determined. The first target frame that meets the first preset condition is an audio frame in which the maximum value of the low frequency part is within a first interval when the sampling rate of the first audio is greater than or equal to a first threshold, or an audio frame that contains a first characteristic sound when the sampling rate of the first audio is less than the first threshold. A second target frame that meets the second preset condition is determined after the first target frame, and the first target frame is used to mark the second target frame; The audio watermark is embedded in the second target frame.

17. A parsing terminal, characterized in that, include: An acquisition unit is used to acquire a first audio, which contains an audio watermark. The audio watermark is associated with a playback terminal, and the playback terminal is used to embed the audio watermark into the first audio in real time. A parsing unit is used to parse the audio watermark from the first audio obtained by the acquisition unit; An execution unit is used to determine the playback terminal based on the audio watermark parsed by the parsing unit; The parsing unit is also used for: A first target frame in the first audio that satisfies a first preset condition is identified. The first target frame that satisfies the first preset condition is an audio frame in which the maximum value of the low frequency part is within a first interval when the sampling rate of the first audio is greater than or equal to a first threshold, or an audio frame containing a first characteristic sound when the sampling rate of the first audio is less than the first threshold. A second target frame that meets the second preset condition is determined after the first target frame; The audio watermark is parsed from the second target frame.

18. An electronic device, characterized in that, The electronic device includes: an interactive device, an input / output (I / O) interface, a processor, and a memory, wherein the memory stores program instructions; The interactive device is used to acquire user input commands; The processor is configured to execute program instructions stored in the memory to perform the method as described in any one of claims 1 to 8 or 9 to 15.

19. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a computer device, the computer device causes the computer device to perform the method as described in any one of claims 1 to 8 or 9 to 15.

Images