A multi-path fusion audio copyright tracing annotation method

Abstract

The application belongs to the technical field of audio signal processing, and specifically discloses a multi-path fusion audio copyright tracing labeling method, which comprises the following steps: performing frequency segmentation on original audio, embedding the same sequence through two independent paths of frequency domain masking site amplitude fine tuning and wavelet domain detail component sign modulation of a copyright modulation sequence, adopting double-path parallel extraction and cross comparison error correction when extracting, and even if one path is damaged, the other path can still completely recover the labeling information, so that the anti-interference ability is improved without sacrificing the auditory quality, and at the same time, the copyright tracing information is split into a core field which cannot be modified after being written for the first time and an incremental field which can be added piece by piece, so that the audio can record the source information and circulation time of each node in the propagation process, completely restore the full-link propagation path of the audio from the creation source to each propagation node, and thus the defects that the existing technology cannot trace the source of the propagation node and the circulation path due to information solidification are solved.

Description

Technical Field

[0001] This invention belongs to the field of audio signal processing technology, and specifically discloses a multi-path fusion method for audio copyright traceability and labeling. Background Technology

[0002] With the popularization of audio digitization technology, the issue of copyright protection for audio content such as music, audiobooks, and podcasts has become increasingly prominent. In particular, in recent years, the maturity of artificial intelligence-generated speech technology has significantly lowered the threshold for audio tampering, pirated distribution, and unauthorized secondary creation, making it more difficult to track audio copyright infringement.

[0003] To address the aforementioned issues, digital audio watermarking has become a primary means of audio copyright protection. By covertly embedding copyright information into the audio carrier, it enables copyright declaration and traceability of infringement. For example, Chinese invention patent application CN109344578A proposes an audio watermark embedding and extraction method based on chaos and wavelet transform. This method involves preprocessing the watermark with chaotic encryption and scrambling, then embedding robust and fragile watermarks into the low and high frequencies of the wavelet domain, respectively. During extraction, wavelet decomposition is used to identify the positions of audio elements below a threshold in the low or high frequency components. After chaotic decryption and inverse transform, the embedded watermark is obtained.

[0004] The essence of the above scheme lies in allocating two watermarks with different functions to different frequency bands to achieve copyright protection and attack detection respectively. However, this scheme has the following drawbacks in implementation: First, the two watermarks perform different functions, one for copyright protection and the other for attack detection. They embed different watermark information and cannot replace each other. When the watermark used for copyright protection is attacked and destroyed, the watermark used for attack detection cannot replace it in fulfilling its copyright protection function. The overall anti-attack capability of the system is not improved by the introduction of dual watermarks.

[0005] Second, all copyright information is embedded once when the audio is generated, and no new dissemination nodes can be added during the entire dissemination cycle. When the audio is transferred through multiple platforms or undergoes multiple secondary creations, it is impossible to trace the source and flow path of each dissemination node, making it difficult to locate the source of infringement. Summary of the Invention

[0006] To solve the above-mentioned technical problems, or at least partially solve them, this invention provides a multi-path fusion method for audio copyright tracing and labeling.

[0007] The objective of this invention can be achieved through the following technical solution: a multi-path fusion audio copyright traceability and labeling method, comprising the following steps: frequency segmenting of the original audio to obtain perceptible frequency bands and imperceptible frequency bands, extracting read-only features from the perceptible frequency bands and generating encryption keys;

[0008] The copyright traceability information is split into core fields and incremental fields, which are then redundantly encoded and encrypted to obtain a modulation sequence.

[0009] A dual-path embedding method is adopted: the first path embeds the modulation sequence of the core field at the frequency domain masking point through amplitude fine-tuning; the second path performs discrete wavelet transform on the audio, selects high-frequency detail components, and embeds the same core field modulation sequence through symbol modulation; at the same time, reserved points are set in both paths for subsequent incremental fields.

[0010] Incremental fields are appended to the designated reserved sites. During the append process, the validity of the original core fields is first verified, and then the embedding is performed only on the reserved sites.

[0011] The audio synchronization to be detected is extracted in two paths. Through cross-comparison and verification, the propagation link is restored by parsing the incremental field according to the timestamp.

[0012] Combining all the above technical solutions, the positive effects of this invention are as follows: 1. By splitting copyright traceability information into core fields that cannot be modified after the first writing and incremental fields that can be added one by one, this invention enables the source information and circulation time of each node in the audio transmission process to be dynamically recorded, and the addition operation does not destroy the existing annotations. Finally, by parsing the incremental fields in the order of timestamps, the entire transmission path of the audio from the source of creation to each transmission node can be completely restored, thereby effectively solving the defect of the existing technology that cannot trace the source and circulation path of the transmission node due to information solidification.

[0013] 2. This invention segments the original audio by frequency and embeds the copyright modulation sequence into an identical sequence through two independent paths: frequency domain masking point amplitude fine-tuning and wavelet domain detail component symbol modulation. The two paths are redundant backups for each other. During extraction, parallel extraction and cross-comparison error correction are adopted. Even if one path is damaged, the other path can still completely recover the annotation information. This improves the anti-interference ability without sacrificing the auditory quality and overcomes the shortcomings of existing technologies where multiple watermarks cannot be replaced and the overall anti-attack ability is insufficient. Attached Figure Description

[0014] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0015] Figure 1 This is a diagram illustrating the implementation steps of the method of the present invention;

[0016] Figure 2 This is a flowchart illustrating the implementation of the first path in this invention;

[0017] Figure 3 This is a flowchart illustrating the implementation of the second path in this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] After audio is created, due to the widespread use of editing tools, malicious tampering, pirated distribution, and unauthorized secondary creation are frequent occurrences, making it difficult to confirm the original copyright information and trace the source of infringement. To address this issue, existing technologies typically employ digital audio watermarking to covertly embed copyright information into the audio carrier. However, traditional watermarking schemes are mostly one-time embeddings, unable to add new node records during audio transmission, leading to breaks in the transmission chain. Although some improved solutions attempt to embed two watermarks, one for copyright protection and the other for content authentication, the two watermarks carry different information and have independent functions. If one is damaged, the other cannot be replaced, and the overall resistance to attacks has not been improved.

[0020] Therefore, considering the differences in human hearing among different frequency bands in audio, copyright traceability information can be bound and embedded with the original, imperceptible sound features of the audio, so as to achieve reliable retention of annotation information without compromising the audio listening quality.

[0021] Based on the above considerations, this invention proposes a multi-path fusion method for audio copyright tracing and annotation, see [link to relevant documentation]. Figure 1 As shown, the specific steps include: S1, dividing the original audio into frequency segments to obtain perceptible frequency bands and imperceptible frequency bands, extracting read-only features from the perceptible frequency bands and generating encryption keys;

[0022] The auditory quality of audio is mainly determined by the energy distribution, fundamental frequency, and formant structure in the low to mid frequency range, while the high frequency and infrasound frequency ranges contribute very little to human hearing. If the embedding intensity of traditional watermarks is too high, it will damage the sound quality; if it is too low, it will be difficult to resist compression, transcoding, editing, mixing, and other processing.

[0023] Based on this, the present invention performs frequency segmentation on the original audio and separates the frequency bands that affect sound quality from the redundant frequency bands that can carry information according to the differences in the contribution of different frequency bands to human auditory recognition, thereby providing a safe carrier for subsequent embedding without damaging the auditory quality.

[0024] S11. The original audio is divided into low-frequency band, mid-frequency band, high-frequency band and infrasound band by a bandpass filter bank.

[0025] Among them, the low frequency range, such as 20Hz–2kHz, and the mid frequency range, such as 2kHz–16kHz, carry the fundamental frequency, resonant peaks, and main energy. They are the core ranges for human ear to perceive sound quality and are therefore marked as perceptible frequency ranges.

[0026] High-frequency bands, such as 16kHz–20kHz, and infrasound bands, such as <20Hz, have extremely low sensitivity to human hearing and are therefore marked as imperceptible frequency bands.

[0027] S12. Extract short-time average energy, zero-crossing rate, fundamental frequency candidate value and formant frequency parameters frame by frame within the perceptible frequency band. These parameters together constitute the essential auditory attributes of audio, which are unique and stable. Any modification will significantly change the listening experience. Concatenate these parameters in chronological order into read-only features, keeping the original values ​​unchanged throughout the embedding process, thus ensuring that the audio quality is not affected from the source.

[0028] S13. Perform a hash operation on the read-only features of all frames. The hash operation maps the read-only feature sequence of variable length to a hash value of fixed length, i.e., the hash value, which serves as the encryption key bound to the audio. This process is irreversible, and the differences in read-only features of different audio will lead to significant differences in the hash value. The key generated in this way is strongly bound to the audio content and is used for the encryption protection of core fields during the initial annotation. It cannot be stripped or transferred to other audio, thereby ensuring the security of copyright annotation information.

[0029] Signal components in the imperceptible frequency band contribute very little to human hearing, and even moderate modifications will not cause a perceptible decrease in sound quality. Therefore, they can be used as a carrier for embedding copyright information. Since the infrasound band carries very little information, this solution will not extract or utilize it.

[0030] S2. The copyright traceability information is split into core fields and incremental fields, and redundant encoding and encryption are performed on them respectively to obtain the modulation sequence;

[0031] The aforementioned steps, through frequency segmentation and feature extraction, divide the audio into perceptible frequency bands that affect sound quality and imperceptible frequency bands that can carry information, providing a secure carrier for the subsequent embedding of copyright information. In copyright protection, the most important requirement is to solidify and confirm ownership and dynamically record the dissemination path. Achieving this goal presupposes ensuring that the embedded information has a clear structure, distinct permissions, and strong resistance to interference.

[0032] To this end, the present invention splits the copyright tracing information into two types of fields with different read and write attributes: Core field: Defined with a fixed length, containing the copyright holder identifier, work identifier, authorization information, and a fixed-length symmetric encryption key randomly generated during the initial annotation. This field is used to uniquely identify the copyright ownership and authorization scope, and provides an independent key for the encryption of subsequent incremental fields. It is written only once during the initial annotation of the audio, and is then marked as read-only and unmodifiable, thereby ensuring the immutability of the initial copyright information.

[0033] Incremental field: Defined with a fixed length, it includes a node identifier (identifying the propagation node), node type (such as distribution platform, secondary creator, end user), and propagation timestamp. This field is appended sequentially according to the order of propagation. Each incremental field is independently encoded and does not overwrite others. It is used to record the source and time information of each propagation node in the entire lifecycle of the audio, realizing full-chain traceability.

[0034] After the data is split, to ensure that these fields can resist interference, be securely stored, and reliably retrieved during audio transmission, further redundant coding, encryption, and modulation mapping are required. Specifically, the process is as follows: 1) Redundant Coding: Redundant coding is performed on the core field data, with the ratio of the output coded block length to the original data length being a preset fixed ratio, such as 4:1. This process, by adding parity bits, enables the coded block to have error correction capabilities; even if some bits are damaged during transmission or processing, the original information can still be recovered through the parity bits. This yields the core coded block. The same proportion of redundant coding is then performed independently on the data of each incremental field to obtain the incremental coded block corresponding to that incremental field.

[0035] 2) Encryption processing: Symmetric encryption is performed on the core coded blocks using an encryption key generated from the read-only features of the perceptible frequency band, with each ciphertext block accompanied by an incrementing block number as an initialization vector;

[0036] Symmetric encryption is performed on each incremental encoded block using the encryption key stored in the core field, with each ciphertext block accompanied by an incrementing block number as an initialization vector;

[0037] The encryption serves to prevent unauthorized parties from directly reading or tampering with the field content. Furthermore, the core field and the incremental field are encrypted using different keys to prevent updates to the incremental field from affecting the decryption security of the core field.

[0038] This yields the core ciphertext sequence and each incremental ciphertext sequence, reflecting the secure encapsulation state of the copyright information after encoding.

[0039] 3) Modulation mapping: Each ciphertext sequence is expanded into a bit stream, and bit 1 is mapped to a positive fixed modulation amplitude, and bit 0 is mapped to a negative fixed modulation amplitude. This mapping process converts the binary ciphertext into a real-valued signal sequence that can be continuously superimposed, which facilitates the subsequent embedding step to physically superimpose it in the frequency domain or wavelet domain, and finally obtains the modulation sequence corresponding to each field.

[0040] S3. A dual-path embedding method is adopted: the first path embeds the modulation sequence of the core field at the frequency domain masking point through amplitude fine-tuning; the second path performs discrete wavelet transform on the audio, selects high-frequency detail components, and embeds the same core field modulation sequence through symbol modulation; at the same time, reserved points are set in both paths for subsequent incremental fields.

[0041] After generating the modulation sequences for the core and incremental fields, the physical embedding of copyright information can be performed. Since the core field represents the initial copyright ownership information, it needs to be embedded during the initial audio annotation; the incremental fields represent the recording information of each propagation node and are appended sequentially during subsequent propagation. Furthermore, considering the limited anti-interference capability of a single embedding path, this invention adopts a dual-path redundant embedding architecture: two paths embed the exact same core field modulation sequence, serving as backups for each other. Even if one path is damaged due to signal processing or attacks, the other path can still fully recover the core field information, thereby improving anti-interference capability without sacrificing auditory quality. Simultaneously, reserved areas physically isolated from the core field sites are pre-defined in both paths, specifically for the subsequent appending of incremental fields.

[0042] See Figure 2 The first path: frequency domain masking point amplitude fine-tuning embedding.

[0043] This approach utilizes the auditory masking effect of the human ear, fine-tuning the amplitude at frequencies where hearing is insensitive to carry bit information. The specific implementation steps are as follows: First, the original audio is processed by frame segmentation. Then, a Fourier transform is performed frame by frame to convert the time-domain signal into a frequency-domain amplitude spectrum, simultaneously obtaining the frequency position corresponding to each frequency point. The purpose of the Fourier transform is to decompose the signal into different frequency components, facilitating the analysis of the energy distribution and masking relationship at each frequency point.

[0044] It should be noted that the Fourier transform here is performed on the frame-by-frame signal of the entire audio segment. Subsequently, the available embedding sites located in the imperceptible frequency band are automatically filtered out by the masking threshold calculation, without the need to process the imperceptible frequency band separately.

[0045] Then, adjacent frequency points are grouped according to critical frequency bands. The energy distribution within each critical frequency band has an approximate masking characteristic for the human ear. The maximum amplitude value within each critical frequency band is found as the strong signal of that frequency band. This strong signal is used to determine the masking threshold of that frequency band and its neighboring frequency bands.

[0046] Among them, the masking threshold is the maximum allowable energy of the frequency point that the human ear just cannot hear. According to the psychoacoustic principle, the masking threshold of the frequency band where the strong signal is located and its adjacent frequency bands is higher, that is, the strong signal will mask the nearby weak signal. It is set to the amplitude of the strong signal minus a preset decibel value, such as 18dB; the masking threshold of other frequency bands is lower, and the pre-stored absolute hearing threshold is taken, that is, the minimum audible sound pressure level of the human ear at different frequencies.

[0047] The actual amplitude of each frequency point is compared with the masking threshold. If the actual amplitude is lower than the masking threshold, it indicates that the frequency point is in the blind spot of human hearing. Modifying its amplitude will not cause a perceptible decrease in sound quality. Therefore, it is marked as a usable embedding site. Since the amplitude of the imperceptible frequency band is usually lower than the masking threshold, the selected usable sites mainly fall within this frequency band range, with the high frequency band being the main one.

[0048] Next, all available embedding sites are sequentially allocated to each bit in the core field modulation sequence in ascending frequency order. This frequency-order allocation ensures uniform embedding and repeatability during extraction. Simultaneously, a set of dedicated sites are designated at higher frequencies, such as above 18 kHz, specifically for subsequent incremental field writing. These sites are physically isolated from the core field sites to avoid interference. No information is written to these dedicated sites in this step; they are merely reserved as markers.

[0049] Finally, for each bit in the core field modulation sequence, the original amplitude of the corresponding frequency point is adjusted by increasing or decreasing the preset step size, thereby superimposing the value in the modulation sequence onto the frequency point amplitude.

[0050] Since the modulation sequence is a real-valued sequence mapped from copyright information after encoding and encryption, where positive and negative values ​​correspond to bits 1 and 0 respectively, and the frequency amplitude itself does not carry any bit information, the copyright information is written into the frequency domain of the audio in the form of amplitude variation patterns by superimposing the modulation sequence onto the frequency amplitude of each available embedding point, thus completing the copyright embedding. During extraction, by detecting the direction of change of the amplitude of each frequency point, such as increase or decrease, the modulation sequence can be reconstructed, and the copyright information can be decoded.

[0051] The specific adjustment method is as follows: If the current bit is 1, corresponding to the positive modulation amplitude, the original amplitude of the corresponding frequency point is increased by a preset step size; if the bit is 0, corresponding to the negative modulation amplitude, the same preset step size is decreased. The preset step size is a fixed small amplitude value, such as 1.5dB, which is preset by the system. The above process is repeated until all bits in the modulation sequence are embedded into the corresponding available sites.

[0052] It should be noted that when a reduction operation is required and the original amplitude value is less than or equal to the preset step size, performing the subtraction operation would result in a negative amplitude value, which is physically meaningless. Therefore, in such cases, the current site is abandoned and the embedding is performed on the next available embedding site, thus ensuring that the adjusted amplitude value is always non-negative.

[0053] Furthermore, in the first frame of every fixed number of frames (e.g., every 10 frames), multiple frequency points at pre-defined fixed frequency positions are selected, and a preset synchronization bit sequence, such as 0x55AA, is embedded. Here, the fixed position refers to a set of frequency point indices pre-agreed by the system, which remain consistent between the embedding and extraction ends. The synchronization header is used to locate the frame start position and embedding point during extraction, resisting frame loss or clipping and ensuring the synchronization of the extraction.

[0054] See Figure 3 The second path: symbol modulation embedding of detail components in the wavelet transform domain.

[0055] This approach leverages the multi-resolution characteristics of discrete wavelet transform to superimpose the modulation sequence of the core field onto the high-frequency detail components using symbolic modulation, achieving a redundant backup embedding with the frequency domain path. Simultaneously, the third-layer detail components are reserved as a dedicated embedding region for the incremental field. Since the human ear is insensitive to minute changes in the high-frequency detail components, this approach can carry copyright information without compromising auditory quality.

[0056] The specific implementation steps are as follows: Step 1: Perform a three-level discrete wavelet transform on the entire audio time-domain signal. Through step-by-step decomposition, obtain the first-level detail component, the second-level detail component, the third-level detail component, and the lowest-level approximation component. Among them, the detail component corresponds to the high-frequency part of the signal, and the approximation component corresponds to the low-frequency main part;

[0057] Step 2: Select the first and second layer detail components as the embedding carriers of the core field, because these two layers of detail components contain high-frequency detail information of the audio, and the human ear is not sensitive to the amplitude fine-tuning. Calculate the maximum absolute value of the amplitude of each of these two layers of detail components, which reflects the overall energy range of the signal of that layer. Set the modulation step size of each component to: the maximum absolute value of the amplitude of that layer divided by a preset fixed divisor, and then take the integer part. The modulation step size determines the magnitude of the change in the detail component value during symbol modulation. The preset divisor (e.g., 4 or 8) is used to control the amount of change within the range of auditory tolerance.

[0058] Step 3: Embed the modulation sequence corresponding to the core field into the sequence points of the first and second layer detail components through symbol modulation;

[0059] For each component point, the symbol modulation rule is adopted: if the original component value is positive, the modulation step size is added or subtracted: when the modulation sequence value is +Δ, the modulation step size is increased, and when it is -Δ, the modulation step size is decreased.

[0060] If the original component value is negative, the opposite operation is performed: when the modulation sequence value is +Δ, the modulation step size is reduced to decrease the absolute value of the negative value; when it is -Δ, the modulation step size is increased to increase the absolute value of the negative value.

[0061] This rule ensures that the modified component value maintains the same sign as the original value, avoiding perceptible abrupt changes due to sign reversal;

[0062] If the modification causes the symbol to flip unexpectedly, then use a smaller modulation step size, such as halving it each time, and remodulate until the symbol remains unchanged;

[0063] Step 4: Since the center frequency of the third layer detail component is relatively low, the human ear is relatively sensitive to it, and there is a certain risk of auditory perception. However, the number of times the incremental field is appended is limited, and by using a sufficiently small modulation step size, the risk of auditory perception can be controlled within an acceptable range. Therefore, this layer can be used as a reserved embedding carrier for incremental fields that need to be appended multiple times.

[0064] The third layer detail component is physically isolated from the first and second layer detail components of the core field. Its modulation step size is fixed at the maximum absolute value of the amplitude of this layer divided by another preset divisor. This divisor is usually greater than the divisor of the core field to ensure the detectability of incremental information.

[0065] In this step, no information is written to the third layer detail component; only the available area is defined. When additional propagation nodes are needed later, the modulation sequence of each increment field will occupy a continuous segment of the third layer detail component in the order of addition, that is, a continuous sequence of points. Several zero-value points are reserved between segments as separators to distinguish different increment nodes during extraction.

[0066] Step 5: Perform inverse wavelet transform on the modified first and second layer detail components, the unmodified third layer detail components, and the bottommost approximate components to reconstruct the time-domain audio signal. At this point, the core field modulation sequence has been completely embedded into the high-frequency detail components in the wavelet domain through symbol modulation, completing the copyright embedding of the second path.

[0067] S4. Perform incremental field appending in the designated reserved positions. When appending, first verify the validity of the original core fields, and then only perform embedding on the reserved positions.

[0068] After the core fields of the dual-path method are embedded as described above, the audio already carries unalterable initial copyright information. During subsequent propagation, whenever the audio passes through a new propagation node, the recorded information of that node needs to be appended to the audio to form a complete propagation chain. At this point, the incremental field embedding operation needs to be performed in the reserved positions defined in step S3.

[0069] As a specific implementation of the present invention, the appending of incremental fields is performed according to the following process: First, the embedded core fields are extracted from the audio to be annotated. In addition to the copyright holder's identifier, work identifier, and authorization information, the core fields also contain a fixed-length symmetric encryption key randomly generated during the initial annotation. This key is independent of the audio content and does not change due to audio signal processing. The hash key generated from the audio-perceptible features in S1 is only used for the encryption protection of the core fields during the initial annotation and does not participate in the encryption of subsequent incremental fields.

[0070] Next, the authorization information in the core fields is read, including the authorization type and validity period, and the current append operation is verified to ensure it is within the authorized scope and has not exceeded the validity period. If the verification fails, the append process is terminated.

[0071] After verification, the new propagation node information is structured and packaged into a fixed length, and redundantly encoded. Then, it is encrypted using the symmetric encryption key in the core field to obtain the modulation sequence of the new variable field.

[0072] Find idle embedding sites in the reserved high-frequency band in the frequency domain and write new modulation sequences in sequence; in the reserved blank continuous segments of the third layer detail components in the wavelet domain, add new modulation sequences in sequence, with each incremental segment separated by a zero point.

[0073] After writing is complete, all fields, including core fields and all incremental fields, are extracted from the audio again. The integrity of each field and the continuity of the timestamp order of each incremental field are verified. If the verification passes, the newly annotated audio is output; if it fails, the writing operation is rolled back to keep the audio as is.

[0074] S5. Perform dual-path extraction on the audio synchronization to be detected, and restore the propagation link by parsing the incremental field according to the timestamp through cross-comparison and verification error correction.

[0075] After the incremental field is appended, the audio carries complete copyright information and dissemination records. When it is necessary to trace and verify the source of unknown audio, this information needs to be extracted and restored from the audio. Considering that dual-path embedding may cause one of the paths to be damaged due to signal processing, this invention adopts a synchronous extraction, cross-comparison and error correction mechanism to ensure the reliability of information recovery.

[0076] As a specific implementation of the present invention, step S5 is performed according to the following process:

[0077] S51, Dual-path parallel extraction

[0078] S511. Perform the same frequency segmentation, framing, Fourier transform, and multi-level discrete wavelet transform preprocessing on the audio to be detected as on the embedding end, so that the subsequent extraction operation is aligned with the embedding end.

[0079] S512. Search for periodically occurring synchronization header sequences frame by frame in the frequency domain path, lock the frame synchronization start position, locate the embedding point of each frame, read the current amplitude value of each embedding point, and compare it with the reference amplitude, wherein the reference amplitude is the original amplitude estimate obtained by interpolation or median estimation of the amplitude value of the unmodified frequency point (such as the frequency point occupied by the synchronization header or the adjacent frequency point of the non-embedded area) near the point.

[0080] Bit interpretation is performed based on the difference between the current amplitude and the reference amplitude: if the difference is positive and the absolute value is greater than the preset judgment threshold, it is interpreted as bit 1;

[0081] If the difference is negative and its absolute value is greater than the preset judgment threshold, it is interpreted as bit 0;

[0082] If the absolute value of the difference is less than or equal to the preset judgment threshold, it is interpreted as invalid, indicating that the site may not be embedded or may have been damaged.

[0083] The preset judgment threshold is set by the system in advance, for example, 50% of the modulation step size. The purpose of setting the judgment threshold is to avoid misjudgment caused by background noise or small amplitude fluctuations and improve the reliability of extraction.

[0084] The first core field bit sequence is obtained through the above method.

[0085] Then, based on the pre-agreed frequency domain location, such as a specific frequency range with a frequency higher than 18kHz, the dedicated site reserved for the incremental field is located. For each dedicated site, the reference amplitude estimation method and bit interpretation rules that are exactly the same as those for the core field are used to read the bit values ​​of each dedicated site in sequence to obtain the bit sequence of the first incremental field. This sequence consists of multiple incremental field segments arranged in chronological order, and each segment is separated by a preset idle site pattern, such as multiple consecutive zero values ​​or a specific bit sequence.

[0086] For gaps caused by frame loss or site damage, the redundant information of the preceding and following frames is used to fill the gaps, such as copies of the same bit that are repeatedly embedded in adjacent frames. The purpose of this step is to restore the bit sequence embedded in the frequency domain path.

[0087] S513. In the wavelet domain path, read the current value of each component point from the first and second layer detail components, and compare it with the reference value estimated by the unmodified approximate component or the adjacent unmodified detail component. Using the same bit interpretation rule as S512, obtain the bit sequence of the second core field.

[0088] For the third-level detail components, the zero-value point separator markers are searched to locate the boundaries of each increment segment. Within each increment segment, the bit stream is read using the same reference value estimation method and interpretation rules to obtain the bit sequence of the second increment field.

[0089] The above operations yield two core field bit sequences and two corresponding incremental field bit sequences extracted from the frequency domain path and wavelet domain path, which serve as the basis for subsequent cross-matching.

[0090] S52, Cross-comparison and Verification / Error Correction

[0091] S521, Cross-validation of core fields

[0092] The core field bit sequence extracted from the frequency domain path is denoted as the first sequence, and the core field bit sequence extracted from the wavelet domain path is denoted as the second sequence.

[0093] Compare the corresponding bits one by one: if the two are the same, directly use the bit value;

[0094] If a sequence of bits is determined to be valid (meaning that the bit was not marked as missing or incorrect during the extraction process), and another sequence is determined to be invalid, then the bit value of the valid sequence is used.

[0095] If both are valid but have different values, locate the redundant coding block where the bit is located, use the set of parity bits in the coding block to locate the error, for example, calculate the error pattern through the parity equation, determine the location where the error occurs, and flip the bit value at that location to achieve correction;

[0096] After completing the bit-by-bit cross-checking, the corrected core field bit sequence is obtained. This sequence is then redundantly decoded to obtain the plaintext core field, which reflects the initial copyright information of the audio and is completely consistent with the information embedded. When appending incremental fields in subsequent steps, the core field relied upon by the verification process should be based on the corrected plaintext from this step.

[0097] S522, Cross-validation of Incremental Fields

[0098] For each incremental field, the frequency domain path extraction result and the wavelet domain path extraction result are cross-compared and corrected segment by segment in the same way as the core field to obtain the corrected incremental field bit sequence. Then, redundant decoding is performed to obtain the incremental field plaintext, which reflects the audio recording information at each propagation node.

[0099] S53, Reconstructing the Propagation Link

[0100] Sort all incremental plaintext fields in ascending order of their recorded timestamps, extract the node identifier, node type code, and propagation timestamp from each field, and output the propagation node list in the sorted order.

[0101] This list fully reflects the entire transmission trajectory of audio from its source of creation to the current detection node, enabling the restoration and tracing of the transmission path.

[0102] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0103] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-path fused audio copyright provenance annotation method, characterized in that, Includes the following steps: The original audio is segmented by frequency to obtain perceptible and imperceptible frequency bands. Read-only features are extracted from the perceptible frequency bands and an encryption key is generated. The copyright traceability information is split into core fields and incremental fields, which are then redundantly encoded and encrypted to obtain a modulation sequence. A dual-path embedding method is adopted: the first path embeds the modulation sequence of the core field at the frequency domain masking point through amplitude fine-tuning; the second path performs discrete wavelet transform on the audio, selects high-frequency detail components, and embeds the same core field modulation sequence through symbol modulation; at the same time, reserved points are set in both paths for subsequent incremental fields. Incremental fields are appended to the designated reserved sites. During the append process, the validity of the original core fields is first verified, and then the embedding is performed only on the reserved sites. The audio synchronization to be detected is extracted in two paths. Through cross-comparison and verification, the propagation link is restored by parsing the incremental field according to the timestamp.

2. The multi-path fused audio provenance annotation method of claim 1, wherein: The process of frequency segmenting the original audio to obtain perceptible and imperceptible frequency bands, extracting read-only features from the perceptible frequency bands, and generating an encryption key includes the following: The original audio is divided into low-frequency band, mid-frequency band, high-frequency band and infrasound band. The low-frequency band and mid-frequency band are marked as perceptible frequency bands, and the high-frequency band and infrasound band are marked as imperceptible frequency bands. Within the perceptible frequency band, short-time average energy, zero-crossing rate, fundamental frequency candidate value and formant frequency parameters are extracted frame by frame, and these parameters are spliced ​​together in time order to form read-only features. Perform a hash operation on the read-only characteristics of all frames and output the hash value as an encryption key bound to the audio.

3. The multi-path fusion audio copyright tracing and annotation method as described in claim 1, characterized in that: The core fields and incremental fields are split according to the following process: Define core fields, including copyright holder identifier, work identifier, authorization information and a fixed-length symmetric encryption key, which are written only once during the initial annotation and are subsequently marked as read-only and unmodifiable; Define incremental fields, which include node identifier, node type, and propagation timestamp. These fields are added sequentially in the order they occur during propagation. Each incremental field is independently encoded and does not overwrite others.

4. The multi-path fusion audio copyright tracing and annotation method as described in claim 3, characterized in that: The modulation sequence is generated according to the following process: Redundant encoding is performed on the data of the core field and each incremental field to obtain the core encoding block and each incremental encoding block; Symmetric encryption is performed on the core coding block using an encryption key generated with read-only features of the perceptible frequency band. Symmetric encryption is performed on each incremental coded block using the encryption key stored in the core field; Each ciphertext block is accompanied by an incrementing block number as an initialization vector, resulting in the core ciphertext sequence and each incremental ciphertext sequence; Each ciphertext sequence is expanded bit by bit into a bit stream, and bit 1 is mapped to a positive fixed modulation amplitude, and bit 0 is mapped to a negative fixed modulation amplitude, thus obtaining the modulation sequence.

5. The multi-path fusion audio copyright tracing and annotation method as described in claim 1, characterized in that: The first path is implemented according to the following procedure: Perform a Fourier transform on each frame of the original audio to obtain the frequency domain amplitude spectrum and the frequency position corresponding to each frequency point; Adjacent frequency points are grouped according to critical frequency bands, and the maximum amplitude value within each critical frequency band is taken as the strong signal. The masking threshold for each frequency point is determined according to preset rules: the masking threshold for the frequency band where the strong signal is located and its adjacent frequency bands is the amplitude of the strong signal minus the preset decibel value, and the masking threshold for other frequency bands is the pre-stored absolute hearing threshold. The actual amplitude of each frequency point is compared with the masking threshold of that frequency point. If the actual amplitude is lower than the masking threshold, the frequency point is marked as a usable embedding site. Each bit in the core field modulation sequence is sequentially allocated from all available embedding sites in frequency order, and dedicated sites are designated in the high-frequency band for use by the incremental field. For each allocated bit, the original amplitude of the corresponding frequency point is adjusted by increasing or decreasing the preset step size, thereby superimposing the value in the modulation sequence onto the frequency point amplitude, and confirming that the adjusted amplitude is still lower than the masking threshold. Otherwise, it is carried over to the next available embedding site until all bits in the core field modulation sequence are embedded into the corresponding available embedding sites.

6. The multi-path fusion audio copyright tracing and annotation method as described in claim 1, characterized in that: The second path is implemented according to the following procedure: A three-level discrete wavelet transform is performed on the entire audio time-domain signal to obtain the first, second, and third level detail components and the lowest level approximate component; The first and second layer detail components are selected as the embedding carriers for the core fields; The modulation sequence corresponding to the core field is embedded into the sequence points of the first and second layer detail components through symbol modulation; The third-layer detail component is used as a reserved embedding carrier for the incremental field. The modulation sequence of each incremental field occupies a continuous segment of the third-layer detail component, and the segments are separated by zero points. The modified first and second layer detail components, along with the unmodified third layer detail components and the lowest layer approximation components, are subjected to inverse wavelet transform to reconstruct the time-domain audio signal.

7. The multi-path fusion audio copyright tracing and annotation method as described in claim 3, characterized in that: The appending to the incremental field is performed according to the following procedure: For audio with additional annotations, the authorization information is extracted from the core fields for verification, and the symmetric encryption key is obtained from the core fields. After successful verification, the new propagation node information is packaged, encoded, and then encrypted using the symmetric encryption key in the core field and mapped to a new modulation sequence. Find available sites within the reserved high-frequency band in the frequency domain and write the new modulation sequence sequentially. New modulation sequences are sequentially appended into the blank continuous segments reserved for the third layer of detail components in the wavelet domain, with each segment separated by a zero point; After writing is complete, all fields are extracted again, and the integrity and timestamp order are verified. Once verified, the newly annotated audio is output.

8. The multi-path fusion audio copyright tracing and annotation method as described in claim 1, characterized in that: The dual-path extraction includes the following steps: The audio to be detected undergoes the same frequency segmentation, framing, Fourier transform, and wavelet transform preprocessing as the embedded end. Search the synchronization header sequence in the frequency domain path to lock the frame start, locate the embedding point, read the amplitude value to judge the bit, fill the missing position with the redundant information of the previous and next frames to obtain the core field bit sequence, and read the amplitude value of the incremental field according to the preset frequency domain reserved position to obtain the incremental field bit sequence. In the wavelet domain path, bits are read from the first and second layer detail components to obtain the core field bit sequence. Zero-value point separators are searched for in the third layer detail components to locate the boundaries of each incremental segment and read the bit stream to obtain the incremental field bit sequence.

9. The multi-path fusion audio copyright tracing and annotation method as described in claim 1, characterized in that: The cross-alignment and error correction are performed according to the following process: Compare the core field bit sequences extracted from the frequency domain path and the wavelet domain path bit by bit: If the two are the same, then the bit value is used directly; If the bits of one sequence are determined to be valid and the bits of the other sequence are determined to be invalid, then the bit values ​​of the valid sequence shall be used. If both are deemed valid but have different values, the redundant coding block containing the bit is located, and the error location is determined and the bit value is corrected using the check bits in the coding block. After completing the bit-by-bit cross-comparison and correction, the plaintext of the core fields is obtained; Perform the same cross-comparison and error correction on each incremental field segment by segment to obtain the plaintext of each incremental field.

10. The multi-path fusion audio copyright tracing and annotation method as described in claim 9, characterized in that: The restoration propagation link includes the following process: Sort all incremental plaintext fields in ascending order by record timestamp, extract the node identifier, node type, and propagation timestamp of each field in sequence, and output the propagation node list in order.

Images