Audio watermark embedding method and device, computer device and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses an audio watermark embedding method and device, computer equipment and a storage medium, which comprise the following steps: obtaining original audio data, performing feature extraction on the original audio data to obtain original audio features; performing watermark generation on the original audio features according to a preset watermark type to obtain a watermark disturbance factor; performing watermark embedding on the original audio features based on the watermark disturbance factor to obtain initial audio features; performing watermark detection on the initial audio features to obtain an audio watermark detection confidence; and generating target audio data containing a watermark based on the audio watermark detection confidence and the initial audio features. The application can be applied to the audio watermark processing scene of financial technology and medical technology, and the reliability of audio is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and is applicable to the fields of fintech and medical technology, and particularly to an audio watermark embedding method, apparatus, computer equipment and storage medium. Background Technology

[0002] Audio watermarking embedding technology can be used to integrate specific identification information (such as copyright owner information, content authentication codes, and tamper-proof marks) into audio signals to ensure the traceability and reliability of the audio signals. Audio watermarking embedding technology can be applied to multiple scenarios. In fintech scenarios, it can be used to embed watermarks into audio such as online customer authentication voice messages and recordings of customer calls with financial customer service. In healthcare technology scenarios, it can be used to embed watermarks into audio such as remote consultation voice messages from medical personnel and diagnostic audio from monitoring devices.

[0003] Currently, audio watermarking embedding technology mainly adopts a single type of embedding method (such as time domain perturbation, frequency domain perturbation, phase perturbation, spread spectrum, etc.) to integrate the identification information as a watermark into a specific audio signal. However, in actual use scenarios, the watermark information is easily lost after the audio signal is subjected to attacks such as audio reconstruction, which affects the reliability of the audio. Summary of the Invention

[0004] This application provides an audio watermark embedding method, apparatus, computer device, and storage medium to solve the technical problem that the reliability of audio is affected by the easy loss of watermark information after the audio signal is subjected to attacks such as audio reconstruction.

[0005] Firstly, an audio watermark embedding method is provided, including: Obtain raw audio data, and extract features from the raw audio data to obtain raw audio features; Watermark generation is performed on the original audio features according to the preset watermark type to obtain the watermark perturbation factor; The original audio features are watermarked based on the watermark perturbation factor to obtain the initial audio features. Watermark detection is performed on the initial audio features to obtain the audio watermark detection confidence level; Based on the audio watermark detection confidence level and the initial audio features, target audio data containing the watermark is generated.

[0006] Secondly, an audio watermark embedding device is provided, comprising: The audio data acquisition module is used to acquire raw audio data and extract features from the raw audio data to obtain raw audio features. The watermark generation module is used to generate a watermark for the original audio features according to a preset watermark type, and obtain a watermark perturbation factor. The watermark embedding module is used to embed a watermark into the original audio features based on the watermark perturbation factor to obtain initial audio features. The watermark detection module is used to perform watermark detection on the initial audio features and obtain the audio watermark detection confidence level. An audio generation module is used to generate target audio data containing a watermark based on the audio watermark detection confidence level and the initial audio features.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described audio watermark embedding method.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described audio watermark embedding method.

[0009] The aforementioned audio watermark embedding method, apparatus, computer equipment, and storage medium can be applied to audio watermarking processing scenarios in fintech and medical technology. First, the original audio data is acquired, and features are extracted to obtain original audio features. This shifts the watermarking operation from the time domain to the feature domain, helping to reduce the complexity of watermark embedding. Next, a watermark is generated from the original audio features according to a preset watermark type, resulting in a watermark perturbation factor. Based on this perturbation factor, the original audio features are then embedded to obtain initial audio features, achieving more accurate and controllable watermark embedding and effectively avoiding perceptible distortion caused by direct superposition on the waveform. Furthermore, watermark detection is performed on the initial audio features to obtain an audio watermark detection confidence level, enabling precise quantitative evaluation of the watermark embedding quality. Finally, based on the audio watermark detection confidence and initial audio features, target audio data containing the watermark is generated, realizing adaptive adjustment based on the detection results. While ensuring the detectability of the watermark, perceptual distortion is minimized, thus balancing robustness and imperceptibility. This improves the reliability of audio watermark embedding and solves the technical problem that the reliability of audio is affected by the easy loss of watermark information after audio signal is subjected to attacks such as audio reconstruction. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for an audio watermark embedding method in one embodiment of this application; Figure 2 This is a flowchart illustrating an audio watermark embedding method in one embodiment of this application; Figure 3 yes Figure 2 A schematic diagram of a specific implementation method for step S20; Figure 4 yes Figure 2 A schematic diagram of a specific implementation method for step S30; Figure 5 yes Figure 2 A flowchart illustrating another specific implementation of step S30; Figure 6 yes Figure 2 A flowchart illustrating another specific implementation of step S30; Figure 7 yes Figure 2 A schematic diagram of a specific implementation of step S40; Figure 8 This is a schematic diagram of the structure of an audio watermark embedding device in one embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device according to one embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] First, let's analyze some of the terms used in this application: Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0014] Audio watermarking embedding is a digital signal processing method that aims to integrate specific identification information (such as copyright owner information, content authentication codes, and anti-tampering marks) into audio signals. Audio watermarking embedding techniques typically perform minute amplitude or phase adjustments on audio samples in the time domain, frequency domain, or transform domain (such as discrete cosine transform and wavelet transform), while utilizing auditory masking effects to ensure that the auditory quality of the embedded audio remains essentially unchanged. Watermarking embedding requires balancing three core indicators: robustness (the ability to resist attacks such as MP3 compression, filtering, resampling, and neural codecs), imperceptibility (transparency), and information capacity. During detection, corresponding extraction algorithms are used to recover the watermark information from potentially attacked or manipulated audio. It is widely used in scenarios such as copyright protection, content tracing, broadcast monitoring, and authentication of generative audio.

[0015] Audio watermarking embedding technology can be applied to multiple scenarios. In fintech, it can be used to embed watermarks into audio recordings such as online customer verification voice messages and call recordings between customers and financial customer service representatives. In healthcare, it can be used to embed watermarks into audio recordings such as remote consultations by medical staff and diagnostic audio from monitoring devices.

[0016] Currently, audio watermarking embedding technology mainly adopts a single type of embedding method (such as time domain perturbation, frequency domain perturbation, phase perturbation, spread spectrum, etc.) to integrate the identification information as a watermark into a specific audio signal. However, in actual use scenarios, the watermark information is easily lost after the audio signal is subjected to attacks such as audio reconstruction or neural network attacks, which affects the reliability of the audio.

[0017] Based on this, embodiments of this application provide an audio watermark embedding method, apparatus, computer device, and storage medium to solve the technical problem that the reliability of audio is affected by the easy loss of watermark information after the audio signal is subjected to attacks such as audio reconstruction.

[0018] The audio watermark embedding method, apparatus, computer device, and storage medium provided in this application are specifically described through the following embodiments. First, the audio watermark embedding method in this application is described.

[0019] The audio watermark embedding method provided in this application embodiment can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can obtain the original audio data from the client, extract features from the original audio data to obtain the original audio features; generate a watermark from the original audio features according to a preset watermark type to obtain a watermark perturbation factor; embed the watermark into the original audio features based on the watermark perturbation factor to obtain the initial audio features; perform watermark detection on the initial audio features to obtain the audio watermark detection confidence level; generate target audio data containing the watermark based on the audio watermark detection confidence level and the initial audio features, and feed the target audio data back to the client.

[0020] In this application, the audio watermarking process, applied to fintech and medical technology scenarios, first acquires the original audio data, extracts features from the original audio data to obtain original audio features, shifting the watermarking operation from the time domain to the feature domain, which helps reduce the complexity of watermark embedding. Next, watermark generation is performed on the original audio features according to a preset watermark type, obtaining a watermark perturbation factor. Based on the watermark perturbation factor, the original audio features are then embedded with the watermark to obtain initial audio features, achieving more accurate and controllable watermark embedding and effectively avoiding perceptible distortion caused by direct superposition on the waveform. Furthermore, watermark detection is performed on the initial audio features to obtain an audio watermark detection confidence level, which can accurately quantify and evaluate the watermark embedding quality. Finally, target audio data containing the watermark is generated based on the audio watermark detection confidence level and the initial audio features, achieving adaptive adjustment based on the detection results. This ensures watermark detectability while minimizing perceptible distortion, thus balancing robustness and imperceptibility, improving the reliability of audio watermark embedding, and solving the technical problem that audio reliability is affected by the easy loss of watermark information after audio signal attacks such as audio reconstruction.

[0021] The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The following detailed description uses specific embodiments to illustrate this application.

[0022] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the audio watermark embedding method provided in this application embodiment includes the following steps: S10. Obtain the raw audio data, extract features from the raw audio data, and obtain the raw audio features; Raw audio data is audio signal without any watermarking. In fintech scenarios, raw audio data can be recordings of customer calls with bank customer service or voice messages from customers providing identity information during online verification. In healthcare scenarios, raw audio data can be voice messages from remote consultations between doctors and patients or diagnostic audio generated by monitoring devices.

[0023] It should be noted that feature extraction in step S10 refers to using methods such as Mel-frequency cepstral coefficients (MFCC), Fourier transform (FFT), and linear frequency cepstral coefficients (LFCC) to transform the original audio data from a time-domain waveform to a representation space more suitable for watermarking operations. The resulting high-dimensional feature vector is the original audio feature. Feature extraction can reduce the computational complexity of subsequent watermark embedding, and the original audio features are more robust to common attacks (such as MP3 compression and noise superposition) because the features themselves have already compressed redundant information.

[0024] S20. Generate a watermark for the original audio features according to the preset watermark type to obtain the watermark perturbation factor. It is important to understand that watermark generation here refers to constructing a perturbation signal for embedding in the feature space using a specific algorithm based on the characteristics of a preset watermark type. This perturbation factor represents the amount of modification to the original audio features after the watermark is embedded.

[0025] It should be noted that preset watermark types are predefined watermark categories, including: Low-frequency watermarking embeds watermark information into the low-frequency region of the audio signal. This is because the embedded watermark perturbation factors are easily masked by low-frequency components. At the same time, low-frequency components are more likely to be preserved and survive in audio compression (such as MP3 compression), thereby improving the robustness of the watermark.

[0026] Mid-to-high frequency (MHF) watermarking directly embeds the watermark into the amplitude, echo timbre, or time coordinate of the sampling points on the time-domain waveform of the original audio. MHF watermarks are embedded in the mid-to-high frequency region of the audio signal because they often involve introducing small-amplitude energy changes over a short period, which are closer to mid-to-high frequency noise in the frequency domain. Compared to the low-frequency region, the human ear is relatively less sensitive to changes in the mid-to-high frequency region, and the mid-to-high frequency components are more likely to remain relatively intact during bandpass filtering attacks.

[0027] By setting different preset watermark types (low-frequency watermark / medium-high frequency watermark), low-frequency watermarks and medium-high frequency watermarks complement each other, and this complementarity can optimize the embedding strategy for different business needs.

[0028] In some embodiments, in addition to low-frequency watermarks and medium-to-high-frequency watermarks, the preset watermark types can also include new types of watermarks according to actual business scenarios, which are not limited here.

[0029] Among them, such as Figure 3 As shown, step S20, which involves generating a watermark from the original audio features based on a preset watermark type to obtain a watermark perturbation factor, includes the following steps: S21. Determine the watermark embedding rules according to the preset watermark type; S22. Predict the watermark based on the watermark embedding rules to obtain the audio watermark embedding features. S23. Calculate the difference between the audio watermark embedding features and the original audio features to obtain the watermark perturbation factor.

[0030] It should be noted that the watermark embedding rules are preset watermark generation strategies corresponding to the watermark type. Each watermark embedding rule includes parameters such as embedding domain, embedding position, embedding strength, and modulation method, so as to flexibly adapt to different watermarking algorithms, avoid the limitations of a single rule, and improve the universality and scene adaptability of audio watermark embedding.

[0031] After clarifying the watermark embedding rules for each preset watermark type, the target feature representation that the audio should have in the feature domain after watermark embedding is predicted based on the watermark embedding rules through a generative model or mapping function, thus obtaining the audio watermark embedding features. It is understandable that by predicting in the feature domain rather than directly superimposing, the distribution of watermark information and audio features is highly consistent, thereby significantly reducing embedding distortion, improving watermark concealment, and providing a clear target reference for subsequent difference calculations.

[0032] Next, the obtained audio watermark embedding features are subtracted dimension-by-dimensionally from the original audio features to obtain the watermark perturbation factor. This perturbation factor represents the minimum feature perturbation required to embed the watermark into the original audio. By calculating the difference, the watermark signal is independently extracted in the form of the watermark perturbation factor, decoupling the watermark from the original audio features. This facilitates its use in different watermark embedding algorithms and improves the accuracy of watermark embedding.

[0033] For the specific calculation process, please refer to formula (1): (1); in, This is the watermark perturbation factor. The original audio features, For watermark embedding rules, This is an audio watermark embedding feature.

[0034] S30. Watermark embedding is performed on the original audio features based on the watermark perturbation factor to obtain the initial audio features; It is important to understand that watermark embedding here refers to fusing the watermark perturbation factor into the original audio features to obtain initial audio features that already contain watermark information but whose embedding quality has not yet been confirmed.

[0035] In some embodiments, the watermark embedding strategy includes, but is not limited to: The Perceptual Adaptive Time-Frequency Multiplexing (PA-TFM) strategy integrates the ideas of frequency division multiplexing and time division multiplexing, and introduces a human auditory masking model to dynamically select the optimal time-frequency embedding region for each watermark. PA-TFM first calculates the time-frequency masking map of the audio, reflecting the sensitivity of the human ear to different time-frequency points. Then, based on the characteristics of different watermarks (such as a preference for low or mid-high frequencies), it routes the perturbation of each watermark to the region with the strongest masking effect and automatically adjusts the local embedding gain. PA-TFM maximizes the robustness of the watermark (selecting the region least susceptible to attack and destruction) while ensuring auditory transparency (hiding the watermark in a place that is not easily detected by the human ear).

[0036] Frequency Division Multiplexing (FDM) divides the entire frequency range of an audio signal into several non-overlapping sub-bands, assigning different watermarks to different sub-bands for independent embedding. In practice, each watermark only adds perturbation within its own frequency band, ensuring that watermarks from different bands do not interfere with each other. This strategy fully utilizes the frequency domain resources of audio, avoids frequency domain conflicts between watermarks, and is particularly suitable for resisting frequency-selective attacks such as MP3 compression and bandpass filtering.

[0037] Time-division multiplexing (TDM) divides the timeline of an audio signal into several consecutive or interleaved time segments, embedding different watermarks into different time segments. Each watermark only adds perturbation within its assigned time window, and watermarks in different time segments do not affect each other. TDM effectively counters temporal attacks such as time clipping, frame dropping, and time stretching because as long as the attack does not destroy all time segments, at least one time segment's watermark may be preserved.

[0038] Parallel multiplexing involves directly weighting and superimposing the perturbation signals generated by multiple independent watermarks on the original audio, and then adding them to the original audio all at once. The core idea of ​​parallel multiplexing is "simultaneous embedding and linear combination," where perturbations from different watermarks exist simultaneously in the time or frequency domain without competing for time or frequency resources. This strategy is simple to implement, requires no training, and preserves the original characteristics of each watermark. However, a drawback is that different perturbations may conflict in the same time-frequency region, causing mutual interference or energy cancellation.

[0039] The serial multiplexing strategy involves embedding multiple watermarks sequentially into audio. First, the first watermark is embedded into the original audio to obtain a watermarked intermediate audio file. This intermediate audio file is then used as input for the second watermark, and so on. This strategy simulates a pipeline-like processing flow. While subsequent watermarks may partially cover or alter the embedding signal of previous watermarks, under certain attacks (such as recoding and filtering), later-embedded watermarks may actually provide additional protection, enhancing overall robustness.

[0040] In some embodiments, a perceptual adaptive time-frequency multiplexing (PA-TFM) strategy is used for watermark embedding, such as... Figure 4 As shown, in step S30, the original audio features are watermarked based on the watermark perturbation factor to obtain the initial audio features, including the following steps: S3011. Perform masking calculations on the original audio features to obtain audio noise masking features; Specifically, masking calculations are performed on the original audio features to calculate the degree of masking to the human ear at each time-frequency point. The greater the noise, the stronger the masking, and the more suitable it is for embedding a watermark, thus obtaining audio noise masking features. These audio noise masking features are time-frequency masking intensity maps of the same size as the original audio features. The larger the value, the more suitable the location is for embedding a watermark without being perceived. This provides a precise embedding area for watermark embedding from a perceptual perspective, avoiding audible distortion caused by embedding in areas sensitive to the human ear (such as mid-frequency unrecorded sounds in quiet segments).

[0041] S3012. Obtain frequency band data of the original audio features; Specifically, the frequency domain features of the original audio features are obtained and frequency domain partitioned to obtain frequency band data, which includes low frequency band, mid frequency band and high frequency band. Subsequent region allocation provides fine-grained information at the frequency domain level, enabling the system to allocate different watermarks to the most matching frequency bands according to their spectral characteristics, thereby achieving frequency division multiplexing.

[0042] For example: the low frequency range can be 0-500Hz, mainly carrying the fundamental frequency and speech energy; the mid frequency range can be 500-2000Hz, mainly carrying the intelligibility of consonants; and the high frequency range can be above 2000Hz, mainly carrying the breath sound and details.

[0043] S3013. Based on frequency band data and audio noise masking features, the watermark perturbation factor is assigned to a region to obtain the masking region of each watermark perturbation factor. After obtaining the frequency band data, based on the frequency band requirements of different preset watermark types and the audio noise masking characteristics, the watermark perturbation factors corresponding to different preset watermark types are assigned to regions to obtain the masking region for each watermark perturbation factor. Specifically, the watermark perturbation factors for low-frequency watermarks are assigned to the corresponding masking regions in the low-frequency band, while the watermark perturbation factors for mid-to-high-frequency watermarks are assigned to the corresponding masking regions in the mid-to-high-frequency bands.

[0044] By allocating regions, time-frequency reuse of multiple watermarks can be achieved, avoiding superposition distortion caused by multiple watermarks being embedded in the same region. At the same time, the imperceptibility and robustness of each watermark can be maximized by matching watermark characteristics with masking features.

[0045] S3014. Watermark embedding is performed on the original audio features based on the masking region and watermark perturbation factor to obtain the initial audio features.

[0046] Specifically, after obtaining the masking region of each watermark perturbation factor, the corresponding watermark perturbation factor is embedded into the original audio features based on each masking region to obtain the initial audio features after watermark embedding. This ensures that the perturbation is concentrated in the area that is not sensitive to the human ear, thereby minimizing auditory perceptibility while ensuring the robustness of multiple watermarks and effectively improving the audio quality after embedding.

[0047] In some embodiments, a frequency division multiplexing (FDM) strategy is used for watermark embedding, such as... Figure 5 As shown, step S30, which involves embedding a watermark into the original audio features based on the watermark perturbation factor to obtain the initial audio features, includes the following steps: S3021. Obtain frequency band data of the original audio features; Specifically, the frequency domain features of the original audio features are obtained and frequency domain partitioned to obtain frequency band data, which includes low frequency band, mid frequency band and high frequency band. Subsequent region allocation provides fine-grained information at the frequency domain level, enabling the system to allocate different watermarks to the most matching frequency bands according to their spectral characteristics, thereby achieving frequency division multiplexing.

[0048] For example: the low frequency range can be 0-500Hz, mainly carrying the fundamental frequency and speech energy; the mid frequency range can be 500-2000Hz, mainly carrying the intelligibility of consonants; and the high frequency range can be above 2000Hz, mainly carrying the breath sound and details.

[0049] It should be noted that the specific implementation of step S3021 is basically the same as the specific implementation of step S3011 described above.

[0050] S3022. Based on the frequency band data, the watermark perturbation factor is allocated to a frequency band to obtain the allocated frequency band region for each watermark perturbation factor. After obtaining the frequency band data, based on the frequency band requirements of different preset watermark types, the watermark perturbation factors corresponding to different preset watermark types are allocated to frequency bands, resulting in the allocated frequency band region for each watermark perturbation factor. Specifically, watermark perturbation factors for low-frequency watermarks are allocated to the low-frequency band region, while watermark perturbation factors for mid-to-high-frequency watermarks are allocated to the mid-to-high-frequency band regions.

[0051] By allocating frequency bands, orthogonal isolation of multiple watermarks in frequency bands can be achieved, with each watermark occupying an independent frequency band. This fundamentally avoids mutual interference and distortion accumulation caused by multiple watermarks overlapping in the same area. At the same time, the optimal frequency band can be matched according to the functional characteristics of each watermark, taking into account both robustness and imperceptibility.

[0052] S3023. Watermark embedding is performed on the original audio features based on the allocated frequency band region and watermark perturbation factor to obtain the initial audio features.

[0053] Specifically, after obtaining the allocated frequency band region for each watermark perturbation factor, the corresponding watermark perturbation factor is embedded into the original audio features based on each allocated frequency band region, resulting in the initial audio features with completed watermark embedding. It is understandable that, since the perturbation of each watermark is strictly limited to its allocated frequency band region, different watermarks do not intrude on each other in the frequency band, resulting in high reliability and effectively improving the capacity and robustness of the audio watermarking system.

[0054] In some embodiments, a time-division multiplexing (TDM) strategy is used for watermark embedding, such as... Figure 6 As shown, step S30, which involves embedding a watermark into the original audio features based on the watermark perturbation factor to obtain the initial audio features, includes the following steps: S3031. Obtain the duration data of the original audio features; It should be noted that the duration data is the total length information of the original audio features in the time dimension, which provides a global time boundary for subsequent time segmentation, ensuring that the allocation scheme covers the entire audio time segment and does not exceed the boundary.

[0055] S3032. Based on the duration data, the watermark perturbation factor is allocated to a time period to obtain the allocation time period for each watermark perturbation factor; Specifically, the data is first segmented based on duration to obtain multiple time periods, each of which does not overlap. Then, the watermark perturbation factor is assigned to different time periods, resulting in an assigned time period for each watermark perturbation factor. This assigned time period characterizes the time interval within which a particular watermark perturbation factor is allowed to be embedded. An indicator function can be used to indicate whether embedding is allowed within that time interval; a value of 1 indicates that embedding is allowed, and a value of 0 indicates that embedding is not allowed.

[0056] Understandably, time-separation allows different watermarks to be completely separated in time periods, fundamentally eliminating the mutual interference caused by the superposition of multiple watermarks, so that each watermark does not affect the others.

[0057] S3033. Watermark embedding is performed on the original audio features based on the allocated time period and watermark perturbation factor to obtain the initial audio features.

[0058] Specifically, after obtaining the allocation time period for each watermark perturbation factor, the corresponding watermark perturbation factor is embedded into the original audio features based on each allocation time period, resulting in the initial audio features with completed watermark embedding. It is understandable that, since the perturbation of each watermark is strictly limited to its allocation time period, different watermarks do not intrude on each other in the frequency band, resulting in high reliability and effectively improving the capacity and robustness of the audio watermarking system.

[0059] In some embodiments, a parallel multiplexing strategy is used for watermark embedding. In step S30, the influence of each watermark perturbation factor on the original audio features is first determined, and the superposition weight of each watermark perturbation factor is obtained. Then, based on the superposition weight, all watermark perturbation factors are weighted and superimposed on the original audio features to obtain the final initial audio features.

[0060] Understandably, since each watermark perturbation factor is directly superimposed on the original audio features, rather than on the output of other watermark perturbation factors, there is no mutual influence between watermarks. Furthermore, by adjusting different superposition weights during weighted superposition, the strength of each watermark can be more precisely controlled.

[0061] In some embodiments, a serial multiplexing strategy is used for watermark embedding. In step S30, all watermark perturbation factors are directly embedded sequentially into the output of the previous watermark of the original audio feature to obtain the initial audio feature.

[0062] Understandably, since the watermark is embedded later, it acts on the audio features that already contain the previous watermark, thus creating a "masking protection" effect on the previously embedded watermark. This makes the watermark harder to detect and remove, and gives it a higher resistance to attacks.

[0063] It should be noted that the five watermark embedding strategies can be selected for use individually or in combination depending on the business scenario, which provides greater flexibility. Furthermore, combining them can further enhance robustness or adapt to complex attack scenarios.

[0064] For example: The serial multiplexing and frequency division multiplexing strategies are used in combination. Specifically, the frequency band data of the original audio features are first acquired. In each frequency band, all watermark perturbation factors are sequentially embedded into the output of the previous watermark in that band. Finally, the signals of each frequency band are added together to recover the full-band audio features, thus obtaining the initial frequency band features. By embedding different watermarks into different frequency bands and serially superimposing them within each band, the combined use of serial multiplexing and frequency division multiplexing strategies can simultaneously resist narrowband filtering attacks and coverage destruction that may occur during serial embedding, significantly improving overall robustness.

[0065] The parallel multiplexing strategy and the time-division multiplexing strategy are used in combination. Specifically, the duration data of the original audio features is first acquired, and then segmented based on the duration data to obtain multiple time periods. Then, for each time period, the influence of each watermark perturbation factor on the audio features of that time period is determined, and the superposition weight of each watermark perturbation factor is obtained. Next, based on the superposition weight, all watermark perturbation factors are weighted and superimposed onto the audio features of that time period to obtain the final audio features of that time period. Finally, the audio features of each time period are concatenated to obtain the initial frequency band features. Even if a time period is truncated or corrupted, the combined use of the parallel multiplexing and time-division multiplexing strategies may still retain at least one watermark in other time periods.

[0066] S40. Perform watermark detection on the initial audio features to obtain the audio watermark detection confidence level; It is important to understand that watermark detection here refers to extracting watermark information from the initial audio after watermark embedding and determining its probability of existence based on the watermark detection algorithm corresponding to the preset watermark type, thereby obtaining the audio watermark detection confidence level.

[0067] Among them, such as Figure 7 As shown, step S40, which involves watermark detection of the initial audio features to obtain the audio watermark detection confidence level, includes the following steps: S41. Perform watermark recognition on the initial audio features to obtain the original watermark recognition score of the preset watermark type. S42. Perform a logarithmic transformation on the original watermark recognition score to obtain the initial watermark recognition score; S43. The initial watermark recognition scores are weighted and fused to obtain the audio watermark detection confidence level.

[0068] Specifically, for each preset watermark type (low-frequency watermark / mid-to-high-frequency watermark), the watermark detection algorithm corresponding to the preset watermark type is invoked to perform watermark detection on the initial audio features, obtaining the original watermark recognition score for each preset watermark type. The original watermark recognition score is the raw probability value output by the watermark detection algorithm, typically between 0 and 1, representing the probability of the watermark of that preset watermark type existing. By performing targeted recognition for specific watermark types (low-frequency watermark / mid-to-high-frequency watermark), quantified raw results are obtained, providing a foundation for subsequent processing.

[0069] After obtaining the original watermark recognition score corresponding to each preset watermark type, a log-odds transformation is performed on the original watermark recognition score corresponding to each preset watermark type to obtain the initial watermark recognition score corresponding to each preset watermark type, thereby mapping the bounded probability value to the entire real number domain.

[0070] Specifically, the calculation process for the initial watermark recognition score is shown in formula (2): (2); in, For the first The original watermark recognition score corresponding to each preset watermark type For the first The initial watermark recognition score corresponds to each preset watermark type.

[0071] After obtaining the initial watermark recognition score for each preset watermark type, the initial watermark recognition scores are weighted and fused based on the weight of each preset watermark type to obtain the final audio watermark detection confidence score. This audio watermark detection confidence score is used to measure the credibility of the presence of a valid watermark in the initial audio features. By combining the detection results of multiple preset watermark types, it can avoid the misjudgment of a single detector and achieve the technical effect of "correct detection as long as any watermark exists," significantly improving the accuracy of watermark detection. This provides data support for audio watermark embedding and is better suited for scenarios with high watermark reliability, such as embedding watermarks into the voice of customers providing identity information during online identity verification in fintech scenarios, and embedding watermarks into diagnostic audio generated by monitoring devices in medical technology scenarios.

[0072] Understandably, as long as any one watermark exists, even if the other watermarks are poorly embedded, it can still ensure that the watermark is successfully embedded in the original audio data, significantly improving the robustness of watermark embedding.

[0073] In addition, the weights of each preset watermark type can be flexibly configured according to different actual business scenarios, which improves the integration of watermark embedding, watermark detection and business scenarios.

[0074] S50. Generate target audio data containing watermark based on audio watermark detection confidence and initial audio features.

[0075] After obtaining the audio watermark detection confidence level of the initial audio features, the value of the audio watermark confidence level data is compared with the preset detection confidence threshold to determine the quality of the watermark embedding. Then, based on the quality, it is determined whether to adaptively adjust the initial audio features and inversely transform them back to the time domain waveform.

[0076] It should be noted that the preset detection confidence threshold is a pre-set judgment threshold value used to compare with the audio watermark detection confidence, thereby determining the subsequent processing path.

[0077] The selection of the preset detection confidence threshold needs to balance the watermark detection rate and false detection rate, and is usually adjusted according to the security level of the actual business scenario. For example, in fintech scenarios, considering the extremely high legal validity requirements of customer identity verification voice messages, the threshold can be set to 0.8 to ensure that the watermark is only considered valid when the confidence level is extremely high, avoiding improper subsequent processing due to misjudgment; in medical technology scenarios, for diagnostic audio from monitoring devices, since medical liability tracing is involved, the threshold can be set to 0.75 to ensure reliability while also considering processing efficiency.

[0078] Understandably, the configurable threshold mechanism enables watermark embedding to adapt to application scenarios with different security levels, balancing flexibility and reliability.

[0079] Specifically, step S50, which generates target audio data containing the watermark based on the audio watermark detection confidence and initial audio features, includes the following processing steps: Processing Operation 1: If the audio watermark detection confidence exceeds the preset detection confidence threshold, filter the initial audio features to obtain the target audio features; Audio conversion is performed based on the target audio features to obtain target audio data containing a watermark.

[0080] Processing Operation 2: If the audio watermark detection confidence level does not exceed the preset detection confidence level threshold, the original audio data is re-embedded with the audio watermark, that is, the above steps S10 to S50 are re-executed to finally obtain the target audio data containing the watermark.

[0081] For processing step one: First, the audio watermark detection confidence level is compared with a preset detection confidence threshold. If the audio watermark detection confidence level exceeds the preset threshold, the initial audio features are filtered to remove noise components or redundant information, retaining the effective feature components related to the watermark, thus obtaining the target audio features. This improves the quality of subsequent audio conversion, making the output audio sound more natural while retaining the watermark. Next, a vocoder (such as HiFi-GAN, WaveGlow, etc.) is used to perform audio conversion on the filtered target audio features, restoring the target audio features to playable waveform audio data. This waveform audio data is the target audio data containing the watermark.

[0082] For example, in fintech scenarios, the target audio data could be call recordings with identity verification features, which can be directly used for subsequent distribution or archiving. In healthcare scenarios, the target audio data could be consultation recordings with doctor's ID watermarks, which could be used for medical liability tracing.

[0083] Understandably, the target audio data simultaneously satisfies both business distribution needs and watermark traceability requirements, achieving a balance between audio content availability and copyright protection.

[0084] For processing operation 2: First, compare the audio watermark detection confidence with the preset detection confidence threshold. If the audio watermark detection confidence does not exceed the preset detection confidence threshold, it means that the watermark in the current audio does not meet the effective standard. At this time, do not process the existing initial audio features, but fall back to the original audio data and re-embed the audio watermark on the original audio data, that is, re-execute the above steps S10 to S50, and finally obtain the target audio data containing the watermark.

[0085] Understandably, processing operation two, through a closed-loop mechanism of "re-embedding if watermark detection fails," fundamentally ensures the validity of the watermark in the final output audio, avoiding the risk of directly outputting audio with unreliable watermarks. Re-executing S10 to S50 means completely re-running the entire process from the initial watermark embedding steps: feature extraction, watermark embedding, fusion, detection, and output, until the final target audio data with satisfactory watermark confidence is obtained. This avoids the cumulative errors that might be introduced by overlaying processing on top of existing failed embeddings, while ensuring the integrity and consistency of the watermark embedding, thus improving the applicability of watermark embedding in scenarios with high reliability requirements.

[0086] As can be seen, in the above scheme applied to audio watermarking scenarios in fintech and medical technology, the original audio data is first acquired, and features are extracted from the original audio data to obtain original audio features. This shifts the watermarking operation from the time domain to the feature domain, which helps reduce the complexity of watermark embedding. Next, watermark generation is performed on the original audio features according to a preset watermark type, resulting in a watermark perturbation factor. Based on the watermark perturbation factor, the original audio features are then embedded with the watermark to obtain initial audio features, achieving more accurate and controllable watermark embedding and effectively avoiding perceptible distortion caused by directly superimposing on the waveform. Furthermore, watermark detection is performed on the initial audio features to obtain the audio watermark detection confidence level, which can accurately quantify and evaluate the watermark embedding quality. Finally, based on the audio watermark detection confidence and initial audio features, target audio data containing the watermark is generated, realizing adaptive adjustment based on the detection results. While ensuring the detectability of the watermark, perceptual distortion is minimized, thus balancing robustness and imperceptibility. This improves the reliability of audio watermark embedding and solves the technical problem that the reliability of audio is affected by the easy loss of watermark information after audio signal is subjected to attacks such as audio reconstruction.

[0087] In addition to its applications in audio watermarking processing in fintech and medical technology, the audio watermarking embedding method provided in this application can also be used in digital copyright protection (such as infringement tracking and copyright authentication of music, podcasts, audiobooks, etc.), multimedia content tracing and anti-tampering (such as authenticity verification of news broadcasts and film and television works), smart device interaction (such as device identification and user command authentication in smart speakers), broadcast advertising monitoring (statistical analysis of ad playback frequency and coverage by embedding watermarks), audio content management of in-vehicle entertainment systems, and secure communication in voice interaction scenarios (such as preventing voice commands from being maliciously forged or replay attacks), among many other fields, demonstrating significant scenario adaptability and practical value.

[0088] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0089] In one embodiment, an audio watermark embedding device is provided, which corresponds one-to-one with the audio watermark embedding method in the above embodiments. For example... Figure 8 As shown, the audio watermark embedding device includes an audio data acquisition module 101, a watermark generation module 102, a watermark embedding module 103, a watermark detection module 104, and an audio generation module 105. Detailed descriptions of each functional module are as follows: The audio data acquisition module 101 is used to acquire raw audio data and extract features from the raw audio data to obtain raw audio features. The watermark generation module 102 is used to generate a watermark from the original audio features according to a preset watermark type, and obtain a watermark perturbation factor. The watermark embedding module 103 is used to embed watermarks into the original audio features based on the watermark perturbation factor to obtain the initial audio features. Watermark detection module 104 is used to perform watermark detection on the initial audio features and obtain the audio watermark detection confidence level; The audio generation module 105 is used to generate target audio data containing a watermark based on the audio watermark detection confidence and initial audio features.

[0090] In one embodiment, the watermark generation module 102 is specifically used for: Determine the watermark embedding rules based on the preset watermark type; Based on the watermark embedding rules, watermark prediction is performed on the original audio features to obtain the audio watermark embedding features; The difference between the embedded audio watermark features and the original audio features is calculated to obtain the watermark perturbation factor.

[0091] In one embodiment, the watermark embedding module 103 is specifically used for: Masking calculations are performed on the original audio features to obtain audio noise masking features; Obtain frequency band data of raw audio features; Based on frequency band data and audio noise masking features, the watermark perturbation factor is assigned a region to obtain the masking region for each watermark perturbation factor. The original audio features are embedded with a watermark based on the masking region and the watermark perturbation factor to obtain the initial audio features.

[0092] In one embodiment, the watermark embedding module 103 is specifically used for: Obtain frequency band data of raw audio features; Based on the frequency band data, the watermark perturbation factor is allocated to a frequency band to obtain the allocated frequency band region for each watermark perturbation factor. The original audio features are watermarked by embedding the watermark based on the allocated frequency band region and the watermark perturbation factor to obtain the initial audio features.

[0093] In one embodiment, the watermark embedding module 103 is specifically used for: Obtain the duration data of the raw audio features; Based on the duration data, the watermark perturbation factor is allocated to a time period to obtain the allocated time period for each watermark perturbation factor; The original audio features are watermarked by the allocation time period and watermark perturbation factor to obtain the initial audio features.

[0094] In one embodiment, the watermark detection module 104 is specifically used for: Watermark recognition is performed on the initial audio features to obtain the original watermark recognition score of the preset watermark type; The original watermark recognition score is logarithmically transformed to obtain the initial watermark recognition score. The initial watermark recognition scores are weighted and fused to obtain the audio watermark detection confidence score.

[0095] In one embodiment, the audio generation module 105 is specifically used for: If the audio watermark detection confidence exceeds the preset detection confidence threshold, the initial audio features are filtered to obtain the target audio features; Audio conversion is performed based on the target audio features to obtain target audio data containing a watermark.

[0096] This application provides an audio watermark embedding device for audio watermark processing scenarios in fintech and medical technology. First, it acquires raw audio data and extracts features from it to obtain raw audio features, shifting the watermarking operation from the time domain to the feature domain, which helps reduce the complexity of watermark embedding. Next, it generates a watermark based on a preset watermark type, obtaining a watermark perturbation factor. Then, it embeds the watermark into the raw audio features based on the perturbation factor, obtaining initial audio features, achieving more accurate and controllable watermark embedding and effectively avoiding perceptible distortion caused by directly superimposing on the waveform. Furthermore, it performs watermark detection on the initial audio features to obtain an audio watermark detection confidence level, enabling precise quantitative evaluation of the watermark embedding quality. Finally, based on the audio watermark detection confidence and initial audio features, target audio data containing the watermark is generated, realizing adaptive adjustment based on the detection results. While ensuring the detectability of the watermark, perceptual distortion is minimized, thus balancing robustness and imperceptibility. This improves the reliability of audio watermark embedding and solves the technical problem that the reliability of audio is affected by the easy loss of watermark information after audio signal is subjected to attacks such as audio reconstruction.

[0097] Specific limitations regarding the audio watermark embedding device can be found in the limitations of the audio watermark embedding method described above, and will not be repeated here. Each module in the aforementioned audio watermark embedding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0098] Please see Figure 9 , Figure 9 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the audio watermark embedding method of the embodiments of this application, including: Obtain the raw audio data, extract features from the raw audio data, and obtain the raw audio features; Watermark generation is performed on the original audio features according to the preset watermark type to obtain the watermark perturbation factor; The original audio features are embedded with a watermark based on the watermark perturbation factor to obtain the initial audio features. Watermark detection is performed on the initial audio features to obtain the audio watermark detection confidence level; Target audio data containing watermarks is generated based on audio watermark detection confidence and initial audio features.

[0099] The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0100] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the raw audio data, extract features from the raw audio data, and obtain the raw audio features; Watermark generation is performed on the original audio features according to the preset watermark type to obtain the watermark perturbation factor; The original audio features are embedded with a watermark based on the watermark perturbation factor to obtain the initial audio features. Watermark detection is performed on the initial audio features to obtain the audio watermark detection confidence level; Target audio data containing watermarks is generated based on audio watermark detection confidence and initial audio features.

[0101] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0104] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0105] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An audio watermark embedding method, characterized in that, The method includes: Obtain raw audio data, and extract features from the raw audio data to obtain raw audio features; Watermark generation is performed on the original audio features according to the preset watermark type to obtain the watermark perturbation factor; The original audio features are watermarked based on the watermark perturbation factor to obtain the initial audio features. Watermark detection is performed on the initial audio features to obtain the audio watermark detection confidence level; Based on the audio watermark detection confidence level and the initial audio features, target audio data containing the watermark is generated.

2. The audio watermark embedding method according to claim 1, characterized in that, The process of embedding a watermark into the original audio features based on the watermark perturbation factor to obtain initial audio features includes: The original audio features are subjected to masking calculations to obtain audio noise masking features; Obtain the frequency band data of the original audio features; Based on the frequency band data and the audio noise masking features, the watermark perturbation factor is assigned a region to obtain the masking region for each watermark perturbation factor. The original audio features are watermarked based on the masking region and the watermark perturbation factor to obtain the initial audio features.

3. The audio watermark embedding method according to claim 1, characterized in that, The process of embedding a watermark into the original audio features based on the watermark perturbation factor to obtain initial audio features includes: Obtain the frequency band data of the original audio features; Based on the frequency band data, the watermark perturbation factor is allocated to a frequency band to obtain the allocated frequency band region for each watermark perturbation factor. The original audio features are watermarked based on the allocated frequency band region and the watermark perturbation factor to obtain the initial audio features.

4. The audio watermark embedding method according to claim 1, characterized in that, The process of embedding a watermark into the original audio features based on the watermark perturbation factor to obtain initial audio features includes: Obtain the duration data of the original audio features; Based on the duration data, the watermark perturbation factor is allocated to a time period to obtain the allocation time period for each watermark perturbation factor; The original audio features are watermarked based on the allocated time period and the watermark perturbation factor to obtain the initial audio features.

5. The audio watermark embedding method according to any one of claims 1 to 4, characterized in that, The step of performing watermark detection on the initial audio features to obtain the audio watermark detection confidence level includes: Perform watermark recognition on the initial audio features to obtain the original watermark recognition score of the preset watermark type; The original watermark recognition score is logarithmically transformed to obtain the initial watermark recognition score. The initial watermark recognition score is weighted and fused to obtain the audio watermark detection confidence level.

6. The audio watermark embedding method according to any one of claims 1 to 4, characterized in that, The step of generating a watermark from the original audio features according to a preset watermark type to obtain a watermark perturbation factor includes: The watermark embedding rules are determined based on the preset watermark type; Based on the watermark embedding rules, watermark prediction is performed on the original audio features to obtain audio watermark embedding features; The difference between the embedded audio watermark feature and the original audio feature is calculated to obtain the watermark perturbation factor.

7. The audio watermark embedding method according to any one of claims 1 to 4, characterized in that, The step of generating target audio data containing a watermark based on the audio watermark detection confidence level and the initial audio features includes: If the audio watermark detection confidence exceeds a preset detection confidence threshold, the initial audio features are filtered to obtain the target audio features; Audio conversion is performed based on the target audio features to obtain the target audio data containing the watermark.

8. An audio watermark embedding device, characterized in that, The device includes: The audio data acquisition module is used to acquire raw audio data and extract features from the raw audio data to obtain raw audio features. The watermark generation module is used to generate a watermark for the original audio features according to a preset watermark type, and obtain a watermark perturbation factor. The watermark embedding module is used to embed a watermark into the original audio features based on the watermark perturbation factor to obtain initial audio features. The watermark detection module is used to perform watermark detection on the initial audio features and obtain the audio watermark detection confidence level. An audio generation module is used to generate target audio data containing a watermark based on the audio watermark detection confidence level and the initial audio features.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the audio watermark embedding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the audio watermark embedding method as described in any one of claims 1 to 7.

Images