Methods and systems for adding and extracting audio hidden watermarks

Abstract

This application provides a method and system for adding and extracting hidden watermarks in audio, including: obtaining a first waveform file corresponding to the audio to be synthesized; generating user data corresponding to the first waveform file; performing frequency domain transformation on the first waveform file to obtain a first Mel spectrogram; adding the user data to the first Mel spectrogram to obtain a second Mel spectrogram containing the hidden watermark; and performing restoration processing on the second Mel spectrogram to obtain a second waveform file containing the hidden watermark. By injecting a hidden watermark into the Mel spectrogram of the synthesized audio, the original user data can be obtained through watermark extraction. This facilitates tracing the source based on the user data, understanding information such as the data producer or user, enhancing the monitoring and management capabilities of disseminated audio content, and providing effective copyright protection for audio content. This avoids the illegal application of synthesized audio content and reduces the harm caused to the speaker by synthesized audio content.

Description

Technical Field

[0001] This application relates to the field of audio synthesis technology, and in particular to a method, system, computer device, and computer-readable storage medium for adding and extracting audio hidden watermarks. Background Technology

[0002] With the rapid development of speech synthesis technology, it has been widely applied in various fields, such as voice interaction (e.g., intelligent assistants or intelligent customer service), audiobook reading, and order announcements (e.g., ride-hailing apps, restaurant queuing systems). However, speech synthesis technology has several drawbacks:

[0003] (1) Speech synthesis technology uses machines to generate audio data instead of humans. It can freely synthesize audio content containing sensitive information and spread it at will, making it difficult to monitor and manage these audios; (2) The data collection of speech synthesis technology comes from real speakers and has a very human-like effect. Under normal circumstances, it is difficult to distinguish whether it is the speaker's audio or not, which may cause harm to the speaker; (3) Sound and music works are protected by copyright, but in the Internet age, it is possible to spread audio by making copies, which is very unfavorable to the copyright protection of audio. Summary of the Invention

[0004] The purpose of this application is to provide a method, system, computer device, and computer-readable storage medium for adding and extracting audio watermarks, in order to solve the following problems: existing solutions have poor monitoring and management capabilities for disseminated audio content, cannot effectively protect the copyright of audio content, and the synthesized audio content may cause harm to the speaker.

[0005] One aspect of this application provides a method for adding a hidden audio watermark, including:

[0006] Obtain the first waveform file corresponding to the audio to be synthesized;

[0007] Generate user data corresponding to the first waveform file;

[0008] The first waveform file is converted into a frequency domain to obtain the first Mel spectrum.

[0009] The user data is added to the first Mel spectrogram to obtain a second Mel spectrogram containing a hidden watermark.

[0010] The second Mel spectrum is restored to obtain a second waveform file containing a hidden watermark.

[0011] Optionally, obtaining the first waveform file corresponding to the audio to be synthesized includes:

[0012] In response to the received speech synthesis command, the first waveform file corresponding to the audio to be synthesized is obtained.

[0013] Optionally, adding the user data to the first Mel spectrogram to obtain a second Mel spectrogram containing a hidden watermark includes:

[0014] Based on a preset hidden watermark generation model, the user data is added to the first Mel spectrogram to obtain the second Mel spectrogram.

[0015] Optionally, after the step of generating user data corresponding to the first waveform file, the method further includes:

[0016] The user data is encrypted.

[0017] Optionally, the user data includes: user ID, digital signature, and authorization period.

[0018] One aspect of this application provides a method for extracting audio hidden watermarks, including:

[0019] Obtain the second waveform file containing the hidden watermark corresponding to the audio to be traced;

[0020] The second waveform file is converted in the frequency domain to obtain a second Mel spectrum containing a hidden watermark;

[0021] Watermark extraction is performed on the second Mel spectrogram to obtain user data corresponding to the second waveform file.

[0022] Optionally, obtaining the second waveform file corresponding to the audio to be traced includes:

[0023] In response to the received audio tracing command, the second waveform file corresponding to the audio to be traced is obtained.

[0024] Optionally, it also includes:

[0025] The user data is then decrypted.

[0026] One aspect of this application provides an audio hidden watermarking system, including:

[0027] The first waveform file acquisition module is used to acquire the first waveform file corresponding to the audio to be synthesized.

[0028] The user data generation module is used to generate user data corresponding to the first waveform file;

[0029] The first waveform file conversion module is used to perform frequency domain conversion on the first waveform file to obtain the first Mel spectrum.

[0030] The hidden watermarking module is used to add the user data to the first Mel spectrogram to obtain a second Mel spectrogram containing the hidden watermark.

[0031] The Mel spectrum conversion module is used to restore the second Mel spectrum to obtain a second waveform file containing a hidden watermark.

[0032] One aspect of this application provides an audio hidden watermark extraction system, comprising:

[0033] The second waveform file acquisition module is used to acquire the second waveform file containing the hidden watermark corresponding to the audio to be traced.

[0034] The second waveform file conversion module is used to perform frequency domain conversion on the second waveform file to obtain a second Mel spectrum containing a hidden watermark.

[0035] The watermark extraction module is used to extract the watermark from the second Mel spectrogram to obtain user data corresponding to the second waveform file.

[0036] One aspect of this application provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for adding and extracting audio hidden watermarks.

[0037] Another aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by at least one processor to implement the steps of the above-described method for adding and extracting audio hidden watermarks when the at least one processor executes the computer program.

[0038] The audio watermarking method, system, device, and computer-readable storage medium provided in this application inject a hidden watermark into the Mel spectrum corresponding to the synthesized audio. Because the audio with the added watermark is indistinguishable from the audio without the added watermark, and even if the audio file is attacked using methods such as downsampling (vertical cropping), white noise, audio cropping (horizontal cropping), or occlusion, the hidden watermark information can still be completely preserved. The original user data can be obtained through watermark extraction, facilitating tracing based on user data, understanding the data producer or user, enhancing the monitoring and management capabilities of disseminated audio content, and providing effective copyright protection for audio content. This prevents the illegal application of synthesized audio content and reduces the harm caused to the speaker by the synthesized audio content. Attached Figure Description

[0039] Figure 1 This diagram illustrates the application environment of the method for adding and extracting audio hidden watermarks according to an embodiment of this application.

[0040] Figure 2 The flowchart illustrating the method for adding an audio hidden watermark according to Embodiment 1 of this application is shown in the illustration.

[0041] Figure 3 The flowchart illustrating the method for adding an audio hidden watermark according to Embodiment 1 of this application is shown in the illustration.

[0042] Figure 4 The flowchart illustrating the audio hidden watermark extraction method according to Embodiment 2 of this application is shown in the schematic diagram.

[0043] Figure 5 The flowchart illustrating the audio hidden watermark extraction method according to Embodiment 2 of this application is shown in the schematic diagram.

[0044] Figure 6 A block diagram illustrating an audio watermarking device according to Embodiment 3 of this application is shown schematically.

[0045] Figure 7 A block diagram of an audio hidden watermark extraction device according to Embodiment 4 of this application is shown schematically.

[0046] Figure 8 The illustration shows a schematic diagram of the hardware architecture of a computer device suitable for implementing the method of adding and extracting audio hidden watermarks according to Embodiment 5 of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0048] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0049] In existing technologies, speech synthesis technology uses machines to generate audio data instead of humans. This allows for the free synthesis of audio content containing sensitive information, which can then be widely disseminated and difficult to regulate. The audio synthesis technology is based on the voice of a real speaker, creating a highly lifelike effect that often makes it difficult to distinguish from the actual speaker's audio. This can potentially harm the speaker; for example, if criminals use the speaker's voice to impersonate them and defraud family, friends, or colleagues, it could pose unforeseen risks. If the speaker is a public figure, it could have a negative impact on society and unpredictable consequences. While audio and musical works are protected by copyright, in the internet age, simply creating copies of audio files allows for widespread distribution, severely hindering copyright protection. Therefore, it is crucial to determine the copyright and source of audio content and to track those who distribute it.

[0050] In view of this, this application aims to propose a method for adding a hidden watermark to audio, which involves: obtaining a first waveform file corresponding to the audio to be synthesized; generating user data corresponding to the first waveform file; performing frequency domain transformation on the first waveform file to obtain a first Mel spectrogram; adding the user data to the first Mel spectrogram to obtain a second Mel spectrogram containing the hidden watermark; and transforming the second Mel spectrogram to obtain a second waveform file containing the hidden watermark. Correspondingly, a method for extracting a hidden watermark from audio is also provided, which involves: obtaining a second waveform file containing the hidden watermark corresponding to the audio to be traced; performing frequency domain transformation on the second waveform file to obtain a second Mel spectrogram containing the hidden watermark; and extracting the watermark from the second Mel spectrogram to obtain user data corresponding to the second waveform file.

[0051] By injecting a hidden watermark into the Mel spectrogram corresponding to the synthesized audio, the audio with the hidden watermark sounds indistinguishable from the audio without it. Even if the audio file is attacked using methods such as downsampling (vertical cropping), white noise, audio cropping (horizontal cropping), or occlusion, the hidden watermark information can still be completely preserved. The original user data can be obtained by extracting the watermark, which facilitates tracing the source of the data and understanding the data producer or user. This enhances the ability to monitor and manage the disseminated audio content and provides effective copyright protection for the audio content, thereby preventing the illegal use of synthesized audio content and reducing the harm caused to the speakers.

[0052] This application provides several embodiments to further illustrate the methods for adding and extracting audio hidden watermarks, as detailed below.

[0053] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0054] The following is a definition of the terminology used in this application:

[0055] Watermark: A semi-transparent logo or icon added to an image to prevent others from stealing it.

[0056] Blind-watermark: refers to a watermark that is not visible and requires certain technical means to become visible.

[0057] Source tracing: Locating and tracking publicly released documents, products, etc., to find the creators, producers, or distributors.

[0058] TTS: Text To Speech, is a technology that uses machine learning to convert input text into corresponding audio readings, typically outputting audio files in formats such as PCM, WAV, MP3, and AAC.

[0059] Digital signature: also known as public key digital signature, is a string of numbers that only the sender of the information can generate and that cannot be forged by others. This string of numbers also serves as valid proof of the authenticity of the information sent by the sender.

[0060] Music copyright refers to the rights that the creators of musical works enjoy in accordance with the law. Music copyright mainly includes property rights such as the right to perform, reproduce, broadcast, and transmit the music online, as well as moral rights such as the right of attribution and the right to protect the integrity of the work.

[0061] Figure 1 A schematic diagram illustrating an environmental application according to an embodiment of this application is provided. Figure 1 As shown:

[0062] Computer device 10000 can connect to client 30000 via network 20000.

[0063] Computer device 10000 can provide services such as network debugging, or return audio watermarking addition and extraction results data to client 30000, etc.

[0064] Computer equipment 10000 may be located in a data center, such as a single site, or distributed across different geographical locations (e.g., multiple sites). Computer equipment 10000 may provide services via one or more networks 20000. Network 20000 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, agent devices, and / or similar devices. Network 20000 may include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, combinations thereof, etc. Network 20000 may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0065] Computer device 10000 can be implemented by one or more compute nodes. One or more compute nodes can include virtualized compute instances. Virtualized compute instances can include virtual machines, such as emulations of computer systems, operating systems, servers, etc. The compute nodes can load virtual machines based on virtual images and / or other data defining specific software (e.g., operating systems, dedicated applications, servers) used for emulation. As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more compute nodes. A hypervisor can be implemented to manage the use of different virtual machines on the same compute node.

[0066] Client 30000 can be configured to access the content and services of computer device 10000. Client 30000 can include any type of electronic device, such as mobile devices, tablets, laptops, workstations, virtual reality devices, gaming devices, set-top boxes, digital streaming media devices, vehicle terminals, smart TVs, etc.

[0067] The client 30000 can output (e.g., display, render, and present) the results of adding and extracting audio watermarks to the user.

[0068] The following will describe a network debugging solution through several embodiments. This solution can be implemented using computer equipment 10000.

[0069] Example 1

[0070] Figure 2 A flowchart illustrating an audio hidden watermark addition method according to Embodiment 1 of this application is shown schematically. It includes steps S200-S208, wherein...

[0071] Step S200: Obtain the first waveform file corresponding to the audio to be synthesized;

[0072] In this embodiment, the audio to be synthesized is the audio that needs to be synthesized, and the first waveform file is the original waveform file corresponding to the audio to be synthesized, which can be in formats such as WAV, AU, and VOC.

[0073] Specifically, the audio to be synthesized can be music, speech materials (such as story reading), synthesized speech, or other audio materials. The synthesized speech can be speech synthesized based on a basic timbre using some speech synthesis algorithms, such as Tacotron2, Wavenet, Fastspeech, etc. This application does not impose specific limitations on this aspect.

[0074] Step S202: Generate user data corresponding to the first waveform file;

[0075] In this embodiment, user data may include information such as user ID, digital signature, and authorization period. The user ID is a unique identifier for the user, and each synthesized audio file has a unique ID. The authorization period can be the configured duration for the legal use of the audio, for example, 1 year or 5 years. The digital signature is the signature information corresponding to the synthesized audio file; the algorithm used to generate the digital signature can be RSA, DSA, ECDSA, MD5, SHA-1, etc., and this embodiment does not impose specific limitations on this.

[0076] In a preferred embodiment of this application, after step S202, the following step may be included: encrypting the user data.

[0077] Specifically, user data can be further encrypted using algorithms such as AES, DES, 3DES, RSA, and ECC. This application does not impose specific limitations on this.

[0078] Step S204: Perform frequency domain transformation on the first waveform file to obtain the first Mel spectrum.

[0079] In this embodiment, before adding the hidden watermark, the first waveform file can be frequency domain transformed to obtain a first Mel spectrogram. A Mel spectrogram is a spectrogram that converts the frequency of audio information into a Mel-scale spectrum, where the Mel scale is a unit of pitch.

[0080] It should be noted that the execution order of steps S202 and S204 is not important, and this application embodiment does not impose any specific restrictions on this.

[0081] Step S206: Add the user data to the first Mel spectrogram to obtain a second Mel spectrogram containing a hidden watermark;

[0082] In this embodiment, user data can be added as a hidden watermark to the first Mel spectrogram to obtain a second Mel spectrogram containing the hidden watermark.

[0083] Step S208: The second Mel spectrum is restored to obtain a second waveform file containing a hidden watermark.

[0084] In this embodiment, after adding the hidden watermark, the second Mel spectrogram can be restored to obtain a second waveform file containing the hidden watermark, so that users can use the second waveform file for playback or dissemination. Since the second waveform file contains the hidden watermark, it is convenient to perform some source tracing operations by extracting the hidden watermark from the second waveform file to understand the audio producer or user information.

[0085] Specifically, a phase reconstruction algorithm model can be used to reconstruct the second Mel spectrogram to obtain the second waveform file. For example, the phase reconstruction algorithm model can be Waveglow, HifiGan, Wavenet, etc. These neural network vocoders are widely used in speech synthesis and offer better and more realistic waveform reconstruction results. Alternatively, the Griffin-lim algorithm can be used, but this algorithm is based on FFT for iterative phase reconstruction, resulting in a longer generation time. In specific implementations, the required phase reconstruction algorithm model can be selected according to the actual situation to reconstruct the second Mel spectrogram and obtain the second waveform file; this application does not impose specific limitations on this.

[0086] The following are several optional embodiments for optimizing the method of adding the audio hidden watermark, as follows:

[0087] In a preferred embodiment of this application, step S200 may include the following steps:

[0088] In response to the received speech synthesis command, the first waveform file corresponding to the audio to be synthesized is obtained.

[0089] In this embodiment, a voice synthesis function can be provided to the user through a terminal device. For example, when a user uploads a story reading audio, music, or synthesized audio through an algorithm, triggering the voice synthesis control will generate a voice synthesis command. After receiving the voice synthesis command, the client responds to the command, obtains the first waveform file corresponding to the audio to be synthesized, and adds a hidden watermark to the first waveform file.

[0090] In a preferred embodiment of this application, step S206 may include the following steps:

[0091] Based on a preset hidden watermark generation model, the user data is added to the first Mel spectrogram to obtain the second Mel spectrogram.

[0092] In this embodiment, the pre-defined hidden watermark generation model can be a hidden watermark injection algorithm pre-deployed in the client. Based on the pre-defined hidden watermark generation model, user data is added as a hidden watermark to the first Mel spectrogram, resulting in a second Mel spectrogram containing the hidden watermark. As an example, the pre-defined hidden watermark generation model can be the BlindWatermark algorithm.

[0093] To further explain the method for adding an audio hidden watermark in Embodiment 1 of this application, as follows: Figure 3 A flowchart illustrating a method for adding a hidden watermark to audio is provided. The method involves acquiring the original waveform file of audio material such as speech synthesis, music, or story reading, converting the original waveform file to obtain a Mel spectrogram (without watermark); simultaneously configuring user information such as the digital signature, authorization period, and user ID of the audio material, and encrypting this user information to obtain encrypted user data; inputting the Mel spectrogram (without watermark) and the encrypted user data into a hidden watermark generation algorithm model, processing it through the algorithm model, and outputting a Mel spectrogram (with watermark); finally, using phase reconstruction technology to restore the Mel spectrogram (with watermark) to obtain a waveform file (with watermark).

[0094] Example 2

[0095] Figure 4 The flowchart illustrating the audio hidden watermark extraction method according to Embodiment 2 of this application is shown schematically. It includes steps S400-S404, wherein...

[0096] Step S400: Obtain the second waveform file containing the hidden watermark corresponding to the audio to be traced;

[0097] In this embodiment, the audio to be traced is the audio that needs to be tracked, and the second waveform file is the waveform file containing a hidden watermark corresponding to the audio to be synthesized, which can be in formats such as WAV, AU, and VOC.

[0098] Step S402: Perform frequency domain transformation on the second waveform file to obtain a second Mel spectrum containing a hidden watermark;

[0099] In this embodiment, the second waveform file can be frequency domain transformed to obtain a second Mel spectrogram containing a hidden watermark. A Mel spectrogram is a spectrogram that converts the frequency of audio information into a Mel-scale, where the Mel scale is a unit of pitch.

[0100] Step S404: Extract the watermark from the second Mel spectrum to obtain user data corresponding to the second waveform file.

[0101] In this embodiment, a watermark can be extracted from the second Mel spectrogram to obtain user data corresponding to the second waveform file. This user data is the hidden watermark in the second Mel spectrogram. The user data includes information such as user ID, digital signature, and authorization period.

[0102] The following are several optional embodiments for optimizing the method of adding the audio hidden watermark, as follows:

[0103] In a preferred embodiment of this application, step S400 may include the following steps:

[0104] In response to the received audio tracing command, the second waveform file corresponding to the audio to be traced is obtained.

[0105] In this embodiment, a terminal device can provide users with an audio tracing function. For example, when a user uploads an audio file that needs to be traced, triggering the audio tracing control will generate an audio tracing command. After receiving the audio tracing command, the client responds to the audio tracing request and obtains the second waveform file corresponding to the audio to be traced, so as to perform the corresponding tracing operation using the second waveform file.

[0106] In a preferred embodiment of this application, after step S404, the following step may be included: decrypting the user data.

[0107] Specifically, when user data is added as a hidden watermark to the second Mel spectrogram, some encryption processing is performed. When parsing the data, the user data can be decrypted to facilitate tracing the source and understanding information such as the data producer or user.

[0108] To further explain the audio hidden watermark extraction method in Embodiment 2 of this application, as follows: Figure 5 A flowchart of an audio hidden watermark extraction method is shown. The method involves acquiring a waveform file (with watermark), converting the waveform file (with watermark) into a Mel spectrogram (with watermark), then using a hidden watermark extraction algorithm model to extract the watermark. The extracted user data is then decrypted to obtain the required user data, which can be used for tracing to understand information such as the data producer or user.

[0109] Example 3

[0110] Figure 6The diagram illustrates a block diagram of an audio watermarking system according to Embodiment 3 of this application. This audio watermarking system can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in the embodiments of this application.

[0111] like Figure 6 As shown, the audio watermarking system 600 may include the following modules:

[0112] The first waveform file acquisition module 610 is used to acquire the first waveform file corresponding to the audio to be synthesized.

[0113] User data generation module 620 is used to generate user data corresponding to the first waveform file;

[0114] The first waveform file conversion module 630 is used to perform frequency domain conversion on the first waveform file to obtain the first Mel spectrum.

[0115] The hidden watermark adding module 640 is used to add the user data to the first Mel spectrum to obtain a second Mel spectrum containing a hidden watermark.

[0116] Mel spectrum conversion module 650 is used to restore the second Mel spectrum to obtain a second waveform file containing a hidden watermark.

[0117] In a preferred embodiment of this application, the first waveform file acquisition module 610 includes:

[0118] The first waveform file acquisition submodule is used to acquire the first waveform file corresponding to the audio to be synthesized in response to the received speech synthesis command.

[0119] In a preferred embodiment of this application, the hidden watermark adding module 640 includes:

[0120] Based on a preset hidden watermark generation model, the user data is added to the first Mel spectrogram to obtain the second Mel spectrogram.

[0121] In a preferred embodiment of this application, the system further includes:

[0122] The data encryption module is used to encrypt the user data.

[0123] In a preferred embodiment of this application, the user data includes: user ID, digital signature, and authorization period.

[0124] Example 4

[0125] Figure 7 The diagram illustrates a block diagram of an audio hidden watermark extraction system according to Embodiment 4 of this application. This system can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in the embodiments of this application.

[0126] like Figure 7 As shown, the audio hidden watermark extraction system 700 may include the following modules:

[0127] The second waveform file acquisition module 710 is used to acquire the second waveform file containing the hidden watermark corresponding to the audio to be traced.

[0128] The second waveform file conversion module 720 is used to perform frequency domain conversion on the second waveform file to obtain a second Mel spectrum containing a hidden watermark.

[0129] The watermark extraction module 730 is used to extract the watermark from the second Mel spectrum to obtain user data corresponding to the second waveform file.

[0130] In a preferred embodiment of this application, the second waveform file acquisition module 710 includes:

[0131] The second waveform file acquisition submodule is used to acquire the second waveform file corresponding to the audio to be traced in response to the received audio tracing command.

[0132] In a preferred embodiment of this application, the apparatus further includes:

[0133] The data decryption module is used to decrypt the user data.

[0134] Example 5

[0135] Figure 8 This illustration schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a method for adding and extracting audio hidden watermarks according to Embodiment 5 of this application. In this embodiment, the computer device 10000 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. For example, it can be a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including independent FEN servers, or server clusters composed of multiple servers), etc. Figure 8As shown, the computer device 10000 includes, but is not limited to, at least the following: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked to each other via a system bus. Wherein:

[0136] The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for adding and extracting audio watermarks. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0137] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0138] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0139] It should be pointed out that, Figure 8 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0140] In this embodiment, the method for adding and extracting audio hidden watermarks stored in memory 10010 can be further divided into one or more program modules and executed by one or more processors (processor 10020 in this embodiment) to complete the embodiment of this application.

[0141] Example 6

[0142] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the audio hidden watermark addition and extraction method in the embodiment.

[0143] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the computer-readable storage medium can also include both internal storage units and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code for adding and extracting audio watermarks in this embodiment. Furthermore, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0144] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0145] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An audio steganographic embedding method, characterized by, The method comprises: obtaining a first waveform file corresponding to the audio to be synthesized; generating user data corresponding to the first waveform file; performing frequency domain conversion on the first waveform file to obtain a first Mel spectrum graph; adding the user data as a hidden watermark to the first Mel spectrum graph to obtain a second Mel spectrum graph containing a hidden watermark; performing restoration processing on the second Mel spectrum graph to obtain a second waveform file containing a hidden watermark; wherein the user data comprises a unique identity of a user, a digital signature, and an authorization period, the digital signature being signature information corresponding to the audio to be synthesized, and the authorization period being a configured period of legal use of the audio.

2. The audio steganographic embedding method of claim 1, wherein, The method comprises: in response to a received voice synthesis instruction, obtaining a first waveform file corresponding to the audio to be synthesized.

3. The audio steganographic embedding method of claim 1, wherein, The method comprises: based on a preset hidden watermark generation model, adding the user data to the first Mel spectrum graph to obtain the second Mel spectrum graph.

4. The audio steganographic embedding method of claim 1, wherein, The method further comprises, after the step of generating user data corresponding to the first waveform file: performing encryption processing on the user data.

5. An audio steganographic extraction method, characterized by, The method comprises: obtaining a second waveform file containing a hidden watermark corresponding to the audio to be traced, the second waveform file being obtained according to the method for adding an audio hidden watermark according to any one of claims 1 to 4; performing frequency domain conversion on the second waveform file to obtain a second Mel spectrum graph containing a hidden watermark; performing watermark extraction on the second Mel spectrum graph to obtain user data corresponding to the second waveform file; wherein the user data comprises a unique identity of a user, a digital signature, and an authorization period, the digital signature being signature information corresponding to the audio to be synthesized, and the authorization period being a configured period of legal use of the audio.

6. The audio steganographic extraction method of claim 5, wherein, The method comprises: in response to a received audio tracing instruction, obtaining a second waveform file corresponding to the audio to be traced.

7. The method of claim 5, wherein, The method further comprises: performing decryption processing on the user data.

8. A system for adding hidden audio watermarks, characterized in that, The method comprises: a first waveform file obtaining module configured to obtain a first waveform file corresponding to the audio to be synthesized; a user data generating module configured to generate user data corresponding to the first waveform file; a first waveform file converting module configured to perform frequency domain conversion on the first waveform file to obtain a first Mel spectrum graph; a hidden watermark adding module configured to add the user data as a hidden watermark to the first Mel spectrum graph to obtain a second Mel spectrum graph containing a hidden watermark; a Mel spectrum graph converting module configured to perform restoration processing on the second Mel spectrum graph to obtain a second waveform file containing a hidden watermark; wherein the user data comprises a unique identity of a user, a digital signature, and an authorization period, the digital signature being signature information corresponding to the audio to be synthesized, and the authorization period being a configured period of legal use of the audio.

9. An audio steganographic extraction system, characterized by The method comprises: A second waveform file obtaining module is configured to obtain a second waveform file containing a hidden watermark corresponding to the audio to be traced, the second waveform file being obtained according to the method for adding an audio hidden watermark according to any one of claims 1 to 4; A second waveform file converting module is configured to perform frequency domain conversion on the second waveform file to obtain a second Mel spectrogram containing a hidden watermark; A watermark extracting module is configured to perform watermark extraction on the second Mel spectrogram to obtain user data corresponding to the second waveform file. The user data includes a unique identity of a user, a digital signature, and an authorization period, the digital signature being signature information corresponding to the audio to be synthesized, and the authorization period being a configured period of time for legal use of the audio.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor is configured to execute the computer program to implement the steps of the method for adding an audio hidden watermark according to any one of claims 1 to 4, or the steps of the method for extracting an audio hidden watermark according to any one of claims 5 to 7.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which can be executed by at least one processor to enable the at least one processor to execute the steps of the method for adding an audio hidden watermark according to any one of claims 1 to 4, or the steps of the method for extracting an audio hidden watermark according to any one of claims 5 to 7.

12. A computer program product comprising computer instructions, characterised in that, The computer instructions are executed by the processor to implement the steps of the method for adding an audio hidden watermark according to any one of claims 1 to 4, or the steps of the method for extracting an audio hidden watermark according to any one of claims 5 to 7.

Images