Microphone for resisting AI voice cloning by pseudo tone based on acoustic shielding

Abstract

The invention belongs to the technical field of audio equipment design, and relates to a microphone for AI voice cloning based on pseudo tone of acoustic shielding, and the microphone comprises an audio collection module which is used for collecting an environment audio signal in real time; the analog-to-digital converter is used for converting the environment audio signal into a digital signal; the feature extraction module is used for performing frequency domain conversion on the digital signals and extracting frequency spectrum features and tone features; the acoustic shielding calculation module is used for calculating a shielding threshold value of the environment audio signal and determining a frequency band which cannot be heard by human ears in the environment audio signal according to the shielding threshold value; the pseudo tone generation module is used for generating pseudo tones based on the extracted tone features; the synthesis module is used for replacing a corresponding frequency band of the environment audio signal with the generated pseudo tone to form a synthesized audio; and the audio output module is used for outputting the synthesized audio. According to the invention, the pseudo tone can be generated in real time in the audio recording process, and it is ensured that a user is always protected during audio recording.

Description

Technical Field

[0001] The present application belongs to the technical field of audio equipment design, and in particular, relates to a microphone for counteracting AI voice cloning by using pseudo-timbre based on acoustic masking. Background Art

[0002] With the advancement of artificial intelligence (AI) technology, voice cloning technology has developed rapidly in recent years. Voice cloning technology uses deep learning algorithms to generate cloned voices that are highly similar to the voices of specific people by training a large amount of voice data. This technology has broad application prospects in voice interaction, virtual assistants, entertainment, and education. However, the abuse of voice cloning technology also brings risks such as privacy security and identity forgery. For example, malicious users can clone other people's voices to commit telecommunications fraud, spread false information, and even affect the authenticity of evidence in the judicial field. Therefore, how to prevent AI models from cloning sensitive audio information has become a technical problem that needs to be solved urgently.

[0003] There are already some defense technologies and products against voice cloning on the market, mainly including noise addition method, sound watermark method and adversarial method based on feature perturbation.

[0004] The noise addition method adds background noise or adversarial noise to the original audio, making it difficult for the AI ​​voice cloning model to accurately identify the characteristics of the audio. Adversarial noise is generally generated by a generative adversarial network (GAN), which can interfere with the AI ​​model without significantly affecting human hearing. However, the noise addition method often requires the introduction of higher-intensity noise into the audio to achieve the ideal defense effect, which will cause a significant decrease in sound quality and affect the user experience. In addition, when the user is recording or talking in real time, the technology cannot provide protection in time, resulting in the user's voice privacy cannot be guaranteed at critical moments.

[0005] Sound watermarking technology embeds imperceptible watermark features in audio for subsequent detection of whether the audio has been cloned. Watermarks are usually embedded in audio through small changes in high or low frequencies and can only be detected in synthetic audio. However, the detection of sound watermarks depends on the integrity of the audio. Once the audio is compressed or format converted, the watermark is easily damaged, which affects its detection effect. In addition, the audio with embedded watermarks can still be trained and reproduced by AI models, which fails to fundamentally prevent cloning. In fact, sound watermarks will only work when forged audio causes serious public opinion.

[0006] The adversarial method based on feature perturbation refers to perturbing the audio features so that the AI ​​model cannot correctly restore the features when generating cloned voices. This method interferes with specific frequency bands of the audio signal, such as modulating the fundamental frequency or harmonic characteristics, so that the cloned voice is distorted. However, since this method requires frequent adjustment of the interference features, it is difficult to adapt to different voice cloning algorithms, and the defense effect is unstable.

[0007] The above methods usually require high professional knowledge and technical support. Users need to perform complex settings and debugging to ensure the effective operation of the defense system. This is difficult for ordinary users and easily causes technical barriers, limiting the widespread application of defense technology. Summary of the invention

[0008] In order to solve the above problems, the present application provides a microphone that uses acoustic masking-based pseudo-timbre to combat AI voice cloning, so as to protect the audio from AI voice cloning during the audio recording process.

[0009] The microphone provided in this application for counteracting AI voice cloning with pseudo-timbre based on acoustic masking mainly includes:

[0010] Audio acquisition module, used to collect environmental audio signals in real time;

[0011] an analog-to-digital converter for converting the ambient audio signal into a digital signal;

[0012] A feature extraction module is used to perform frequency domain conversion on digital signals and extract spectrum features and tone features;

[0013] An acoustic masking calculation module, used to calculate a masking threshold of an ambient audio signal, and determine a frequency band in the ambient audio signal that is inaudible to human ears according to the masking threshold;

[0014] A pseudo-timbre generating module, used for modulating the frequency band inaudible to human ears in the backup ambient audio signal based on the extracted pitch features to generate a pseudo-timbre;

[0015] A synthesis module, used to replace the corresponding frequency band of the ambient audio signal with the generated pseudo tone to form a synthetic audio;

[0016] The audio output module is used to output the synthesized audio.

[0017] Preferably, the audio acquisition module has a programming interface for receiving a set sampling rate and sampling duration.

[0018] Preferably, the acoustic shielding calculation module includes:

[0019] The masking threshold calculation unit for each frequency band is used to divide the spectrum into multiple frequency bands and determine the masking threshold of each frequency band;

[0020] The frequency band index obtaining unit is used to obtain the frequency band index whose power is lower than the masking threshold.

[0021] Preferably, the microphone further comprises:

[0022] The amplifier is used to enhance the strength of the collected ambient audio signal.

[0023] Preferably, the microphone further comprises:

[0024] The digital signal processing module is used to equalize and compress the synthesized audio.

[0025] Preferably, in the pseudo-timbre generating module, the phase and amplitude of the pitch features are adjusted, and the pitch features at least include fundamental frequency, formant and harmonics.

[0026] This application can generate pseudo-timbre in real time during audio recording, ensuring that users are always protected during audio recording. This application can effectively interfere with the recognition of AI models through acoustic masking and pseudo-timbre generation, while maintaining high-quality audio output and improving user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is an organizational chart of a preferred embodiment of a microphone for counteracting AI voice cloning based on pseudo-timbre of acoustic masking in the present application.

[0028] Figure 2 This is a schematic diagram of the principle of pseudo-timbre synthesis based on masking threshold in this application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the implementation of this application will be described in more detail in combination with the drawings in the implementation of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described implementation is a part of the implementation of this application, not all of the implementations. The implementation described below with reference to the drawings is exemplary and is intended to be used to explain this application, and cannot be understood as a limitation on this application. Based on the implementation in this application, all other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The implementation of this application is described in detail below in combination with the drawings.

[0030] This application provides a microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre, such as Figure 1 As shown, it mainly includes:

[0031] Audio acquisition module, used to collect environmental audio signals in real time;

[0032] an analog-to-digital converter for converting the ambient audio signal into a digital signal;

[0033] A feature extraction module is used to perform frequency domain conversion on digital signals and extract spectrum features and tone features;

[0034] An acoustic masking calculation module, used to calculate a masking threshold of an ambient audio signal, and determine a frequency band in the ambient audio signal that is inaudible to human ears according to the masking threshold;

[0035] A pseudo-timbre generating module, used for modulating the frequency band inaudible to human ears in the backup ambient audio signal based on the extracted pitch features to generate a pseudo-timbre;

[0036] A synthesis module, used to replace the corresponding frequency band of the ambient audio signal with the generated pseudo tone to form a synthetic audio;

[0037] The audio output module is used to output the synthesized audio.

[0038] The audio acquisition module of the present application is responsible for capturing the ambient audio signal in real time. The hardware is equipped with a high-sensitivity condenser microphone element and has good frequency response characteristics (20Hz-20kHz) to ensure that rich audio details can be captured. In some optional embodiments, the audio acquisition module has a programming interface for receiving the set sampling rate and sampling duration.

[0039] The output end of the audio acquisition module is connected to an analog-to-digital converter, which is used to convert analog audio signals into digital signals for subsequent processing. The analog-to-digital converter ADC of this application supports multiple sampling rates to ensure the quality of the converted signal, and converts it according to the set sampling rate and outputs it as digital audio data.

[0040] In some optional embodiments, the microphone further includes: an amplifier for enhancing the strength of the collected ambient audio signal and improving the signal-to-noise ratio of the signal. In this embodiment, the amplifier is located between the audio acquisition module and the analog-to-digital converter. The present application uses a low-noise amplifier for amplifying the weak signal from the audio acquisition module and amplifying it to a suitable level for processing by the analog-to-digital converter ADC.

[0041] The output end of the analog-to-digital converter ADC is connected to the feature extraction module, which is used to analyze the collected audio signal and extract key audio features. Analyzing the collected audio signal mainly refers to performing time domain and frequency domain analysis. The algorithms used are the conventional fast Fourier transform algorithm (FFT) and Mel-frequency cepstral coefficient algorithm (MFCC).

[0042] The features extracted by the feature extraction module mainly include spectral features and pitch features. Tonality features include fundamental frequency, resonance peaks and harmonics. Tonality features are mainly used for the subsequent generation of pseudo-timbres, while spectral features are mainly used to determine which frequency bands can be added with pseudo-timbres based on the masking threshold.

[0043] The acoustic masking calculation module is used to determine the masking threshold of each frequency band, so as to determine the frequency band in the ambient audio signal that is inaudible to the human ear. In some optional embodiments, the acoustic masking calculation module includes: a masking threshold calculation unit for each frequency band, which is used to divide the spectrum into multiple frequency bands and determine the masking threshold of each frequency band; a frequency band index acquisition unit, which is used to obtain the frequency band index whose power is lower than the masking threshold. These frequency band indexes will be used in the pseudo-timbre generation module and the synthesis module. As suitable embedded frequency bands, they can ensure that the interference effect of the synthesized audio is maximized after the pseudo-timbre is added.

[0044] Figure 2 The relationship between the frequency spectrum and the sound intensity is given. Figure 2 In the figure, the horizontal axis is the frequency value of the audio signal, and the vertical axis is the sound intensity of the audio signal, which can be regarded as the energy value of the frequency. When the sound intensity of a frequency is less than a certain threshold, the human ear cannot hear it, such as Figure 2 As shown by the dotted line in , for example, at a frequency of 0.02KHz, corresponding to a frequency of 20Hz, the sound intensity must reach 70 decibels to be heard. If the sound intensity is lower than 70 decibels, it cannot be heard. In addition, when the sound signal of a certain frequency has a large energy, that is, when the sound signal intensity of a certain frequency is very large, the masking threshold near this frequency will increase a lot. For example, at about 0.3kHz, the sound signal energy of this frequency is very large, about 60 decibels. At this time, the masking threshold of the signal near this frequency (0.3KHz) will increase, as shown in the figure. Figure 2 The solid line shows.

[0045] Based on this, this application constructs a frequency domain based on the acoustic masking effect of the human ear and the curve of the relationship between volume and frequency. Figure 2 As shown in the masking curve, in this embodiment, the initial masking threshold can be determined first, and then the sound intensity of each frequency band of the collected audio can be determined to correct the masking threshold, and finally, it is determined whether each frequency band is an inaudible frequency band.

[0046] The pseudo-timbre generation module is used to generate pseudo-timbre according to the pitch characteristics. The material used for the pseudo-timbre is also the collected audio file. Since the collected audio file needs to be processed, a backup design is made here. In the process of processing the backup ambient audio signal, when it is determined that the intensity of certain frequency bands of the spectrum data of the ambient audio signal is lower than the masking threshold, the intensity of these frequency bands of the corresponding backup ambient audio signal is also lower than the masking threshold. Therefore, it is only necessary to use the frequency band index to process the data of the corresponding frequency band of the backup ambient audio signal to obtain the pseudo-timbre. The so-called pseudo-timbre is a false feature, which means that the phase and amplitude of the audio signals in these frequency bands are slightly adjusted so that the modulated pseudo-timbre is inconsistent with the original audio.

[0047] The reason why this application processes the backup ambient audio signal instead of the original collected ambient audio signal is mainly because of the concern that in the process of pseudo-timbre processing, the spectrum below the masking threshold is processed to be above the masking threshold, which affects the audible part of the original collected ambient audio signal. Therefore, in the process of generating pseudo-timbre, it is necessary to ensure that it is imperceptible to the human ear. Conventional audio processing methods such as weighted mapping or taking the maximum value can be used to fine-tune the intensity of the pseudo-timbre to ensure that the best interference effect is achieved without affecting the perception of the human ear.

[0048] The synthesis module is used to replace part of the characteristic data of the original collected ambient audio signal with the characteristic data of the pseudo-timbre according to the frequency band index, and finally output it to the outside through the audio output module. The audio output module is used to output the processed audio signal to the connected device to ensure high-quality audio output. It mainly includes an audio interface, supports multiple output interfaces such as USB and 3.5mm, and adapts to the needs of different devices, so as to ensure the compatibility of the output signal with the user's device and provide a plug-and-play experience.

[0049] In some optional implementations, the microphone further includes: a digital signal processing module for equalizing and compressing the synthesized audio. This embodiment uses a high-performance DSP chip to execute various preset audio effect processing algorithms, such as an equalizer and a compressor, to optimize the audio signal after pseudo-tone embedding and acoustic masking processing to enhance the overall sound quality.

[0050] In some optional embodiments, the microphone further includes a control and interface module for providing control logic and a user interface of the system. The control module is, for example, a microcontroller, which is responsible for coordinating the work of each module and processing user input and settings. The interface module includes, for example, buttons, LED indicators, and a display screen for users to manage and set the device, such as starting recording, adjusting settings, and viewing status information.

[0051] This application improves the out-of-box user experience. Through modular design, users only need to connect a microphone to automatically enable the privacy protection function without complicated settings, greatly reducing the threshold for use.

[0052] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A microphone for counteracting AI voice cloning with pseudo-timbre based on acoustic masking, characterized in that: include: Audio acquisition module, used to collect environmental audio signals in real time; an analog-to-digital converter for converting the ambient audio signal into a digital signal; A feature extraction module is used to perform frequency domain conversion on digital signals and extract spectrum features and tone features; An acoustic masking calculation module, used to calculate a masking threshold of an ambient audio signal, and determine a frequency band in the ambient audio signal that is inaudible to human ears according to the masking threshold; A pseudo-timbre generating module, used for modulating the frequency band inaudible to human ears in the backup ambient audio signal based on the extracted pitch features to generate a pseudo-timbre; A synthesis module, used to replace the corresponding frequency band of the ambient audio signal with the generated pseudo tone to form a synthetic audio; The audio output module is used to output the synthesized audio.

2. The microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre as claimed in claim 1, characterized in that: The audio acquisition module has a programming interface for receiving a set sampling rate and sampling duration.

3. The microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre as claimed in claim 1, characterized in that: The acoustic shielding calculation module includes: The masking threshold calculation unit for each frequency band is used to divide the spectrum into multiple frequency bands and determine the masking threshold of each frequency band; The frequency band index obtaining unit is used to obtain the frequency band index whose power is lower than the masking threshold.

4. The microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre as claimed in claim 1, characterized in that: The microphone also includes: The amplifier is used to enhance the strength of the collected ambient audio signal.

5. The microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre as claimed in claim 1, characterized in that: The microphone also includes: The digital signal processing module is used to equalize and compress the synthesized audio.

6. The microphone for counteracting AI voice cloning based on acoustic masking pseudo-timbre as claimed in claim 1, characterized in that: In the pseudo-timbre generating module, the phase and amplitude of the pitch features are adjusted, and the pitch features at least include fundamental frequency, formant and harmonics.

Images