A microphone for anti-audio cloning based on acoustic masking

Abstract

The application belongs to the technical field of audio device design, and relates to a microphone for pseudo-tone color confrontation AI voice cloning based on acoustic masking, which comprises an audio acquisition module, an analog-to-digital converter, a feature extraction module, an acoustic masking calculation module, a pseudo-tone color generation module, a synthesis module and an audio output module. The audio acquisition module is used for acquiring an environmental audio signal in real time. The analog-to-digital converter is used for converting the environmental audio signal into a digital signal. The feature extraction module is used for performing frequency domain conversion on the digital signal, and extracting frequency spectrum features and tone features. The acoustic masking calculation module is used for calculating a masking threshold of the environmental audio signal, and determining a frequency band that is inaudible to human ears in the environmental audio signal according to the masking threshold. The pseudo-tone color generation module is used for generating a pseudo-tone color based on the extracted tone features. The synthesis module is used for replacing the corresponding frequency band of the environmental audio signal with the generated pseudo-tone color to form a synthesized audio. The audio output module is used for outputting the synthesized audio externally. The application can generate a pseudo-tone color in real time during audio recording, and ensures that a user is always protected during audio recording.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of audio device design, and particularly relates to a microphone for anti-AI voice cloning based on pseudo-tone color confrontation. BACKGROUND

[0002] With the progress of artificial intelligence (AI) technology, voice cloning technology has developed rapidly in recent years. Voice cloning technology uses deep learning algorithms to train a large amount of voice data to generate cloned voices that are highly similar to the voices of specific individuals. This technology has broad application prospects in voice interaction, virtual assistants, entertainment, and education. However, the misuse of voice cloning technology also poses risks such as privacy security and identity forgery. For example, malicious users can commit telecommunications fraud, spread false information, or even affect the authenticity of evidence in the judicial field by cloning the voices of others. Therefore, how to prevent AI models from cloning sensitive audio information has become a technical problem that needs to be solved urgently.

[0003] There are some defense technologies and products against voice cloning on the market at present, mainly including noise addition method, sound watermarking method, and feature perturbation-based confrontation method.

[0004] The noise addition method adds background noise or adversarial noise to the original audio to make it difficult for AI voice cloning models to accurately identify the characteristics of the audio. Adversarial noise is generally generated by a generative adversarial network (GAN), which can interfere with AI models without significantly affecting human hearing. However, the noise addition method often needs to introduce high-intensity noise into the audio to achieve the desired defense effect, which will significantly degrade the audio quality and affect the user experience. In addition, this technology cannot provide protection in real time when the user is recording or talking, resulting in the inability to protect the user's voice privacy in critical moments.

[0005] Sound watermarking technology embeds imperceptible watermark features in audio for subsequent detection of whether the audio has been cloned. Watermarks are usually embedded in audio through high-frequency or low-frequency micro changes and can only be detected in synthesized audio. However, the detection of sound watermarking relies on the integrity of the audio, and once the audio is compressed or converted, the watermark is easily damaged, affecting its detection effect. In addition, the audio after watermark embedding can still be trained and copied by AI models, failing to fundamentally prevent cloning generation. In fact, sound watermarking only works when the fake audio causes serious public opinion.

[0006] The feature perturbation-based adversarial method refers to perturbing the audio features so that the AI model cannot correctly restore the features when generating cloned speech. This method interferes with specific frequency bands of the audio signal, such as modulating the fundamental frequency or harmonic characteristics, so that the cloned generated speech is distorted. However, since such a method needs to frequently adjust the interference features, it is difficult to adapt to different speech cloning algorithms, and the defense effect is unstable.

[0007] The above methods usually require high professional knowledge and technical support. Users need to perform complex settings and debugging to ensure the effective operation of the defense system. This is difficult for ordinary users and may cause technical obstacles, limiting the widespread application of defense technology. SUMMARY

[0008] To solve the above problems, the application provides a microphone for anti-AI speech cloning based on acoustic masking of pseudo timbre, which can protect the audio from AI speech cloning during audio recording.

[0009] The microphone for anti-AI speech cloning based on acoustic masking of pseudo timbre provided by the application mainly includes:

[0010] An audio acquisition module for acquiring environmental audio signals in real time;

[0011] An analog-to-digital converter for converting the environmental audio signals into digital signals;

[0012] A feature extraction module for performing frequency domain conversion on the digital signals to extract spectral features and pitch features;

[0013] An acoustic masking calculation module for calculating the masking threshold of the environmental audio signals and determining the frequency band that is inaudible to human ears in the environmental audio signals according to the masking threshold;

[0014] A pseudo timbre generation module for modulating the frequency band that is inaudible to human ears in the backup environmental audio signals based on the extracted pitch features to generate a pseudo timbre;

[0015] A synthesis module for replacing the corresponding frequency band of the environmental audio signals with the generated pseudo timbre to form a synthesized audio;

[0016] An audio output module for outputting the synthesized audio externally.

[0017] Preferably, the audio acquisition module has a programming interface for receiving a set sampling rate and sampling duration.

[0018] Preferably, the acoustic masking calculation module includes:

[0019] A frequency band masking threshold calculation unit for dividing the spectrum into multiple frequency bands and determining the masking threshold of each frequency band;

[0020] a frequency band index acquisition unit configured to acquire a frequency band index whose power is lower than a masking threshold.

[0021] Preferably, the microphone further comprises:

[0022] an amplifier configured to enhance the intensity of the collected ambient audio signal.

[0023] Preferably, the microphone further comprises:

[0024] a digital signal processing module configured to perform equalization and compression processing on the synthesized audio.

[0025] Preferably, in the pseudo timbre generation module, the pitch features are adjusted in phase and amplitude, and the pitch features at least include a fundamental frequency, a formant and a harmonic.

[0026] The present application can generate a pseudo timbre in real time during audio recording, ensuring that the user is always protected during audio recording. The present application can effectively interfere with the recognition of an AI model through acoustic masking and pseudo timbre generation, while maintaining high-quality audio output and improving user satisfaction. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The figure is a structural diagram of a preferred embodiment of the microphone for pseudo timbre against AI voice cloning based on acoustic masking of the present application.

[0028] Figure 2 The figure is a schematic diagram of pseudo timbre synthesis based on a masking threshold of the present application. DETAILED DESCRIPTION

[0029] To make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The described embodiments are part of the embodiments of the present application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0030] The present application provides a microphone for pseudo timbre against AI voice cloning based on acoustic masking, as shown in Figure 1 The microphone mainly comprises:

[0031] An audio acquisition module is configured to acquire an ambient audio signal in real time.

[0032] An analog-to-digital converter is configured to convert the ambient audio signal into a digital signal.

[0033] A feature extraction module is configured to perform frequency domain conversion on the digital signal, extract frequency spectrum features and tone features.

[0034] An acoustic masking calculation module is configured to calculate a masking threshold of the ambient audio signal, and determine a frequency band that is inaudible to human ears in the ambient audio signal according to the masking threshold.

[0035] A pseudo timbre generation module is configured to modulate the frequency band that is inaudible to human ears in the backup ambient audio signal based on the extracted tone features, and generate a pseudo timbre.

[0036] A synthesis module is configured to replace the corresponding frequency band of the ambient audio signal with the generated pseudo timbre, and form a synthesized audio.

[0037] An audio output module is configured to output the synthesized audio externally.

[0038] The audio acquisition module of the present application is responsible for capturing the ambient audio signal in real time. A high-sensitivity condenser microphone element is configured on the hardware, which has good frequency response characteristics (20Hz-20kHz) and can capture rich audio details. In some optional embodiments, the audio acquisition module has a programming interface for receiving a set sampling rate and sampling duration.

[0039] The audio acquisition module is connected to an analog-to-digital converter at the output end. The analog-to-digital converter ADC is configured to convert the analog audio signal into a digital signal for subsequent processing. The analog-to-digital converter ADC of the present application supports multiple sampling rates to ensure the quality of the converted signal, and converts the signal according to the set sampling rate to output digital audio data.

[0040] In some optional embodiments, the microphone further includes an amplifier configured to enhance the intensity of the collected ambient audio signal and improve the signal-to-noise ratio of the signal. In this embodiment, the amplifier is located between the audio acquisition module and the analog-to-digital converter. The present application uses a low-noise amplifier to amplify the weak signal from the audio acquisition module to an appropriate level for processing by the analog-to-digital converter ADC.

[0041] The output end of the analog-to-digital converter ADC is connected to a feature extraction module. The feature extraction module is configured to analyze the collected audio signal and extract key audio features. The analysis of the collected audio signal mainly refers to time domain and frequency domain analysis, and the algorithms used are the conventional fast Fourier transform algorithm (FFT) and the mel frequency cepstral coefficient algorithm (MFCC).

[0042] The features extracted by the feature extraction module mainly include spectral features and pitch features, such as fundamental frequency, formants and harmonics. The pitch features are mainly used for subsequent generation of the pseudo timbre, and the spectral features are mainly used for subsequent determination of which frequency bands can be added to the pseudo timbre according to the masking threshold.

[0043] The acoustic masking calculation module is used to determine the masking threshold of each frequency band, so as to determine the frequency band that is inaudible to the human ear in the environmental audio signal. In some optional embodiments, the acoustic masking calculation module comprises: a masking threshold calculation unit for each frequency band, which is used to divide the spectrum into a plurality of frequency bands and determine the masking threshold of each frequency band; and a frequency band index acquisition unit, which is used to acquire the frequency band index whose power is lower than the masking threshold. These frequency band indexes will be used in the pseudo timbre generation module and the synthesis module as suitable embedding frequency bands, which can ensure that the interference effect of the synthesized audio is maximized after adding the pseudo timbre.

[0044] Figure 2 The relationship between the spectrum and the sound intensity is given, Figure 2 The horizontal coordinate is the frequency value of the audio signal, and the vertical coordinate is the sound intensity of the audio signal, which can be regarded as the energy value of the frequency. When the sound intensity of a frequency is less than a certain threshold, the human ear cannot hear it, as shown by the dashed line in Figure 2 For example, at a frequency of 0.02 kHz, the sound intensity corresponding to a frequency of 20 Hz needs to reach 70 decibels to be heard. If the sound intensity is lower than 70 decibels, it cannot be heard. In addition, when the sound signal of a certain frequency has a large energy, i.e. the sound signal intensity of a certain frequency is large, the masking threshold near the frequency will be greatly increased, for example, at a position of about 0.3 kHz, the sound signal energy of the frequency is large, about 60 decibels. At this time, the masking threshold of the signal near the frequency (0.3 kHz) will be increased, as shown by the solid line in Figure 2 .

[0045] Accordingly, based on the acoustic masking effect of the human ear and considering the curve of the relationship between the volume and the frequency, a masking curve shown in Figure 2 is constructed in the frequency domain. In this embodiment, the initial masking threshold can be determined first, and then the sound intensity of each frequency band of the collected audio signal can be determined to correct the masking threshold, and finally, it is determined whether each frequency band is an inaudible frequency band.

[0046] The pseudo timbre generation module is configured to generate a pseudo timbre according to the tonal features. The pseudo timbre is also generated from the collected audio file. Since the collected audio file needs to be processed, a backup design is provided. During the processing of the backup ambient audio signal, when it is determined that the intensity of some frequency bands of the spectral data of the ambient audio signal is lower than the masking threshold, the intensity of the corresponding frequency bands of the backup ambient audio signal is also lower than the masking threshold. Therefore, only the frequency band index is needed to process the data of the corresponding frequency bands of the backup ambient audio signal to obtain the pseudo timbre. The pseudo timbre is a false feature, which means that the phase and amplitude of the audio signal of the frequency bands are slightly adjusted so that the modulated pseudo timbre is inconsistent with the original audio.

[0047] The backup ambient audio signal is processed instead of the original collected ambient audio signal, mainly because it is worried that the frequency spectrum below the masking threshold is processed to be higher than the masking threshold during the processing of the pseudo timbre, which affects the part of the original collected ambient audio signal that can be heard by the human ear. Therefore, during the generation of the pseudo timbre, it is necessary to ensure that it is imperceptible to the human ear. The intensity of the pseudo timbre can be slightly adjusted using conventional audio processing methods such as weighted mapping or maximum value to ensure the best interference effect without affecting the perception of the human ear.

[0048] The synthesis module is configured to replace part of the feature data of the original collected ambient audio signal with the feature data of the pseudo timbre according to the frequency band index, and finally output the processed audio signal to the connected device through the audio output module. The audio output module is configured to output the processed audio signal to the connected device to ensure high-quality audio output. It mainly includes an audio interface, supports multiple output interfaces such as USB and 3.5mm, and adapts to the needs of different devices to ensure the compatibility of the output signal with the user device and provide a plug-and-play experience.

[0049] In some optional embodiments, the microphone further includes a digital signal processing module for equalization and compression processing of the synthesized audio. This embodiment uses a high-performance DSP chip to execute various preset audio effect processing algorithms such as equalizer and compressor to optimize the audio signal processed by the pseudo timbre embedding and acoustic masking to enhance the overall sound quality.

[0050] In some optional embodiments, the microphone further includes a control and interface module for providing control logic and user interface of the system. The control module is a microcontroller, for example, which is responsible for coordinating the work of each module and processing user input and settings. The interface module includes buttons, LED indicator lights and display screens, for example, for users to manage and set the device, such as starting recording, adjusting settings and viewing status information.

[0051] The application improves the out-of-box user experience, through the modular design, the user only needs to connect the microphone, and the privacy protection function can be automatically enabled, without complex setting, greatly reducing the use threshold.

[0052] The above merely provides a specific implementation of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which shall be covered within the protection scope of the application. Therefore, the protection scope of the application shall be subject to the protection scope of the claims.

Claims

1. A microphone for acoustic-shield-based pseudo timbre counter-AI voice cloning, characterized in that, The microphone comprises: an audio acquisition module configured to acquire an ambient audio signal in real time; an analog-to-digital converter configured to convert the ambient audio signal into a digital signal; a feature extraction module configured to perform frequency domain conversion on the digital signal, and extract a frequency spectrum feature and a pitch feature; an acoustic masking calculation module configured to calculate a masking threshold of the ambient audio signal, and determine a frequency band in the ambient audio signal that is inaudible to human ears according to the masking threshold; a pseudo timbre generation module configured to modulate the frequency band in the backup ambient audio signal that is inaudible to human ears based on the extracted pitch feature, and generate a pseudo timbre; a synthesis module configured to replace a corresponding frequency band of the ambient audio signal with the generated pseudo timbre, and form a synthesized audio; an audio output module configured to output the synthesized audio externally.

2. The microphone for acoustic-shield-based, anti-voice-cloning AI voice cloning of spurious timbres of claim 1, wherein, The audio acquisition module has a programming interface configured to receive a set sampling rate and a sampling time length.

3. The microphone for acoustic-shield-based, anti-voice-cloning AI voice cloning of spurious timbres of claim 1, wherein, The acoustic masking calculation module comprises: a frequency band masking threshold calculation unit configured to divide a frequency spectrum into a plurality of frequency bands, and determine a masking threshold of each frequency band; a frequency band index acquisition unit configured to acquire a frequency band index whose power is lower than the masking threshold.

4. The microphone of claim 1, wherein the AI voice cloning based on acoustic masking of false timbre counter AI voice cloning is based on a neural network. The microphone further comprises: an amplifier configured to enhance the intensity of the acquired ambient audio signal.

5. The microphone of claim 1, wherein, The microphone further comprises: a digital signal processing module configured to perform equalization and compression processing on the synthesized audio.

6. The microphone of claim 1, wherein the AI voice cloning based on acoustic masking of false timbre counter AI voice cloning is based on a neural network. In the pseudo timbre generation module, the pitch feature is adjusted in phase and amplitude, and the pitch feature at least includes a fundamental frequency, a formant, and a harmonic.

Images