A forgery speech detection algorithm and system based on principal component filtering

Abstract

The existing counterfeit speech detection method has weak robustness in the re-encoding and noise mismatch scene, in order to improve the robustness of the existing method, the counterfeit speech detection research work puts forward the strategy of data augmentation to the training data set. However, the data augmentation strategy can increase the training data quantity, reduce the model training efficiency, and can only be used for known encoding algorithm and noise difference scene. The present application relates to the field of counterfeit speech detection, especially to the field of counterfeit speech detection for re-encoding and noise interference scene, specifically relates to a counterfeit speech detection algorithm and system based on subject filtering, mainly designs a subject signal filtering module based on the relationship between the human ear hearing masking effect and the signal-to-noise energy ratio, which can eliminate the part causing the distribution difference in the spectrogram feature, at the same time, without increasing the training data quantity, can improve the robustness of the model in the unknown encoding algorithm and noise difference scene, and has good universality.

Description

Technical fields:

[0001] This invention relates to the field of forged speech detection, and particularly to the field of forged speech detection in scenarios of recoding and noise interference. Specifically, it relates to a forged speech detection algorithm and system based on subject filtering. Technical background:

[0002] The lack of robustness in spoofing detection models refers to the sharp decline in detection performance when there is a mismatch between the training and evaluation datasets. Mismatches between training and evaluation datasets can be categorized into several scenarios, including speaker mismatch, spoofing algorithm mismatch, recoding mismatch, and noise interference mismatch. Speaker mismatch refers to the presence of speakers in the evaluation dataset that are not present in the training dataset. Spoofing algorithm mismatch refers to the use of spoofing synthesis algorithms in the evaluation dataset that were not used in the training dataset. Recoding mismatch refers to the possibility that the speech data in the evaluation dataset has undergone processing with various unknown encoding algorithms, while the speech data in the training dataset has not undergone encoding processing or involves limited processing algorithms. Noise interference mismatch refers to the presence of various noise interferences in the speech data in the evaluation dataset, while the training dataset contains only clean, noise-free speech. These mismatch scenarios can coexist; for example, the LA evaluation dataset of ASVspoof2019 exhibits both speaker and spoofing algorithm mismatches with the training dataset.

[0003] Improving the robustness of forged speech detection models is a gradual process. Early research focused on the robustness of forged speech detection models in scenarios of speaker mismatch and algorithm mismatch, proposing many targeted detection models and novel loss functions. These methods achieved high accuracy in these scenarios. However, existing methods do not adequately address the robustness of forged speech detection models in scenarios of recoding mismatch and noise mismatch. Encoding is a common processing method for digital audio, and audio inevitably encounters various noise interferences during acquisition, transmission, and playback. Therefore, forged speech detection models for practical applications should consider performance in scenarios of recoding mismatch and noise mismatch.

[0004] Experiments show that existing spoofing detection models exhibit poor robustness in recoding mismatch and noise mismatch scenarios. The main distributed mismatch scenario between the ASVspoof2021 LA evaluation dataset and the ASVspoof2019 LA training dataset is the recoding mismatch scenario. Existing spoofing detection models trained on the ASVspoof2019 LA training dataset achieve an EER of approximately 4%-7% when evaluated on the ASVspoof2019 LA evaluation dataset, while the EER is generally close to 20% when evaluated on the ASVspoof2021 LA evaluation dataset. The ADD LF dataset simulates real-world noise mismatch scenarios. Existing spoofing detection models achieve an EER close to 10% when trained on the ADD dataset and approximately 30% during evaluation, indicating weak decision-making ability in noise mismatch scenarios. Therefore, the performance of existing algorithms in recoding mismatch and noise mismatch scenarios needs further improvement. Summary of the Invention

[0005] The above-mentioned technical problems of the present invention are mainly solved by the following technical solutions:

[0006] A forged speech detection algorithm based on subject filtering, including

[0007] After collecting voice data and extracting features, the data is divided into training and testing sets.

[0008] Masking filtering and amplitude filtering are performed on the training set and the test set respectively to extract the subject of speech data. Interference data due to recoding and noise in the spectrogram are removed to obtain the extracted training set and test set.

[0009] The trained detection model is obtained by training the detection model with the training set;

[0010] The trained detection model is used to perform real-time detection of fake speech.

[0011] In the aforementioned spoofing detection algorithm based on subject filtering, feature extraction uses a general feature extraction algorithm to extract spectrogram features of the speech to be tested.

[0012] In the aforementioned subject-based forged speech detection algorithm, during the subject extraction process using masking filtering:

[0013] The masking measures of spectrogram features are performed, and the spectrogram feature masking curves are calculated.

[0014] By removing the masked frequency components from the original spectrogram features based on the masking curve, the unmasked power spectrum is obtained.

[0015] In the aforementioned forgery detection algorithm based on subject filtering, during amplitude filtering subject extraction:

[0016] The unmasked power spectrum is divided into frequency bands according to the characteristics of human vocalization and hearing;

[0017] An adaptive amplitude filtering algorithm is used to remove noise signals from each frequency band to obtain the power spectrum of the main signal.

[0018] In the aforementioned spoofing detection algorithm based on subject filtering, the Bark domain spectrum, the sound pressure level (SPL) of the spectrum amplitude, and the local peak points of the frequency curve are calculated based on the speech power spectrum.

[0019] The masking curve is calculated by substituting the sound pressure level (SPL) of the Bark band, the spectrogram amplitude, and the local peak points into the masking transfer function.

[0020] The unmasked speech power spectrum is obtained by removing frequency components with amplitudes lower than the masking curve.

[0021] In the aforementioned spoofing detection algorithm based on subject filtering, the unmasked power spectrum is divided into three frequency bands: high, medium, and low.

[0022] Amplitude filtering is performed on different frequency bands according to the adaptive energy levels within the frequency band region.

[0023] In the aforementioned spoofing detection algorithm based on subject filtering, the Bark domain spectrogram is calculated, i.e., the frequency domain to Bark domain conversion is calculated as shown in Equation 1.

[0024]

[0025] Where F hz f represents the frequency value. bark This represents the frequency domain value of the Buck scale.

[0026] In the aforementioned spoofing detection algorithm based on subject filtering, the core of adaptive filtering is shown in Equation 2:

[0027]

[0028] f abs Represents the absolute value of the amplitude, Top 10％ This indicates that the amplitudes of all frequency components within the frequency band are sorted in descending order, F. h There are two parameters: the first parameter is all the frequency components of this frequency band, and the second parameter is the Top. 10％ The calculated amplitude point, F hSet the amplitudes of all frequency components whose amplitudes are lower than the second parameter to singular values.

[0029] A forged speech detection system based on subject filtering, including

[0030] The first module is configured to collect speech data and, after feature extraction, divide it into a training set and a test set.

[0031] The second module is configured to perform masking filtering for subject extraction and amplitude filtering for subject extraction on the training set and the test set, respectively, to remove interference data in the speech data due to recoding and noise in the spectrogram, and to obtain the extracted training set and test set.

[0032] The third module is configured to train the detection model using the training set to obtain a trained detection model.

[0033] The fourth module is configured to perform real-time detection of fake speech using a pre-trained detection model.

[0034] Therefore, this invention has the following advantages: 1. Addressing the common data distribution mismatch problem in the field of spoofed speech detection, the subject extraction module removes easily changing non-subject parts from the speech signal, effectively improving the robustness of existing spoofed speech detection models in scenarios involving recoding and noise interference. 2. The subject extraction module utilizes the auditory masking effect, filtering out non-subject parts of the speech content while retaining the main body of the speech, thus maintaining the semantics and naturalness of the original speech. When the training dataset and evaluation dataset are not mismatched, the subject extraction module does not significantly reduce the detection accuracy of existing spoofed speech detection models. Attached Figure Description

[0035] Figure 1 This is a schematic diagram comparing the subject extraction scheme with general methods.

[0036] Figure 2 It is a subject extraction calculation process based on the auditory masking effect.

[0037] Figure 3 This is the specific calculation process for the subject extraction scheme based on spectrogram amplitude filtering. Detailed Implementation

[0038] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0039] Example:

[0040] This invention, through in-depth analysis of the speech coding process and the characteristics of signal and noise in the spectrogram, proposes masking filtering and amplitude filtering schemes for subject extraction based on the masking effect of human hearing and the high energy difference between signal and noise. These schemes eliminate interference components caused by recoding and noise in the spectrogram, preserving the main vocal component, thus achieving more robust forged speech detection. Specifically, it includes:

[0041] 1. Using the STFT algorithm, extract the original spectrogram features of the speech sample to be tested.

[0042] 2. Process the original spectrogram features using the main filtering module, including:

[0043] 2.1 Calculate the masking curve of the original spectrogram features according to the formula for calculating the masking effect.

[0044] 2.2 Based on the masking curve, the masked frequency components in the original spectrogram features are removed to obtain the unmasked power spectrum.

[0045] 2.3 Based on the characteristics of human hearing, adaptive amplitude filtering is applied to different frequency bands in the unmasked power spectrum to remove noise signals and obtain the main signal.

[0046] 3. Use the main signal as input to train a fake speech detection model (there are many deep neural networks that can be used for fake speech detection; any network can be selected here, which is not covered by this invention).

[0047] 4. The trained model can then be used for spoofing detection. However, before detection, the main signal still needs to be extracted using the main filtering method described in step 2.

[0048] The core of this method lies in the main feature extraction module, and its overall structure is as follows: Figure 1 As shown, its workflow is as follows: First, spectrogram features of the test speech are extracted using a general feature extraction algorithm. Second, a masking metric is performed on the spectrogram features to calculate their masking curve. Then, the masking removal module removes masked frequency components from the original spectrogram features based on the masking curve, obtaining the unmasked power spectrum. Finally, the unmasked power spectrum is divided into frequency bands according to the characteristics of human vocalization and hearing, and an adaptive amplitude filtering algorithm is used to remove noise signals from each frequency band. The main body extraction module processes the spectrogram features of both training and evaluation data, removing interference information caused by recoding and noise signals between features. This prevents the fake speech detection model from relying on unstable interference information during training and evaluation, thereby improving its robustness.

[0049] The main body extraction module first removes interference signals imperceptible to the human ear based on the masking effect. Then, it performs amplitude filtering based on the amplitude relationship between the noise and the main signal. This processing order is because the calculation of the masking curve needs to maintain the integrity of the original speech signal. If amplitude filtering is performed first, the relationship between the signals is disrupted, and the calculated masking curve will lose its original meaning. Masking filtering only removes the imperceptible portion, which does not affect the subsequent noise removal based on amplitude. Therefore, the processing order of the main body extraction module is masking filtering first, followed by amplitude filtering.

[0050] The subject extraction module includes a subject extraction scheme based on auditory masking effect and a subject extraction scheme based on amplitude filtering, which will be introduced separately below.

[0051] 1. Subject extraction scheme based on auditory masking effect (masking filter subject extraction).

[0052] The specific processing flow of the subject extraction scheme based on the masking effect is as follows: Figure 2 As shown. First, the Bark domain spectrum, the sound pressure level (SPL) of the spectrum amplitude, and the local peak points of the frequency curve are calculated based on the speech power spectrum. The frequency domain to Bark domain conversion is shown in Equation 1, where f hz f represents the frequency value. bark The frequency domain value represents the Bark scale. The frequency domain is converted to the Bark domain because the Bark region better matches the human auditory system. There are 24 Bark sub-bands corresponding to 24 regions in the human ear. The physiological basis of the masking effect is the mutual interference of speech frequency components within each of the 24 regions of the human ear. The unit of SPL value is dB, which represents the ratio of the sound pressure level at that point to the standard sound pressure level. The formula for converting frequency to SPL is shown in Equation 2, where... N represents the square of the absolute value of the power, which represents the energy of that frequency component. fft The number of levels used in the Fourier transform is generally slightly larger than the speech frame window length and a power of 2, to facilitate the use of the Fast Fourier Transform algorithm. Local peaks are frequency points in the frequency curve where the frequency component is higher than its surrounding frequency components. Algorithms for finding local peaks in a sequence are very mature; this paper uses the `find_peak` function from the scipy library to locate peaks. During the calculation of the masking curve, each peak is considered as a non-noise component. The reason for calculating peaks is that in auditory masking effects, the masking effect of non-noise components on noise components is different from the masking effect of noise components on noise components; generally, only the masking effect of non-noise components on noise components needs to be calculated.

[0053] The masking curve can be calculated by substituting the Bark band, SPL, and local peak points into the masking transfer function. The masking transfer function is an iterative process that calculates and accumulates the masking effect of each non-noise point on the surrounding signals to obtain the masking curve of the entire frequency spectrum. The formula for calculating the masking effect of each non-noise point on the global spectrum in the masking transfer function is shown in Equation 3. SPL i This represents the SPL value at the peak point of the current iteration. A masking effect only occurs when the SPL of the non-noise portion is greater than 40. j This is used to temporarily store the masking effect of the i-th peak on the j-th frequency component globally. dz is the difference between the Bark scale value of the i-th frequency and the Bark scale value of the j-th frequency; its absolute value is used. θ depends on the value of dz; it is set to 1 if dz is positive, and 0 otherwise. This means that a masking effect only exists if the energy of the local peak point i is greater than the energy of j. The masking removal module removes frequency components with amplitudes lower than the masking curve to obtain the unmasked speech power spectrum.

[0054]

[0055]

[0056] Sf j =abs(dz)·(-27+0.37·max(SPL i -40,0)·θ) (3)

[0057] 2. Subject extraction scheme based on amplitude filtering (amplitude filtering subject extraction).

[0058] The specific process of the subject extraction scheme based on spectrogram amplitude filtering is as follows: Figure 3 As shown. The unmasked power spectrum is first divided into three frequency bands: high, medium, and low. For different frequency bands, amplitude filtering is performed according to the adaptive energy level within the band region. The core of the adaptive filtering is shown in Equation 4. abs Represents the absolute value of the amplitude, Top 10％ This indicates that the amplitudes of all frequency components within the frequency band are sorted in descending order, and the amplitude that happens to fall at the 10th percentile is selected as the Top value. 30％ and Top 5％ Similarly. h There are two parameters: the first parameter is all the frequency components of this frequency band, and the second parameter is the Top. 10％ The calculated amplitude point, F h Set the amplitudes of all frequency components whose amplitudes are lower than the second parameter to singular values.

[0059] In different frequency bands, different percentages are retained according to the amplitude distribution of each speech sound. Low frequencies are the main components of human vocalization, so only frequencies with sufficient energy need to be retained. Mid frequencies are an important basis for the human ear to distinguish different sounds, so more frequency components need to be retained. High frequencies are mostly consonants and noise, so only the minimum information needs to be retained.

[0060]

[0061] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A spoof speech detection algorithm based on principal component filtering, characterized in that, The method comprises the steps of: Collecting voice data and extracting features, and then dividing the voice data into a training set and a test set; Masking filter main body extraction and amplitude filter main body extraction are performed on the training set and the test set respectively, and interference data of the voice data in the spectrogram due to re-encoding and noise is removed to obtain the extracted training set and test set; During masking filter main body extraction: Masking measurement is performed on the spectrogram features to calculate a spectrogram feature masking curve; According to the masking curve, the frequency components that are masked in the original spectrogram features are removed to obtain a non-masked power spectrogram; During amplitude filter main body extraction: The non-masked power spectrogram is divided into multiple frequency band parts according to the characteristics of human voice production and hearing; An adaptive amplitude filtering algorithm is used to remove noise signals in each frequency band part to obtain a main signal power spectrogram; The training set is used to train a detection model to obtain a trained detection model; The trained detection model is used to perform real-time detection on the fake voice.

2. The spoof speech detection algorithm based on host filtering according to claim 1, wherein, The feature extraction is performed on the spectrogram features of the voice to be detected by using a general feature extraction algorithm.

3. The fake voice detection algorithm based on main body filtering according to claim 1, wherein: The Bark domain spectrogram, the sound pressure level SPL of the spectrogram amplitude, and the local peak points of the frequency curve are calculated according to the voice power spectrogram; The Bark frequency band, the sound pressure level SPL of the spectrogram amplitude, and the local peak points are brought into a masking transfer function to calculate a masking curve; The frequency components with amplitudes lower than the masking curve are removed to obtain a non-masked voice power spectrogram.

4. The fake voice detection algorithm based on main body filtering according to claim 1, wherein: The non-masked power spectrogram is divided into high, medium, and low frequency bands; Amplitude filtering is performed on different frequency bands according to the adaptive energy level in the frequency band region.

5. The fake voice detection algorithm based on main body filtering according to claim 1, wherein: The Bark domain spectrogram is calculated according to the formula 1, （1） wherein denotes a frequency value, denotes a frequency domain value in Bark scale.

6. The spoof speech detection algorithm based on host filtering according to claim 1, wherein, The core of the adaptive filtering is shown in the formula 2: （2） an absolute value representing the amplitude, all the frequency components in the frequency band are sorted in descending order of their amplitudes, there are two parameters, the first parameter is all the frequency components of the frequency band, and the second parameter is the calculated amplitude point, all the amplitudes of the frequency components whose amplitudes are lower than the second parameter are set to be singular values.

7. A spoof speech detection system based on host filtering, characterized by, The method comprises the steps of: The first module is configured to collect voice data, extract features, and then divide the voice data into a training set and a test set; The second module is configured to perform masking filter main body extraction and amplitude filter main body extraction on the training set and the test set respectively, remove interference data of the voice data in the spectrogram due to re-encoding and noise, and obtain the extracted training set and test set; During masking filter main body extraction: Masking measurement is performed on the spectrogram features to calculate a spectrogram feature masking curve; According to the masking curve, the frequency components that are masked in the original spectrogram features are removed to obtain a non-masked power spectrogram; During amplitude filter main body extraction: The non-masked power spectrogram is divided into multiple frequency band parts according to the characteristics of human voice production and hearing; An adaptive amplitude filtering algorithm is used to remove noise signals in each frequency band part to obtain a main signal power spectrogram; The third module is configured to train a detection model by using the training set to obtain a trained detection model; The fourth module is configured to use the trained detection model to perform real-time detection on the fake voice.

Images