Secure Automatic Speaker Verification System

Through multi-level classification and sm-ALTP features combined with unbalanced bagging technology, the ASV system's susceptibility to audio spoofing attacks is solved, and high-accurate audio classification and speaker verification security are achieved.

CN115280410BActive Publication Date: 2024-11-26THE RGT UNIV OF MICHIGAN
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202180020058.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-01-13
Filing Date
2021-01-12
Publication Date
2024-11-26
Estimated Expiration
2041-01-12

AI Technical Summary

Technical Problem

Existing Automatic Speaker Verification (ASV) systems are vulnerable to audio spoofing attacks, especially voice playback attacks and voice cloning attacks, resulting in unreliable authentication.

Method used

Using multi-level and multi-category classification methods, by extracting pattern and power spectrum features in audio signals, constructing feature vectors, and using a classifier set for classification, identifying real, recorded or computer-generated audio signals, and modifying acoustic localities in combination with symbols. Three-valued mode (sm-ALTP) features and unbalanced bagging technology optimized classifier training to combat cloned audio generated by unseen algorithms.

Benefits of technology

Significantly improves the ability to combat voice cloning and replay attacks, achieves high-accuracy real and spoofed audio classification, ensuring the security and reliability of speaker verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115280410B_ABST
    Figure CN115280410B_ABST
Patent Text Reader

Abstract

Traditional speaker verification systems are vulnerable to voice spoofing attacks, such as voice replay attack, voice cloning attack, and clone replay attack. To overcome these drawbacks, a secure automatic speaker verification system based on a novel sign-modified acoustic local ternary pattern (sm‑ALTP) feature and an ensemble of classifiers based on unbalanced bagging with an enhanced attack vector machine is proposed. The proposed audio representation method clusters the high-frequency and low-frequency components in an audio frame by normally distributing them on a convex function. Afterwards, neighborhood statistics are applied to capture user-specific vocal tract information.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 960,356, filed on January 13, 2020. The entire disclosure of the above application is incorporated herein by reference.

[0003] Government Terms

[0004] This invention was made with government support under Grant CNS1816019 from the National Science Foundation. The government has certain rights in this invention. Technical Field

[0005] The present disclosure relates to secure automatic speaker verification. Background Art

[0006] Automatic speaker verification (ASV) is an essential component of voice biometric applications. These applications authenticate speakers based on their unique voice characteristics and protect user accounts from identity theft. However, security vulnerabilities have emerged that cause ASV systems to fail due to synthetic audio generation algorithms and digitally processed fake audio, making voice biometric applications unreliable. Similarly, smart speakers and many voice-enabled IoT devices that rely on automatic speaker verification and / or voice recognition are also vulnerable to audio spoofing attacks.

[0007] Audio spoofing attacks on ASV systems can be categorized into speech replay attacks, speech synthesis (or voice cloning) attacks, and voice conversion (VC) attacks. In a replay attack, the pre-recorded speech of the real target speaker is played back to deceive the ASV system. Replay attacks pose the greatest threat because they are easy to launch, and the only prerequisite for launching these attacks is to have the pre-recorded speech of the speaker. In addition, since the feature representations of the original and recorded signals are highly similar, the ASV system cannot distinguish them.

[0008] Voice cloning techniques collect pre-recorded voice samples of a speaker and aim to generate voice samples that are perceptually indistinguishable from real voices. Voice samples generated by voice cloning algorithms are even more difficult to detect than voice replay attacks, due to the lack of nonlinear components in cloned voices that appear in subsequent recordings and also serve as signs of replay.

[0009] The present disclosure regards audio spoofing attack detection as a multi-level and multi-category classification problem, rather than treating the problem as a binary classification problem in which audio samples are labeled as real or spoofed. The goal of the proposed system is to combat replay attacks while identifying the target speaker and the cloning algorithm used for the attack. The reason for performing cloning algorithm detection is that the present disclosure hopes to find the cause of the ASV system's failure on certain spoofed audio through cloning algorithm detection. In addition, through algorithm analysis, the present disclosure aims to analyze the characteristics of cloning algorithms in order to develop a secure ASV system that can also combat cloned audio generated by unseen algorithms. In this regard, the present disclosure emphasizes the following points: (1) analysis of speaker-specific cloning algorithm behavior; (2) addressing the deficiencies of ASV component and classifier optimization to combat unseen algorithms; and (3) addressing the class imbalance problem to distinguish real audio from spoofed audio.

[0010] This section provides background information related to the present disclosure which is not necessarily prior art. Summary of the invention

[0011] This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.

[0012] In one aspect, a computer-implemented method for speaker verification is provided. The method includes: receiving an audio signal from an unknown speaker; extracting features from the audio signal, wherein the features are based on patterns in the audio signal and a representation of a power spectrum of the audio signal; constructing a feature vector based on the extracted features; and classifying the audio signal using the feature vector, wherein the audio signal is classified as one of real, recorded, or computer-generated.

[0013] One or more of the extracted features represent patterns in the amplitudes of data samples obtained from the audio signal, and the amplitudes of the data samples may be quantized using a local ternary pattern. In some embodiments, the data samples are quantized using an adaptive threshold, wherein the data samples are grouped into multiple frames and the adaptive threshold for a given frame is a mathematical function of a standard deviation of the amplitudes of the data samples comprising the given frame.

[0014] In an embodiment, the representation of the power spectrum of the audio signal is further defined as mel-frequency cepstrum coefficients and is further defined as a support vector machine (Support Vector Machine).

[0015] In some embodiments, at least one classifier is further defined as a set of classifiers, wherein a first classifier in the set of classifiers is trained to recognize recorded audio signals and a second classifier is trained to recognize computer-generated audio signals. The first classifier and the second classifier can be trained using asymmetric bagging and subspace sampling.

[0016] In another aspect, an audio signal from an unknown speaker is classified using a set of classifiers. In a similar manner, a first feature is extracted from the audio signal, wherein the first feature represents a pattern in the amplitude of data samples obtained from the audio signal; an additional feature is extracted from the audio signal, wherein the additional feature represents a power spectrum of the audio signal. A feature vector is constructed by concatenating the first feature and the additional feature. The audio signal is classified by a first classifier, wherein the first classifier is trained to recognize recorded audio signals; the audio signal is classified by a second classifier, wherein the second classifier is trained to recognize computer-generated audio signals; and the audio signal is classified by a third classifier, wherein the third classifier is trained to recognize real audio signals. Finally, based on the outputs from the first classifier, the second classifier, and the third classifier, the audio signal is labeled as one of real, recorded, or computer-generated.

[0017] In yet another aspect, a system for secure speaker verification is provided. The system includes a feature extractor, an audio library, and one or more classifiers. The feature extractor is configured to receive an audio signal from an unknown speaker. The feature extractor operates to extract features from the audio signal and construct a feature vector based on the extracted features. The extracted features include a first feature and an additional feature, the first feature representing a pattern in the amplitude of a data sample obtained from the audio signal, and the additional feature representing a power spectrum of the audio signal.

[0018] The audio library stores audio samples from a plurality of registered speakers, and the audio library includes audio samples from unknown speakers.

[0019] The classifier is connected to the audio library and configured to receive the feature vector from the feature extractor. Then, the classifier classifies the audio signal by comparing the feature vector with the audio samples in the audio library, wherein the audio signal is classified as one of a real signal, a recorded signal or a computer-generated signal.

[0020] Further areas of applicability will become apparent from the description provided herein.The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.

[0022] Figures 1A to 1C Different arrangements of a secure automatic speaker verification (SASV) system are shown.

[0023] Figure 2 A flow chart providing an overview of an improved speaker verification method performed by a speaker verification subsystem is provided.

[0024] Figure 3 is a diagram illustrating an example architecture of a speaker verification subsystem.

[0025] Figure 4A and Figure 4B These are the spectrum analysis images of the original speech and the cloned speech respectively.

[0026] Figure 5 A diagram depicting the sign modified acoustic local ternary pattern (sm-ALTP) representation of the real audio and the cloned audio.

[0027] Figure 6 Figure 2 is a graph of a sample audio frame.

[0028] Figure 7 Confusion matrix for voice clone and voice algorithm detection using a 70-30 ratio.

[0029] Figure 8 Confusion matrix for voice clone and voice algorithm detection using 30-70 ratio.

[0030] Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings.

[0032] Figures 1A to 1CDifferent arrangements of a secure automatic speaker verification (SASV) system 10 are shown. The SASV system 10 generally consists of a microphone 12, an application 14, and a speaker verification subsystem 16. The microphone 12 collects an audio signal from a speaker and transmits the audio signal as input to the application 14. In one example, the application 14 is an online shopping application. The application 14 is then connected to the speaker verification subsystem 16. As will be further described below, the speaker verification subsystem 16 uses the audio signal transmitted by the application 14 to verify the identity of the speaker. The speaker's identity is then transmitted back to the application 14 by the speaker verification subsystem 16. An online shopping application is just one example of an application that may require speaker verification.

[0033] exist Figure 1A In FIG. 1 , microphone 12, application 14, and speaker verification subsystem 16 are located on the same computing device. Figure 1B In FIG. 1 , microphone 12 is located on a client computing device; and application 14 and speaker verification subsystem 15 are located on a server computing device. Figure 1C In the embodiment of the present invention, microphone 12 and application 14 are located on a client computing device; while speaker verification subsystem 16 is located on a server computing device. In both cases, the client computing device and the server computing device are interconnected via a network such as the Internet. Other arrangements of SASV system 10 may also be considered by the present disclosure.

[0034] Figure 2 An overview of an improved speaker verification method performed by the speaker verification subsystem 16 according to the present disclosure is provided. As a starting point, an audio signal from an unknown speaker is received by a signal processor at 21. It is assumed that the input audio signal is from a speaker that has been previously registered in the speaker verification subsystem 16.

[0035] At 22, features are extracted from the input audio signal. In an example embodiment, the extracted features include a first feature and an additional feature, the first feature representing a pattern in the amplitude of a data sample obtained from the audio signal, and the additional feature representing a power spectrum of the audio signal. More specifically, the first feature is constructed by quantizing the data samples using a local ternary pattern. Preferably, the data samples are quantized using an adaptive threshold, wherein the data samples are grouped into multiple frames, and the adaptive threshold for a given frame is a function of the standard deviation of the amplitude of the data samples comprising the given frame. The method for constructing the first feature will be described in more detail below. In one example, the additional feature is also defined as a Mel-frequency cepstral coefficient, although other indicators of the power spectrum also fall within the scope of the present disclosure.

[0036] Then, at 23, a feature vector is constructed based on the extracted features. In an exemplary embodiment, the feature vector includes the first feature and the additional feature. That is, the first feature is concatenated with the additional feature to form the feature vector.

[0037] At 24, based on the feature vector, the audio signal is classified using at least one classifier. In particular, the audio signal is classified as one of a real audio signal, a recorded audio signal, or a computer-generated audio signal. Although the classifier is preferably implemented by a support vector machine, neural networks, decision trees, and other types of machine learning techniques are also contemplated by the present disclosure.

[0038] In an example embodiment, an audio signal is labeled as one of a real audio signal, a recorded audio signal, or a computer-generated audio signal using a set of classifiers. In this example, a first classifier is trained to recognize a recorded audio signal, a second classifier is trained to recognize a computer-generated audio signal, and a third classifier is trained to recognize a real audio signal. The outputs of each of these classifiers are then combined and used to label the audio signal. Preferably, the first classifier, the second classifier, and the third classifier are trained using non-balanced bagging and subspace sampling.

[0039] In addition, at 25, the feature vector may also be used to identify the speaker of the audio signal. In one example, the speaker is identified by comparing the feature vector with audio samples from registered speakers stored in an audio library. In a similar manner, a classifier may also be used to accomplish speaker identification. It will be appreciated that Figure 2 Only relevant steps of the method are discussed, but other software implementation instructions may be required to control and manage the overall operation of the speaker verification subsystem.

[0040] Figure 3 An example architecture of a speaker verification subsystem 16 is further shown. In this example embodiment, the speaker verification subsystem 16 includes an audio library 32, a feature extractor 34, and a set of classifiers 36. The speaker verification subsystem 16 may also include additional classifiers 37 for speaker identification and / or clone algorithm detection. A profile generator 38 receives outputs from each of these classifiers and generates a profile of an unknown speaker. These components may be located in and / or implemented by one or more signal processors.

[0041] During operation, the feature extractor 34 receives an audio signal from an unknown speaker, extracts features from the audio signal, and then constructs a feature vector based on the extracted features, for example, as described further below. One or more classifiers in the classifier set 36 use the feature vector to classify the audio signal. In an example embodiment, the classifier set 36 includes three classifiers, although more or fewer classifiers are also contemplated by the present disclosure.

[0042] The audio library 22 includes replayed speech, cloned speech, and real speaker speech. In one example, for each registered speaker, cloned speech is generated by multiple speech cloning algorithms. Therefore, for m real speakers and p speech cloning algorithms, there are (m×p) cloned speaker categories. Regarding adversarial cloned audio samples generated by any unseen speech cloning algorithm (on which the model has not been trained), the model may incorrectly predict the cloning algorithm type, but the model will still detect a cloning attack. In this case, the model will mark the input audio as cloned audio. Similarly, for replay attack detection, the input audio sample will be marked as replay / real audio. Therefore, there are q=m+(m×p)+2+2 speaker categories to be identified.

[0043] Automatic Speaker Verification (ASV) systems are vulnerable to replay attacks, voice cloning attacks, and voice conversion attacks, so an effective audio representation mechanism should take into account the following facts. First, microphones add a layer of nonlinearity due to intermodulation distortion, introducing multiple detectable patterns; therefore, during audio fingerprinting, the audio representation mechanism should be able to characterize these patterns in order to well discern between the original audio and the replayed audio. Second, subsequent recordings of the same recording (very common in audio splicing attacks) introduce high-order nonlinearities and make the audio signal easier to discern through pattern-based analysis. Third, voice cloning algorithms also introduce obvious artifacts / distortions, and neighborhood statistics need to be applied to capture these artifacts. Fourth, the audio representation mechanism of the ASV system should be less sensitive to noise in the acoustic environment that is not related to the audio processing.

[0044] For feature extraction, a novel feature extraction method is proposed, which is referred to as sign modified acoustic local ternary patterns (sm-ALTP) in this article. In an example embodiment, these features are based on patterns in the audio signal (e.g., local ternary patterns) and a representation of the power spectrum of the audio signal. The representation of the power spectrum can be limited to Mel-frequency cepstral coefficients, although other representations are also considered in the present disclosure. The sm-ALTP feature is an extension of the ALTP feature, which is further described in Adnan et al. "Fall Detection Through Acoustic Local Ternary Patterns" Applied Acoustics, vol. 140, pp. 296-300 (2018), which is incorporated in its entirety.

[0045] For the ALTP feature, the input audio signal Y[n] with N samples is divided into i={1; 2; :::; k} non-overlapping frames or windows F with length l=9 (i) In each frame F (i) In this example, c represents the center sample in a frame and has z j adjacent samples, where j represents F (i) The indices of neighboring samples in the frame. To compute the ALTP response, we apply the parameter t around sample c. h To calculate c and z j The difference in the amplitude spectrum between . Parameter t h The value of is between 0 and 1 and is obtained by performing a linear search operation. (i) Located around c ±t h Sample values ​​within the width range are quantized to zero, while values ​​greater than c+t h The sample value is quantized to 1, which is less than ct h The sample value of is quantized to -1. Therefore, a three-valued function is obtained:

[0046]

[0047] Then, the function p(c,z j ,t h ) is decomposed into two mode categories, namely, the upper mode P up (.) and the next pattern P lw (.)for:

[0048]

[0049] Similarly,

[0050]

[0051] These upper and lower patterns are then used for upper and lower ALTP representation generation. The upper ALTP feature A is calculated using Equation 4. U .

[0052]

[0053] The ALTP feature A is calculated by Equation 5. L .

[0054]

[0055] Then, as described in Equations 6 and 7, A is calculated by applying the Kronecker delta function δ(.) U and A L Histogram of .

[0056]

[0057]

[0058] Where b represents the bin and a represents the frame index. u (b) and H l (b) Then, by concatenating (||) the two histograms, we obtain the ALTP feature:

[0059] H A =[H u (b)||H l (b)] (8)

[0060] The ALTP feature was originally proposed for indoor applications (i.e., fall detection); and due to its tolerance to noise, it has shown very good performance as a feature descriptor against state-of-the-art feature extraction methods. However, when applied to speaker verification systems, ALTP has some drawbacks that need to be overcome. These drawbacks are—(a) non-static pattern detection—e.g. Figure 4A and Figure 4B As shown, spectral analysis of the cloned audio shows that the artifact has a non-static repetitive pattern, which can be more effectively captured by a dynamic threshold mechanism. However, ALTP only has a static threshold (i.e., ±t h), therefore, there is room for improvement in speaker verification applications for ALTP. (b) Signal volatility - In order to effectively capture artifacts in cloned and replayed audio, it is important to understand how fast the signal changes with respect to artifacts. ALTP features lack this property. Therefore, performance against spoofed audio is degraded. (c) Brute-force optimization - In ALTP, a brute-force approach is required for threshold optimization; therefore, error reduction cannot be guaranteed in time critical applications. (d) Noise uniformity - ALTP is robust to uniform noise that remains consistent in audio scenes (e.g., indoor audio); however, in outdoor environments, due to the non-uniform noise, feature extraction based on static thresholds becomes unstable, requiring a different approach for noise suppression.

[0061] The ALTP feature was originally proposed for sound classification, and for sound classification, it is not necessary to collect the vocal tract of the sound source. Therefore, the ALTP feature performs very well against the most advanced feature extraction methods. However, in speaker verification, it is necessary to identify the speaker in a text-independent scenario, so it is important to model the speaker's vocal tract.

[0062] The sm-ALTP feature overcomes the limitations of the ALTP feature by defining a dynamically optimizable threshold and collecting the speaker's vocal tract. In sm-ALTP, the three-value function is calculated as:

[0063]

[0064] Where σ is F (i) The standard deviation of , α is the scaling factor, i.e. (0<α<1). σ can be calculated as:

[0065]

[0066] By replacing t with (σ×α) h , which overcomes the limitations of ALTP features (a), (b), and (d), which requires incorporating signal variance in neighborhood statistics. Another limitation of ALTP features is that t h A brute force optimization is required through linear search. However, by defining the following convex function, the new threshold (i.e. (σ×α)) can be optimized.

[0067]

[0068] Where J(.) is the cost function, θ is the classification weight, q = {1, 2, ..., M} is the total number of records in the training set, g is the classification function used (i.e., rectified linear unit (relu), sigmoid, hyperbolic tangent (tanh), etc.), y q represents the actual classification label of the audio recording. The probability of the cost function is expressed as:

[0069]

[0070] Then, the parameter σ can be optimized by applying the gradient descent algorithm as:

[0071]

[0072] in

[0073]

[0074] therefore

[0075]

[0076] Or in shorthand, it can be written as:

[0077]

[0078] Therefore, the partial derivative will return:

[0079]

[0080] or

[0081]

[0082] By replacing equations 2 to 5 with (σ×α), H is obtained using equations 6 and 7. u (b) and H l (b), and generate the feature representation as:

[0083] H=[H u (b)||H l (b)] (19)

[0084] The feature representation H captures the patterns present in the input signal, but this representation lacks vocal tract information that can be captured, for example, by Mel-scaled cepstral coefficients. For example, at 1000 Hz, the cepstral coefficients of a particular speaker always appear negative due to the phonemic representation of the phonological structure of the particular speaker, and this frequency occurs very frequently; in the case of sm-ALTP, a large positive histogram peak will appear, but the peak will not provide any information about the phonological behavior at this particular frequency. Therefore, the present disclosure proposes further processing of the sm-ALTP representation using Equation 20.

[0085] H s =H×sgn(μ t (C γ t))×β (20)

[0086] Among them, C γ (t) is the yth th Frame t th Cepstral coefficient, μ t C γ (t), and t = {1, 2, ..., 20}.

[0087] As shown in Equation 21, C is implemented by calculating the frame energy E(f) using the index f γ (t).

[0088]

[0089] In Equation 20, the parameter β = 0.1 is used for H s Then, the final representation of sm-ALTP features can be expressed as:

[0090] H sm =[μ t (C γ (t))||H s (twenty two)

[0091] Although reference has been made to Mel-frequency cepstral coefficients, other representations of the power spectrum of an audio signal also fall within the broader aspects of the present disclosure.

[0092] No matter how powerful the feature extraction method can be, the characteristics of the data in terms of data quality, data collection mechanism, and dataset size will affect the classification performance in speaker verification systems. For example, if the training set consists of fewer true representations and much more spoofed representations, the classifier will tend to favor the spoofing class. In this specific case, a higher classification accuracy may be the result of a bias towards the spoofing class; in fact, the classifier is providing much lower performance for true samples, which is the main goal of any speaker verification system. Therefore, even a higher classification accuracy will become meaningless. At this point, it is crucial to find out what causes the classifier to produce incorrect outputs. To achieve this goal, for clone attack detection, the cloning algorithm used to generate spoofed audio is identified. The classification model can be further improved by collecting correlations between spoof samples and cloning algorithms. In this example embodiment, the classification model is designed for the following tasks: 1) Speaker identification for registered users; 2) Detection of cloning attacks by seen and unseen algorithms; 3) Detection of algorithms used for clone audio generation; 4) Target speaker identification in order to identify users with enhanced security requirements; 5) Replay and clone replay attack detection. While approaching an efficient classification model, the complexity of the testing process can be ensured not to increase in a way that makes the classification model unsuitable for real-time applications. Therefore, tasks with balanced data representation (i.e., tasks 1, 3, and 4) are handled by a single SVM classifier with a polynomial kernel; while for tasks with highly imbalanced datasets (i.e., tasks 2 and 5), multiple classifiers are trained using unbalanced bagging and subspace sampling.

[0093] In order to generate multiple classifiers, unbalanced bagging and subspace sampling are used in the proposed SASV system 10. In unbalanced bagging, boosting is performed on the deception class samples because there are many more deception samples than real samples. In this way, each classifier is trained on a balanced set using a complete real set and a subset of deception samples, thereby improving the unstable SVM classification performance. The stable SVM classifier becomes able to distinguish well even unseen real and deception samples; however, if other data balancing methods (e.g., upsampling or downsampling) are used instead of unbalanced bagging, the classifier becomes either overfitting or underfitting and exhibits very low performance for unseen samples. After unbalanced bagging, aggregation of multiple classifiers is performed, such as aggregation of multiple classifiers by weighted normalized voting rule (wNVR) on the development set. Other techniques for aggregating the outputs of multiple classifiers are also within the consideration of the present disclosure.

[0094] After training multiple classifiers, the results of the classifiers are integrated. In this example embodiment, a weighted normalized voting rule (wNVR) is applied to aggregate the results of all classifiers, although other aggregation rules including majority voting rule (MVR) are within the consideration of the present disclosure. wNVR is a new concept proposed for ensemble generation. The reason for choosing wNVR instead of MVR is that MVR cannot take advantage of the precise classifiers and gives the same weight to all classifiers.

[0095] Assume that w = {1, 2, ..., Q}, the set classifier is generated by applying the weighted cross-entropy function:

[0096]

[0097] Where λ is a weight so that for k = {1, 2, ..., K} categories to be classified, the advantage of more accurate classifiers is taken, and b = {1, 2, ..., M} is the number of instances x in the development set. b The final category label C*(x) is generated as:

[0098]

[0099] The parameter s is a normalization factor to control bias / variance effects.

[0100] After training and model optimization, the trained model can be used for evaluation. The evaluation set consists of instances with seen real speakers and unseen real speakers, and in the case of voice cloning attacks, the evaluation set consists of instances with seen real speakers and unseen real speakers, as well as instances of samples generated by seen and unseen algorithms. After model evaluation, any query audio samples can be passed to the final model, and the final model can perform speaker verification tasks in real-time scenarios.

[0101] Existing methods ignore some important signal characteristics during feature extraction, which ultimately degrades the performance of these methods. For example, the first three limitations emphasize that during playback and voice cloning, intermodulation and algorithmic artifacts appear, which exhibit discernible patterns. The proposed method effectively captures these artifacts to distinguish spoofed signals from real signals due to pattern analysis of the input signal. For example, Figure 5 As shown, the real audio and the cloned audio show spikes at the same feature points, but due to the differences in these spikes, the signals are still easily distinguishable. In addition, at certain feature points (such as Figure 5 At feature 16 in , the real signal and the spoofed signal show spikes in opposite directions. Figure 5 The difference in eigenvalues ​​in indicates that the cloned audio, although behaving similarly to the real audio, the essential signal components (i.e., pitch, volume, etc.) are not perfectly replicated. However, such differences can be easily revealed by a lower-level analysis of the input signal using the proposed method.

[0102] Another limitation of audio representation methods is that their robustness to noise is not easily quantified. However, the proposed method is robust to noise and this claim can be easily verified. Figure 6 It can be seen that additive noise (which can either increase or decrease F (i) The value of the center sample c in the frame, which can be the reason for generating an error code for c) will become invalid. The reason is that the value of sample c is now within the range between the upper and lower thresholds, and thus becomes more tolerant to noise additive values. In addition, since the features are not too complex, fast model retraining is possible; thus making the proposed method effective for applications with the need for continuous user registration.

[0103] The performance of the proposed system is evaluated on the ASVspoof-2019 dataset and the voicespoofing detection corpus (VSDC). The ASVspoof-2019 dataset (Table 1) also includes two datasets, namely the logical access (LA) dataset for voice cloning attack detection and the physical access (PA) dataset for replay attack detection. The LA dataset has 25,380 samples for training, 24,844 samples for development, and 71,933 samples for evaluation. The training and development sets contain speech samples of 20 speakers (different speakers in both datasets) as the true category; while the spoofing set has clone samples of the same speaker's utterance generated by 2 voice conversion algorithms and 4 speech synthesis algorithms, which consist of 120 (20×6) spoofed speaker-plus-algorithm categories. The voice conversion algorithms are based on (i) neural network-based methods and (ii) transfer function-based methods. The speech synthesis algorithms are implementations of (i) waveform concatenation, (ii) neural network-based parametric speech synthesis using source filter vocoder, and (iii) neural network-based parametric speech synthesis using Wavenet. The evaluation set includes unseen real speech samples and unseen spoof speech samples collected from multiple speakers. The spoof set includes samples generated by 19 algorithms, including methods based on generative adversarial networks (GANs) and methods based on deep neural networks. The PA dataset consists of 54,000 training samples, 33,534 development samples, and 153,522 evaluation samples (Table 1).

[0104] VSDC is designed for replay and clone replay attack detection. Clone replay represents the recording of a cloned voice sample; for this purpose, ASV is used to spoof the clone sample to generate the replay sample in a similar way to the real voice recording. The samples in the dataset differ in terms of environment, configuration, speaker type, recording, replay equipment, and number of speakers (Table 2). More specifically, the samples also contain noise and interference. To generate the replay audio, different replay equipment is used to mitigate the impact of the specific replay equipment. VSDC includes voice samples of ten male and five female speakers who volunteered to provide their services for data collection.

[0105] Next, the performance of the proposed method is evaluated for real speaker verification. Real speaker verification is the main task performed by any ASV system. As shown in Table 3, the proposed method achieves an average precision, recall, f1 score, and accuracy value of 99% on the ASV Spoof-2019 dataset using a 70-30 ratio (i.e., 70% of the data is used for training and 30% of the data is used for testing). For most categories, the evaluation rate is 100%; while no category may have more than 1 misclassified sample; there are only 7 misclassified samples out of 774 test samples corresponding to 20 speaker categories, which is negligible. In addition, even if the training and testing ratio is changed to 30-70 (i.e., 30% of the data is used for training and 70% of the data is used for testing), the proposed method still gives an average precision, recall, f1 score, and accuracy value of 98% using 1806 samples for testing out of 2580 total real samples, which clearly shows that the method is robust to the ASV task.

[0106] In this experiment, the performance of the proposed method was evaluated for the synthetic audio generation algorithm test. The synthetic audio generation algorithm includes the speech conversion algorithm and the speech synthesis algorithm as described above. From the results shown in Table 4, it can be seen that the proposed method gives approximately 100% performance in all performance evaluation levels. If we compare Figure 7 and Figure 8 The confusion matrix shown shows that the algorithmic detection performance of the proposed method remains the same even when the number of test samples increases from 6803 to 15874. Therefore, these results confirm the hypothesis that these algorithms introduce their own unique properties / artifacts in the generated cloned audio, which are usually different from those introduced by other audio generation algorithms; a good audio representation with an effective classification mechanism can exploit these artifacts to perform algorithmic level detection; therefore, the attack detection profile becomes more reliable. This feature can be particularly beneficial for audio forensics applications, especially in litigation cases, by inspiring higher confidence.

[0107] In order to protect registered users and ASV-based applications, it is important to identify which registered users' voices have been stolen. With this information, additional security measures can be taken to further protect these specific speakers, as well as applications. Therefore, in this experiment, the algorithm and speaker information were combined together and then used as training and test labels as shown in Table 5. In this way, 6 audio synthesis algorithms were used for 20 registered speakers with 120 audio categories. In Table 5, the results of 30 randomly selected categories are shown; from these results, it can be seen that the proposed method gives an accuracy of 97%, and the average of all performance evaluation levels is also 97%. The difference between the accuracy values ​​of Table 6 and Table 5 is about 2.6%, which is attributed to the probability that the sample is partially associated with a specific output label. For example, a sample that is misclassified for a real speaker can still be associated with the correct voice cloning algorithm. In addition, as in the case of algorithm detection (Table 6), since there are only 6 categories, the error bound is lower in this case. However, if a method still gives a high performance even by applying a drill down operation to further diversify the classification problem, then the method will be considered reliable. Therefore, based on these results, it can be said that the proposed method reliably provides us with information about the stolen speakers, which is also a unique property of the proposed method.

[0108] For this experiment, 76,236 unseen instances were selected for evaluation purposes. Of these instances, 9,902 instances were real instances and 66,334 instances were cloned instances. However, these 76,236 instances consisted of 5,000 instances from the ASV Spoof-2019 development set and 71,236 instances from the evaluation set, which have never been used for training. All of these instances have unseen speakers (20 speakers from the development set and 67 speakers from the evaluation set), and 19 different voice cloning algorithms and voice conversion algorithms (including the 6 algorithms mentioned in Table 4 and the remaining 13 algorithms in Table 7) are used for cloned audio generation of these 87 speakers. Since the algorithms used for audio synthesis have never been used for training the proposed method, the method cannot predict the algorithm labels. Therefore, for this experiment, the model was trained using a training set with two labels (i.e., real and cloned). Therefore, the purpose of this experiment is to evaluate whether the proposed method can distinguish any real / cloned audio, no matter who the speaker is or how the cloning is performed.

[0109] From the results presented in Table 6, it can be seen that the proposed method gives an overall accuracy of 88%. By further applying a drill-down operation on the accuracy values, it can be found that the accuracy of the true class is 86%, while the average accuracy of the clone class is 90%; therefore, the overall accuracy becomes 88%. Among the 87 speakers, the average accuracy remains above 90% for 72 speakers, which is fair considering that only 20 speakers are used for training purposes and these 20 speakers are not considered for evaluation purposes in this experiment. Similarly, as shown in Table 7, if the 13 algorithms that were not used for training are analyzed, it can be seen that the accuracy of 8 algorithms is close to 100%; while the accuracy of 2 algorithms exceeds 90%. The most problematic algorithms are A17 to A19, whose accuracy drops significantly. However, it can be seen from Table 7 that the number of samples in all these algorithm categories is the least. A17, which gives the lowest accuracy, is only about 27% (in terms of sample size) of A09, which has the highest accuracy of 100% and also contains the largest number of samples. Therefore, we can infer that model optimization is positively correlated with the number of samples; although no external algorithm labels are used, our model still identifies correlations between unique types of artifacts introduced by any synthesis algorithm; and the model returns correct outputs for most samples.

[0110] For a good algorithm, a high accuracy value is one of the many requirements for algorithm performance in class-dependent scenarios, including precision, recall, and f1-score. The reason for class-dependent analysis is that in the case of unbalanced data, if the classifier even ignores the secondary class, the classifier will still give a high overall accuracy and other performance evaluation measures. However, since the secondary class is usually the class of interest that must be considered, such a high evaluation value is unacceptable. For example, for this experiment, the true class in the training set has only about 13% of the samples; if our algorithm marks every sample as cloned, the overall accuracy is still 87%. However, since real speaker verification (especially for registered users) is a basic activity performed by any ASV system, such an accuracy is considered useless. Moreover, in real-world scenarios, failure to verify the true speaker will lead to frustration of registered users and will cause financial losses to companies providing ASV services. However, by observing the results shown in Table 6, it can be analyzed that the proposed method gives a precision of 67% for the true class and a precision of 97% for the clone class. Since the precision also takes into account the false positive rate, for highly imbalanced data (e.g., in the case where the ratio of the two classes is 13:87), the precision of the true class decreases; while the false positives in the clone class are less, so these highly imbalanced data do not have a very large adverse impact on the precision of the clone class. However, in the case of recall, since we only consider the correctly classified instances of a particular class for all the relevant instances of that class; therefore, in the case of the true class, the recall is 91%, which is about 24% higher than the precision. Similarly, for the clone class, the recall decreases by 6% and becomes 91%. Therefore, the proposed method provides good performance in terms of recall for the true class; while the method provides good performance in terms of precision for the clone class. Combining the precision and recall through the f1 score, we obtain f1 scores of 81% and 94% for the true class and clone class, respectively. The difference in f1 scores indicates that the model needs to enhance the training set to better classify unseen real instances. However, in real-world scenarios, since the proposed SASV system is required to correctly classify only registered real speakers (on which the model is trained) as real (as shown in Table 3), it is a good thing from a security perspective to misclassify unregistered users (although they are real speakers).

[0111] In a replay attack, pre-recorded speech of any real speaker is played back in front of the ASV system. Since the speech sample belongs to the real speaker, artifacts that occur during speech synthesis are lost in the replay sample; therefore, the audio fingerprint matches the real speaker exactly and the fake appears. However, a deeper analysis of the replay samples shows that the recorded speech also contains nonlinear components that can be used as signs for replay attack detection. In order to detect replay attacks, we first elaborate on what a replay sample consists of: 1) replay and clone replay patterns, and 2) replay and clone replay attack detection.

[0112] To carry out a replay attack, a microphone chain is involved (i.e. one microphone is used to record the speaker's voice, and the next microphone provided by the ASV system is used for system interaction). However, during the conversion of acoustic energy into electrical signals, the electromechanical components present in the microphone (such as carbon fiber, circuit system, etc.) introduce nonlinearities in the input signal; any nonlinearity in these components will cause the output to be slightly distorted. In general, the stiffness of the mechanical suspension and the acoustic damping are the main causes of nonlinear distortion in most microphones, which can be classified as harmonic distortion, intermodulation distortion, and difference frequency distortion. Harmonic distortion is the effect of nonlinearity on pure tone excitation, resulting in harmonic components in the output. Intermodulation distortion is the effect of nonlinearity at the output caused by excitation as the sum of stronger high-frequency components and weaker low-frequency components. Difference frequency distortion is the effect of nonlinearity at the output caused by excitation of a sine wave of the same amplitude. Considering that speakers also behave in a nonlinear manner, a first-order speech replay attack can be modeled as a processing chain of microphone-speaker-microphone (MSM), which is equivalent to three cascaded second-order systems. Therefore, the processing chain representing the first-order replay attack is expected to introduce higher-order nonlinearities due to the cascade of the MSM processing chain. Therefore, higher-order harmonic distortion can be used to distinguish between real audio and spoofed audio. However, in the case of clone replay (introduced in VSDC), the speech cloning artifacts also contain nonlinear components and therefore have similar behavior to the deeper chain of MSM. In addition, clone replay can be detected by capturing both nonlinear components and cloning artifacts through an effective audio representation mechanism. The implementation of clone replay can be found in text-related scenarios, where the forger has speech samples of real speakers, but these samples are different from the samples used to launch the attack on the ASV system. Therefore, it can be said that clone replay attack detection is a basic requirement for secure ASV systems.

[0113] In this experiment, the performance of the proposed method is evaluated on the VSDC and ASV-spoof-2019 datasets for replay and clone replay attack detection. From the results shown in Table 8, it can be seen that the proposed method achieves significant performance on both datasets for audio replay attack detection. More specifically, 98.3% and 99% average precision, 98.5% and 99% average recall, and 98.4% and 99% average f1 scores are obtained on the VSDC and ASV-spoof-2019 datasets, respectively. From the results, it can be seen that the performance of the proposed method on the ASV-spoof-2019 dataset is slightly better than that on VSDC, which is due to the fact that the samples of VSDC are generated under more challenging and diverse conditions compared to the ASV-spoof-2019 dataset. In VSDC, the proposed method achieves better performance for clone replay attack detection compared to the first-order replay attack (which confirms the conclusion that the clone signal becomes more distorted compared to the normal sample after replay); therefore, the clone signal also becomes more discernible.

[0114] The present disclosure proposes a secure automatic speaker verification (SASV) system that can identify registered ASV users and can also fight against audio synthesis attacks, voice playback attacks, and audio injection attacks. The audio synthesis detection module distinguishes original speech from algorithmically generated synthetic audio, and also provides information about the algorithm previously used for cloned audio generation. The replay detection module fights against voice replay and cloned voice replay attacks. The audio injection detection module fights against scenarios where the communication channel is compromised and hackers attempt to infiltrate the system by skipping the ASV step. The proposed framework is based on novel sm-ALTP features and a collection of learning through unbalanced bagging. Through unbalanced bagging, the problem of class imbalance is overcome, and the dynamic range of the SVM classifier is generated. Evaluations on the ASV Spoof-2019 and VSDC datasets show that the method effectively captures deception patterns (even those generated by unseen algorithms), thereby providing a comprehensive security solution for ASV systems.

[0115] The technology described herein can be implemented by one or more computer programs executed by one or more processors. These computer programs include processor executable instructions stored on non-transitory tangible computer readable media. These computer programs may also include stored data. Non-limiting examples of non-transitory tangible computer readable media are non-volatile memory, magnetic storage, and optical storage.

[0116] Certain portions of the above description present the techniques described herein in terms of algorithms and symbolic representations of information operations. These algorithmic descriptions and representations are the means used by those skilled in the art of data processing to most effectively communicate the essence of their work to other skilled in the art. Although these operations are described functionally or logically, they should be understood to be implemented by computer programs. In addition, without loss of generality, it sometimes proves to be advantageous to refer to these operation arrangements as modules or to call these operation arrangements by functional names.

[0117] Unless otherwise expressly stated, as is apparent from the above discussion, it should be recognized that throughout the description, discussions using terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" refer to the actions and processes of a computer system or similar electronic computing device that operates on and transforms data represented as physical (electronic) quantities in a computer system memory or registers or other such information storage, transmission or display devices.

[0118] Certain aspects of the described techniques include processing steps and instructions described herein in algorithmic form. It should be noted that the described processing steps and instructions can be embodied in software, firmware, or hardware, and when embodied in software, can be downloaded to reside on different platforms used by real-time network operating systems and operated on these different platforms.

[0119] The present disclosure also relates to a device for performing the operations herein. The device may be specially constructed for the desired purpose, or the device may include a general-purpose computer that is selectively activated or reconfigured by a computer program stored on a computer-readable medium and accessible to a computer. Such a computer program may be stored in a tangible computer-readable storage medium, such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, an application-specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each medium is coupled to a computer system bus. In addition, the computer involved in this specification may include a single processor or may be an architecture designed with multiple processors for increasing computing power.

[0120] The algorithms and operations proposed herein are not inherently related to any particular computer or other device. Various general purpose systems may also be used together with programs according to the teachings herein, or it may prove to be convenient to construct multiple specialized devices to perform the required method steps. For those skilled in the art, the structures required for various these systems together with equivalent variations will be apparent. In addition, the present disclosure is described without reference to any particular programming language. It should be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present disclosure as described herein.

[0121] The above description of these embodiments has been provided for the purpose of illustration and description. This description is not intended to be exhaustive or to limit the present disclosure. The individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but where applicable these elements or these features are interchangeable and can be used in selected embodiments, even if not specifically shown or described. The same situation can also be varied in many ways. These changes should not be considered as departing from the present disclosure, and all of these modifications are intended to be included within the scope of the present disclosure.

[0122] appendix

[0123] Table 1

[0124]

[0125] Table 2 Table II: Detailed information of the Speech Spoofing Detection Corpus (VSDC)

[0126]

[0127] Table 3

[0128] Speaker ID Accuracy Recall F1 score LA_0079_bon 1.00 1.00 1.00 LA_0080_bon 0.97 0.94 0.96 LA_0081_bon 0.95 0.97 0.96 LA_0082_bon 1.00 1.00 1.00 LA_0083_bon 1.00 1.00 1.00 LA_0084_bon 1.00 1.00 1.00 LA_0085_bon 0.98 0.98 0.98 LA_0086_bon 0.95 1.00 0.97 LA_0087_bon 1.00 0.98 0.99 LA_0088_bon 1.00 0.97 0.99 LA_0089_bon 1.00 0.97 0.99 LA_0090_bon 1.00 1.00 1.00 LA_0091_bon 1.00 1.00 1.00 LA_0092_bon 1.00 0.98 0.99 LA_0093_bon 1.00 0.97 0.98 LA_0094_bon 1.00 1.00 1.00 LA_0095_bon 0.95 1.00 0.98 LA_0096_bon 1.00 1.00 1.00 LA_0097_bon 1.00 0.98 0.99 LA_0098_bon 0.95 1.00 0.97 Accuracy 0.99

[0129] Table 4

[0130] Algorithm ID algorithm Accuracy Recall F1 score A01 Neural Waveform Model 0.998 0.996 0.997 A02 Source Filter Vocoder-1 0.996 0.999 0.997 A03 Source Filter Vocoder-2 0.994 1.000 0.997 A04 Waveform splicing 0.990 0.987 0.989 A05 Source Filter Vocoder-3 0.997 0.995 0.996 A06 Spectral filtering 0.998 0.997 0.997 Accuracy 0.996

[0131] Table 5

[0132] Algorithm + Speaker ID Accuracy Recall F1 score A01_LA_0079_spoof 0.99 1.00 0.99 A01_LA_0081_spoof 0.98 0.98 0.98 A02_LA_0086_spoof 0.98 1.00 0.99 A02_LA_0088_spoof 0.98 1.00 0.99 A03_LA_0091_spoof 0.98 1.00 0.99 A03_LA_0092_spoof 1.00 1.00 1.00 A04_LA_0095_spoof 1.00 1.00 1.00 A04_LA_0097_spoof 1.00 0.98 0.99 A05_LA_0079_spoof 1.00 1.00 1.00 A05_LA_0081_spoof 1.00 1.00 1.00 A06_LA_0094_spoof 0.94 0.94 0.94 A06_LA_0095_spoof 0.96 0.91 0.94 Accuracy of 120 categories 0.97

[0133] Table 6

[0134]

[0135] Table 7

[0136] Algorithm ID algorithm Sample size Accuracy A07 Vocoder + Generative Adversarial Network (GAN) 4823 0.98 A08 Neural Waveform 4855 0.99 A09 Source Filter Vocoder-4 4893 1.00 A10 Neural Waveform 4878 0.99 A11 Griffin lim 4882 0.99 A12 Neural Waveform 4603 0.94 A13 Waveform splicing + waveform filtering 4908 1.00 A14 Source Filter Vocoder-5 4904 1.00 A15 Neural Waveform 4747 0.97 A16 Waveform splicing 4442 0.90 A17 Waveform filtering 1352 0.28 A18 Source Filter Vocoder-6 1855 0.38 A19 Spectral filtering 2345 0.48

[0137] Table 8

[0138]

[0139] Table 9

[0140]

[0141] Table 10

[0142]

Claims

1. A computer-implemented method for speaker verification, comprising: receiving, by a signal processor, an audio signal from an unknown speaker; extracting features from the audio signal by the signal processor, wherein the features are based on patterns in the audio signal and a representation of a power spectrum of the audio signal, wherein one or more of the extracted features represent patterns in amplitudes of data samples taken from the audio signal; constructing a feature vector by the signal processor according to the extracted features; and The audio signal is classified by at least one classifier using the feature vector, wherein the audio signal is classified as one of a real audio signal, a recorded audio signal, or a computer-generated audio signal.

2. The computer-implemented method of claim 1 , wherein: The amplitude of the data samples is quantized using a local three-valued pattern.

3. The computer-implemented method of claim 2, wherein: The data samples are quantized using an adaptive threshold, wherein the data samples are grouped into multiple frames and the adaptive threshold for a given frame is a mathematical function of the standard deviation of the magnitudes of the data samples comprising the given frame.

4. The computer-implemented method of claim 1 , wherein: The representation of the power spectrum of the audio signal is further defined as Mel-frequency cepstral coefficients.

5. The computer-implemented method of claim 1 , wherein: The at least one classifier is further defined as a support vector machine.

6. The computer-implemented method of claim 1 , wherein: The at least one classifier is further defined as a set of classifiers, wherein a first classifier in the set of classifiers is trained to recognize the recorded audio signal and a second classifier is trained to recognize the computer-generated audio signal.

7. The computer-implemented method of claim 6, further comprising: The first classifier and the second classifier are trained using unbalanced bagging and subspace sampling.

8. The computer-implemented method of claim 1 , further comprising: The unknown speaker is identified using the feature vector.

9. A computer-implemented method for speaker verification, comprising: receiving, by a signal processor, an audio signal from an unknown speaker; extracting, by the signal processor, a first feature from the audio signal, wherein the first feature represents a pattern in amplitudes of data samples obtained from the audio signal; extracting, by the signal processor, additional features from the audio signal, wherein the additional features represent a power spectrum of the audio signal; The signal processor constructs a feature vector by concatenating the first feature and the additional feature; classifying the audio signal using the feature vector by a first classifier, wherein the first classifier is trained to recognize recorded audio signals; classifying the audio signal using the feature vector by a second classifier, wherein the second classifier is trained to recognize computer-generated audio signals; classifying the audio signal using the feature vector by a third classifier, wherein the third classifier is trained to recognize real audio signals; and The audio signal is labeled as one of the real audio signal, the recorded audio signal, or the computer-generated audio signal based on outputs from the first classifier, the second classifier, and the third classifier.

10. The computer-implemented method of claim 9, wherein: The first feature is extracted by quantizing the data samples using a local three-value pattern using an adaptive threshold, such that the data samples are grouped into multiple frames, and the adaptive threshold for a given frame is a mathematical function of the standard deviation of the amplitudes of the data samples comprising the given frame.

11. The computer-implemented method of claim 9, wherein: The additional features are further defined as Mel-frequency cepstral coefficients.

12. The computer-implemented method of claim 9, wherein: At least one of the first classifier, the second classifier, or the third classifier is further defined as a support vector machine.

13. The computer-implemented method of claim 12, further comprising training at least one of the first classifier, the second classifier, or the third classifier using unbalanced bagging and subspace sampling.

14. A system for secure speaker verification, comprising: a feature extractor configured to receive an audio signal from an unknown speaker, the feature extractor operating to extract features from the audio signal and construct a feature vector based on the extracted features, wherein the extracted features include a first feature and an additional feature, the first feature representing a pattern in the amplitude of data samples obtained from the audio signal, and the additional feature representing a power spectrum of the audio signal; an audio library storing audio samples from a plurality of registered speakers, the audio library including audio samples from the unknown speaker; and One or more classifiers connected to the audio library and configured to receive the feature vector from the feature extractor, wherein the classifier classifies the audio signal by comparing the feature vector with the audio samples in the audio library, wherein the audio signal is classified as one of a real audio signal, a recorded audio signal, or a computer-generated audio signal.

15. The system of claim 14, wherein: The first feature is extracted by quantizing the data samples using a local three-value pattern using an adaptive threshold, such that the data samples are grouped in multiple frames, and the adaptive threshold for a given frame is a mathematical function of the standard deviation of the amplitudes of the data samples comprising the given frame.

16. The system of claim 14, wherein: The additional features are also defined as Mel-frequency cepstral coefficients.

17. The system of claim 14, wherein: One or more of the classifiers include: a first classifier configured to receive the feature vector from the feature extractor and classify the audio signal using the feature vector, wherein the first classifier is trained to recognize the recorded audio signal; a second classifier configured to receive the feature vector from the feature extractor and classify the audio signal using the feature vector, wherein the second classifier is trained to recognize the computer-generated audio signal; and A third classifier is configured to receive the feature vector from the feature extractor and classify the audio signal using the feature vector, wherein the third classifier is trained to recognize the real audio signal.

18. The system of claim 17, further comprising a profile generator configured to receive outputs from the first classifier, the second classifier, and the third classifier; and based on the outputs from the first classifier, the second classifier, and the third classifier, label the audio signal as one of the real audio signal, the recorded audio signal, or the computer-generated audio signal.