An anti-noise synthesized speech detection method based on multi-scale features and residual network

Abstract

The present application relates to a kind of anti-noise synthetic voice detection methods based on multi-scale feature and residual network, comprising the following steps: first, construct the two-dimensional Log-Fbank feature extraction model that original one-dimensional voice signal is converted into two-dimensional Log-Fbank acoustic feature map, then the output of two-dimensional Log-Fbank feature extraction model is directly input as multi-scale two-dimensional residual network, construct the anti-noise synthetic voice detection base model, and then train the detection model shaped;Afterwards, the voice to be detected is input into the anti-noise synthetic voice detection model shaped, to output the authenticity classification result.The present application solves the problem that depth counterfeit voice brings huge security threat to system under real complex noise environment, establishes the voice anti-counterfeiting detection model with high noise robustness, and then accurately classifies whether the target voice is the original true voice issued by natural person or the false voice synthesized by counterfeiting.

Description

Technical Field

[0001] This invention belongs to the field of voice anti-spoofing detection in audio processing within information security, specifically relating to a noise-resistant synthesized speech detection method based on multi-scale features and residual networks. Background Technology

[0002] With the rapid development of deep learning and speech processing technologies, Automatic Speaker Verification (ASV) systems have been widely applied in many information security fields, such as identity authentication, financial payments, access control systems, and intelligent customer service. However, ASV systems are highly vulnerable to various deceptive speech attacks, posing a serious challenge to their security. Among the many types of deception attacks, logical access attacks based on Text-to-Speech (TTS) and Voice Conversion (VC) technologies are particularly prominent. Attackers can easily use existing open-source algorithms to generate highly realistic forged speech from a small number of real recordings of target speakers, thereby deceiving the ASV system or concealing their true identity, causing serious security risks. Therefore, developing efficient and accurate synthetic speech detection (anti-spoofing) models is crucial.

[0003] Despite significant progress in anti-spoofing research for ASV systems in recent years, and the high accuracy of many detection models on standard, clean, high-quality datasets, these systems still have fatal vulnerabilities in real-world deployments. In complex real-world application scenarios, speech signals are inevitably affected by environmental noise, channel distortion, and other factors. Existing detection models often overfit to training data, and when faced with test speech containing unknown environmental noise, the effectiveness of feature extraction decreases dramatically, leading to a sharp decline in detection performance. The system's anti-interference capability and robustness remain a pressing technical challenge that needs to be addressed.

[0004] In addition, the relevant prior art background information is as follows:

[0005] 1) A method for deception detection using optical convolutional neural networks (LCNN) to extract acoustic features. (Lavrentyeva A, et al. Audio replay attack detection with deep learning frameworks[C], 2017 INTERSPEECH).

[0006] 2) Lai et al. proposed a method combining a squeeze-excitation module and a residual network (SENet-ResNet) for detecting real and fake speech. (Lai CI, et al. Assert: Anti-spoofing with squeeze-excitation and residual networks[C], 2019 INTERSPEECH).

[0007] 3) Hua et al. proposed an algorithm for extracting end-to-end synthetic speech based on short-time Fourier transform (STFT) spectrograms. (Hua Y, et al. Towards end-to-end synthetic speech detection[J],IEEE Signal Processing Letters, 2021).

[0008] 4) Zhang et al. proposed a spoofing detection algorithm based on ResNet architecture combined with single-class boundary optimization. (Zhang Z, et al. One-class learning towards synthetic voice spoofing detection[J],IEEE Signal Processing Letters,2021).

[0009] The algorithms described above all utilize deep convolutional neural networks to extract acoustic features across the entire frequency band, training the network with models such as cross-entropy or single classification for detection. However, existing deep networks perform indiscriminate processing across the entire frequency band when extracting features, lacking protection mechanisms for the low-frequency core fundamental frequency and complex noise reduction designs. Furthermore, features are easily distorted under strong noise masking, resulting in a lack of robustness.

[0010] Alharbi et al. proposed DeepRawNet, an audio deepfake detection model based on an end-to-end deep residual network (Alharbi LA, et al. DeepRawNet: empowering deepfake audio detection through dynamic enhancements[J]. PeerJ Computer Science,2026) [cite:92,93,101,120,121]. The experimental approach also relies on deep learning to extract acoustic features for anti-spoofing detection, and the basic data used in the experiment is the ASVspoof 2019 LA speech dataset [cite:113,117]. Their experimental data shows that when Gaussian noise and other acoustic distortion interference are introduced into the test set, the error rate (EER) of this deep learning model degrades to 5.20%. However, in this experiment, the network architecture lacks dedicated channel protection for the low-frequency core fundamental frequency (F0) and fails to incorporate noise-resistant designs such as large-interval edge pushing mechanisms in its internal structure, resulting in performance degradation even under strong noise masking. Our work, using the same database and Gaussian white noise testing, achieves a 2.46% improvement. Summary of the Invention

[0011] This invention addresses the shortcomings of existing technologies by providing a noise-resistant synthesized speech detection method based on multi-scale features and residual networks. This method aims to solve the significant security threat posed by deepfake speech to systems in real, complex noise environments. The goal is to establish a highly noise-resistant and robust speech anti-spoofing detection model, thereby accurately classifying whether the target speech is the original, genuine speech spoken by a natural person or a fake speech synthesized through forgery.

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions.

[0013] A noise-resistant synthesized speech detection method based on multi-scale features and residual networks mainly includes the following steps:

[0014] First, a two-dimensional Log-Fbank feature extraction model is constructed to transform the original one-dimensional speech signal into a two-dimensional Log-Fbank acoustic feature map.

[0015] The two-dimensional Log-Fbank feature extraction model is mainly implemented through the following process:

[0016] (1) Pre-emphasis: for the original one-dimensional speech signal By using a first-order high-pass filter to compensate for energy loss in the high-frequency range, the high-frequency acoustic characteristics are highlighted. Its time-domain difference formula is as follows: ,in, This is the index of the discrete-time sampling points; The current original speech sample value; This is the speech sample value from the previous sampling point; This is the pre-emphasized output signal; This is the pre-weighting factor, typically set to 0.97;

[0017] (2) Framing and Windowing: The pre-emphasized continuous speech signal is divided into short speech segments of fixed length (corresponding to 200 time frames), and each frame signal is processed. Applying a Hamming window smooths signal edges and reduces spectral leakage. The windowed signal... Represented as: ,in, This represents the total number of sampling points contained in each frame. The time index of the sampling point within the current short frame (the value range is...) ; The current frame signal after applying the Hamming window;

[0018] (3) Fast Fourier Transform and Power Spectrum Calculation: Perform a Short Time Fourier Transform (STFT) on the windowed time-domain signal of each frame to map it to the frequency domain, and calculate its discrete power spectrum. : ,in, The calculated discrete power spectrum; For discrete frequency indices in the frequency domain; The imaginary unit; is the base of the natural logarithm;

[0019] (4) Mel filterbank filtering: set up a Mel filterbank consisting of 40 triangular bandpass filters. This is then overlaid onto the power spectrum for energy integration. Actual frequency With Mel frequency The formula for nonlinear mapping is: ,in, This is the actual linear physical frequency (in Hz). This is the corresponding Mel frequency after conversion;

[0020] (5) Logarithmic Operation: The natural logarithm of the frequency band energy value after passing through each Mel filter is taken to simulate the nonlinear characteristics of human hearing, while smoothing out amplitude abrupt changes, and finally obtaining a two-dimensional feature matrix of size 200×40 Log-F bank. : ,in, For the first Logarithmic energy characteristic output of each Mel band; Group number index for Mel filter ; For the first A Mel filter at frequency The frequency domain response function at that point.

[0021] Compared to directly inputting one-dimensional speech data or highly compressed cepstral features (such as MFCC) after Discrete Cosine Transform (DCT), the Log-Mel filter bank acoustic feature map strictly preserves the physical topological distribution of frequencies from low to high in spatial dimension. This characteristic ensures that the low-frequency core fundamental frequency (F0) information is densely and completely preserved in the first 12 dimensions of the feature matrix, avoiding the irreversible destruction of low-frequency machine forgery defects caused by the DCT dimensionality reduction operation during MFCC extraction. This provides high-quality physical input for subsequent targeted dimensionality-free subband feature extraction by the dual-stream residual network.

[0022] Second, the output of the two-dimensional Log-Fbank feature extraction model is used as the direct input of the multi-scale two-dimensional residual network (Res2Net2D), thereby constructing a noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks.

[0023] To enhance noise resistance, a convolutional block attention model is cascaded and embedded after multi-scale residual feature extraction in a multi-scale two-dimensional residual network. This model can adaptively infer attention weights along the channel and spatial dimensions of the features, dynamically suppress redundant regions that are heavily contaminated by noise, and guide the network to forcefully focus the receptive field on the residual, most discriminative effective boundary features of true and false speech.

[0024] In the tail feature processing structure of the multi-scale two-dimensional residual network, the high-fidelity fundamental frequency features output by the sub-band channel and the multi-scale denoising features output by the global backbone channel are cascaded and fused in the channel dimension. The fused high-dimensional feature tensor is then subjected to dimensionality reduction and representation enhancement through a statistical pooling layer. This not only preserves the global average distribution of features but also effectively captures the abnormal fluctuation variance of machine-faked speech on the time-frequency scale. The dimensionality-reduced features are finally fed into a fully connected layer equipped with an additional angular interval classifier. This classifier forcibly increases the inter-class isolation margin between real and fake speech in the cosine space, thereby outputting a real / fake classification result with high confidence and noise robustness.

[0025] Third, the speech dataset is input into a noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks for training, resulting in a mature noise-resistant synthetic speech detection model.

[0026] Fourth, the speech to be detected is input into the established noise-resistant synthetic speech detection model, thereby outputting the true / false classification result.

[0027] As a further improvement to the above scheme, when inputting the raw speech data into the two-dimensional Log-Fbank feature extraction model, the data needs to be standardized. The standardization formula is as follows: ,in, The input is the raw feature data; The standardized feature data; The mathematical expectation (mean) of the feature data; The variance of the feature data.

[0028] First, mean normalization is performed. For each feature of the given data, its mean is subtracted, normalizing the data center to zero. This reduces the computational cost of the algorithm. Then, the mean is divided by the variance of that feature, a normalization process that normalizes the magnitudes of each dimension of the dataset to the same range. This speeds up training, accelerates weight convergence, stabilizes the loss function, prevents vanishing or exploding gradients during training, and improves algorithm performance.

[0029] Furthermore, to address background interference in real-world physical environments and improve the robustness of the detection algorithm, this invention employs extreme noise augmentation on the acoustic feature maps during the preprocessing stage: Random time-frequency masking (SpecAugment) is used on the generated feature maps, randomly selecting certain consecutive time frames and frequency bands and setting their feature values ​​to 0; simultaneously, additive white Gaussian noise with a certain signal-to-noise ratio is superimposed on the feature matrix. By artificially constructing harsh training data containing missing features and background noise, the microscopic flaws on the surface of the forged speech are smoothed out, forcing the network to avoid overfitting and learn deeper, more robust physical features.

[0030] This invention proposes a deep learning architecture based on a multi-scale two-dimensional residual network (Res2Net2D) specifically for the task of deep fake speech detection (speech anti-spoofing), designed for accurately distinguishing between original, genuine speech and machine-generated speech. Compared to traditional networks, this model achieves significant automation and refinement in feature extraction. Its core lies in the innovative introduction of a multi-scale grouping connection mechanism, where feature maps are grouped by channel within a residual block, and a rich, multi-scale receptive field is constructed through hierarchical residual connections. The ingenuity of this multi-scale architecture lies in its ability to perform multi-level deep analysis of two-dimensional acoustic feature maps, from macroscopic temporal sequences to microscopic local imperfections, without significantly increasing the number of model parameters and computational complexity. This avoids the repetitive learning of redundant shallow features, making the entire network extremely easy to train and exhibiting excellent convergence. Simultaneously, the dedicated two-dimensional convolutional kernels and statistical pooling settings perfectly match the time-frequency physical characteristics of acoustic feature maps and the need to capture outlier variance. Furthermore, addressing the pain point of detection models being prone to overfitting, this invention deeply integrates a dynamic time-frequency masking (SpecAugment) strategy with an additional angle-spaced (AM-Softmax) loss function during the network training phase. This joint optimization strategy forcibly cuts off the model's dependence on single locally known features and significantly increases the inter-class isolation margin between real and fake speech in the feature space, giving the model extremely strong cross-domain generalization ability and ensuring that it maintains extremely high and stable detection accuracy when facing highly realistic unknown spoofing attacks.

[0031] In contrast, traditional deepfake detection networks are typically limited by a single-scale receptive field or coarse global mean pooling, and rely excessively on conventional cross-entropy optimization objectives, making it difficult to deeply mine high-dimensional and complex machine forgery clues. This traditional structure inherently cannot avoid dependence on local features of known attack samples, leading to severe performance degradation and generalization bottlenecks when dealing with complex, variable, and unknown zero-day attacks. Attached Figure Description

[0032] Figure 1This is a flowchart of the two-dimensional Log-F bank feature extraction process.

[0033] Figure 2 This is a schematic diagram of a multi-scale two-dimensional residual network structure. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0035] The experimental data in this embodiment uses Log-Mel filter bank (Log-Fbank) acoustic feature maps. Compared to directly inputting one-dimensional speech data or highly compressed cepstral features (such as MFCC) after Discrete Cosine Transform (DCT), Log-Fbank features strictly preserve the physical topological distribution of frequencies from low to high in spatial dimensions. This characteristic ensures that the low-frequency core fundamental frequency (F0) information is densely and completely preserved in the first 12 dimensions of the feature matrix, avoiding the irreversible destruction of low-frequency machine forgery defects caused by the DCT dimensionality reduction operation during MFCC extraction. This provides high-quality physical input for subsequent targeted dimensionality-free subband feature extraction by the dual-stream residual network.

[0036] A noise-resistant synthesized speech detection method based on multi-scale features and residual networks mainly includes the following steps:

[0037] First, a two-dimensional Log-Fbank feature extraction model is constructed to transform the original one-dimensional speech signal into a two-dimensional Log-Fbank acoustic feature map.

[0038] Reference Figure 1 The two-dimensional Log-Fbank feature extraction model is mainly implemented through the following process:

[0039] (1) Pre-emphasis: for the original one-dimensional speech signal By using a first-order high-pass filter to compensate for energy loss in the high-frequency range, the high-frequency acoustic characteristics are highlighted. Its time-domain difference formula is as follows: ,in, This is the index of the discrete-time sampling points; The current original speech sample value; This is the speech sample value from the previous sampling point; This is the pre-emphasized output signal; This is the pre-weighting factor, typically set to 0.97;

[0040] (2) Framing and Windowing: The pre-emphasized continuous speech signal is divided into short speech segments of fixed length (corresponding to 200 time frames), and each frame signal is processed. Applying a Hamming window smooths signal edges and reduces spectral leakage. The windowed signal... Represented as: ,in, This represents the total number of sampling points contained in each frame. The time index of the sampling point within the current short frame (the value range is...) ; The current frame signal after applying the Hamming window;

[0041] (3) Fast Fourier Transform and Power Spectrum Calculation: Perform a Short Time Fourier Transform (STFT) on the windowed time-domain signal of each frame to map it to the frequency domain, and calculate its discrete power spectrum. : ,in, The calculated discrete power spectrum; For discrete frequency indices in the frequency domain; The imaginary unit; is the base of the natural logarithm;

[0042] (4) Mel filterbank filtering: set up a Mel filterbank consisting of 40 triangular bandpass filters. This is then overlaid onto the power spectrum for energy integration. Actual frequency With Mel frequency The formula for nonlinear mapping is: ,in, This is the actual linear physical frequency (in Hz). This is the corresponding Mel frequency after conversion;

[0043] (5) Logarithmic Operation: The natural logarithm of the frequency band energy value after passing through each Mel filter is taken to simulate the nonlinear characteristics of human hearing, while smoothing out amplitude abrupt changes, and finally obtaining a two-dimensional feature matrix of size 200×40 Log-F bank. : ,in, For the first Logarithmic energy characteristic output of each Mel band; Group number index for Mel filter ; For the first A Mel filter at frequency The frequency domain response function at that point.

[0044] Compared to directly inputting one-dimensional speech data or highly compressed cepstral features (such as MFCC) after Discrete Cosine Transform (DCT), the Log-Mel filter bank acoustic feature map strictly preserves the physical topological distribution of frequencies from low to high in spatial dimension. This characteristic ensures that the low-frequency core fundamental frequency (F0) information is densely and completely preserved in the first 12 dimensions of the feature matrix, avoiding the irreversible destruction of low-frequency machine forgery defects caused by the DCT dimensionality reduction operation during MFCC extraction. This provides high-quality physical input for subsequent targeted dimensionality-free subband feature extraction by the dual-stream residual network.

[0045] Second, refer to Figure 2 The output of the two-dimensional Log-Fbank feature extraction model is used as the direct input of the multi-scale two-dimensional residual network (Res2Net2D), thereby constructing a noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks.

[0046] Given that existing speech synthesis and conversion algorithms struggle to accurately reconstruct the core human vocal frequency (F0) band covering 80Hz-600Hz, and that this low-frequency energy concentration area is not easily swallowed up by high-frequency white noise, this network creatively introduces a dual-stream feature physical separation mechanism at the input end. For example... Figure 1 As shown at the input end, the network precisely segments the preprocessed Log-Fbank two-dimensional acoustic feature matrix with a size of 200×40 (i.e., 200 consecutive time frames, 40 Mel frequency band dimensions): it peels off the first 12 dimensions of the corresponding ultra-low frequency band features and sends them separately to the independent "F0 sub-band channel" for shallow two-dimensional convolution operation to achieve dimensionality reduction-free protection of key physical clues in low frequency; at the same time, the complete 200×40 original two-dimensional feature matrix is ​​sent to the "global backbone channel" for deep abstract feature parsing.

[0047] Combination Figure 2As can be seen from the backbone, this invention employs a multi-scale two-dimensional residual network (Res2Net2D) as the basic backbone for global feature extraction. Unlike the single receptive field of ordinary residual networks, multi-scale residual blocks, through group convolutions and hierarchical connections, can extract multi-granularity acoustic features within the same network layer, thereby keenly capturing minute machine flaws and macroscopic intonation distortions of varying scales in forged speech. Furthermore, to enhance noise resistance, the core design of this invention involves cascading and embedding a Convolutional Block Attention Model (CBAM) after the backbone network completes multi-stage (Stage 1-4) multi-scale residual feature extraction. Faced with time-frequency masking (SpecAugment) and extreme Gaussian white noise interference introduced during data preprocessing, the CBAM module can adaptively infer attention weights along the channel and spatial dimensions of the features, dynamically suppressing redundant regions severely contaminated by noise, and guiding the network to forcefully focus the receptive field on the remaining, most discriminative, effective boundary features of true and false speech.

[0048] Finally, refer to Figure 2 The network employs a tail-end feature processing structure, concatenating and fusing the high-fidelity fundamental frequency features output from the "F0 sub-band channel" with the multi-scale denoising features output from the "global backbone channel" along the channel dimension. The fused high-dimensional feature tensor is then subjected to dimensionality reduction and representation enhancement via **statistical pooling (which simultaneously calculates the mean and standard deviation of the feature maps in the spatial dimension and concatenates them)**. This design not only preserves the global average distribution of features but also effectively captures the anomalous variance of machine-generated spoofed speech on the time-frequency scale. The dimensionality-reduced features are finally fed into a fully connected layer equipped with an additional angular interval classifier (AM-Softmax). This classifier forcibly increases the inter-class isolation margin between real and spoofed speech in cosine space, thereby outputting a true / false classification result with extremely high confidence and noise robustness.

[0049] Third, the speech dataset is input into a noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks for training, resulting in a mature noise-resistant synthetic speech detection model.

[0050] Fourth, the speech to be detected is input into the established noise-resistant synthetic speech detection model, thereby outputting the true / false classification result.

[0051] During this process, when inputting the raw speech data into the two-dimensional Log-Fbank feature extraction model, the data needs to be standardized. The standardization formula is as follows: ,in, The input is the raw feature data; The standardized feature data; The mathematical expectation (mean) of the feature data; The variance of the feature data.

[0052] First, mean normalization is performed. For each feature of the given data, its mean is subtracted, normalizing the data center to zero. This reduces the computational cost of the algorithm. Then, the mean is divided by the variance of that feature, a normalization process that normalizes the magnitudes of each dimension of the dataset to the same range. This speeds up training, accelerates weight convergence, stabilizes the loss function, prevents vanishing or exploding gradients during training, and improves algorithm performance.

[0053] Furthermore, to address background interference in real-world physical environments and improve the robustness of the detection algorithm, this invention employs extreme noise augmentation on the acoustic feature maps during the preprocessing stage: Random time-frequency masking (SpecAugment) is used on the generated feature maps, randomly selecting certain consecutive time frames and frequency bands and setting their feature values ​​to 0; simultaneously, additive white Gaussian noise with a certain signal-to-noise ratio is superimposed on the feature matrix. By artificially constructing harsh training data containing missing features and background noise, the microscopic flaws on the surface of the forged speech are smoothed out, forcing the network to avoid overfitting and learn deeper, more robust physical features.

[0054] In this embodiment of the invention, the specific network parameters are shown in Table 1 below.

[0055]

[0056] Generally, the horizontal axis of a time-frequency graph represents the time dimension, and the vertical axis represents the frequency dimension. In conventional two-dimensional image recognition tasks, neural networks typically employ single-scale convolutional kernels similar to a standard 3×3 kernel because local pixel values ​​in an image usually exhibit high spatial correlation. However, considering the time-frequency physical characteristics of speech signals, the artifacts produced by machine-generated forged speech in the spectrum are often cross-band and of varying scales—including both local frequency abrupt changes and macroscopic pitch distortions. Furthermore, existing speech synthesis algorithms struggle to accurately reconstruct the fundamental frequency (F0) of human speech, which covers the low-frequency region. Therefore, the detection model must be able to accurately extract multi-scale forged spectral features while effectively protecting the low-frequency fundamental frequency under complex noise masking. To address this, our network employs a multi-scale residual module (Bottle2neck) with a scale grouping factor of s=4 in its backbone structure and segments the input into the first 12 dimensions of F0 sub-band channels. This design allows the network to acquire multi-scale receptive fields within the same convolutional layer, thereby adaptively capturing forged distortions of different scales on the time-frequency graph. Furthermore, compared to simply increasing the kernel size, this type of multi-scale grouped convolutional structure effectively controls the number of model parameters while improving feature representation capabilities, thus effectively avoiding network overfitting. Additionally, in the extraction of low-frequency physical features of the F0 subband (F0_Stem), this network employs an asymmetric pooling downsampling strategy. In this invention, downsampling of the low-frequency channel is only performed in the time dimension of the time-frequency map (i.e., the stride is set to (1,2)), and the frequency dimension is not downsampled throughout the entire low-frequency extraction process. This aligns the time feature dimension of the backbone network without causing the loss of frequency dimension features in the core fundamental frequency region, preserving the most original acoustic traces for subsequent classification. To address the issue that time-frequency maps are easily contaminated by Gaussian white noise in real-world environments, this network incorporates a CBAM attention module at the end of feature extraction, guiding the network to adaptively assign higher weights to key frequency bands. These targeted structural designs are extremely beneficial for the network to achieve better classification results in extreme environments.

[0057] During network training, the network in this embodiment of the invention is trained using mini-batch iterative stochastic gradient descent with AM-Softmax loss. Supervised learning methods are employed to optimize the hyperparameters on the validation set. Table 2 lists some important hyperparameters used for network training. Under this structure, the proposed two-stream multi-scale noise-resistant network model provides fairly good recognition accuracy. The training hyperparameters are shown in Table 2 below; where... , These are the parameters for the ADAM optimizer.

[0058]

[0059] In the experimental setup of this embodiment of the invention, the experimental data used in this network structure comes from the ASVspoof2019 Logical Access (LA) dataset. During the experiment, this dataset was divided into three sets for training, validating, and testing network performance: the training set contains 25,380 speech segments, of which 2,580 are real speech segments and 22,800 are fake speech segments; the validation set contains 24,986 speech segments, of which 2,548 are real speech segments and 22,438 are fake speech segments; and the evaluation test set contains 71,933 speech segments, of which 7,355 are real speech segments and 64,578 are fake speech segments.

[0060] In the experiments, spoofing methods such as text-to-speech (TTS) and voice conversion (VC) were employed. The training and validation sets included six known spoofing attack algorithms; the evaluation test set included 13 unknown spoofing attack algorithms (A07 to A19) that the network had never encountered during the training phase. This was to verify the algorithm's versatility against different unknown spoofing methods.

[0061] In the experiments, this invention uses equal error rate (EER) as the evaluation metric. To evaluate the proposed speech detection network, we conducted three classification experiments and compared the results with those of the network structure proposed by Lubna A. Alharbi et al.: Experiment 1: Detection of known attacks with the same distribution; Experiment 2: Generalized detection of unknown attacks across distributions; Experiment 3: Validation of the core module's effectiveness. Details are as follows.

[0062] Experiment 1: Detection of known attacks with the same distribution.

[0063] This section of the experiments utilizes the validation set (Development Set) of the ASVspoof 2019 LA dataset for basic performance testing. In this test scenario, the fake speech in the validation set is consistent with that in the training set, generated by six known speech conversion (VC) and text-to-speech (TTS) algorithms (numbered A01-A06). The core objective of this experiment is to conduct a "uniform distribution stress test": rigorously evaluating the network model's ability to fit, extract, and identify features with known forgery traces (such as basic spectral phase breaks and unnatural transitions of high-frequency formants) when faced with attack samples whose acoustic feature distribution is completely identical to that during the training phase. This is a fundamental step in measuring whether a deep forgery detection network possesses adequate underlying learning capabilities. The experimental comparison results are shown in Table 3 below:

[0064]

[0065] As can be seen from the experimental results in Table 3, in the known attack test (Dev set) with the same distribution, the error rate of the official traditional baseline method (LFCC-GMM) remained at 2.71%. This type of traditional method relies too much on shallow acoustic features extracted manually, making it difficult to fully capture the deep, high-dimensional and complex machine forgery patterns in speech signals, resulting in a relatively high false positive rate.

[0066] In comparison, the method proposed in this invention achieves an extremely low error rate of 0.52%, significantly outperforming traditional baseline methods (with an error rate reduction of 2.19%). This excellent basic detection performance fully demonstrates that the dual-stream multi-scale residual network designed in this invention not only effectively avoids the loss of core physical speech features caused by conventional downsampling and can accurately locate local speech defects, but also, combined with the AM-Softmax loss function's forced isolation of the boundary between real and fake speech in the feature space, enables the model to exhibit extremely keen and accurate low-level feature discrimination capabilities when faced with highly realistic, deceptive speech known during the training phase.

[0067] Experiment 2: Generalized detection of unknown attacks across distributions.

[0068] In real-world business deployment environments, defense systems often face "zero-day attacks" launched by hackers using the latest iterations of voice spoofing algorithms that the system has never encountered before. Therefore, the network's ability to generalize detection of unknown synthesis algorithms is a core indicator for evaluating the practical application value of anti-spoofing systems. This experiment uses the ASVspoof 2019 LA evaluation set (Eval Set) for rigorous cross-distribution stress testing. This test set includes 13 unknown attack algorithms (A07-A19) that the network had not encountered during the training phase. The underlying acoustic logic of these algorithms is completely different from that of the training set, posing a significant cross-domain challenge to the generalization and robustness of various detection networks. We compared our invention with several classic and state-of-the-art network models in the prior art, and the experimental results are shown in Table 4 below:

[0069]

[0070] As can be seen from the table, the experimental results of the official baseline method and Lubna A. Alharbi's method are 8.09% and 2.80%, respectively. Therefore, the algorithm model of this invention has better generalization ability and detection accuracy when dealing with unknown deceptive voice attacks.

[0071] Experiment 3: Validation of the effectiveness of the core innovation module.

[0072] To further demonstrate the substantial progress brought about by the newly added core module in the technical solution of this invention, we conducted a comparative test with controlled variables on the ASVspoof 2019 LA evaluation set. We compared the basic Res2Net network constructed in previous studies with the complete upgraded network architecture of this invention using the same error rate. The experimental results are shown in Table 5 below:

[0073]

[0074] As can be clearly seen from the internal ablation experiment results in Table 5, the initial basic Res2Net model had an equal error rate of 6.21%. However, the complete architecture of this invention, by introducing the AM-Softmax loss function to forcibly expand the isolation margin between real and fake speech classes and by employing a dynamic time-frequency masking strategy (SpecAugment) to cut off local feature dependencies, significantly reduced the equal error rate to 2.46% (an absolute decrease of 3.75%). This internal comparison result, excluding interference from the basic network architecture, directly and powerfully demonstrates that the newly added core feature module claimed in this invention effectively overcomes the performance bottlenecks of previous technologies and brings significant technical progress to the system.

[0075] In this experiment, our method has certain advantages over previous work:

[0076] (1) A multi-scale residual network (Res2Net) classification model was constructed using deep learning methods. This network, through a fine-grained multi-scale receptive field mechanism, is not only easier to optimize, but also effectively captures minute machine forgery flaws of different dimensions in speech signals, overcoming the limitations of traditional networks in extracting single features;

[0077] (2) This algorithm has extremely strong generalization ability in practical applications. When faced with up to 13 unknown speech spoofing methods not seen in the training phase (such as the latest TTS and VC technologies), it still maintains an extremely low error rate and has excellent detection results for new attacks with cross-domain distribution.

[0078] (3) The network of this invention effectively overcomes the bottleneck of overfitting of the basic network by introducing the AM-Softmax loss function and the dynamic time-frequency masking (SpecAugment) strategy. The model can forcibly increase the inter-class distance between real and fake speech in the feature space, thereby easily obtaining a huge improvement in accuracy from a limited dataset and producing better results than the previous basic network;

[0079] (4) The two-dimensional convolution kernel and multi-scale connection settings used in the network structure are more in line with the acoustic feature analysis of two-dimensional speech temporal signals, which is conducive to the network more comprehensively and accurately representing the underlying physical clues of complex deceptive speech.

[0080] (5) The original dual-stream network structure design of this invention can perform deeper multi-path feature analysis of speech signals. While ensuring extremely high detection accuracy, it effectively enhances the robustness of the system in real complex channel transmission scenarios (such as communication compression, environmental background noise, etc.), taking into account both detection efficiency and accuracy. It has extremely high engineering implementation and practical deployment value.

[0081] Compared to traditional networks, this model achieves significant automation and refinement in feature extraction. Its core lies in the innovative introduction of a multi-scale grouping connection mechanism. Within a residual block, feature maps are grouped by channel, and a rich, multi-scale receptive field is constructed through hierarchical residual connections. The brilliance of this multi-scale architecture lies in its ability to perform multi-level deep analysis of 2D acoustic feature maps, from macroscopic temporal sequences to microscopic local imperfections, without significantly increasing the number of model parameters or computational complexity. This avoids redundant learning of shallow features, making the entire network extremely easy to train and exhibiting excellent convergence. Simultaneously, the dedicated 2D convolutional kernels and statistical pooling settings perfectly match the time-frequency physical characteristics of acoustic feature maps and the need to capture outlier variance. Furthermore, addressing the pain point of detection models being prone to overfitting, this invention deeply integrates a dynamic time-frequency masking (SpecAugment) strategy and an additional angular interval (AM-Softmax) loss function during the network training phase. This joint optimization strategy forcibly cuts off the model's dependence on a single local known feature and significantly increases the inter-class isolation margin between real and fake speech in the feature space, giving the model extremely strong cross-domain generalization ability and ensuring that it maintains extremely high and stable detection accuracy when facing highly realistic unknown spoofing attacks.

[0082] The examples described above clearly illustrate instances of the present invention and are intended to better explain the technical concept of the present invention. For those skilled in the art, various changes and modifications can be made based on this specification, and such obvious and reasonable extensions of changes and modifications still fall within the protection scope of the present invention.

Claims

1. A noise-resistant synthesized speech detection method based on multi-scale features and residual networks, characterized in that, The main steps include: First, a two-dimensional Log-Fbank feature extraction model is constructed to transform the original one-dimensional speech signal into a two-dimensional Log-Fbank acoustic feature map; The two-dimensional Log-Fbank feature extraction model is implemented through the following process: (1) Pre-emphasis: for the original one-dimensional speech signal By using a first-order high-pass filter to compensate for energy loss in the high-frequency range, the high-frequency acoustic characteristics are highlighted. Its time-domain difference formula is as follows: ,in, This is the index of the discrete-time sampling points; The current original speech sample value; This is the speech sample value from the previous sampling point; This is the pre-emphasized output signal; This is the pre-weighting factor, typically set to 0.97; (2) Framing and Windowing: The pre-emphasized continuous speech signal is divided into short speech segments of fixed length (corresponding to 200 time frames), and each frame signal is processed. Applying a Hamming window smooths signal edges and reduces spectral leakage. The windowed signal... Represented as: ,in, This represents the total number of sampling points contained in each frame. The time index of the sampling point within the current short frame (the value range is...) ; The current frame signal after applying the Hamming window; (3) Fast Fourier Transform and Power Spectrum Calculation: Perform a Short Time Fourier Transform (STFT) on the windowed time-domain signal of each frame to map it to the frequency domain, and calculate its discrete power spectrum. : ,in, The calculated discrete power spectrum; For discrete frequency indices in the frequency domain; The imaginary unit; is the base of the natural logarithm; (4) Mel filterbank filtering: set up a Mel filterbank consisting of 40 triangular bandpass filters. This is then overlaid onto the power spectrum for energy integration. Actual frequency With Mel frequency The formula for nonlinear mapping is: ,in, This is the actual linear physical frequency (in Hz). This is the corresponding Mel frequency after conversion; (5) Logarithmic Operation: The natural logarithm of the frequency band energy value after passing through each Mel filter is taken to simulate the nonlinear characteristics of human hearing, while smoothing out amplitude abrupt changes, and finally obtaining a two-dimensional feature matrix of size 200×40 Log-F bank. : ,in, For the first Logarithmic energy characteristic output of each Mel band; Group number index for Mel filter For the first A Mel filter at frequency Frequency domain response function at; Second, the output of the two-dimensional Log-Fbank feature extraction model is used as the direct input of the multi-scale two-dimensional residual network, thereby constructing a noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks. To enhance noise resistance, a convolutional block attention model is cascaded and embedded after multi-scale residual feature extraction in a multi-scale two-dimensional residual network. This model can adaptively infer attention weights along the channel and spatial dimensions of the features, dynamically suppress redundant regions that are heavily contaminated by noise, and guide the network to forcefully focus the receptive field on the residual, most discriminative effective boundary features of true and false speech. In the tail feature processing structure of the multi-scale two-dimensional residual network, the high-fidelity fundamental frequency features output by the sub-band channel and the multi-scale denoising features output by the global backbone channel are cascaded and fused in the channel dimension. The fused high-dimensional feature tensor is reduced in dimensionality and enhanced in representation by the statistical pooling layer. This not only preserves the global average distribution of features, but also effectively captures the abnormal fluctuation variance of machine-faked speech in the time-frequency scale. The dimensionality-reduced features are finally fed into a fully connected layer equipped with an additional angle interval classifier. This classifier forcibly increases the inter-class isolation margin between real speech and fake speech in the cosine space, thereby outputting a real and fake classification result with high confidence and noise robustness. Third, the speech dataset is input into the noise-resistant synthetic speech detection basic model based on multi-scale features and residual networks for training, resulting in a mature noise-resistant synthetic speech detection model. Fourth, the speech to be detected is input into the established noise-resistant synthetic speech detection model, thereby outputting the true / false classification result.

2. The noise-resistant synthesized speech detection method based on multi-scale features and residual networks according to claim 1, characterized in that, When inputting raw speech data into a two-dimensional Log-Fbank feature extraction model, the data needs to be standardized. The standardization formula is as follows: ,in, The input is the raw feature data; The standardized feature data; The mathematical expectation (mean) of the feature data; The variance of the feature data.

Images