Method and apparatus for detecting a counterfeit speech based on extended residual connection

Abstract

The application discloses a kind of based on the method for detecting counterfeit speech of extended residual connection, comprising the following steps: S1: obtaining the speech signal to be detected;S2: the front-end acoustic feature extraction is carried out to the speech signal to be detected, and frame-level acoustic feature sequence is obtained;S3: the frame-level acoustic feature sequence is input into the ERC-Conformer coding network containing Conformer main branch and extended residual connection ERC branch, and the synergistic modeling of Conformer main branch and extended residual connection ERC branch is passed through, and the enhanced coding feature is obtained;S4: the speech-level aggregation and classification discrimination of enhanced coding feature are carried out, and the corresponding counterfeit speech detection result is output.The application is based on the ability of Conformer main network global context modeling is retained, in the residual path of target sublayer Introducing ERC branch of extended residual connection, make residual path from pure information transmission structure expand into conditional enhancement path with input dependent ability, to improve the modeling ability of local time-frequency abnormal clues in counterfeit speech of model.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing and artificial intelligence security technology, specifically relating to a method and apparatus for detecting forged speech based on extended residual connections. Background Technology

[0002] With the rapid development of generative speech technologies such as text-to-speech, speech cloning, and speech conversion, the naturalness, fluency, and speaker similarity of deepfake speech are constantly improving, making it more deceptive and covert in scenarios such as identity authentication, voice interaction, online dissemination, and content moderation. Therefore, accurately distinguishing between real and fake speech has become a crucial research issue in the field of speech security. Existing methods for detecting fake speech typically employ a combination of front-end acoustic feature extraction and back-end deep model discrimination, achieving real vs. fake speech detection through frame-level representation learning and speech-level classification of the input speech.

[0003] In recent years, the Conformer architecture has been widely used in audio deepfake detection tasks due to its combined global context modeling and local convolution modeling capabilities. This architecture can model long-range dependencies of speech signals at a macroscopic level and extract local variation features between adjacent time frames at a microscopic level, making it well-suited for jointly characterizing global spectral anomalies and local temporal artifacts in deepfake speech. Conformer provides a good modeling foundation for common anomalies in deepfake speech, such as waveform splicing glitches, local phase discontinuities, and excessive spectral smoothing.

[0004] However, existing Conformer-based forgery detection methods typically employ residual fusion based on identity mappings across multiple sublayers. While such residual paths help alleviate gradient degradation during deep network training, they primarily serve as cross-layer information transfer mechanisms and lack the ability to filter and enhance input-dependent features. For local fine-grained artifacts with weak amplitude, strong concealment, and short duration in forgery detection tasks, traditional static residual connections struggle to fully leverage their enhancement effect on local time-frequency anomalies, thus limiting the model's ability to represent weak forgery cues.

[0005] Furthermore, anomalous cues in deepfake speech are often distributed across a complex time-frequency space. Relying solely on the backbone network for unified modeling can easily lead to insufficient responses to local artifacts. Especially when the residual path lacks a task-oriented enhancement mechanism, although the model can extract some macroscopic speech structure information, its ability to supplement the modeling of local artifacts remains limited. Summary of the Invention

[0006] (1) Technical problems to be solved To address the shortcomings of existing technologies, the present invention aims to provide a method and apparatus for detecting forged speech based on extended residual connections, which addresses the problem that existing Conformer-based forged speech detection methods mainly employ identity mapping in the residual path and lack sufficient ability to enhance local fine-grained forged artifacts.

[0007] (2) Technical solution To address the aforementioned technical problems, this invention provides a method for detecting forged speech based on extended residual connections, comprising the following steps: S1: Acquire the speech signal to be detected; S2: Perform front-end acoustic feature extraction on the speech signal to be detected to obtain a frame-level acoustic feature sequence; S3: Input the frame-level acoustic feature sequence into an ERC-Conformer coding network containing a Conformer backbone branch and an extended residual connection ERC branch, and obtain the enhanced coding features through collaborative modeling of the Conformer backbone branch and the extended residual connection ERC branch. S4: Perform speech-level aggregation and classification on the enhanced encoded features, and output the corresponding fake speech detection results.

[0008] Preferably, in step S2, the front-end acoustic features adopt linear frequency cepstral coefficients (LFCC) features. In step S2, the frame-level acoustic feature sequence is mapped into a continuous feature representation that can be processed by the subsequent coding network, thereby completing the conversion from the original speech signal to the detection feature input.

[0009] Furthermore, the ERC branch in step S3 includes a collaborative processing procedure of "high-dimensional expansion - local modeling - dimensionality reduction mapping - dynamic gating", specifically including the following steps: First, the input features are extended to a high-dimensional representation space through linear projection; Subsequently, local convolution operations are applied to the expanded features along the time axis to obtain the local modeling results; Then, nonlinear activation and dimensionality reduction mapping are performed on the local modeling results to obtain candidate enhancement features; Finally, channel-wise dynamic gating weights are generated based on the statistical information of the input features in the time dimension, adaptively modulate the candidate enhancement features, and combine them with a global scaling factor to form a residual enhancement term.

[0010] Furthermore, the local modeling unit employs a one-dimensional depth convolution operation along the time axis, which enhances its ability to detect forgeries and artifacts such as short-term discontinuities, local disturbances, and transient anomalies.

[0011] Furthermore, the statistical description information includes mean information and fluctuation information.

[0012] Further, step S3 specifically involves: the frame-level acoustic feature sequence is first normalized and then fed into the original sub-layer branch and the ERC branch of the Conformer backbone branch, respectively. The original sub-layer branch is used to perform standard attention modeling or convolution modeling operations to extract global contextual dependencies or structural information between local adjacent frames in the speech signal; the ERC branch performs high-dimensional expansion, local modeling, dimensionality reduction mapping, and dynamic gating modulation on the same input feature to generate residual enhancement terms related to forgery detection. Subsequently, the input features, the output of the original sub-layer, and the residual enhancement terms output by the ERC branch are fused to obtain the enhanced coded features.

[0013] Furthermore, the Conformer backbone includes multi-head self-attention sub-layers, convolutional sub-layers, and feedforward network sub-layers; The parallel integration method of the ERC branch in the Conformer backbone is as follows: the ERC branch is embedded in the multi-head self-attention sub-layer and convolutional sub-layer in the form of parallel residual branches, so as to enhance the supplementary modeling capability for local weak forgery artifacts while maintaining the stability of the backbone network; while the feedforward network sub-layer maintains its original structure to avoid unnecessary parameter redundancy and maintain the stability of the overall network topology.

[0014] Furthermore, step S4 specifically involves: firstly, performing discourse-level aggregation on the enhanced encoded features to obtain a discourse-level embedding representation with fixed dimensions; then, inputting the discourse-level embedding representation into the classification module to perform discrimination between real and fake speech in the speech to be detected, and outputting the corresponding detection results.

[0015] Furthermore, the discourse-level aggregation process is implemented using statistical pooling or attention statistical pooling to allocate aggregation weights based on the importance of different time frames for true / false speech discrimination; the classification module is preferably trained using a classification optimization objective with interval constraints to enhance the intra-class compactness and inter-class separability of the output features and improve the performance of fake speech detection.

[0016] The present invention also provides a forged speech detection device based on extended residual connections, comprising: The voice acquisition module is used to receive the voice signal to be detected; The front-end feature extraction module is used to extract acoustic features from the speech signal to be detected and construct a frame-level input representation; The encoding enhancement module is used to input the frame-level input representation into the encoding network and enhance the modeling of local time-frequency artifacts through parallel fusion of the Conformer backbone branch and the extended residual connection ERC branch to obtain the enhanced encoding features; The aggregation module is used to convert the enhanced encoded features into utterance-level embedded representations; The classification and discrimination module is used to output the detection result of real speech or fake speech based on the utterance-level embedding representation.

[0017] Beneficial effects Compared with the prior art, the beneficial effects of the present invention are as follows: This invention introduces an extended residual connection ERC branch into the Conformer residual path, which extends the residual path from a fixed identity mapping to a learnable, input-dependent enhanced path, thereby improving the residual branch's ability to represent local fine-grained forgeries. This invention improves cross-channel information interaction capabilities and local abnormal pattern perception capabilities through a structural design of "high-dimensional expansion - local modeling - dimensionality reduction mapping", which helps to capture short-term discontinuities, local phase perturbations and subtle time-frequency anomalies in forged speech. This invention further introduces a dynamic gating mechanism based on statistical information, which enables the enhancement strength to be adaptively adjusted according to the input features, thereby achieving effective enhancement of forgery detection-related features without disrupting the stable transmission of the original residual path. While maintaining the global modeling advantages of the Conformer backbone network, this invention enhances the ability to characterize the subtle differences between real and fake speech, thus possessing good application value in fake speech detection.

[0018] This invention, while retaining the global context modeling capability of the Conformer backbone network, introduces an extended residual connection ERC branch into the residual path of the target sublayer. This expands the residual path from a simple information transmission structure into a conditional enhancement path with input dependency capability, thereby improving the model's ability to model local time-frequency anomalies in forged speech. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of a forged speech detection method based on extended residual connections according to the present invention; Figure 2 This is a schematic diagram illustrating an integration method of an ERC module within a Conformer block according to the present invention; Figure 3 This is a schematic diagram of an extended residual connection ERC module structure according to the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Various modifications or equivalent substitutions made to the present invention by those skilled in the art without departing from the spirit and substance of the present invention should fall within the scope of protection of the present invention.

[0021] This embodiment provides a method for detecting spoofed speech based on extended residual connections, the overall process of which is as follows: Figure 1 As shown, it mainly includes four stages: acquiring the speech signal to be detected, front-end acoustic feature extraction, ERC-Conformer-based encoding enhancement modeling, and back-end discrimination output.

[0022] Specifically, the method for detecting forged speech based on extended residual connections includes the following steps: S1: Acquire the speech signal to be detected.

[0023] S2: Perform front-end acoustic feature extraction on the speech signal to be detected to obtain a frame-level acoustic feature sequence; Specifically, in the front-end acoustic feature extraction stage, the system first reads and preprocesses the input speech signal to be detected to complete the front-end acoustic feature extraction, and converts the original speech signal into a frame-level acoustic feature sequence suitable for subsequent coding network processing. Preferably, the front-end acoustic features are linear frequency cepstral coefficients (LFCC) features. The LFCC features adopted use a linear frequency band division method, which is more conducive to preserving local anomaly information and spectral envelope change details in the high-frequency region than the traditional Mel-scale-based feature representation. This enhances the ability to preserve high-frequency anomaly clues and spectral detail change information in forged speech, thus providing a more sufficient low-level input representation for forged speech detection tasks. Through this step, the conversion from the original speech signal to a frame-level acoustic feature sequence is realized. Therefore, step S2 can map the frame-level acoustic feature sequence into a frame-level acoustic feature sequence that can be processed by the subsequent coding network, thereby completing the conversion from the original speech signal to the detection feature input.

[0024] S3: Input the frame-level acoustic feature sequence into an ERC-Conformer coding network containing a Conformer backbone branch and an extended residual connection ERC branch, and obtain the enhanced coding features through collaborative modeling of the Conformer backbone branch and the extended residual connection ERC branch.

[0025] The frame-level acoustic feature sequence is input into the ERC-Conformer coding network for deep feature extraction. The ERC-Conformer coding network is based on the Conformer backbone structure. While maintaining the global context modeling capability, it introduces extended residual connection ERC branches in the residual path of the target sub-layer to supplement and enhance the local fine-grained time-frequency artifacts in the forged speech, thereby obtaining enhanced coded features.

[0026] Through the synergistic effect of the main branch and the ERC branch, the ERC-Conformer coding network can simultaneously characterize the macroscopic acoustic structure information and local anomalous perturbation features in the speech signal, thereby improving the ability to represent subtle differences between real speech and fake speech.

[0027] The ERC branch in step S3 includes a collaborative processing procedure of high-dimensional expansion, local temporal modeling, dimensionality reduction mapping, and dynamic gating. The specific steps of the collaborative processing procedure are as follows: First, the input features are extended to a high-dimensional representation space through linear projection; Subsequently, local convolution operations are applied to the expanded features along the time axis to model local temporal structures such as short-term discontinuities, local phase perturbations, and transient anomalies. Subsequently, candidate enhancement features are obtained through nonlinear activation and dimensionality reduction mapping; Meanwhile, channel-wise dynamic gating weights are generated based on the statistical information of the input features in the time dimension, adaptively modulating the candidate enhancement features, and combining them with a global scaling factor to form a residual enhancement term; Finally, the residual enhancement term is fused with the input features and the original sublayer output to obtain the enhanced coding features.

[0028] In step S3, the local modeling process of the ERC branch adopts a one-dimensional deep convolution operation along the time axis to enhance the model's ability to perceive short-term mutations, local discontinuities, and fine-grained fake perturbations; the local modeling results are then nonlinearly activated and mapped back to the original feature dimension to form candidate enhanced features. The dynamic gating strategy for the ERC branch in step S3 is as follows: To enable the extended residual connection to adaptively enhance different input speech samples, statistical descriptive information should be calculated along the time dimension based on the input features, and channel-by-channel gating coefficients should be generated based on the statistical descriptive information. The statistical descriptive information includes at least mean information and fluctuation information. The gating coefficients are used to perform channel-by-channel scaling modulation on the candidate enhancement features, and combined with global scaling parameters to control the overall contribution strength of the residual enhancement term, so as to avoid excessive perturbation to the original residual propagation path. The Conformer network includes a multi-head self-attention sub-layer, a convolutional sub-layer, and a feedforward network sub-layer; The parallel integration method of the ERC branch in Conformer is as follows: the ERC branch is embedded into multi-head self-attention sub-layers and convolutional sub-layers in a parallel residual branch manner to enhance the supplementary modeling capability for local weak forgeries while maintaining the stability of the backbone network; while the feedforward network sub-layers maintain their original structure to avoid unnecessary parameter redundancy and maintain the stability of the overall network topology; the input features are normalized and then fed into the original sub-layers and ERC modules respectively, and the two outputs are fused with the input residuals to achieve collaborative modeling of the backbone branch and the enhancement branch. S4: Perform speech-level aggregation and classification on the enhanced encoded features, and output the corresponding fake speech detection results.

[0029] Step S4 specifically involves: firstly, performing discourse-level aggregation on the enhanced encoded features to obtain a fixed-dimensional discourse-level embedding representation; then, inputting the discourse-level embedding representation into the classification module to perform discrimination between real and fake speech in the speech to be detected, and outputting the corresponding detection results.

[0030] The speech-level aggregation process is preferably implemented using statistical pooling, and more preferably attention-based statistical pooling, to allocate aggregation weights according to the importance of different time frames for true / false discrimination; the classification module is preferably trained using a classification optimization objective with interval constraints to enhance the intra-class compactness and inter-class separability of the output features and improve the performance of fake speech detection.

[0031] like Figure 3 As shown in the figure, this embodiment further explains the structural composition and working principle of the extended residual connection ERC branch in this invention.

[0032] The ERC branch is used to supplement and enhance the local fine-grained time-frequency artifacts in the forged speech detection task while maintaining the stable transmission function of the original residual path. This improves the coding network's ability to represent weak forgery clues. Unlike traditional residual connections, which mainly use identity mapping and only undertake cross-layer information transmission, the ERC branch in this embodiment introduces a learnable enhancement branch into the residual path, expanding the residual path from a fixed transmission structure to an input-dependent conditional enhancement structure.

[0033] Specifically, let the features input to the ERC branch be represented as input features, which have time and channel dimensions. Traditional residual connections usually only add the input features directly to the output of the backbone branch. In order to enhance the modeling ability of the residual path for local artifacts, this embodiment further introduces residual enhancement terms on the basis of the original residual connection, so that the output features simultaneously include the original input information, the information extracted by the backbone network, and the enhancement information generated by the ERC branch. Thus, the ERC branch can enhance the characterization of local time-frequency anomaly patterns related to forgery detection without destroying the stability of the original network.

[0034] In terms of structural composition, the ERC branch includes a feature expansion unit, a local modeling unit, a feature reflection unit, and a gated modulation unit. The ERC branch also includes a collaborative processing procedure of "high-dimensional expansion - local perception - dimensionality reduction mapping - dynamic gating", as detailed below: First, the feature expansion unit performs a high-dimensional expansion mapping on the input features. By using linear projection, the input features are mapped from the current dimension to a higher-dimensional representation space. This high-dimensional expansion operation can improve the representational freedom of the residual branch and enhance the interaction between different channels and different frequency bands, thereby providing a richer feature combination space for subsequent local structure modeling.

[0035] Preferably, in this embodiment, the expansion factor can be 4.

[0036] Subsequently, the local modeling unit performs local temporal modeling operations on the expanded features along the time axis to obtain local modeling results. This step can enhance the model's ability to perceive forgeries and artifacts such as short-term discontinuities, local disturbances, and transient anomalies.

[0037] Preferably, the local modeling process is implemented using one-dimensional depth convolution. Since depth convolution can independently perform time-dimensional modeling on each channel, it can more effectively extract pseudo-speculiar responses in fake speech that are short in duration, weak in amplitude, but have local structural anomalies.

[0038] After obtaining the local modeling results, the feature replay unit first performs nonlinear activation processing on it, preferably using the SiLU activation function, and then compresses the high-dimensional representation back to the original feature dimension through dimensionality reduction mapping to obtain candidate enhanced features.

[0039] Through the processing path of "high-dimensional expansion - local modeling - dimensionality reduction mapping", the ERC branch can generate candidate enhancement features containing local artifact response information, providing a foundation for subsequent adaptive modulation.

[0040] In this embodiment, setting a gated modulation unit in the ERC branch allows the enhancement level to be adaptively adjusted according to different input speech samples.

[0041] To enable the extended residual connection to adaptively enhance different input speech samples, the gated modulation unit first performs statistical analysis on the input features along the time dimension, calculating their mean and fluctuation information, preferably a mean vector and a standard deviation vector. Then, the statistical information is concatenated to form a global statistical description, which is input into a multilayer perceptron for nonlinear mapping to generate channel-by-channel gated weights, which are then used to adaptively modulate the candidate enhancement features. Through this process, the enhancement intensity of different channels can be controlled within a reasonable range, allowing the ERC branch to highlight the channel responses related to forgery detection while avoiding excessive perturbation of the original feature distribution due to overly strong enhancement.

[0042] Building upon this, this embodiment further introduces a global scaling factor to control the overall strength of the residual enhancement term. Finally, the gate weights and candidate enhancement features are modulated channel-by-channel, and the residual enhancement term is generated in conjunction with the global scaling factor. Thus, the ERC branch is a conditional enhancement mechanism resulting from the combined action of local structural transformation and statistical gating. Specifically, the input features undergo high-dimensional expansion, local convolution, and dimensionality reduction mapping to generate candidate artifact responses, while simultaneously generating channel-level modulation coefficients through statistical information. Both factors jointly determine the final expression of the residual enhancement term. When the gating weights tend to be constant and the output of the enhancement branch is weak, the impact of the ERC branch on the overall mapping decreases accordingly. At this point, the network behavior can degenerate into a form close to the standard residual connection. Therefore, the ERC branch in this embodiment does not completely replace the original residual path, but rather introduces learnable enhancement capabilities for local time-frequency artifacts in forged speech while retaining the role of residual stable transmission. Through this combined setting, the residual path not only undertakes the function of cross-layer information transmission, but also can perform targeted supplementary modeling of local anomaly clues based on the statistical attributes and local structural features of the input speech, thereby providing more sufficient feature support for the subsequent authenticity judgment of the coding network.

[0043] This embodiment further illustrates the integration method of the extended residual connection ERC branch in the Conformer coding network of the present invention. For example... Figure 2 As shown, in order to introduce enhancement capabilities for local time-frequency artifacts while preserving the stability of the original Conformer backbone structure as much as possible, this invention adopts a selective parallel integration strategy to embed the ERC branch into the target sub-layer residual path of the Conformer block, thereby realizing the collaborative modeling of the backbone branch and the enhancement branch.

[0044] Specifically, in a Conformer block, the input features are first processed sequentially through a feedforward sub-layer, a multi-head self-attention sub-layer, a convolutional sub-layer, and a second feedforward sub-layer. Cross-layer feature fusion is achieved through residual connections. Although the traditional Conformer can simultaneously take into account global context modeling and local convolutional modeling, its residual path usually only uses a fixed identity mapping, mainly undertaking the role of information transmission, and lacks the ability to adaptively enhance input features. To address this issue, this embodiment embeds ERC branches in the multi-head self-attention sub-layer and the convolutional sub-layer in the form of parallel residual branches, so that the residual path can not only maintain a stable transmission role, but also further have the ability to supplement the modeling of local fine-grained forgery artifacts.

[0045] In the specific integration process, the input features are first normalized, and the normalized features are then fed into the original sub-layer branch and the ERC branch. The original sub-layer branch performs standard attention modeling or convolution modeling operations to extract global contextual dependencies or structural information between adjacent local frames in the speech signal. The ERC branch performs high-dimensional expansion, local temporal modeling, dimensionality reduction mapping, and dynamic gating modulation on the same input features to generate residual enhancement terms related to forgery detection. Subsequently, the input features, the output of the original sub-layer, and the residual enhancement terms from the ERC branch are fused to obtain the enhanced encoded features.

[0046] In this embodiment, the ERC branch is preferably applied to the multi-head self-attention sublayer and the convolutional sublayer in a parallel residual branch manner, rather than to the two feedforward network sublayers. This is because the multi-head self-attention sublayer is mainly responsible for modeling long-range contextual dependencies, and the convolutional sublayer is mainly responsible for extracting local pattern information. Both have a more direct correspondence with macroscopic acoustic structural anomalies and local time-frequency artifacts in the forged speech detection task. In contrast, the main function of the feedforward network sublayer is to perform position-wise nonlinear mapping. If an ERC branch emphasizing local time-frequency enhancement is also introduced into this type of sublayer, it may cause functional overlap and introduce unnecessary parameter redundancy. Therefore, this embodiment adopts a selective parallel integration strategy, so that the ERC branch focuses on the target sublayer that is more relevant to local anomaly modeling.

[0047] Through the aforementioned integration method, the Conformer backbone branch and the ERC enhancement branch form a complementary relationship. The backbone branch is mainly responsible for extracting global contextual information and macroscopic acoustic dependencies of the speech signal, while the ERC branch supplements the modeling of local time-frequency artifacts through conditional enhancement processes in the residual paths. After element-wise fusion, the model's ability to represent subtle differences between real and fake speech can be improved while maintaining the original network topology and training stability. In other words, this embodiment does not achieve improvement by replacing the original Conformer structure, but rather by performing task-oriented local enhancements on key residual paths while retaining its main framework, thereby balancing model stability, scalability, and detection performance.

[0048] This embodiment further explains the backend aggregation and classification output process in this invention. After processing by the aforementioned ERC-Conformer coding network, the frame-level feature sequence corresponding to the input speech already contains both the global acoustic dependency information extracted by the backbone network and the local time-frequency artifact information enhanced by the ERC branch. Since the duration of different speech samples usually varies, the length of the frame-level feature sequence output by the coding network is not fixed. Therefore, it is necessary to further convert the variable-length frame-level feature sequence into a fixed-dimensional discourse-level representation for subsequent unified real / fake speech discrimination. To this end, this embodiment sets up a discourse-level aggregation and classification output module to complete the conversion from frame-level features to discourse-level embedded representation and output the corresponding fake speech detection results.

[0049] Specifically, the backend aggregation module receives the frame-level feature sequence output by the ERC-Conformer coding network and performs statistical aggregation on the frame-level features to obtain a fixed-dimensional discourse-level embedding representation.

[0050] Preferably, the aggregation method can be implemented using statistical pooling, and more preferably attention-based statistical pooling. Unlike traditional global average pooling, attention-based statistical pooling can adaptively allocate aggregation weights according to the importance of different time frames in distinguishing between real and fake speech, thereby giving a higher proportion of local temporal segments that are more critical to the detection task in the final representation. This not only preserves the first-order statistical information of the speech signal, but also preserves the second-order statistical information related to the dynamic changes over time, making the output speech-level representation more suitable for forgery detection tasks.

[0051] Furthermore, in a preferred embodiment, the attention statistical pooling module first calculates the attention weights corresponding to each time frame based on the frame-level features, then performs weighted aggregation on all frame-level features based on the attention weights to obtain a weighted mean representation and a weighted fluctuation representation, preferably a weighted standard deviation representation. Finally, the two types of statistics are concatenated to form a fixed-dimensional discourse-level embedding representation. In this way, the model can not only reflect the average feature distribution of the overall speech, but also further characterize common local abnormal fluctuations, energy instability, and fine-grained statistical biases in fake speech, thereby enhancing the discourse-level representation's ability to represent the differences between real and fake speech.

[0052] After obtaining the utterance-level embedding representation, the classification output module further performs real / fake speech discrimination. Specifically, the utterance-level embedding representation is input into the classification layer, which outputs a discrimination result indicating whether the detected speech belongs to real or fake speech. In a preferred embodiment, the classification layer can be trained using a classification optimization objective with margin constraints to enhance the intra-class compactness and inter-class separability of the output features. More preferably, the classification optimization objective can use the AM-Softmax loss function. Compared to ordinary Softmax, the classification optimization method with margin constraints can further widen the discrimination boundary between different categories in the feature space, making the utterance-level embedding representation learned by the model more conducive to distinguishing between real and fake speech.

[0053] Through the aforementioned backend aggregation and classification output process, this embodiment can map the variable-length frame-level feature sequence output by the aforementioned ERC-Conformer coding network into a fixed-dimensional speech-level embedding representation, and output the forged speech detection result accordingly.

[0054] It should be noted that the core innovation of this invention lies mainly in the extended residual connection ERC branch and its integration method in the Conformer coding network, while the backend aggregation and classification output module is mainly used to complete the closed-loop implementation of the detection process. Therefore, without departing from the core idea of ​​this invention, the utterance-level aggregation method and classification output method can also adopt other equivalent implementation forms that can achieve variable-length sequence aggregation and true / false discrimination according to actual application needs.

[0055] In summary, this embodiment, by setting up a backend aggregation and classification output module, further converts the frame-level features extracted by the ERC-Conformer coding network into a speech-level representation suitable for discrimination, and completes the classification output of real speech and fake speech. Thus, together with the aforementioned front-end feature extraction process and ERC-Conformer coding enhancement process, it constitutes a complete fake speech detection method flow.

[0056] To verify the effectiveness of the forged speech detection method based on extended residual connections proposed in this invention, this embodiment was tested on the ASVspoof 2019 LA evaluation set.

[0057] This dataset is a commonly used publicly available evaluation dataset for audio deepfake detection tasks. It contains various types of unknown logical access attacks and can effectively reflect the model's detection capabilities under complex forgery conditions. This embodiment mainly focuses on analyzing the effectiveness of the internal components of the ERC branch and the rationality of the integration method of the ERC branch in the Conformer coding network to verify the performance improvement of the technical solution described in this invention for fake speech detection.

[0058] First, to verify the role of each component within the ERC module, this embodiment uses a standard Conformer backbone network equipped with Attention Statistical Pooling (ASP) and AM-Softmax loss function as the baseline system. On this basis, different components in the ERC branch are gradually introduced, and the detection performance changes under the corresponding model configurations are compared. The detection results are shown in Tables 1 and 2.

[0059] Table 1. Performance verification results of internal components of the ERC module

[0060] Table 1 shows the experimental results. When only linear high-dimensional extended projection is added to the residual path, the model's equal error rate decreases from 3.68% to 3.05%, indicating that the high-dimensional representation space helps enhance the model's ability to represent anomalous spectral patterns. Further introducing temporal deep convolution further reduces the model's equal error rate to 2.44%, demonstrating that local temporal modeling plays a positive role in capturing fine-grained artifacts such as short-term discontinuities and local perturbations. Adding a dynamic gating mechanism further reduces the model's equal error rate to 2.38%, indicating that adaptive modulation based on global statistical information can further improve the fusion effect between the residual enhancement term and the original features. Therefore, the three components of the ERC branch—high-dimensional extension, local temporal modeling, and dynamic gating—all have a positive effect on improving the performance of forged speech detection, thus verifying the rationality of the ERC branch structure design described in this embodiment.

[0061] Furthermore, to verify the integration method of the ERC module in the Conformer coding network, this embodiment compares the detection performance of the model under different integration strategies, and the detection results are shown in Table 2.

[0062] Table 2. Results of ERC module integration effectiveness verification.

[0063] Table 2 shows the experimental results: without the ERC module, the baseline model's equal error rate (EMR) is 3.68%; when the ERC module is applied to all residual connections, the EMR decreases to 2.95%; and when the ERC module is integrated in parallel only into the multi-head self-attention sub-layer and convolutional sub-layer, the EMR further decreases to 2.38%. These results indicate that, compared to uniformly applying enhancements across all residual paths, introducing the ERC branch only into the target sub-layer (attention sub-layer and convolutional sub-layer), which is more relevant to context modeling and local pattern extraction, is more effective in enhancing the local time-frequency artifact enhancement capabilities of extended residual connections. This demonstrates that the selective parallel integration strategy employed in this invention is both reasonable and effective, improving the model's ability to perceive fine-grained forgery cues while avoiding the introduction of functional redundancy in unnecessary sub-layers.

[0064] The results of the above verification show that the extended residual connection ERC branch proposed in this invention not only improves the modeling ability of the residual path for local forgery artifacts through high-dimensional extension, local temporal modeling, and dynamic gating mechanisms, but also further enhances the model's ability to detect forged speech in complex and unknown attack scenarios by selectively integrating it into the target sublayer of the Conformer coding network. This demonstrates that the forged speech detection method based on extended residual connections proposed in this invention, while maintaining the advantages of global modeling of the backbone network and the overall structural stability, enhances the supplementary modeling ability for local fine-grained forgery artifacts, providing a more sufficient feature basis for subsequent speech-level aggregation and true / false discrimination. It can effectively improve the representation effect of subtle differences between real and forged speech and has good application value.

[0065] This embodiment also provides a spoofing detection device based on extended residual connections, which is used to implement the spoofing detection method based on extended residual connections described in the foregoing embodiments.

[0066] The device includes a voice acquisition module, a front-end feature extraction module, an encoding enhancement module, an aggregation module, and a classification and discrimination module.

[0067] The speech acquisition module receives the speech signal to be detected; the front-end feature extraction module extracts acoustic features from the speech signal to be detected to obtain a frame-level acoustic feature sequence; the encoding enhancement module inputs the frame-level acoustic feature sequence into the ERC-Conformer coding network and performs collaborative enhancement modeling on local time-frequency artifacts through parallel fusion of the Conformer backbone branch and the extended residual connection ERC branch to obtain enhanced encoded features; the aggregation module converts the enhanced encoded features into a speech-level embedding representation; and the classification and discrimination module outputs the detection result of real speech or fake speech based on the speech-level embedding representation.

[0068] The ERC branch in the encoding enhancement module includes a feature expansion unit, a local modeling unit, a feature reversion unit, and a gated modulation unit. The feature expansion unit maps input features to a high-dimensional representation space; the local modeling unit extracts local anomalous patterns along the time axis; the feature reversion unit maps high-dimensional features back to the original dimension; and the gated modulation unit generates channel-wise gate weights based on statistical information of the input features to adaptively adjust candidate enhancement features. Through this module configuration, this embodiment can enhance the modeling ability for local fine-grained forgery artifacts while maintaining the stability of the backbone network structure, thereby improving the performance of forged speech detection.

[0069] The above embodiments are preferred implementations of the present invention. In addition, the present invention can be implemented in other ways. Any obvious substitutions without departing from the concept of the present technical solution are within the protection scope of the present invention.

Claims

1. A method for detecting forged speech based on extended residual connections, characterized in that, Includes the following steps: S1: Acquire the speech signal to be detected; S2: Perform front-end acoustic feature extraction on the speech signal to be detected to obtain a frame-level acoustic feature sequence; S3: Input the frame-level acoustic feature sequence into an ERC-Conformer coding network containing a Conformer backbone branch and an extended residual connection ERC branch, and obtain the enhanced coding features through collaborative modeling of the Conformer backbone branch and the extended residual connection ERC branch; S4: Perform speech-level aggregation and classification on the enhanced encoded features, and output the corresponding fake speech detection results.

2. The method for detecting forged speech based on extended residual connections according to claim 1, characterized in that, In step S2, the front-end acoustic features adopt linear frequency cepstral coefficient (LFCC) features. In step S2, the frame-level acoustic feature sequence is mapped into a continuous feature representation that can be processed by the subsequent coding network, thereby completing the conversion from the original speech signal to the detection feature input.

3. The method for detecting forged speech based on extended residual connections according to claim 1, characterized in that, The ERC branch in step S3 includes a collaborative processing procedure of "high-dimensional expansion - local modeling - dimensionality reduction mapping - dynamic gating", specifically including the following steps: First, the input features are extended to a high-dimensional representation space through linear projection; Subsequently, local convolution operations are applied to the expanded features along the time axis to obtain the local modeling results; Then, nonlinear activation and dimensionality reduction mapping are performed on the local modeling results to obtain candidate enhancement features; Finally, channel-wise dynamic gating weights are generated based on the statistical information of the input features in the time dimension, adaptively modulate the candidate enhancement features, and combine them with a global scaling factor to form a residual enhancement term.

4. The method for detecting forged speech based on extended residual connections according to claim 3, characterized in that, The local modeling unit employs a one-dimensional depthwise convolution operation along the time axis, which enhances its ability to detect forgeries and artifacts such as short-term discontinuities, local disturbances, and transient anomalies.

5. The method for detecting forged speech based on extended residual connections according to claim 3, characterized in that, The statistical description information includes mean information and fluctuation information.

6. The method for detecting forged speech based on extended residual connections according to claim 3, characterized in that, Step S3 specifically involves: the frame-level acoustic feature sequence is first normalized and then fed into the original sub-layer branch and the ERC branch of the Conformer backbone. The original sub-layer branch is used to perform standard attention modeling or convolution modeling operations to extract global contextual dependencies or structural information between local adjacent frames in the speech signal. The ERC branch performs high-dimensional expansion, local modeling, dimensionality reduction mapping, and dynamic gating modulation on the same input feature to generate residual enhancement terms related to forgery detection. Subsequently, the input features, the output of the original sub-layer, and the residual enhancement terms output by the ERC branch are fused to obtain the enhanced coded features.

7. The method for detecting forged speech based on extended residual connections according to claim 1, characterized in that, The Conformer backbone includes multi-head self-attention sub-layers, convolutional sub-layers, and feedforward network sub-layers; The parallel integration method of the ERC branch in the Conformer backbone is as follows: the ERC branch is embedded in the multi-head self-attention sub-layer and convolutional sub-layer in the form of parallel residual branches, so as to enhance the supplementary modeling capability for local weak forgery artifacts while maintaining the stability of the backbone network; while the feedforward network sub-layer maintains its original structure to avoid unnecessary parameter redundancy and maintain the stability of the overall network topology.

8. The method for detecting forged speech based on extended residual connections according to claim 1, characterized in that, Step S4 specifically involves: firstly, performing discourse-level aggregation on the enhanced encoded features to obtain a fixed-dimensional discourse-level embedding representation; then, inputting the discourse-level embedding representation into the classification module to perform discrimination between real and fake speech in the speech to be detected, and outputting the corresponding detection results.

9. The method for detecting forged speech based on extended residual connections according to claim 1, characterized in that, The speech-level aggregation process is implemented using statistical pooling or attention statistical pooling to allocate aggregation weights based on the importance of different time frames for true / false speech discrimination. The classification module is preferably trained using a classification optimization objective with interval constraints to enhance the intra-class compactness and inter-class separability of the output features and improve the performance of fake speech detection.

10. A forged speech detection device based on extended residual connections, used to implement the forged speech detection method based on extended residual connections according to any one of claims 1-9, characterized in that, include: The voice acquisition module is used to receive the voice signal to be detected; The front-end feature extraction module is used to extract acoustic features from the speech signal to be detected and construct a frame-level input representation; The encoding enhancement module is used to input the frame-level input representation into the encoding network and enhance the modeling of local time-frequency artifacts through parallel fusion of the Conformer backbone branch and the extended residual connection ERC branch to obtain the enhanced encoding features; The aggregation module is used to convert the enhanced encoded features into utterance-level embedded representations; The classification and discrimination module is used to output the detection result of real speech or fake speech based on the utterance-level embedding representation.

Images