Multi-scale cross-branch feature interaction-based forged voice detection method and device

Abstract

The invention discloses a forged voice detection method and device based on multi-scale cross-branch feature interaction, and belongs to the technical field of forged voice detection. The method comprises the following steps: acquiring and preprocessing a training corpus to obtain a voice segment with a fixed time length; constructing a detection model, and inputting the voice segments into the detection model for training; the detection model comprises a front-end feature extraction module, a multi-scale cross-branch feature interaction module and a classification module; the front-end feature extraction module adopts a pre-training model XLSR-300M to extract high-dimensional time sequence features in the voice segments; the multi-scale cross-branch feature interaction module performs multi-scale division and parallel feature extraction on the high-dimensional time sequence features, and realizes feature interaction and information complementation among different branches through a cross attention mechanism; the classification module is used for integrating the interacted multi-scale features and outputting a classification result of voice authenticity; and obtaining and preprocessing a test corpus, inputting an obtained voice segment into the trained detection model, and outputting a forged voice detection result.

Description

Technical Field

[0001] This invention belongs to the field of forged speech detection technology, specifically relating to a forged speech detection method and apparatus based on multi-scale cross-branch feature interaction. Background Technology

[0002] With the deep integration of speech synthesis, conversion technology and deep learning, the naturalness and human-likeness of forged speech continue to improve, making it extremely easy to deceive Automatic Speaker Verification (ASV) systems and causing security risks such as identity forgery and fraudulent transactions. Therefore, forged speech detection has become a research focus in the field of speech security.

[0003] The development of forged speech detection technology has undergone multiple stages of evolution: In the early stages, it relied on manually designed acoustic features and traditional classifiers, but manual features were limited by prior knowledge, easily lost fine-grained deception clues, and had poor robustness to unknown attacks; With the advancement of the ASVspoof series of tasks, models gradually shifted to end-to-end architectures, and attention mechanisms such as Residual Network (ResNet), Squeeze and Excitation Network (SENet) were widely used, enhancing the ability to extract local features and select channels; In recent years, pre-trained models such as wav2vec 2.0 and its variant XLSR-300M have become the mainstream for front-end feature extraction, significantly improving the generalization ability of the models.

[0004] However, existing technologies still have some shortcomings: First, there is insufficient modeling of global and long-range dependencies. Convolutional Neural Networks (CNNs) are limited by the receptive field of convolution and have difficulty capturing the temporal correlation between the two ends of speech. Although there have been related explorations based on Transformers, they have not fully combined multi-scale feature collaboration. Second, multi-scale features are not fully utilized. Single-scale features or simple channel fusion are easily interfered with by redundant information and cannot take into account both fine-grained temporal-frequency details and coarse-grained global semantics. Third, cross-branch information interaction is lacking. The complementary information of features at different scales has not been effectively mined, resulting in limited detection accuracy and generalization in complex scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method and apparatus for detecting forged speech based on multi-scale cross-branch feature interaction, thereby solving the problems in the prior art.

[0006] The objective of this invention can be achieved through the following technical solutions: Forged speech detection methods based on multi-scale cross-branch feature interactions include: Acquire training data, including real speech and fake speech; The acquired training data is preprocessed to obtain speech segments of fixed duration; A detection model is constructed by inputting preprocessed speech segments into the model for training. Hyperparameters are adjusted to minimize the target loss function until a set number of iterations are reached, resulting in a trained detection model. The detection model includes a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module. The front-end feature extraction module uses a pre-trained model XLSR-300M to extract high-dimensional temporal features from the speech segments. The multi-scale cross-branch feature interaction module performs multi-scale partitioning and parallel feature extraction on the high-dimensional temporal features, and uses a cross-attention mechanism to achieve feature interaction and information complementarity between different branches. The classification module integrates the interacted multi-scale features and outputs the classification result for speech authenticity. Acquire test data, including real speech and fake speech. Preprocess the test data to obtain speech segments of fixed duration, input them into the trained detection model, and output fake speech detection results.

[0007] Furthermore, the preprocessing process for the training corpus and the test corpus is the same: each speech segment in the corpus is trimmed or spliced ​​according to the number of sample points to obtain a speech segment of fixed duration.

[0008] Furthermore, in the front-end feature extraction module, the preprocessed speech segment sequentially passes through an XLSR-300M pre-trained model, a two-dimensional convolutional layer, a dimension expansion operation, and a dimension transpose operation.

[0009] Furthermore, the pre-trained model XLSR-300M includes: one feature input layer, a 24-layer Transformer encoder, and one feature output layer; the feature input layer converts the input speech temporal signal into a linear feature sequence that the model can process; each Transformer encoder layer contains one multi-head self-attention sub-layer and one feedforward neural network sub-layer, and both sub-layers adopt residual connections and layer normalization mechanisms. The multi-head self-attention sub-layer models the association of information at different positions in the speech feature sequence by computing multiple attention heads in parallel. The feedforward neural network sub-layer adopts a two-layer fully connected network structure and enhances the model's feature expression capability through a non-linear activation function.

[0010] Furthermore, the multi-scale cross-branch feature interaction module adopts a dual-branch parallel architecture, including: a dual-branch feature embedding submodule, a dual-branch encoding submodule, a bidirectional cross-attention submodule, and a feature integration submodule; The high-dimensional temporal features are first input into the dual-branch feature embedding submodule, which generates branch features that focus on fine-grained artifacts and branch features that focus on global modeling through multi-scale partitioning. Then, the two types of branch features are input into the dual-branch encoding submodule, which completes the deep semantic encoding within the branch through a self-attention layer and an MLP feedforward layer. The encoded dual-branch features enter the bidirectional cross-attention submodule, which realizes the complementary update of cross-branch features through a bidirectional cross-attention mechanism, and integrates local fine-grained information and global context information. Finally, the feature integration submodule aligns and concatenates the updated dual-branch features according to their temporal length, and outputs temporal features of unified dimension and global fused features.

[0011] Furthermore, the dual-branch feature embedding submodule includes: two two-dimensional convolutional layers, two learnable position embedding layers, and one Dropout layer; wherein, the high-dimensional temporal features output by the front-end feature extraction module... The two branches of the dual-branch feature embedding submodule are input separately, and feature blocks are partitioned and embedded through two-dimensional convolutional layers of different scales to obtain initial embedded features at two scales. and ; for initial embedded features and By performing flattening and transpose operations respectively, the embedding features of the sequence format are obtained. and Then add learnable position embedding vectors respectively. and This yields position-encoded sequence features. and For sequence features and Dropout random deactivation is performed separately, and finally the intra-branch encoded input features are obtained. and ; The dual-branch encoding submodule includes: one single-layer Transformer block and one dual-layer Transformer block. Each Transformer block consists of one multi-head self-attention layer, two layer normalization layers, one MLP feedforward layer, and two residual connections: encoding the input features within each branch. and The single-layer and double-layer Transformer blocks of the dual-branch Transformer encoding submodule are respectively input for deep encoding. During the encoding process, a multi-head self-attention sublayer models the correlation between different positional information of the sequence features, an MLP feedforward sublayer performs nonlinear transformation on the features, and residual connections and layer normalization layers are used for training and to retain the original feature information, resulting in encoded features that include both global and local features. and ; The bidirectional cross-attention submodule includes: two cross-attention blocks, each consisting of two layer normalization layers, four linear layers, one multi-head cross-attention layer, and one Softmax activation function: from encoded features and Extracting global features , and local features , Cross-branch information complementarity is achieved through a bidirectional cross-attention interaction submodule, in order to For query, Calculate cross-attention updates for keys and values get ,by For query, Calculate cross-attention updates for keys and values get ; The feature integration submodule includes two concatenation operations and one linear interpolation operation: ... and , and The complete interactive features are obtained by splicing them together separately. and ,right and In , Features are aligned using linear interpolation to obtain time-series features of uniform length. and splicing at the same time and Obtain global fusion features .

[0012] Furthermore, the classification module includes: one feature concatenation layer, one linear layer, one bidirectional GRU layer, one global average pooling layer, two fully connected layers, two Dropout layers, and one ReLU activation function; the classification process of the classification module includes: right and The features are concatenated to obtain the concatenated temporal features. and will The input linear projection layer is used for dimension mapping to obtain the mapped temporal features. ; Will The input is processed by a bidirectional GRU layer to encode temporal features and then global average pooling is performed to obtain global temporal statistical features. Integrate it with global features The final fused feature is obtained by performing a splicing operation. ; Fusion features The input is processed by the Dropout layer for regularization, then input into the first fully connected layer, which maps the fused feature dimensions from 3072 to 512. After processing by the ReLU activation function, it is input into the second Dropout layer for secondary regularization. Finally, the input is fed into the second fully connected layer, mapped to the predicted log probability feature. , , Indicates batch size, The Middle ( The predicted log probability of ) samples is: in, The predicted log probability corresponding to real speech. The predicted log probability of the corresponding fake speech is directly used to calculate the weighted cross-entropy loss function to complete the model training process.

[0013] Furthermore, the target loss function uses a weighted cross-entropy loss function. : in, Indicates batch size, , Indicates the first The loss weights corresponding to the true class of each sample Indicates the first The true label corresponding to each sample Corresponding to real voice categories, Corresponding to the category of forged voice; Represents the natural logarithm operation. Indicates exponentiation. , The first The predicted log probabilities of each sample for real speech and forged speech, and , , Indicates the first The predicted log probability of the category corresponding to the true label of each sample. If the sample is real speech, then If the sample is fake speech, then .

[0014] The forged speech detection device based on multi-scale cross-branch feature interaction performs the above method, including: Training corpus acquisition module: Acquires training corpus, including real speech and fake speech; Training corpus processing module: preprocesses the acquired training corpus to obtain speech segments of fixed duration; The detection model construction and training module is used to build a detection model. Preprocessed speech segments are input into the detection model for training. Hyperparameters are adjusted to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model. The detection model includes a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module. The front-end feature extraction module uses the pre-trained model XLSR-300M to extract high-dimensional temporal features from the speech segments. The multi-scale cross-branch feature interaction module performs multi-scale partitioning and parallel feature extraction on the high-dimensional temporal features, and uses a cross-attention mechanism to achieve feature interaction and information complementarity between different branches. The classification module integrates the interacted multi-scale features and outputs the classification result for speech authenticity. And, the detection module: acquires test data, which includes real speech and fake speech, preprocesses the test data to obtain speech segments of fixed duration, inputs them into the trained detection model, and outputs fake speech detection results.

[0015] A computer storage medium storing a readable program that, when executed, instructs a computing device to perform the forged speech detection method based on multi-scale cross-branch feature interaction as described above.

[0016] The beneficial effects of this invention are: 1. The multi-scale cross-branch feature interaction module of the present invention divides the high-dimensional temporal features output by the front-end feature extraction module into different scales, and constructs small-scale branches and large-scale branches for parallel modeling. Small-scale branches are more conducive to capturing short-term fine-grained forgery artifacts such as local discontinuities and subtle frequency band anomalies, while large-scale branches are more conducive to modeling long-range context and global structural consistency. Through parallel multi-scale feature extraction, the present invention simultaneously enhances the perception ability of local anomalies and global consistency violations in the same model, and reduces the risk of false detection caused by modeling only at a single scale.

[0017] 2. This invention differs from simple splicing or weighted summation multi-scale fusion methods by introducing a bidirectional cross-attention mechanism between two scale branches. This allows the global convergence marker of one branch to selectively focus on the local feature sequence of the other branch, thereby achieving cross-scale information interaction and complementarity. The small-scale branch can introduce more stable global context constraints from the large-scale branch, while the large-scale branch can obtain more sensitive local artifact clues from the small-scale branch. This interaction mechanism can effectively alleviate the problems of semantic inconsistency and redundancy in multi-scale features, making the fused features more focused and discriminative, thereby improving the model's adaptability and robustness to various types of forgery attacks.

[0018] 3. The classification module of this invention aligns and fuses the interactive dual-scale time series features, and uses a bidirectional GRU to model the temporal dynamics. By effectively aggregating sequence information, it enhances the ability to identify cross-time forgery patterns. At the same time, it jointly models the global summary information carried by the global features of each branch with the bidirectional GRU aggregation results, so that the classifier can use both global semantic summary and temporal dynamic evidence. Thus, even under short-term noise disturbances, local forgery weakening, or uneven distribution of forgery traces, it can still maintain a low risk of misjudgment and more stable output performance, thereby improving detection accuracy and engineering usability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the fake voice detection method of the present invention; Figure 2 This is a network structure diagram of the multi-scale cross-branch feature interaction module of the present invention; Figure 3 This is a network structure diagram of the bidirectional cross-attention submodule of the present invention; Figure 4 This is a network structure diagram of the classification module of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 like Figure 1 As shown, the spoofing detection method based on multi-scale cross-branch feature interaction includes a training phase (S1-S3) and a testing phase (S4). The training phase obtains the detection network and parameters required for spoofing detection, and the testing phase enables the detection of both real and spoofed speech. The specific steps of the spoofing detection method are as follows: Includes the following steps: S1, Obtain training data, including real speech and fake speech; In this embodiment, the training corpus comes from the LA logic access task of the ASVspoof2019 competition, including real speech and 6 types of fake speech generated by speech synthesis and speech conversion algorithms, of which 4 are speech synthesis type and 2 are speech conversion type. All training and verification corpora in the acquired corpus are extracted. There are 25,380 training corpora and 24,986 verification corpora. Most of the corpora are 2-8 seconds long.

[0023] The selected training corpus is used as the input for model training. In each iteration, the validation corpus is input into the model for validation, so that the training results of each round of the model can be presented intuitively, making it easier to adjust the training parameters of the model.

[0024] S2, preprocess the acquired training data to obtain speech segments of fixed duration; In this embodiment, the sampling rate of the training corpus is set to 16kHz and the number of sample points is 64,000. Each speech in the training corpus is trimmed or spliced ​​according to the number of sample points to obtain a speech segment with a fixed duration of 4s.

[0025] S3. Construct a detection model. Input the preprocessed speech segment into the detection model for training. Adjust the hyperparameters to minimize the target loss function until the set number of iterations is reached to obtain a trained detection model. like Figure 1 As shown, the detection model includes: a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module; wherein, the front-end feature extraction module uses the pre-trained model XLSR-300M to extract high-dimensional temporal features from speech segments; the multi-scale cross-branch feature interaction module performs multi-scale division and parallel feature extraction on the high-dimensional temporal features, and realizes feature interaction and information complementarity between different branches through a cross-attention mechanism; the classification module is used to integrate the multi-scale features after interaction and output the classification result of speech authenticity.

[0026] Specifically, the details of each module in the detection model are as follows: (1) Front-end feature extraction module In the front-end feature extraction module, the preprocessed speech segments sequentially pass through one XLSR-300M pre-trained model, one two-dimensional convolutional layer, one dimension expansion operation, and one dimension transpose operation; The XLSR-300M pre-trained model, through self-supervised pre-training on massive amounts of multilingual speech data, can directly extract high-dimensional temporal features from input speech segments, replacing traditional manually designed feature extraction methods and better capturing key acoustic and semantic information in speech signals. Two-dimensional convolutional layers encode the extracted high-dimensional temporal features, modeling the local temporal and feature channel correlation information of speech, enhancing the feature differences between real and fake speech, and suppressing redundant noise. Dimension expansion operations adapt the dimensionality format of the convolutional feature tensors, transforming three-dimensional features into four-dimensional features to meet the input dimensionality requirements of subsequent modules. Dimension transpose operations rearrange the dimensionality order of the four-dimensional feature tensors to adapt to the operational logic of subsequent multi-scale convolution and attention mechanisms, inputting standardized high-dimensional temporal features into the multi-scale cross-branch feature interaction module. The XLSR-300M pre-trained model consists of one feature input layer, 24 Transformer encoder layers, and one feature output layer. The feature input layer converts the input speech temporal signal into a linear feature sequence that the model can process. The Transformer encoder is the core feature extraction unit of the model. Each Transformer encoder layer contains one multi-head self-attention sublayer and one feedforward neural network sublayer. Both sublayers employ residual connections and layer normalization mechanisms. The multi-head self-attention sublayer models the association of information at different positions in the speech feature sequence through parallel computation of multiple attention heads. The calculation formula is as follows: in, This represents the result of single-head attention calculation. , , These represent the query vector, key vector, and value vector, respectively. express transpose, This represents the dimension of the query vector and the key vector. This is a scaling factor used to prevent gradient vanishing during computation; This represents the result of multi-head attention calculation. Indicates the first The computational output of each attention head. From 1 to positive integers, Indicates the number of attention heads. This represents a vector concatenation operation. This represents the output projection weight matrix; The feedforward neural network sublayer adopts a two-layer fully connected network structure, and enhances the model's feature representation ability through a non-linear activation function. The calculation formula is as follows: in, This represents the output of the feedforward neural network. This represents the input features of the sublayer. , These represent the weight matrices of the two fully connected layers, respectively. , These represent the corresponding bias vectors. This represents the ReLU activation function, used to introduce nonlinear feature transformations.

[0027] (2) Multi-scale cross-branch feature interaction module like Figure 2 As shown, the multi-scale cross-branch feature interaction module adopts a dual-branch parallel architecture, including: one dual-branch feature embedding submodule, one dual-branch encoding submodule, one bidirectional cross-attention submodule, and one feature integration submodule. High-dimensional temporal features are first input into the dual-branch feature embedding submodule, which generates small-scale branch features focusing on fine-grained artifacts and large-scale branch features emphasizing global modeling through multi-scale partitioning. Then, the two types of branch features are input into the dual-branch encoding submodule, where deep semantic encoding within each branch is completed through a self-attention layer and an MLP feedforward layer. The encoded dual-branch features enter the bidirectional cross-attention submodule, which uses a bidirectional cross-attention mechanism to achieve complementary updates of cross-branch features, fusing local fine-grained information with global contextual information. Finally, the feature integration submodule aligns and concatenates the updated dual-branch features according to their temporal length, outputting temporal features of a unified dimension and globally fused features.

[0028] The dual-branch feature embedding submodule consists of two two-dimensional convolutional layers, two learnable position embedding layers, and one Dropout layer: First, the high-dimensional temporal features output by the front-end feature extraction module are processed... The two branches of the dual-branch feature embedding submodule are input separately, and feature blocks are partitioned and embedded through two-dimensional convolutional layers of different scales to obtain initial embedded features at two scales. and The formula is as follows: in, This represents a two-dimensional convolution operation. and These are the convolution kernel (8,16) and stride (8,16) for the small-scale branch, respectively. and These are the convolution kernel (16,32) and stride (16,32) for the large-scale branch, respectively. For initial embedded features and By performing flattening and transpose operations respectively, the embedding features of the sequence format are obtained. and Then add learnable position embedding vectors respectively. and This yields position-encoded sequence features. and For sequence features and Dropout random deactivation is performed separately, and finally the intra-branch encoded input features are obtained. and .

[0029] The dual-branch encoding submodule consists of one single-layer Transformer block and one dual-layer Transformer block. Each Transformer block comprises one multi-head self-attention layer, two layer normalization layers, one MLP (Multilayer Perceptron) feedforward layer, and two residual connections: encoding the input features within each branch. and Deep encoding is performed on the single-layer Transformer block and the double-layer Transformer block of the dual-branch Transformer encoding submodule, respectively. During the encoding process, the multi-head self-attention sublayer models the correlation between different positional information of the sequence features, the MLP feedforward sublayer performs nonlinear transformation on the features, and the residual connection and layer normalization layer are used to stabilize training and retain the original feature information, resulting in encoded features that contain both global and local features. and ; The MLP feedforward sublayer consists of two fully connected layers, a GELU activation function, and a Dropout layer. The input features are expanded in dimension by the first fully connected layer, then nonlinear transformation is introduced through the GELU activation function, and the Dropout layer prevents overfitting. The second fully connected layer then compresses the feature dimension. The purpose is to perform nonlinear mapping on the features output by the multi-head self-attention layer in the Transformer block, thereby enhancing the representation ability of speech features.

[0030] like Figure 3 As shown, the bidirectional cross-attention submodule consists of two cross-attention blocks. Each cross-attention block comprises two normalization layers, four linear layers, one multi-head cross-attention layer, and one Softmax activation function: from the encoded features and Extracting global features , and local features , Cross-branch information complementarity is achieved through a bidirectional cross-attention interaction submodule, in order to For query, Calculate cross-attention updates for keys and values get Similarly, For query, Calculate cross-attention updates for keys and values get The formula is as follows: in, This represents cross-attention computation, which specifically includes the linear projection of the query, key, and value, the softmax normalization computation of attention weights, the linear projection of the output features, and the residual connection operation.

[0031] The feature integration submodule includes two concatenation operations and one linear interpolation operation: ... and , and The complete interactive features are obtained by splicing them together separately. and ,right and In , Features are aligned using linear interpolation to obtain time-series features of uniform length. and splicing at the same time and Obtain global fusion features The formula is as follows: in, This represents a vector concatenation operation. This represents a linear interpolation operation. Two branches and Maximum feature length, final output , and As input features for the classification module.

[0032] (3) Classification module like Figure 4As shown, the classification module includes: one feature concatenation layer, one linear layer, one bidirectional GRU layer, one global average pooling layer, two fully connected layers, two Dropout layers, and one ReLU activation function; the classification process of the classification module includes: First of all, and The features are concatenated to obtain the concatenated temporal features. and will The input linear projection layer is used for dimension mapping to obtain the mapped temporal features. Secondly, The input is processed by a bidirectional GRU layer to encode temporal features and then global average pooling is performed to obtain global temporal statistical features. Integrate it with global features The final fused feature is obtained by performing a splicing operation. Then, the fusion features will be... The input is processed by the Dropout layer for regularization, then fed into the first fully connected layer, which maps the fused feature dimension from 3072 to 512. After passing through the ReLU activation function, it is fed into the second Dropout layer for secondary regularization. Finally, it is fed into the second fully connected layer, which maps it to the predicted log probability feature. , , Indicates batch size, The Middle ( The predicted log probability of ) samples is: in, The predicted log probability corresponding to real speech. The predicted log probability of the corresponding fake speech is directly used to calculate the weighted cross-entropy loss function to complete the model training process.

[0033] The target loss function used during training is the weighted cross-entropy loss function. The formula is as follows: in, Indicates batch size, , Indicates the first The loss weights corresponding to the true class of each sample Indicates the first The true label corresponding to each sample ( Corresponding to real voice categories, (Corresponding to the category of fake voice) Represents the natural logarithm operation. Indicates exponentiation. , The first The predicted log probabilities of each sample for real speech and forged speech, and , , Indicates the first The predicted log probability of the category corresponding to the true label of each sample, if the sample is real speech ( ),but Similarly, if the sample is fake speech ( ),but ; During training, the loss from each batch is backpropagated to each layer of the model to update the model parameters, thus completing the training process and detecting the weighted cross-entropy loss function of the model. The smaller the value, the higher the match between the model's prediction results for real and fake speech categories and the sample's real labels, and the higher the model's accuracy in detecting fake speech.

[0034] In this embodiment, the number of iteration rounds is set to 100.

[0035] S4. Obtain test data, which includes real speech and fake speech. Preprocess the test data to obtain speech segments of fixed duration, input them into the trained detection model, and output the fake speech detection results.

[0036] The forged speech is generated by an algorithm that combines speech synthesis, speech conversion, or a combination of both. The preprocessing of the test corpus is the same as that of the training corpus in S2. In this embodiment, the test corpus comes from the logical access tasks of the ASVspoof2019 and ASVspoof2021 competitions. All the obtained test corpus data, totaling 181,566 entries, are extracted. Most of the data are 2-8 seconds long. Unlike the test corpus of the ASVspoof2019 logical access task, the real and fake speech in the ASVspoof2021 test corpus needs to be transmitted through various telephone systems. During transmission, it may be affected by interference caused by unknown codecs, bandwidth differences, etc., which poses a challenge to the research of highly robust speech spoofing detection methods.

[0037] In this embodiment, the trained detection model was used to evaluate the ASVspoof2019 LA test set and the ASVspoof2021 LA test set. The test results are shown in Table 1. Table 1 Evaluation results of the detection model on the ASVspoof series LA test set As shown in Table 1, in the ASVspoof2019 LA test set, the detection model of this invention achieved an EER of only 0.22% and a min t-DCF as low as 0.0068, indicating that the model has extremely high discrimination between real and fake speech in this scenario, with a near-zero risk of misjudgment. In the ASVspoof2021 LA test set, the test corpus is closer to real and complex scenarios, with more diverse types of fake speech and stronger interference. The detection model achieved an EER of 0.84% ​​and a min t-DCF of 0.2077. Although the performance decreased due to the increased task difficulty, it is still at a relatively high level in the industry, demonstrating the model's good robustness and generalization ability.

[0038] In summary, this invention proposes a method for detecting forged speech based on multi-scale cross-branch feature interaction. Specifically, the front-end uses an XLSR-300M pre-trained model to extract high-dimensional speech features, providing semantically rich basic features for subsequent detection. The core network is a multi-scale cross-branch feature interaction module, which consists of two-branch feature embedding, two-branch encoding, bidirectional cross-attention, and feature integration sub-modules. The two-branch feature embedding sub-module divides feature blocks through convolution at different scales, generating fine-grained and coarse-grained features. The two-branch encoding sub-module deeply encodes the two types of features, outputting encoded features containing global and local information. The bidirectional cross-attention sub-module realizes complementary updates of cross-branch global and local features, mining complementary information between scales. The feature integration sub-module aligns through concatenation and linear interpolation to obtain temporal features and global fusion features of a unified dimension. Based on this, a bidirectional GRU is introduced to capture the long-short-term dependencies of temporal features. Finally, the multi-source features are fused and input into the classification sub-network, combined with a weighted cross-entropy loss optimization model, effectively suppressing redundant information and noise interference, and improving the accuracy and generalization of forged speech detection in complex scenarios.

[0039] Based on a similar inventive concept, embodiments of the present invention also provide a computer storage medium storing a readable program that, when run by a processor, can execute the aforementioned method for detecting forged speech based on multi-scale cross-branch feature interaction.

[0040] Based on a similar inventive concept, this invention provides an electronic device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described forgery speech detection method based on multi-scale cross-branch feature interaction.

[0041] Based on a similar inventive concept, embodiments of the present invention also provide a computer program product, including computer instructions, which instruct a computing device to perform the operations corresponding to the above-described method for detecting forged speech based on multi-scale cross-branch feature interaction.

[0042] Example 2 Based on the forged speech detection method based on multi-scale cross-branch feature interaction proposed in Embodiment 1, this embodiment proposes a forged speech detection device based on multi-scale cross-branch feature interaction, specifically including: Training corpus acquisition module: Acquires training corpus, including real speech and fake speech; Training corpus processing module: preprocesses the acquired training corpus to obtain speech segments of fixed duration; The detection model construction and training module is used to build a detection model. Preprocessed speech segments are input into the detection model for training. Hyperparameters are adjusted to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model. The detection model includes a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module. The front-end feature extraction module uses the pre-trained model XLSR-300M to extract high-dimensional temporal features from speech segments. The multi-scale cross-branch feature interaction module performs multi-scale partitioning and parallel feature extraction on the high-dimensional temporal features, and uses a cross-attention mechanism to achieve feature interaction and information complementarity between different branches. The classification module integrates the interacted multi-scale features and outputs the classification result for speech authenticity. And, the detection module: acquires test data, which includes real speech and fake speech, preprocesses the test data to obtain speech segments of fixed duration, inputs them into the trained detection model, and outputs fake speech detection results.

[0043] The methods of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for performing the methods shown herein.

[0044] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for detecting forged speech based on multi-scale cross-branch feature interaction, characterized in that, include: Acquire training data, including real speech and fake speech; The acquired training data is preprocessed to obtain speech segments of fixed duration; A detection model is constructed by inputting preprocessed speech segments into the model for training. Hyperparameters are adjusted to minimize the target loss function until a set number of iterations are reached, resulting in a trained detection model. The detection model includes a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module. The front-end feature extraction module uses a pre-trained model XLSR-300M to extract high-dimensional temporal features from the speech segments. The multi-scale cross-branch feature interaction module performs multi-scale partitioning and parallel feature extraction on the high-dimensional temporal features, and uses a cross-attention mechanism to achieve feature interaction and information complementarity between different branches. The classification module integrates the interacted multi-scale features and outputs the classification result for speech authenticity. Acquire test data, including real speech and fake speech. Preprocess the test data to obtain speech segments of fixed duration, input them into the trained detection model, and output fake speech detection results.

2. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 1, characterized in that, The preprocessing process for the training and testing corpora is the same: each speech segment in the corpus is trimmed or spliced ​​according to the number of sample points to obtain a speech segment of fixed duration.

3. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 1, characterized in that, In the front-end feature extraction module, the preprocessed speech segment sequentially passes through an XLSR-300M pre-trained model, a two-dimensional convolutional layer, a dimension expansion operation, and a dimension transpose operation.

4. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 3, characterized in that, The pre-trained model XLSR-300M includes: one feature input layer, a 24-layer Transformer encoder, and one feature output layer. The feature input layer converts the input speech temporal signal into a linear feature sequence that the model can process. Each Transformer encoder layer contains one multi-head self-attention sub-layer and one feedforward neural network sub-layer. Both sub-layers use residual connections and layer normalization mechanisms. The multi-head self-attention sub-layer models the association of information at different positions in the speech feature sequence by computing multiple attention heads in parallel. The feedforward neural network sub-layer adopts a two-layer fully connected network structure and enhances the model's feature representation capability through a non-linear activation function.

5. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 1, characterized in that, The multi-scale cross-branch feature interaction module adopts a dual-branch parallel architecture, including: a dual-branch feature embedding submodule, a dual-branch encoding submodule, a bidirectional cross-attention submodule, and a feature integration submodule; The high-dimensional temporal features are first input into the dual-branch feature embedding submodule, which generates branch features that focus on fine-grained artifacts and branch features that focus on global modeling through multi-scale partitioning. Then, the two types of branch features are input into the dual-branch encoding submodule, which completes the deep semantic encoding within the branch through a self-attention layer and an MLP feedforward layer. The encoded dual-branch features enter the bidirectional cross-attention submodule, which realizes the complementary update of cross-branch features through a bidirectional cross-attention mechanism, and integrates local fine-grained information and global context information. Finally, the feature integration submodule aligns and concatenates the updated dual-branch features according to their temporal length, and outputs temporal features of unified dimension and global fused features.

6. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 5, characterized in that, The dual-branch feature embedding submodule includes: two 2D convolutional layers, two learnable position embedding layers, and one Dropout layer; wherein, the high-dimensional temporal features output by the front-end feature extraction module... The two branches of the dual-branch feature embedding submodule are input separately, and feature blocks are partitioned and embedded through two-dimensional convolutional layers of different scales to obtain initial embedded features at two scales. and ; for initial embedded features and By performing flattening and transpose operations respectively, the embedding features of the sequence format are obtained. and Then add learnable position embedding vectors respectively. and This yields position-encoded sequence features. and For sequence features and Dropout random deactivation is performed separately, and finally the intra-branch encoded input features are obtained. and ; The dual-branch encoding submodule includes: one single-layer Transformer block and one dual-layer Transformer block. Each Transformer block consists of one multi-head self-attention layer, two layer normalization layers, one MLP feedforward layer, and two residual connections: encoding the input features within each branch. and The single-layer and double-layer Transformer blocks of the dual-branch Transformer encoding submodule are respectively input for deep encoding. During the encoding process, a multi-head self-attention sublayer models the correlation between different positional information of the sequence features, an MLP feedforward sublayer performs nonlinear transformation on the features, and residual connections and layer normalization layers are used for training and to retain the original feature information, resulting in encoded features that include both global and local features. and ; The bidirectional cross-attention submodule includes: two cross-attention blocks, each consisting of two layer normalization layers, four linear layers, one multi-head cross-attention layer, and one Softmax activation function: from encoded features and Extracting global features , and local features , Cross-branch information complementarity is achieved through a bidirectional cross-attention interaction submodule, in order to For query, Calculate cross-attention updates for keys and values get ,by For query, Calculate cross-attention updates for keys and values get ; The feature integration submodule includes two concatenation operations and one linear interpolation operation: ... and , and The complete interactive features are obtained by splicing them together separately. and ,right and In , Features are aligned using linear interpolation to obtain time-series features of uniform length. and splicing at the same time and Obtain global fusion features .

7. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 6, characterized in that, The classification module includes: one feature concatenation layer, one linear layer, one bidirectional GRU layer, one global average pooling layer, two fully connected layers, two Dropout layers, and one ReLU activation function; the classification process of the classification module includes: right and The features are concatenated to obtain the concatenated temporal features. and will The input linear projection layer is used for dimension mapping to obtain the mapped temporal features. ; Will The input is processed by a bidirectional GRU layer to encode temporal features and then global average pooling is performed to obtain global temporal statistical features. Integrate it with global features The final fused feature is obtained by performing a splicing operation. ; Fusion features The input is processed by the Dropout layer for regularization, then input into the first fully connected layer, which maps the fused feature dimensions from 3072 to 512. After processing by the ReLU activation function, it is input into the second Dropout layer for secondary regularization. Finally, the input is fed into the second fully connected layer, mapped to the predicted log probability feature. , , Indicates batch size, The Middle ( The predicted log probability of ) samples is: in, The predicted log probability corresponding to real speech. The predicted log probability of the corresponding fake speech is directly used to calculate the weighted cross-entropy loss function to complete the model training process.

8. The method for detecting forged speech based on multi-scale cross-branch feature interaction according to claim 1, characterized in that, The objective loss function uses a weighted cross-entropy loss function. : in, Indicates batch size, , Indicates the first The loss weights corresponding to the true class of each sample Indicates the first The true label corresponding to each sample Corresponding to real voice categories, Corresponding to the category of forged voice; Represents the natural logarithm operation. Indicates exponentiation. , The first The predicted log probabilities of each sample for real speech and forged speech, and , , Indicates the first The predicted log probability of the category corresponding to the true label of each sample. If the sample is real speech, then If the sample is fake speech, then .

9. A forged speech detection device based on multi-scale cross-branch feature interaction, comprising the method described in any one of claims 1-8, characterized in that, include: Training corpus acquisition module: Acquires training corpus, including real speech and fake speech; Training corpus processing module: preprocesses the acquired training corpus to obtain speech segments of fixed duration; The detection model construction and training module is used to build a detection model. Preprocessed speech segments are input into the detection model for training. Hyperparameters are adjusted to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model. The detection model includes a front-end feature extraction module, a multi-scale cross-branch feature interaction module, and a classification module. The front-end feature extraction module uses the pre-trained model XLSR-300M to extract high-dimensional temporal features from the speech segments. The multi-scale cross-branch feature interaction module performs multi-scale partitioning and parallel feature extraction on the high-dimensional temporal features, and uses a cross-attention mechanism to achieve feature interaction and information complementarity between different branches. The classification module integrates the interacted multi-scale features and outputs the classification result for speech authenticity. And, the detection module: acquires test data, which includes real speech and fake speech, preprocesses the test data to obtain speech segments of fixed duration, inputs them into the trained detection model, and outputs fake speech detection results.

10. A computer storage medium storing a readable program, characterized in that, When the program runs, it can instruct the computing device to perform the forged speech detection method based on multi-scale cross-branch feature interaction as described in any one of claims 1-8.

Images