Voice spoofing detection method based on double attention and multi-scale feature fusion

Abstract

The invention discloses a voice deception detection method based on double attention and multi-scale feature fusion, and belongs to the technical field of voice deception detection. The method comprises the steps that training corpora are acquired and preprocessed; inputting the preprocessed training corpus into a detection model for model training; adjusting hyper-parameters to minimize the target loss function to obtain a trained detection model; obtaining a test corpus and preprocessing the test corpus; inputting the preprocessed test corpus into the trained detection model, and outputting a voice deception detection result; the detection module comprises a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used for extracting a preliminary feature X in an input voice segment, the feature extraction module further extracts a multi-scale feature Yi from the preliminary feature X, the feature fusion module aligns and splices the multi-scale feature Yi and then outputs a fusion feature Y, and the classification processing module processes the fusion feature to obtain a classification result.

Description

Technical Field

[0001] The present invention belongs to the technical field of voice deception detection, and in particular relates to a voice deception detection method based on dual attention and multi-scale feature fusion. Background Art

[0002] Speech is the primary communication medium for human communication and human-computer interaction. It not only conveys linguistic content but also contains the unique identity of the speaker. With the rapid development of speech synthesis technology, the generation of forged synthesized speech has become increasingly common. The increasing naturalness and anthropomorphism of forged speech makes Automatic Speaker Verification (ASV) systems vulnerable to spoofing attacks. Therefore, research on speech spoofing detection has important theoretical and practical significance.

[0003] Voice spoofing detection works by analyzing the feature differences between real and forged speech, then using a classifier to determine the authenticity of the speech. Early voice spoofing detection models consisted of a front-end feature extractor that extracts handcrafted acoustic features, and a back-end classifier. The feature extractor used handcrafted acoustic features, while the classifiers included Gaussian mixture models, support vector machines, and deep neural network models. Because the detection model's potential relies on the information captured by the initial features, the handcrafted features used in traditional detection models often result in information loss, which in turn affects the detection of unknown attacks. Therefore, a more efficient and universal feature representation is needed to improve the model's robustness.

[0004] The ASVspoof 2019 Logical Access Task promoted the transition of voice spoofing detection models from separate front-end and back-end systems to integrated, end-to-end detection systems. The Residual Network (ResNet) is a widely used model. Considering the relationships and differences between different discriminative features, some researchers introduced the Squeeze and Excitation Network (SENet) to capture the interactions between feature channels, improving the performance of the detection model. However, in real-world applications, deceptive features cannot exist independently. These studies did not consider integrating contextual information, ignoring contextual clues related to deceptive features. The ASVspoof 2021 Logical Access Task focuses on voice spoofing detection in simulated real-world scenarios, presenting a more challenging task. The improved Residual Network (Res2Net) was first introduced to the field of voice spoofing detection and combined with SENet to generate the SE-Res2Net model. This model can focus on key feature information in the channel dimension, helping to capture deceptive clues and improve the generalization ability of the detection system. However, directly superimposing channel features can cause the discriminative features to be contaminated by redundant information. Moreover, the extracted features are single-minded, which easily overlooks key discriminative information. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a voice deception detection method based on dual attention and multi-scale feature fusion, which solves the problems in the existing technology.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The voice deception detection method based on dual attention and multi-scale feature fusion includes the following steps:

[0008] Obtain training corpus and preprocess it to obtain speech segments of fixed length;

[0009] Build a detection model and input the preprocessed training corpus into the detection model for model training; and adjust the hyperparameters to minimize the target loss function until the set number of iterations is reached to obtain a trained detection model;

[0010] Obtain test corpus and preprocess it to obtain speech segments of fixed length;

[0011] Input the preprocessed test corpus into the trained detection model and output the speech deception detection results;

[0012] The detection module includes: a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used to extract the preliminary feature X from the input speech segment, and the feature extraction module further extracts the multi-scale feature Y from the preliminary feature X i , the feature fusion module combines the multi-scale features Y i After alignment and splicing, the fused feature Y is output. The classification processing module processes the fused feature to obtain the classification result.

[0013] Furthermore, the training corpus and the test corpus both include: real speech and fake speech, the fake speech being generated by speech synthesis and speech conversion algorithms;

[0014] The preprocessing process of the training corpus and the test corpus is the same, and each speech is cut or spliced according to the number of sample points to obtain speech segments of fixed length.

[0015] Furthermore, in the Sinc module, the preprocessed speech segment passes through one Sinc convolution layer and one maximum pooling layer to obtain preliminary features X;

[0016] The Sinc convolution layer filters the input speech segment through a bandpass filter to extract features of different frequency bands. The formula of the bandpass filter is:

[0017]

[0018] Where h(n) represents a bandpass filter, n represents the sample point, f1 and f2 represent the low cutoff frequency and the high cutoff frequency, respectively, L represents the filter length, and sinc(·) represents the normalized sinc function.

[0019] And use the Hamming window to window the bandpass filter:

[0020] h w (n)=h(n)·w(n)

[0021]

[0022] Among them, h w (n) represents the windowed bandpass filter, and w(n) represents the Hamming window function.

[0023] Furthermore, the feature extraction module includes: a multi-scale residual network; the multi-scale residual network consists of 2 basic residual blocks, 4 enhanced residual blocks and 6 dual attention blocks;

[0024] The basic residual block and the enhanced residual block have the same structure, consisting of two one-dimensional convolutional layers, two batch normalization layers, two Leaky ReLU activation functions, and one maximum pooling layer. The basic residual block and the enhanced residual block have different channel configurations, and the first enhanced residual block has an additional one-dimensional convolutional layer compared to other residual blocks to adjust the channel dimension of the skip connection.

[0025] The dual attention block consists of 1 channel attention sub-block and 1 spatial attention sub-block; the channel attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 multi-layer perceptron and 1 Sigmoid activation function; the spatial attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 one-dimensional convolution layer and 1 Sigmoid activation function;

[0026] The residual blocks and dual attention blocks of the feature extraction module are placed alternately to form 6 multi-scale residual blocks. The output feature of the i-th multi-scale residual block is B represents the batch size, C i Represents the i-th output feature Y i The number of channels, T i Represents the i-th output feature Y i The length of the time dimension of the feature.

[0027] Furthermore, the feature fusion module includes two operations: interpolation alignment and splicing fusion. The interpolation alignment operation uses the time dimension T6 of the last feature as the alignment target and performs linear interpolation on other features. The splicing operation splices all aligned features along the channel dimension.

[0028] Furthermore, the classification processing module includes: 1 batch normalization layer, 1 SELU activation function, 1 bidirectional GRU layer and 2 fully connected layers;

[0029] The fusion feature Y is input into the batch normalization layer and the SELU activation function for pre-processing to obtain Then the channel dimension of the feature is exchanged with the time dimension to obtain Next, Input the bidirectional GRU layer to model the temporal features. The forward GRU and the backward GRU generate their own hidden state sequences respectively. These two sequences are concatenated in the hidden layer dimension to obtain H all , so that each time step can contain both past and future information; finally, take the hidden state H of the last time step final As the input of the fully connected layer, the second fully connected layer further compresses the output vector h1 of the first fully connected layer into the final classification space to obtain a two-dimensional vector B represents the batch size, and the two elements in h correspond to the original prediction score vectors for the two categories of speech being real and fake, respectively.

[0030] Furthermore, the objective loss function uses the weighted cross entropy loss function L WCE , the formula is as follows:

[0031]

[0032] Where B represents the batch size, represents the weight, y i Indicates the predicted label vector corresponding to the true and false classification, Represents the predicted label y for the i-th sample i The corresponding model output raw score vector, h i,0 represents the original score vector output by the model for the i-th sample on the forged category 0, h i,1 Represents the original score vector output by the model for the i-th sample in the true category 1.

[0033] The speech deception detection system based on dual attention and multi-scale feature fusion includes:

[0034] Training corpus processing module: obtains training corpus and preprocesses it to obtain speech segments of fixed length;

[0035] Model training module: Builds a detection model and inputs the preprocessed training corpus into the detection model for model training. It also adjusts hyperparameters to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model.

[0036] Test corpus processing module: obtains test corpus and preprocesses it to obtain speech segments of fixed length;

[0037] And, the detection module: inputs the preprocessed test corpus into the trained detection model and outputs the voice deception detection results;

[0038] The detection module includes: a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used to extract the preliminary feature X from the input speech segment, and the feature extraction module further extracts the multi-scale feature Y from the preliminary feature X i , the feature fusion module combines the multi-scale features Y i After alignment and splicing, the fused feature Y is output. The classification processing module processes the fused feature to obtain the classification result.

[0039] A computer storage medium stores a readable program, which, when executed by a processor, can execute the above-mentioned voice deception detection method based on dual attention and multi-scale feature fusion.

[0040] An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0041] The memory is used to store at least one executable instruction, which enables the processor to perform operations corresponding to the above-mentioned voice deception detection method based on dual attention and multi-scale feature fusion.

[0042] Beneficial effects of the present invention:

[0043] 1. The detection model of the present invention proposes a multi-scale residual network combined with a dual attention mechanism in the feature extraction module. The network consists of a basic residual block, an enhanced residual block and a dual attention block. The multi-scale features of each residual block and the dual attention block are retained. Specifically, the first two basic residual blocks are used to extract shallow detail features, and the last four enhanced residual blocks are used to extract deep semantic features. Combining shallow features with deep features can capture important information of speech signals at different levels. The residual block helps to build a deep network, so that the model avoids overfitting during the learning process, and at the same time enhances the ability to learn complex feature relationships; a dual attention block combining a channel and spatial attention mechanism is added after each residual block. This module can effectively integrate multi-scale features, ensure effective information focusing on multi-scale features, suppress unimportant features and background noise, and improve the overall detection performance of the detection model.

[0044] 2. The voice spoofing detection method of the present invention aligns and concatenates multi-scale features processed by residual blocks and dual attention blocks to obtain an enhanced multi-scale feature vector. This method not only effectively extracts local information from the speech signal but also captures global context, forming a more expressive feature representation. This enables the classifier to better parse authentic and forged speech features from the multi-scale features, thereby improving the accuracy of voice spoofing detection and ensuring that the system maintains efficient recognition capabilities in various complex spoofing attack scenarios.

[0045] 3. The detection model of the present invention uses a bidirectional GRU network to extract time series features in the classification processing module. The bidirectional GRU network can simultaneously process the forward and reverse information of the sequence, making the learned feature representation richer and more effective, providing important support for subsequent classification processing. In the face of short-term acoustic feature noise and uncertainty, it can improve the model's robustness to these uncertainties by balancing the forward and reverse feature information, reducing the risk of misjudgment and improving detection accuracy.

[0046] 4. The detection model of this invention fully utilizes a large number of real speech and forged speech generated by various algorithms during the training phase, enabling it to effectively detect forged speech using unknown algorithms during the testing phase. This broadens the model's applicability and improves the performance of speech spoofing detection in open-set scenarios. Furthermore, the number of multi-scale residual blocks in the feature extraction phase can be flexibly adjusted, allowing for the optimal configuration of residual blocks and dual-attention blocks at different levels, further enhancing the model's flexibility and generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 is a flow chart of the voice deception detection method of the present invention;

[0049] Figure 2 : is a multi-scale residual network structure diagram of the present invention;

[0050] Figure 3 : is a network structure diagram of the basic residual block and enhanced residual block of the present invention;

[0051] Figure 4 This is the dual attention block network structure diagram of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] Example 1

[0054] like Figure 1 As shown in FIG, a voice deception detection method based on dual attention and multi-scale feature fusion includes a training phase (S1-S4) and a testing phase (S5-S6). The detection network and parameters required for voice deception detection are obtained in the training phase, and the detection of real and forged voices is achieved in the testing phase. The voice deception detection method specifically includes the following steps:

[0055] S1, obtain training corpus;

[0056] The training corpus includes real speech and fake speech, and the fake speech is generated by speech synthesis and speech conversion algorithms;

[0057] In this embodiment, the training corpus comes from the logical access task of the ASVspoof 2019 competition, including real speech and 6 types of fake speech generated by speech synthesis and speech conversion algorithms; all the training and verification corpora in the acquisition corpus are extracted, with a total of 25,380 training corpora and 24,986 verification corpora, most of which are 2-8 seconds in length.

[0058] The selected training corpus is used as the input for model training. After each iteration, the verification corpus is input into the model for verification, so that the training results of each round of the model can be presented intuitively, which facilitates the adjustment of the model training parameters.

[0059] S2, preprocessing the acquired training corpus to obtain speech segments of fixed length;

[0060] In this embodiment, during the preprocessing process, the sampling rate of the training corpus is set to 16 kHz and the number of sample points is set to 64,000. Each speech in the training corpus is cropped or spliced according to the number of sample points to obtain speech segments with a fixed length of 4 seconds.

[0061] S3, build a detection model and input the preprocessed training corpus into the detection model for model training;

[0062] The detection model includes: a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used to extract the preliminary feature X from the input speech segment, and the feature extraction module further extracts the multi-scale feature Y from the preliminary feature Xi , where i = 1, 2, ···, 6. The feature fusion module aligns and concatenates the input multi-scale features and outputs the fused features Y. The classification processing module processes the fused features to obtain the final classification results;

[0063] (1)Sinc module

[0064] In the Sinc module, the preprocessed speech segment passes through one Sinc convolution layer and one maximum pooling layer to obtain the preliminary feature X;

[0065] The Sinc convolution layer filters the input speech segments by designing a set of ideal bandpass filters based on the Mel scale, extracting features of different frequency bands, thereby replacing the traditional convolution layer and better capturing the key information in the speech signal. The designed filter formula is as follows:

[0066]

[0067] Where h(n) represents a bandpass filter, n represents the sample point, f1 and f2 represent the low cutoff frequency and the high cutoff frequency, respectively, L represents the filter length, and sinc(·) represents the normalized sinc function.

[0068] To suppress spectrum leakage, a Hamming window is used for windowing. The formula is as follows:

[0069] h w (n)=h(n)·w(n)

[0070]

[0071] Among them, h w (n) represents the windowed bandpass filter, and w(n) represents the Hamming window function.

[0072] (2) Feature extraction module

[0073] like Figure 2 As shown, the feature extraction module includes: a multi-scale residual network; the multi-scale residual network consists of 2 basic residual blocks, 4 enhanced residual blocks and 6 dual attention blocks;

[0074] The network structure of the basic residual block and the enhanced residual block is as follows Figure 3 As shown in the figure, the two have the same structure, both consisting of 2 one-dimensional convolutional layers, 2 batch normalization layers, 2 Leaky ReLU activation functions and 1 maximum pooling layer; the difference between the two lies in the number of channels. In addition, the first enhanced residual block has an additional one-dimensional convolutional layer compared to other residual blocks due to the expansion of the number of channels, which is used to adjust the channel dimension of the jump connection.

[0075] In the basic residual block of the feature extraction module, the preliminary feature X is first input, and after the batch normalization layer and the Leaky ReLU activation function, the feature X′ is obtained. The formula is as follows:

[0076] X′=f LReLU (BN(X))

[0077] f LReLU (x)=max(x,αx)

[0078] Among them, BN(·) represents batch normalization, f LReLU (·) represents the Leaky ReLU activation function, max(·) means taking the maximum element, and α is usually a small positive number;

[0079] Then, the feature X′ is input into the one-dimensional convolution layer for a convolution operation. The formula is as follows:

[0080] X″=K1*X′+b1

[0081] Where X″ represents the one-dimensional convolution output feature, * represents the convolution operation, K1 and b1 represent the convolution kernel and bias of the first convolution layer respectively;

[0082] Next, the feature X″ will undergo batch normalization, activation, and one-dimensional convolution for residual connection; finally, the output feature X1 of the first residual block is obtained through the maximum pooling layer, as shown in the following formula:

[0083] X1=MaxPool(f LReLU (BN(K2*X″+b2))+X)

[0084] Among them, MaxPool(·) represents maximum pooling, * represents convolution operation, K2 and b2 represent the convolution kernel and bias of the second convolution layer, respectively.

[0085] The network structure of the dual attention block is as follows Figure 4 As shown in the figure, the dual attention block consists of 1 channel attention sub-block and 1 spatial attention sub-block. The channel attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 multi-layer perceptron and 1 Sigmoid activation function. The spatial attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 one-dimensional convolution layer and 1 Sigmoid activation function.

[0086] In the channel attention sub-block, the output feature X1 of the first basic residual block is taken as input. First, X1 is subjected to global average pooling and global maximum pooling respectively, and then input into a shared multi-layer perceptron. The two outputs of the multi-layer perceptron are added and passed through a Sigmoid activation function to obtain the channel attention weight. The formula is as follows:

[0087]

[0088] Where σ(·) represents the Sigmoid activation function, MLP(·) represents the multi-layer perceptron, AvgPool(·) and MaxPool(·) represent average pooling and maximum pooling, respectively.

[0089] MLP(x)=W2f ReLU (W1x+β1)+β2

[0090] f ReLU (x)=max(0,x)

[0091] Among them, f ReLU (·) represents the ReLU activation function, W1 and β1 represent the weight and bias of the first fully connected layer in the multilayer perceptron, W2 and β2 represent the weight and bias of the second fully connected layer, and max(·) represents the maximum element;

[0092] Finally, the channel attention weight Multiply the input feature X1 by channel, the formula is as follows:

[0093]

[0094] in, represents the output features of the channel attention sub-block, Represents element-wise multiplication;

[0095] In the spatial attention sub-block, the output features of the channel attention sub-block are As input, first Perform average pooling and maximum pooling along the channel dimension respectively, and then concatenate the two pooling outputs to obtain the concatenated features Then the spatial attention weight is obtained through a one-dimensional convolution layer and an activation function Finally, the spatial attention weight With input features The output feature Y1 of the first dual attention block is obtained by element-by-element multiplication. The formula is as follows:

[0096]

[0097] Among them, Concat(·) represents the feature concatenation operation, AvgPool(·) and MaxPool(·) represent average pooling and maximum pooling respectively, σ(·) represents the Sigmoid activation function, W and b represent the convolution kernel and bias of the convolution layer respectively, * represents the convolution operation, Represents element-wise multiplication.

[0098] In addition, in this embodiment, the residual blocks and dual attention blocks of the feature extraction module are placed alternately to form 6 multi-scale residual blocks, and the output feature of the i-th multi-scale residual block is B represents the batch size, C i Represents the i-th output feature Y i The number of channels, T i Represents the i-th output feature Y i The length of the time dimension of the feature.

[0099] (3) Feature fusion module

[0100] The feature fusion module includes two operations: interpolation alignment and splicing fusion;

[0101] The interpolation alignment operation of the feature fusion module takes the time dimension T6 of the last feature as the alignment target and performs linear interpolation on other features. The formula is as follows:

[0102] Y′ i =Interpolate(Y i ,T6), i=1,2,···,5

[0103] Among them, Y′ i represents the feature after the i-th interpolation alignment, Interpolate(·) represents the linear interpolation operation;

[0104] The splicing operation of the feature fusion module splices all aligned features along the channel dimension. The formula is as follows:

[0105] Y=Concat(Y1′,Y2′,Y3′,Y4′,Y5′,Y6)

[0106] in, represents the fused features after splicing, and Concat(·) represents the feature splicing operation.

[0107] (4) Classification processing module

[0108] The classification processing module includes: 1 batch normalization layer, 1 SELU activation function, 1 bidirectional GRU layer and 2 fully connected layers.

[0109] In the classification processing module, first, the fusion feature Y is input into the batch normalization layer and the SELU activation function for pre-processing to obtain Secondly, the channel dimension of the feature is exchanged with the time dimension to obtain To adapt the input of the bidirectional GRU; next, Input the bidirectional GRU layer to model the temporal features. The forward GRU and the backward GRU generate their own hidden state sequences respectively. These two sequences are concatenated in the hidden layer dimension to obtain H all , so that each time step can contain both past and future information; finally, take the hidden state H of the last time step final As the input of the fully connected layer, the second fully connected layer further compresses the output vector h1 of the first fully connected layer into the final classification space to obtain a two-dimensional vector B represents the batch size, and the two elements in h correspond to the original prediction score vectors for the two categories of speech being real and fake, respectively.

[0110] S4, in the process of training the detection model, adjust the hyperparameters to minimize the target loss function until the set number of iterations is reached and a trained detection model is obtained;

[0111] The target loss function uses the weighted cross entropy loss function L WCE , the formula is as follows:

[0112]

[0113] Where B represents the batch size, represents the weight, y i Indicates the predicted label vector corresponding to the true and false classification, Represents the predicted label y for the i-th sample i The corresponding model output raw score vector, h i,0 represents the original score vector output by the model for the i-th sample on the forged category 0, h i,1 Represents the original score vector output by the model for the i-th sample in the true category 1. The smaller the weighted cross entropy loss function of the detection model, the closer the predicted classification label is to the true classification label.

[0114] In this embodiment, the number of iteration rounds is set to 100 rounds.

[0115] S5, obtaining the test corpus and preprocessing it to obtain speech segments of fixed length;

[0116] The test corpus includes real speech and fake speech, and the fake speech is generated by speech synthesis, speech conversion and a hybrid algorithm of the two. S5 uses the same method as S2 to preprocess the obtained test corpus.

[0117] In this embodiment, the test corpus preprocessing process is consistent with the training corpus preprocessing process of S2. The test corpus comes from the logical access task of the ASVspoof 2021 competition. All the test corpora are retrieved, with a total of 181,566 items. Most of the corpora are 2-8 seconds long. Unlike the ASVspoof 2019 logical access task test corpus, the real voice and forged voice of the test corpus need to be transmitted through multiple telephone systems. During the transmission process, they may be affected by interfering changes caused by some unknown codecs, bandwidth differences, etc. This poses a challenge to the research of highly robust voice deception detection methods.

[0118] S6: Input the preprocessed test corpus into the trained detection model for testing to obtain the voice deception detection result.

[0119] In summary, this paper proposes a voice spoofing detection method based on dual attention and multi-scale feature fusion. It proposes a multi-scale residual network that introduces a dual attention mechanism in feature extraction. The network consists of a base residual block, an enhanced residual block, and a dual attention block. The base residual block extracts shallow detail information, while the enhanced residual block extracts deep semantic information. A dual attention block combining channel and space is introduced after each residual block to ensure effective information focusing on multi-scale features. The multi-scale features retained in the multi-scale residual network are aligned and spliced to obtain multi-scale fused features, thereby achieving high-quality voice spoofing detection.

[0120] Based on a similar inventive concept, an embodiment of the present invention also provides a computer storage medium storing a readable program. When the program is run by a processor, it can execute the above-mentioned voice deception detection method based on dual attention and multi-scale feature fusion.

[0121] Based on similar inventive concepts, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0122] The memory is used to store at least one executable instruction, which enables the processor to perform operations corresponding to the above-mentioned voice deception detection method based on dual attention and multi-scale feature fusion.

[0123] Based on similar inventive concepts, an embodiment of the present invention also provides a computer program product, including computer instructions, which instruct a computing device to perform operations corresponding to the above-mentioned voice deception detection method based on dual attention and multi-scale feature fusion.

[0124] Example 2

[0125] Based on the voice deception detection method based on dual attention and multi-scale feature fusion proposed in Example 1, this embodiment proposes a voice deception detection system based on dual attention and multi-scale feature fusion, which specifically includes:

[0126] Training corpus processing module: obtains training corpus and preprocesses it to obtain speech segments of fixed length;

[0127] Model training module: Builds a detection model and inputs the preprocessed training corpus into the detection model for model training. It also adjusts hyperparameters to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model.

[0128] Test corpus processing module: obtains test corpus and preprocesses it to obtain speech segments of fixed length;

[0129] And, the detection module: inputs the preprocessed test corpus into the trained detection model and outputs the voice deception detection results;

[0130] Example 3

[0131] In this embodiment, the detection model of the present invention (baseline + dual attention + multi-scale feature fusion) is compared with other models in the prior art. The experimental process is as follows:

[0132] Dataset: The ASVspoof 2019 logical access task dataset was used as the training and validation sets, and the ASVspoof 2021 logical access task dataset was used as the test set. To ensure experimental fairness, all models used RawBoost data augmentation technology to preprocess the training and validation sets.

[0133] Model Construction: The baseline model uses the existing RawNet2 model. The baseline + dual-attention model builds on the baseline model by adding channel- and spatial-level dual-attention mechanisms. The baseline + multi-scale feature fusion model builds on the baseline model by fusing features of different scales using a multi-scale feature fusion method. The baseline + dual-attention + multi-scale feature fusion model combines these two improvements. All models are implemented using the PyTorch deep learning framework.

[0134] Experimental parameters: The training parameters of all models were kept consistent to ensure the reliability and comparability of the experimental results. The learning rate was set to 0.0001, the batch size was set to 128, the number of iterations was set to 100 epochs, and the Adam optimizer was used.

[0135] Model training: All models are trained using the preprocessed dataset, and the loss function value and accuracy index are recorded during the training process.

[0136] Model evaluation: Use the test set to evaluate the trained model and calculate the equal error rate (EER) and minimum tandem detection cost function (min t-DCF).

[0137] The experimental results are shown in Table 1 below:

[0138] Table 1 Model comparison experimental results

[0139]

[0140] As can be seen from Table 1 above, compared with the baseline model, the EER of the baseline + dual-attention model is reduced by 6.82%, and the min t-DCF is reduced by 4.49%. This shows that the dual-attention mechanism improves the model's feature extraction ability by enhancing the model's attention to key features. The baseline + multi-scale feature fusion model reduces the EER by 9.32% and the mint-DCF by 5.39% compared with the baseline model. This shows that the multi-scale feature fusion module can fuse feature information at different levels and enrich the model's feature expression. Compared with the baseline model, the EER of the baseline + dual-attention + multi-scale feature fusion model is reduced by 19.13%, and the min t-DCF is reduced by 11.68%. This shows that when the dual-attention mechanism and the multi-scale feature fusion module are combined, the two produce a synergistic effect, which further improves the model performance, fully demonstrating the effectiveness and innovation of the present invention in applying dual-attention and multi-scale feature fusion to the voice deception detection model.

[0141] The method of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It will be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0142] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention as claimed.

Claims

1. A voice deception detection method based on dual attention and multi-scale feature fusion, characterized by: The following steps are involved: Obtain training corpus and preprocess it to obtain speech segments of fixed length; Build a detection model and input the preprocessed training corpus into the detection model for model training; and adjust the hyperparameters to minimize the target loss function until the set number of iterations is reached to obtain a trained detection model; Obtain test corpus and preprocess it to obtain speech segments of fixed length; Input the preprocessed test corpus into the trained detection model and output the speech deception detection results; The detection module includes: a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used to extract the preliminary feature X from the input speech segment, and the feature extraction module further extracts the multi-scale feature Y from the preliminary feature X i , the feature fusion module combines the multi-scale features Y i After alignment and splicing, the fused feature Y is output. The classification processing module processes the fused feature to obtain the classification result.

2. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: The training corpus and the test corpus both include: real speech and fake speech, wherein the fake speech is generated by speech synthesis and speech conversion algorithms; The preprocessing process of the training corpus and the test corpus is the same, and each speech is cut or spliced according to the number of sample points to obtain speech segments of fixed length.

3. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: In the Sinc module, the preprocessed speech segment passes through a Sinc convolution layer and a maximum pooling layer to obtain preliminary features X; The Sinc convolution layer filters the input speech segment through a bandpass filter to extract features of different frequency bands. The formula of the bandpass filter is: Where h(n) represents a bandpass filter, n represents the sample point, f1 and f2 represent the low cutoff frequency and the high cutoff frequency, respectively, L represents the filter length, and sinc(·) represents the normalized sinc function. And use the Hamming window to window the bandpass filter: h w (n)=h(n)·w(n) Among them, h w (n) represents the windowed bandpass filter, and w(n) represents the Hamming window function.

4. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: The feature extraction module includes: a multi-scale residual network; the multi-scale residual network consists of 2 basic residual blocks, 4 enhanced residual blocks and 6 dual attention blocks; The basic residual block and the enhanced residual block have the same structure, consisting of two one-dimensional convolutional layers, two batch normalization layers, two LeakyReLU activation functions, and one maximum pooling layer. The basic residual block and the enhanced residual block have different channel configurations, and the first enhanced residual block has an additional one-dimensional convolutional layer compared to other residual blocks to adjust the channel dimension of the skip connection. The dual attention block consists of 1 channel attention sub-block and 1 spatial attention sub-block; the channel attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 multi-layer perceptron and 1 Sigmoid activation function; the spatial attention sub-block consists of 1 average pooling layer, 1 maximum pooling layer, 1 one-dimensional convolution layer and 1 Sigmoid activation function; The residual blocks and dual attention blocks of the feature extraction module are placed alternately to form 6 multi-scale residual blocks. The output feature of the i-th multi-scale residual block is i=1,2,···,6, B represents the batch size, C i Represents the i-th output feature Y i The number of channels, T i Represents the i-th output feature Y i The length of the time dimension of the feature.

5. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: The feature fusion module includes two operations: interpolation alignment and splicing fusion. The interpolation alignment operation uses the time dimension T6 of the last feature as the alignment target and performs linear interpolation on other features. The splicing operation splices all aligned features along the channel dimension.

6. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: The classification processing module includes: 1 batch normalization layer, 1 SELU activation function, 1 bidirectional GRU layer and 2 fully connected layers; The fusion feature Y is input into the batch normalization layer and the SELU activation function for pre-processing to obtain Then the channel dimension of the feature is exchanged with the time dimension to obtain Next, Input the bidirectional GRU layer to model the temporal features. The forward GRU and the backward GRU generate their own hidden state sequences respectively. These two sequences are concatenated in the hidden layer dimension to obtain H all , so that each time step can contain both past and future information; finally, take the hidden state H of the last time step final As the input of the fully connected layer, the second fully connected layer further compresses the output vector h1 of the first fully connected layer into the final classification space to obtain a two-dimensional vector B represents the batch size, and the two elements in h correspond to the original prediction score vectors for the two categories of speech being real and fake, respectively.

7. The voice deception detection method based on dual attention and multi-scale feature fusion according to claim 1 is characterized in that: The target loss function uses the weighted cross entropy loss function L WCE , the formula is as follows: Where B represents the batch size, represents the weight, y i Indicates the predicted label vector corresponding to the true and false classification, Represents the predicted label y for the i-th sample i The corresponding model output raw score vector, h i,0 represents the original score vector output by the model for the i-th sample on the forged category 0, h i,1 Represents the original score vector output by the model for the i-th sample in the true category 1.

8. Voice deception detection system based on dual attention and multi-scale feature fusion, characterized by: include: Training corpus processing module: obtains training corpus and preprocesses it to obtain speech segments of fixed length; Model training module: Builds a detection model and inputs the preprocessed training corpus into the detection model for model training. It also adjusts hyperparameters to minimize the target loss function until the set number of iterations is reached, resulting in a trained detection model. Test corpus processing module: obtains test corpus and preprocesses it to obtain speech segments of fixed length; And, the detection module: inputs the preprocessed test corpus into the trained detection model and outputs the voice deception detection results; The detection module includes: a Sinc module, a feature extraction module, a feature fusion module and a classification processing module; the Sinc module is used to extract the preliminary feature X from the input speech segment, and the feature extraction module further extracts the multi-scale feature Y from the preliminary feature X i , the feature fusion module combines the multi-scale features Y i After alignment and splicing, the fused feature Y is output. The classification processing module processes the fused feature to obtain the classification result.

9. A computer storage medium storing a readable program, characterized in that: When the program is executed by the processor, the voice deception detection method based on dual attention and multi-scale feature fusion as described in any one of claims 1 to 7 can be executed.

10. An electronic device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the voice deception detection method based on dual attention and multi-scale feature fusion as described in any one of claims 1-7.

Images