Intelligent voice spoofing attack detection method based on attention mechanism

Abstract

The application discloses an intelligent speech forgery attack detection method based on an attention mechanism, which comprises the following steps: converting a speech sample from a time domain to a frequency domain for analysis, considering the influence of different filters on feature expression, filtering the frequency spectrum by using multiple filters, and extracting three different voiceprint features of the speech sample, namely, a logarithmic power spectrum, a mel frequency cepstrum coefficient, and a linear frequency cepstrum coefficient; training a forged speech attack detection model based on an attention mechanism and a residual network, using the attention mechanism to perform adaptive feature selection, enhancing effective features with discriminative characteristics, suppressing noise and redundant features, and then performing advanced feature extraction and learning through the residual network; and using the trained forged speech attack detection model to detect the legality of received speech samples, scoring the speech samples, and determining that the samples are real human voices if the scores exceed a threshold value, otherwise, determining that the samples are forged speech. The method has high accuracy, high efficiency and strong generalization ability.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of Internet of Things security, and more particularly relates to a smart voice forgery attack detection method based on an attention mechanism. BACKGROUND

[0002] The task of forged voice detection is to identify whether a given voice sample is a sound made by a real human or a fake voice synthesized by an electronic device. In recent years, with the continuous development of smart voice technology, smart voice assistants have been widely integrated into various mobile devices and Internet of Things devices. However, while smart voice assistants bring great convenience to people, they also face great challenges. Attackers can bypass the identity verification of smart voice systems by maliciously recording and replaying or synthesizing voices, which poses a great threat to the privacy and property safety of users. In response to this, many researchers have conducted research on spoofed voice detection.

[0003] Due to the influence of factors such as complex and diverse actual scenarios, large environmental noise, many and constantly changing attack techniques such as voice synthesis, and the like, the forged voice detection task presents great difficulties and challenges. In order to enhance the robustness and generalization ability of the model and better cope with unknown attacks, some existing methods use physical quantities such as wearable device sensor data, ultrasonic waves, millimeter waves, or wireless signals in the environment for dual identity verification, but the above methods require additional equipment or have strong location restrictions, affecting their usability. In addition, based on the characteristic difference in voiceprint features between human voices and sounds emitted by electronic loudspeakers, many studies use voiceprint features as inputs and use Gaussian Mixture Model (GMM), x-vector, and other machine learning algorithms to classify voice samples, but the detection accuracy of the above machine learning methods is low. In order to better learn feature information, deep learning-based methods have been widely applied, but due to the high-dimensional nature of voice features, there is a lot of redundant information, which can easily cause model overfitting and insufficient ability to cope with unknown attacks. Moreover, methods based on sequence models such as Recurrent Neural Network (RNN) and Long Short-Term Memory Neural Network (LSTM) have the problems of large computational load, long time consumption, inability to parallel computing, and low model efficiency. SUMMARY

[0004] In view of the defects or improvement needs of the prior art, the present application provides a smart voice forgery attack detection method based on an attention mechanism, which aims to enhance the expression of difference features using an attention mechanism in the process of identifying the legality of voice, improve the distinguishability of human voice and machine sound feature matrices, and solve the problems of poor generalization ability and insufficient ability to cope with unknown attacks of existing methods. Through a classifier model based on a residual network, the forged voice is identified, and local feature correlations are extracted using convolution kernels, aiming to improve the detection accuracy while taking into account the model efficiency.

[0005] To achieve the above objectives, this invention provides an intelligent speech forgery attack detection method based on an attention mechanism, comprising the following steps:

[0006] (1) Transform the speech samples from the time domain to the frequency domain for analysis, consider the influence of different filters on the feature performance, use multiple filters to filter the spectrum, and extract three different speech sample voiceprint features: log power spectrum, Mel frequency cepstral coefficient, and linear frequency cepstral coefficient.

[0007] (2) Train a fake speech attack detection model based on attention mechanism and residual network, use attention mechanism for adaptive feature selection, enhance effective features with discriminative power, suppress noise and redundant features, and then use residual network for advanced feature extraction and learning.

[0008] (3) Use the trained fake speech attack detection model to detect the legality of the received speech samples and score the speech samples. If the score exceeds the threshold, it is a real human voice; otherwise, the sample is judged to be fake speech.

[0009] In one embodiment of the present invention, step (1) specifically includes:

[0010] (1.1) The length of all speech samples in the original dataset is unified by padding and truncation to adjust the original speech samples to the same length;

[0011] (1.2) Divide the speech samples into multiple speech frames according to the number of sampling points, and extract the voiceprint features for each individual speech frame;

[0012] (1.3) Let N be the number of sampling points and Y(n) be the speech frame signal. Multiply each speech frame by the Hamming window function W(n) to obtain the windowed signal X. w ():

[0013]

[0014] X w (n) = Y(n) × W(n)

[0015] (1.4) Perform a Fast Fourier Transform on the windowed signal and calculate its spectrum:

[0016]

[0017] (1.5) Considering the impact of different filters on feature performance, speech sample voiceprint features are extracted based on three different filters, including: calculating power spectrum features based on speech spectrum and eliminating convolution noise through logarithmic transformation; calculating Mel-frequency cepstral coefficients; and calculating linear frequency cepstral coefficients.

[0018] In one embodiment of the present invention, step (1.5) specifically includes the following sub-steps:

[0019] (1.5.1) Based on the speech spectrum, calculate the power spectrum features and eliminate convolution noise through logarithmic transformation:

[0020] SPEC = log(X[k]) 2 )

[0021] (1.5.2) Calculate the Mel-spectral coefficients based on the speech spectrum;

[0022] (1.5.2.1) The spectrum is filtered using a Mel filter:

[0023] S mel (i)=ln(Mel(X[k] 2 ),(0≤i≤M)

[0024] The Mel frequency conversion formula is:

[0025]

[0026] Where M is the number of Mel filters;

[0027] (1.5.2.2) Perform discrete cosine transform on the Mel spectrum;

[0028]

[0029] Where L is the MFCC feature order, and in this step, the first 30 coefficients are selected as the MFCC static features;

[0030] (1.5.2.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic features of the speech;

[0031] (1.5.3) Calculate the linear frequency cepstral coefficients based on the speech spectrum;

[0032] (1.5.3.1) A linear triangular filter is used to filter the spectrum:

[0033] S line (i)=ln(LF(X[k)) 2 ),(0≤i≤M)

[0034] The frequency response of a triangular filter is defined as follows:

[0035]

[0036] Where M is the number of triangular filters, and this step uses 60 linear triangular filters to filter the spectrum;

[0037] (1.5.3.2) Perform discrete cosine transform on the linear spectrum to eliminate the correlation between signal values ​​of different orders:

[0038]

[0039] Where L is the characteristic order of LFCC;

[0040] (1.5.3.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic feature representation of speech.

[0041] In one embodiment of the present invention, step (2) specifically includes:

[0042] (2.1) Define a fake speech attack detection model based on attention mechanism and residual network. The fake speech attack detection model consists of attention feature enhancement module, deep feature extraction module and fully connected classification module. Using the voiceprint features extracted in step (1), design loss function, select parameter optimizer, and train the network using gradient descent and backpropagation algorithm to obtain fake speech attack detection model.

[0043] (2.2) The voiceprint features F∈R extracted in (1) H×W As input to the fake speech detection model, the attention feature enhancement module is used to dynamically select and process important information in the high-dimensional features;

[0044] (2.3) Enhance the feature map F A ∈R C×H×W Input the advanced feature extraction module, where C represents feature F. A The number of channels, H and W are the height and width of the high-dimensional feature map, respectively; the receptive field of the model is expanded by using 6 stacked residual blocks to capture the correlation between speech frames and frequency components, and to extract deeper features:

[0045] (2.4) The features obtained by the advanced feature extraction module Logical reasoning is performed using a linear classifier consisting of two fully connected layers, fc1 and fc2; a non-linear transformation is applied between the two fully connected layers using the LeakyReLU activation function; finally, a 2D vector V∈R is output through the Softmax activation function. 2 The probability distribution represented;

[0046] (2.5) Calculate the loss function based on the prediction result vector V, calculate the gradient through backpropagation, and update the parameter model using the Adam optimizer.

[0047] In one embodiment of the present invention, in step (2.1): the attention feature enhancement module consists of two parts: channel attention and spatial attention, used to perform adaptive feature selection from high-dimensional and multi-channel input, enhance important features, and reduce redundant features; the enhanced feature map is input into a deep feature extraction network, which consists of 6 sets of residual blocks, captures the local correlation between speech features through convolution operations, expands the receptive field by using multi-layer convolution, and obtains a comprehensive feature descriptor for the speech sample based on higher-dimensional features; finally, the high-level features are input into a fully connected classification module consisting of two fully connected layers, fc1 and fc2, to perform probability prediction and calculate the score of the speech sample; if the score exceeds the threshold, the sample is judged to be real human voice; otherwise, the sample is judged to be fake speech.

[0048] In one embodiment of the present invention, in step (2.2): the weights and biases of the fake speech detection model are initialized, and the voiceprint features extracted in (1) are used as the input of the attention feature enhancement module in step (2.1). For an input feature map with C input channels and H×W size, the attention feature enhancement module generates an attention map by performing global max pooling and global average pooling operations along both the channel dimension and the spatial dimension, and calculates a weight matrix for the input feature map. The attention feature enhancement module does not change the size of the input feature map, but only adaptively assigns weights to the original features, providing enhanced features for the subsequent advanced feature extraction module.

[0049] In one embodiment of the present invention, step (2.2) specifically includes:

[0050] (2.2.1) For the input feature map, each channel of the feature map is treated as a feature detector. The attention feature enhancement module assigns weights to different channel features through channel attention: for high-dimensional features with respect to the speech map F∈R C×H×W Global max pooling and global average pooling are used to obtain two feature descriptors for speech, Vec. CA_max ∈R C and Vec CA_avg ∈R C Vec CA_max ∈R C and Vec CA_avg ∈R C The features of each channel are globally synthesized to reflect the overall situation of each channel's features; the above two feature descriptors are fed into a shared multilayer perceptron network composed of two 1×1 Convs to obtain the corresponding attention weights. CA_max ∈R C and Weight CA_avg ∈R C ; Calculate Weight CA_max With WeightCA_avg The summation is then used to unify the weight values ​​to the [0,1] range using the Sigmoid function, resulting in the final channel attention weight. CA ∈R C The acquired channel attention maps are used to assign weights to different channel features for the first round of feature enhancement: F CA =F + F × Weight CA ;

[0051] (2.2.2) Calculate the spatial attention map and find effective information in the high-dimensional feature space: The feature map F after channel enhancement... CA ∈R C×H×W Cut along the channel direction into H×W one-dimensional vectors Vec SA ∈R C Calculate Vec respectively SA The maximum and average values ​​are used to obtain the spatial attention map Weight. SA_max ∈H×W and Weight SA_avg ∈H×W; The spatial attention graph Weight SA_max and Weight SA_max The data is spliced ​​together and then fused using Conv7×7 to obtain the final spatial attention weight. SA ∈R H×W The acquired spatial attention map is used to assign different weights to different locations in the high-dimensional features for a second round of feature enhancement: F Attention =F CA +F CA ×Weight SA .

[0052] In one embodiment of the present invention, step (2.3) specifically includes:

[0053] Each residual block consists of a convolutional branch and a residual F. A ∈R C×H×W The branch consists of two parts, feature map F A ∈R C×H×W The inputs are used as inputs to the convolutional and residual branches. The convolutional branch consists of two 3×3 convolutions, the first with a stride of 1, followed by batch normalization and a Dropout operation with a deactivation probability of 0.5, and then a 3×3 Conv with a stride of 3. The residual branch consists of a 3×3 convolution to adapt to changes in feature map size during the convolutional operation. The outputs of the convolutional and residual branches are summed and batch normalized again, with the LeakyReLU activation function used to prevent gradient vanishing. The final high-level representation is obtained by stacking six residual blocks.

[0054] In one embodiment of the present invention, step (2.5) specifically includes:

[0055] Using the labels in the original dataset as the network's expected output and the probabilities predicted in step (2.4) as the network's actual prediction results, a target loss function is designed between the expected output and the actual predicted output for the network model constructed in step (2.1), expressed as follows:

[0056] Lost=-∑y i *log(p i )

[0057] Where, p i y represents the probability value predicted by the model. i The model is identified by its true labels. Based on the designed loss function, the model is iteratively trained using the backpropagation algorithm to minimize the classification loss function and achieve the optimal network model.

[0058] In one embodiment of the present invention, step (3) specifically includes:

[0059] (3.1) Extract the speaker characteristics of the audio to be detected, input them into the attention feature enhancement module, enhance the effective features, and suppress the ineffective features:

[0060] For the kth speech sample I in the dataset I to be detected k The voiceprint features are input into the model trained in step (2), and channel attention weights are generated through global pooling, average pooling operations in the feature space, and MLP convolution. CA The original feature map is multiplied by the channel attention weights, and the spatial attention weights are obtained through global pooling, average pooling operations along the channel direction, and a 7×7 Conv with a stride of 1. SA Feature enhancement is performed using the following formula:

[0061] F CA =F + F × Weight CA

[0062] F Attention =F CA +F CA ×Weight SA

[0063] (3.2) The enhanced feature map is processed by a high-level feature extraction module consisting of six residual blocks and two fully connected layers to obtain the probability distribution of the sample's category;

[0064] (3.3) Calculate the speech sample model score. If the score is greater than the set threshold, the sample is considered to be a real human voice; otherwise, the sample is considered to be fake speech. The score calculation expression is as follows:

[0065] CM=log(p(human|input,θ))-log(p(spoof|input,θ)

[0066] Where p(human|input,θ) represents the probability that the input speech is a real human voice, and p(spoof|input,θ) represents the probability that the input speech is a fake speech.

[0067] In summary, compared with the prior art, the above-described technical solutions conceived by this invention have the following technical effects:

[0068] (1) High accuracy: This invention is based on attention mechanism and residual convolutional neural network to build a model. It adaptively selects important features through attention mechanism, reduces the risk of overfitting by using residual network structure, and expands the receptive field of the model by using multi-layer convolution to learn the context information in speech samples and improve the detection accuracy.

[0069] (2) High efficiency: The model used in this invention consists only of convolution operation, fully connected operation and basic matrix operation, without involving time series calculation module. It has a simple structure, high model efficiency, short time consumption, and can perform matrix parallel operation to improve computational efficiency.

[0070] (3) Strong generalization ability: This invention addresses the characteristics of high dimensionality of speech features, large number of neural network channels, and redundant features that affect model performance. It calculates channel attention and spatial attention maps of speech features and assigns weights to features in different channels and positions. This can more quickly focus on more discriminative information in the input features, reduce the impact of redundancy and noise, and improve the generalization ability of the model. Attached Figure Description

[0071] Figure 1 This is a flowchart of the attention-based intelligent voice spoofing attack detection method in an embodiment of the present invention;

[0072] Figure 2 This is a flowchart of the feature extraction method in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0074] The technical terminology of this invention will be explained and described below:

[0075] Global average pooling: For the input feature map, the average value of the two-dimensional matrix for each channel is calculated as the feature representation of that channel.

[0076] Global max pooling: For the input feature map, the maximum value of the two-dimensional matrix of each channel is selected as the feature representation of that channel.

[0077] The Softmax loss function is a loss function composed of Softmax and cross-entropy loss. It maps the class score output by the model to the [0,1] interval representing the probability distribution. Then, it takes the negative logarithm of the probability value corresponding to the label class of the input sample and sums them up as the loss value predicted by the current model for that sample. The gradient of the model parameter update can be obtained by taking the derivative.

[0078] The LeakyReLU activation function has a gradient of 1 when the input is greater than 0 and a negative value with a small gradient when the input is less than 0. It can effectively solve the problems of gradient vanishing and jagged paths in gradient update direction, and is often used in speech-related fields.

[0079] Dropout: Sets the retention probability of neural network nodes and sets the weights of hidden nodes to 0, so that the network will not be biased towards a certain node during training, thus reducing the network's overfitting behavior.

[0080] like Figure 1 As shown, the intelligent speech forgery attack detection method based on the attention mechanism of the present invention includes the following steps:

[0081] (1) Transform the speech samples from the time domain to the frequency domain for analysis. Consider the impact of different filters on feature representation. Use multiple filters to filter the spectrum and extract three different speech sample speaker features: logarithmic power spectrum, Mel-frequency cepstral coefficients, and linear-frequency cepstral coefficients. The specific steps include the following sub-steps (e.g.) Figure 2 As shown):

[0082] (1.1) The length of all speech samples in the original dataset is unified by padding and truncation to adjust the original speech samples to the same length.

[0083] For the speech playback and speech synthesis datasets provided by the 2019 Automatic Speaker Recognition Spoofing Attack and Defense Countermeasures Challenge (ASVSpoof2019), in order to facilitate batch processing and ensure model efficiency while maintaining detection accuracy, the length of the speech samples was standardized to 48,000 sampling points to extract speaker features of the same size, thus meeting the requirement of consistent input data size for batch training of neural networks.

[0084] (1.2) Divide the speech samples into multiple speech frames according to the number of sampling points, and extract the voiceprint features for each individual speech frame.

[0085] In this step, the number of sampling points N for each frame is set to 512, the sliding window length during frame division is 512, and the window interval length is 128.

[0086] (1.3) Let N be the number of sampling points and Y(n) be the speech frame signal. Multiply each speech frame by the Hamming window function W(n) to obtain the windowed signal X. w ():

[0087]

[0088] X w (n) = Y(n) × W(n)

[0089] (1.4) Perform a Fast Fourier Transform (FFT) on the windowed signal and calculate its spectrum:

[0090]

[0091] (1.5) Considering the impact of different filters on feature representation, speaker features of speech samples are extracted based on three different filters, including: calculating power spectral density (SPEC) features based on the speech spectrum and eliminating convolution noise through logarithmic transformation; calculating Mel-frequency cepstral coefficients; and calculating linear frequency cepstral coefficients. Specifically, this includes:

[0092] (1.5.1) Based on the speech spectrum, calculate the power spectral characteristics (SPEC) and eliminate convolution noise through logarithmic transformation:

[0093] SPEC = log(X[k]) 2 )

[0094] (1.5.2) Calculate Mel-frequency cepstral coefficients (MFCCs) based on the speech spectrum. Specifically, this includes:

[0095] (1.5.2.1) The spectrum is filtered using a Mel filter:

[0096] S mel (i)=ln(Mel(X[k] 2 ),(0≤i≤M)

[0097] The Mel frequency conversion formula is:

[0098]

[0099] Where M represents the number of Mel filters. This step uses 128 Mel filters for filtering.

[0100] (1.5.2.2) Perform Discrete Cosine Transform (DCT) on the Mel spectrum.

[0101]

[0102] Where L is the MFCC feature order, and in this step, the first 30 coefficients are selected as the MFCC static features.

[0103] (1.5.2.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic features of the speech.

[0104] This step involves taking the first and second derivatives of the first 30 static characteristic coefficients to obtain 30 first-order and second-order derivative coefficients, respectively.

[0105] (1.5.3) Calculate the linear frequency cepstral coefficients (LFCC) based on the speech spectrum. Specifically, this includes:

[0106] (1.5.3.1) A linear triangular filter is used to filter the spectrum:

[0107] S line (i)=ln(LF(X[k)) 2 ),(0≤i≤M)

[0108] The frequency response of a triangular filter is defined as follows:

[0109]

[0110] Where M is the number of triangular filters, and this step uses 60 linear triangular filters to filter the spectrum.

[0111] (1.5.3.2) Perform Discrete Cosine Transform (DCT) on the linear spectrum to eliminate the correlation between signal values ​​of different orders:

[0112]

[0113] Where L is the LFCC feature order, and in this step, the first 30 coefficients are selected as LFCC static features.

[0114] (1.5.3.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic feature representation of speech.

[0115] This step involves taking the first and second derivatives of the first 30 static characteristic coefficients to obtain 30 first-order and second-order derivative coefficients, respectively.

[0116] (2) Train a fake speech attack detection model based on attention mechanism and residual network. Use attention mechanism for adaptive feature selection to enhance effective features with discriminative power, suppress noise and redundant features, and then use residual network for advanced feature extraction and learning. The specific steps include the following:

[0117] (2.1) Define a fake speech attack detection model based on attention mechanism and residual network. The fake speech attack detection model consists of attention feature enhancement module, deep feature extraction module and fully connected classification module. Using the voiceprint features extracted in step (1), design loss function, select parameter optimizer, and train the network using gradient descent and backpropagation algorithms to obtain the fake speech attack detection model, where:

[0118] The attention feature enhancement module consists of two parts: channel attention and spatial attention. It is used to adaptively select features from high-dimensional and multi-channel inputs, enhance important features, and reduce redundant features. The enhanced feature map is then input into a deep feature extraction network, which consists of six residual blocks (three convolutional layers Conv1_1, Conv1_2, Conv2_1, two batch normalization layers, and one Dropout layer). This network captures the local correlations between speech features through convolutional operations, expands the receptive field using multiple convolutional layers, and obtains a comprehensive feature descriptor for the speech sample based on higher-dimensional features. Finally, the high-level features are input into a fully connected classification module consisting of two fully connected layers, fc1 and fc2, to predict the probability of the speech sample and calculate a score. If the score exceeds a threshold, the sample is judged to be real human speech; otherwise, the sample is judged to be fake speech.

[0119] (2.2) The voiceprint features F∈R extracted in (1) H×W As input to the fake speech detection model, the attention feature enhancement module dynamically selects and processes important information from the high-dimensional features:

[0120] Initialize the weights and biases of the fake speech detection model. Use the voiceprint features extracted in (1) as the input to the attention feature enhancement module in step (2.1). For an input feature map with C input channels and H×W size, the attention feature enhancement module generates an attention map by performing global max pooling (GMP) and global average pooling (GAP) operations along both the channel dimension and the spatial dimension, and calculates the weight matrix for the input feature map. The attention feature enhancement module does not change the size of the input feature map, but only adaptively assigns weights to the original features to provide enhanced features for the subsequent advanced feature extraction module. Specifically, it includes the following sub-steps:

[0121] (2.2.1) For the input feature map, each channel of the feature map is treated as a feature detector. The attention feature enhancement module assigns weights to different channel features through channel attention: for high-dimensional features with respect to the speech map F∈R C×H×W Global max pooling (GMP) and global average pooling (GAP) are used to obtain two feature descriptors for speech, Vec. CA_max ∈R C and Vec CA_avg ∈R C Vec CA_max ∈R C and Vec CA_avg ∈R C The features of each channel are globally synthesized to reflect the overall situation of each channel's features; the above two feature descriptors are fed into a shared multilayer perceptron (MLP) network composed of two 1×1 Convs to obtain the corresponding attention weights. CA_max ∈R C and Weight CA_avg ∈R C ; Calculate Weight CA_max With Weight CA_avg The summation is then used to unify the weight values ​​to the [0,1] range using the Sigmoid function, resulting in the final channel attention weight. CA ∈R C The acquired channel attention maps are used to assign weights to different channel features for the first round of feature enhancement: F CA =F + F × Weight CA .

[0122] (2.2.2) Calculate the spatial attention map and find effective information in the high-dimensional feature space: The feature map F after channel enhancement... CA ∈R C×H×W Cut along the channel direction into H×W one-dimensional vectors Vec SA ∈R C Calculate Vec respectively SA The maximum and average values ​​are used to obtain the spatial attention map Weight. SA_max ∈H×W and Weight SA_avg ∈H×W; The spatial attention graph Weight SA_max and Weight SA_max The data is spliced ​​together and then fused using Conv7×7 to obtain the final spatial attention weight. SA ∈R H×W The acquired spatial attention map is used to assign different weights to different locations in the high-dimensional features for a second round of feature enhancement: F Attention =FCA +F CA ×Weight SA .

[0123] (2.3) Enhance the feature map F A ∈R C×H×W Input the advanced feature extraction module, where C represents feature F. A The number of channels, H and W are the height and width of the high-dimensional feature map, respectively; the receptive field of the model is expanded by using 6 stacked residual blocks to capture the correlation between speech frames and frequency components, and to extract deeper features:

[0124] Each residual block consists of a convolutional branch and a residual F. A ∈R C×H×W The branch consists of two parts, feature map F A ∈R C ×H×W The inputs are used as inputs to the convolutional and residual branches. The convolutional branch consists of two 3×3 convolutions, the first with a stride of 1, followed by batch normalization and a dropout operation with a deactivation probability of 0.5, and then a 3×3 convolution with a stride of 3. The residual branch consists of a 3×3 convolution to adapt to changes in feature map size during the convolutional operation. The outputs of the convolutional and residual branches are summed and batch normalized again, with the LeakyReLU activation function used to prevent gradient vanishing. The final high-level representation is obtained by stacking six residual blocks.

[0125] (2.4) After multiple convolutions, high-level features With a sufficiently large receptive field, feature map F RA The matrix information corresponding to each feature point corresponds to the features and interdependencies of adjacent frames and adjacent frequency components within a certain region. The features obtained by the advanced feature extraction module are... Logical reasoning is performed using a linear classifier consisting of two fully connected layers, fc1 and fc2; a non-linear transformation is applied between the two fully connected layers using the LeakyReLU activation function; finally, a 2D vector V∈R is output through the Softmax activation function. 2 The probability distribution is represented.

[0126] (2.5) Calculate the loss function based on the predicted result vector V, calculate the gradient through backpropagation, and update the parameter model using the Adam optimizer:

[0127] Using the labels in the original dataset as the network's expected output and the probabilities predicted in step (2.4) as the network's actual prediction results, a target loss function is designed between the expected output and the actual predicted output for the network model constructed in step (2.1), expressed as follows:

[0128] Lost=-∑y i *log(p i )

[0129] Where, p i y represents the probability value predicted by the model. i The model is identified by its true labels. Based on the designed loss function, the model is iteratively trained using the backpropagation algorithm to minimize the classification loss function and achieve the optimal network model.

[0130] (3) Use the trained fake speech attack detection model to perform legality detection on the received speech samples, score the speech samples, and if the score exceeds the threshold, it is a real human voice; otherwise, the sample is judged to be fake speech. The specific steps include the following:

[0131] (3.1) Extract the speaker characteristics of the audio to be detected, input them into the attention feature enhancement module, enhance the effective features, and suppress the ineffective features:

[0132] For the kth speech sample I in the dataset I to be detected k The voiceprint features are input into the model trained in step (2), and channel attention weights are generated through global pooling, average pooling operations in the feature space, and MLP convolution. CA The original feature map is multiplied by the channel attention weights, and the spatial attention weights are obtained through global pooling, average pooling operations along the channel direction, and a 7×7 Conv with a stride of 1. SA Feature enhancement is performed using the following formula:

[0133] F CA =F + F × Weight CA

[0134] F Attention =F CA +F CA ×Weight SA

[0135] (3.2) The enhanced feature map is processed by a high-level feature extraction module consisting of six residual blocks (Conv1_1, Conv1_2, Conv2_1, BN1, BN2, Dropout) and two fully connected layers to obtain the prediction result, namely the probability distribution of the category to which the sample belongs.

[0136] (3.3) Calculate the speech sample model score. If the score is greater than the set threshold, the sample is considered to be a real human voice; otherwise, the sample is considered to be fake speech. The score calculation expression is as follows:

[0137] CM=log(p(human|input,θ))-log(p(spoof|input,θ)

[0138] Where p(human|input,θ) represents the probability that the input speech is a real human voice, and p(spoof|input,θ) represents the probability that the input speech is a fake speech.

[0139] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting intelligent speech spoofing attacks based on an attention mechanism, characterized in that, Includes the following steps: (1) The speech samples are converted from the time domain to the frequency domain for analysis. Considering the influence of different filters on the feature performance, multiple filters are used to filter the spectrum and extract three different speech sample voiceprint features: log power spectrum, Mel frequency cepstral coefficient, and linear frequency cepstral coefficient. (2) Train a fake speech attack detection model based on attention mechanism and residual network, use attention mechanism for adaptive feature selection, enhance effective features with discriminative power, suppress noise and redundant features, and then use residual network for deep feature extraction and learning. Specifically, it includes: (2.1) Define a fake speech attack detection model based on attention mechanism and residual network. The fake speech attack detection model consists of attention feature enhancement module, deep feature extraction module and fully connected classification module. Using the voiceprint features extracted in step (1), design loss function, select parameter optimizer, and train the network using gradient descent and backpropagation algorithm to obtain fake speech attack detection model. (2.2) Voiceprint features extracted from (1) As input to the fake speech detection model, the attention feature enhancement module is used to dynamically select and process important information in the high-dimensional features; (2.3) Enhanced feature map Input the deep feature extraction module, where C represents the feature. The number of channels, H and W are the height and width of the high-dimensional feature map, respectively; the receptive field of the model is expanded by using 6 stacked residual blocks to capture the correlation between speech frames and frequency components, and to extract deeper features: (2.4) The features obtained by the deep feature extraction module Logical reasoning is performed using a linear classifier composed of two fully connected layers, fc1 and fc2; the two fully connected layers communicate using... LeakyRelu The activation function performs a nonlinear transformation; finally, through... Softmax The activation function outputs a 2D vector. The probability distribution represented; (2.5) Based on the predicted result vector V Calculate the loss function, backpropagate to calculate the gradient, and use... Adam The optimizer updates the parameter model; (3) Use the trained fake speech attack detection model to detect the legality of the received speech samples and score the speech samples. If the score exceeds the threshold, it is a real human voice; otherwise, the sample is judged to be fake speech.

2. The intelligent speech spoofing attack detection method based on attention mechanism as described in claim 1, characterized in that, Step (1) specifically includes: (1.1) Unify the length of all speech samples in the original dataset by padding and truncation to make the original speech samples the same length; (1.2) Divide the speech samples into multiple speech frames according to the number of sampling points, and extract the voiceprint features for each individual speech frame; (1.3) Let N be the number of sampling points, For each speech frame signal, multiply each speech frame by a Hamming window function. Receive windowing signal : (1.4) Perform a Fast Fourier Transform on the windowed signal and calculate its spectrum: (1.5) Considering the impact of different filters on feature performance, the voiceprint features of speech samples are extracted based on three different filters, including: calculating power spectrum features based on speech spectrum and eliminating convolution noise through logarithmic transformation; calculating Mel-frequency cepstral coefficients; and calculating linear frequency cepstral coefficients.

3. The intelligent speech spoofing attack detection method based on attention mechanism as described in claim 2, characterized in that, Step (1.5) specifically includes the following sub-steps: (1.5.1) Based on the speech spectrum, calculate the power spectrum features and eliminate convolution noise through logarithmic transformation: (1.5.2) Calculate the Mel-spectral coefficients based on the speech spectrum; (1.5.2.1) The spectrum is filtered using a Mel filter: The Mel frequency conversion formula is: in, M The number of Mel filters; (1.5.2.2) Perform discrete cosine transform on the Mel spectrum; in, L To determine the MFCC feature order, this step selects the first 30 coefficients as the static features of MFCC. (1.5.2.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic features of the speech; (1.5.3) Calculate the linear frequency cepstral coefficients based on the speech spectrum; (1.5.3.1) A linear triangular filter is used to filter the spectrum: The frequency response of a triangular filter is defined as follows: Where M is the number of triangular filters, and this step uses 60 linear triangular filters to filter the spectrum; (1.5.3.2) Perform discrete cosine transform on the linear spectrum to eliminate the correlation between signal values ​​of different orders: Where L is the characteristic order of LFCC; (1.5.3.3) Perform first-order and second-order derivatives on the static features to obtain the dynamic feature representation of speech.

4. The intelligent speech spoofing attack detection method based on attention mechanism as described in claim 1, characterized in that, In step (2.1): the attention feature enhancement module consists of two parts: channel attention and spatial attention. It is used to perform adaptive feature selection from high-dimensional and multi-channel inputs, enhance important features, and reduce redundant features. The enhanced feature map is input into a deep feature extraction network, which consists of 6 sets of residual blocks. The network captures the local correlation between speech features through convolution operations, expands the receptive field by using multi-layer convolution, and obtains a comprehensive feature descriptor for the speech sample based on higher-dimensional features. Finally, the deep features are input into a fully connected classification module consisting of two fully connected layers, fc1 and fc2, to predict the probability of the speech samples and calculate the scores. If the score exceeds the threshold, the sample is judged to be a real human voice; otherwise, the sample is judged to be fake speech.

5. The intelligent speech forgery attack detection method based on attention mechanism as described in claim 1, characterized in that, In step (2.2): the weights and biases of the fake speech detection model are initialized, and the voiceprint features extracted in (1) are used as the input to the attention feature enhancement module in step (2.1). For input channels with a number of channels, C Size is The input feature map is used to generate an attention map by performing global max pooling and global average pooling operations along both the channel dimension and the spatial dimension, and to calculate the weight matrix for the input feature map. The attention feature enhancement module does not change the size of the input feature map, but only adaptively assigns weights to the original features, providing enhanced features for the subsequent deep feature extraction module.

6. The intelligent speech spoofing attack detection method based on attention mechanism as described in claim 1, characterized in that, Step (2.2) specifically includes: (2.2.1) For the input feature map, each channel of the feature map is treated as a feature detector. The attention feature enhancement module assigns weights to different channel features through channel attention: for high-dimensional features with respect to the speech map Global max pooling and global average pooling are used to obtain two feature descriptors for speech. and , and The features of each channel were globally synthesized, reflecting the overall situation of each channel's features; the above two feature descriptors were fed into two... Conv, a shared multilayer perceptron network, acquires the corresponding attention weights. and ;calculate and The summation is then used to unify the weight values ​​to the [0,1] range using the Sigmoid function, resulting in the final channel attention weights. The acquired channel attention maps are used to assign weights to different channel features for the first round of feature enhancement. ; (2.2.2) Calculate the spatial attention map and find effective information in the high-dimensional feature space: the feature map after channel enhancement Cut along the channel direction one-dimensional vectors Calculate separately The spatial attention map is obtained by taking the maximum and average values ​​of the values. and Spatial attention map and splicing, and via Conv7 7. The fusion process is performed to obtain the final spatial attention weights. The acquired spatial attention map is used to assign different weights to different locations in the high-dimensional features for a second round of feature enhancement. .

7. The intelligent speech forgery attack detection method based on attention mechanism as described in claim 1, characterized in that, Step (2.3) specifically includes: Each residual block consists of two parts: a convolutional branch and a residual branch, and the feature map. This will be used as input to the convolution branch and the residual branch; the convolution branch includes two... Convolution, the first The convolution stride is 1, followed by batch normalization and Dropout with a kill probability of 0.5, then the input stride is 3. Conv; the residual branch includes a Convolution is used to adapt to changes in feature map size during convolution operations; the outputs of the convolutional and residual branches are summed and batch normalized again, with the LeakyReLU activation function used to prevent gradient vanishing; the final high-level representation is obtained by stacking six residual blocks. .

8. The intelligent speech spoofing attack detection method based on attention mechanism as described in claim 1, characterized in that, Step (2.5) specifically includes: Using the labels in the original dataset as the network's expected output and the probabilities predicted in step (2.4) as the network's actual prediction results, a target loss function is designed between the expected output and the actual predicted output for the network model constructed in step (2.1), expressed as follows: in, The probability value predicted by the model. The model is identified by its true labels. Based on the designed loss function, the model is iteratively trained using the backpropagation algorithm to minimize the classification loss function and achieve the optimal network model.

9. The intelligent speech forgery attack detection method based on attention mechanism as described in claim 1 or 2, characterized in that, Step (3) specifically includes: (3.1) Extract the speaker features of the audio to be detected, input them into the attention feature enhancement module, enhance the effective features, and suppress the ineffective features: For the dataset to be detected I The first in k One voice sample I k The voiceprint features are input into the model trained in step (2), and channel attention weights are generated through global pooling, average pooling operations in the feature space, and MLP convolution. The original feature map is multiplied by the channel attention weights, and then subjected to global pooling, average pooling, and a stride of 1 along the channel direction. Conv obtains spatial attention weights Feature enhancement is performed using the following formula: (3.2) The enhanced feature map is processed by a deep feature extraction module consisting of six residual blocks and two fully connected layers to obtain the probability distribution of the sample's category; (3.3) Calculate the speech sample model score. If the score is greater than the set threshold, the sample is considered to be a real human voice; otherwise, the sample is considered to be fake speech. The score calculation expression is as follows: in, This represents the probability that the input speech is a real human voice. This represents the probability that the input speech is fake.

Images