Defense method for voiceprint recognition adversarial samples based on cosine similarity and speech denoising
By detecting and defending adversarial samples based on cosine similarity and speech denoising methods, the problem of adversarial sample attack in deep learning systems is solved, accurate detection and defense of adversarial samples is achieved, and the security of the voiceprint recognition model is enhanced.
Patent Information
- Application Number
- CN202210653117.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-09
AI Technical Summary
Deep learning systems are facing adversarial sample attacks, and the existing technology is difficult to effectively detect and defend, resulting in misclassification of models, threatening property, personal privacy and traffic safety.
Adopt the adversarial sample defense method based on cosine similarity and speech denoising. Adversarial samples are detected through data preprocessing, building voiceprint recognition models, designing adversarial sample generators, using wavelet transform reconstruction and cosine similarity calculation, and Use denoising neural networks to remove adversarial perturbations.
Accurately detect adversarial samples and perform label purification, effectively reducing the risks brought by adversarial samples and enhancing the security of voiceprint recognition models.
Smart Images

Figure CN115188384B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a voiceprint recognition adversarial sample defense method, and the present invention belongs to the field of deep learning security. Background Art
[0002] With the rapid development of deep learning, deep learning has become one of the most common technologies in artificial intelligence, affecting and changing people's lives in all aspects. Typical applications include smart home, smart driving, speech recognition, voiceprint recognition and other fields. However, as a very complex software system, deep learning will also face various hacker attacks. Hackers can also threaten property safety, personal privacy, traffic safety and announcement safety through deep learning systems. Attacks against deep learning systems usually include the following. 1. Stealing models. Hackers use various advanced means to steal model files deployed in the server. 2. Data poisoning. Data poisoning for deep learning mainly refers to adding abnormal data to the training samples of deep learning, causing the model to make classification errors when encountering certain conditions. For example, the backdoor attack algorithm adds a backdoor tag to the poisoned data to poison the model. 3. Adversarial samples. Adversarial samples refer to input samples formed by deliberately adding subtle interference to the data set. This sample causes the model to give an erroneous output with high confidence. Simply put, adversarial samples cause deep learning models to make classification errors by superimposing carefully constructed disturbances that are difficult for humans to detect on the element data. The security of deep learning has become an urgent issue that we need to solve today.
[0003] Defense methods are mainly divided into two categories: adversarial sample defense and adversarial sample detection. The main purpose of adversarial sample defense is to restore the classification label of the adversarial sample to the label of the normal sample; the main purpose of adversarial sample detection is to find adversarial samples in the sample set and remove them. Summary of the invention
[0004] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide a voiceprint recognition adversarial sample defense method based on cosine similarity and speech denoising.
[0005] The technical method adopted by the present invention to solve the technical problem is: data preprocessing, preprocessing the speaker voice data we use; building a voiceprint recognition model; using several different adversarial attack methods combined with the voiceprint recognition model to design an adversarial sample generator with malicious information samples; reconstructing the clean data by wavelet transform, calculating the cosine similarity of the output probability vectors obtained by the samples before and after the wavelet transform of the clean data in the classification model, and setting a cosine similarity threshold; reconstructing the adversarial samples by wavelet transform, and also calculating the cosine similarity values of the output vectors before and after the reconstruction, comparing the value with the cosine similarity threshold, those less than the threshold are adversarial samples, and those greater than the threshold are adversarial samples that are not detected; training a speech denoising neural network, inputting the undetected adversarial samples into the denoising network for denoising, and removing adversarial disturbances.
[0006] A method for defending against adversarial samples of voiceprint recognition based on cosine similarity and speech denoising, comprising the following steps:
[0007] Step 1: Perform data preprocessing on the speaker's speech signal;
[0008] Step 2: Build a voiceprint recognition model;
[0009] Step 3: Design adversarial samples based on the voiceprint model;
[0010] Step 4: Obtain the detection threshold based on the clean data and wavelet transform reconstruction method;
[0011] Step 5: Detect adversarial samples based on decision thresholds; Perform wavelet transform and then get the cosine similarity value C′ i , compare the cosine similarity value of the adversarial sample with the decision threshold T, if C′ is satisfied i <T then judge For adversarial examples;
[0012] Step 6: Denoising the neural network to defend against adversarial examples;
[0013] Further, step 1 specifically includes:
[0014] First, extract data from the existing voice file (.WAV format), and use the librosa audio processing Python toolkit to extract voice data from the speaker's voice, as follows:
[0015] X i ,sr=librosa(T i ,sr=None),i=1,2,...n+m (1)
[0016] Where X iis the speech data of the i-th speaker’s speech file extracted, sr is the sampling rate of the speech data, T i is the speech file of the i-th speaker;
[0017] Normalize the speech signal data set and divide the data set D into training set D train With the test set D test , where the dataset
[0018] D train ={(X1,Y1),(X2,Y2),...,(X n ,Y n )}
[0019] D test ={(X n+1 ,Y n+1 ),(X n+2 ,Y n+2 ),...,(X n+m ,Y n+m )}
[0020] X i =(X i1 ,X i2 ,...,X id ),d represents X i The data length, Represents the c-class label; the normalization formula is:
[0021]
[0022] in represents the normalized sample, max(X i ) represents the maximum value among the d sampling points in the sample, after normalization
[0023] Further, step 2 specifically includes:
[0024] The structure and parameters of the classification model are pre-specified and do not change; the classification model structure used in the present invention mainly includes 1D convolution layer, maximum pooling layer, batch normalization layer, and fully connected layer. The specific structure is shown in Table 1. The voiceprint recognition classification model is trained using the training data set as follows:
[0025] Target model:
[0026]
[0027] F target (·) represents the probability vector of the model output;
[0028] Further, step 3 specifically includes:
[0029] Adversarial examples are defined as:
[0030]
[0031] Among them, δ i is the perturbation added to the original sample;
[0032] Here, we take the optimization-based adversarial attack method as an example to illustrate the generation of adversarial samples. The optimization-based attack method is essentially a gradient-based adversarial sample generation method.
[0033] The optimization function is defined as:
[0034]
[0035]
[0036] Further, step 4 specifically includes:
[0037] The detection threshold is obtained according to the clean data and wavelet transform reconstruction method;
[0038] Suppress disturbances i (t), and get the real signal f i (t), after the signal is transformed by wavelet, the signal f can be removed to the greatest extent i (t), most of the energy is concentrated on a few wavelet coefficients with larger amplitudes; while the perturbation e i (t) After wavelet transformation, it will be distributed on all time axes at all scales, and the amplitude is not very large; the purpose of speech signal noise reduction can be achieved by threshold filtering;
[0039] The steps of wavelet threshold denoising are as follows:
[0040] I. For speech signal f i (t) Perform wavelet transform; select an orthogonal wavelet and the number of decomposition layers N, and transform the signal f i (t) Perform N-layer wavelet decomposition;
[0041] II. For the sample signal f i The wavelet transform coefficients of (t) are subjected to linear threshold processing; the high-frequency coefficients of each layer from the first layer to the Nth layer are processed by the threshold function, and the low-frequency coefficients of each layer are not processed; the threshold formula is as follows:
[0042]
[0043] Where w is the wavelet coefficient and λ is the selected threshold;
[0044] III. Reconstruct the processed wavelet coefficients; reconstruct the speech signal based on the low-frequency coefficients of the Nth layer of wavelet decomposition and the high-frequency coefficients of the first to Nth layers after processing, so as to obtain the denoised speech signal f i de (t); then:
[0045]
[0046] The original clean sample and the clean samples after wavelet transformation Input the voiceprint recognition model to get the corresponding output probability vector and Then the cosine similarity between the two vectors is calculated as
[0047]
[0048] Then, a decision threshold is selected from the cosine similarity value set C of the clean samples. For example, if the false detection rate is set to b%, the decision threshold T is the b% value in sort(C) (sort(C) means sorting C from small to large).
[0049] Further, step 6 specifically includes:
[0050] First, a denoising neural network is trained. The denoising neural network used is DCCRN. The DCCRN network first transforms the noisy speech through short-time Fourier transform to obtain the speech complex spectrum with real and imaginary parts, and then defines the input complex matrix I = I r +jI i , complex convolution filter W = W r +jW i , where the matrix W r and W i represents the real and imaginary parts of the complex convolution kernel, and the output feature F of the complex layer out for:
[0051] F out =(I r *W r -I i *W i )+j(I r *W i +I i *W r ) (10)
[0052] Similar to complex convolution, given a complex input X r and X i The real and imaginary parts of the complex LSTM output L out Can be defined as:
[0053] L rr =LSTM r (X r );L ii =LSTM i (X i ) (11)
[0054] L ri =LSTM i (X r );L ir =LSTM r (X i ) (12)
[0055] L out =(L rr -L ii )+(L ri +L ir ) (13)
[0056] Among them, LSTM r and LSTM i are the real and imaginary parts of two regular LSTM modules;
[0057] In the process of training the network, the purpose of DCCRN is to optimize a complex mask matrix M = M r +jM i , the noisy speech can be obtained after the mask matrix is passed through to obtain the denoised speech
[0058] S=M*Y (14)
[0059] Where S = S r +jS i , which is the complex spectrum of clean speech, Y = Y r +jY i is the complex spectrum of noisy speech;
[0060] Given S and Y, M can be calculated as follows:
[0061]
[0062] M is expressed in polar coordinates as:
[0063]
[0064]
[0065] M phase =arctan2(M i ,M r ) (18)
[0066] The noisy speech can be estimated as:
[0067]
[0068] The loss function of DCCRN is SI-SNR, and the calculation formula is as follows:
[0069]
[0070]
[0071]
[0072] Where <·,·> represents the dot product between two vectors. When it is very close to the clean speech S, e n ≈0;
[0073] When the DCCRN denoising network is trained, the undetected adversarial samples are input into the denoising network to obtain the denoised speech, which will be restored to the original correct label.
[0074] The working principle of the present invention is:
[0075] The speech data set used was preprocessed: the original waveform time domain data of each speech segment was obtained, the data was divided into a training set and a test set, and normalized.
[0076] Steps to build a voiceprint recognition model: The structure and parameters of the voiceprint recognition model are pre-specified and will not change. The data set suitable for the recognition model is also pre-given, that is, the speaker's speech sample, including the input time domain waveform data and corresponding classification labels for speaker recognition. The sample set in the data set should be able to be predicted and output by the model with high accuracy.
[0077] Design adversarial samples based on the voiceprint recognition model: Select several commonly used white-box adversarial sample attack methods. Adjust the gradient direction of the input data according to the voiceprint recognition model parameters, so that when the input sample changes slightly, the voiceprint recognition model generates an incorrect label.
[0078] The detection threshold is obtained according to the clean data and wavelet transform reconstruction method: the clean samples are decomposed by wavelet and then reconstructed to achieve the purpose of sample denoising by wavelet transform. The clean samples before and after wavelet transform are input into the voiceprint recognition network to obtain two model output probability vectors, and then the cosine similarity value between the corresponding vectors is calculated. A threshold is set in the cosine similarity value, and the threshold is selected based on the false detection rate.
[0079] Detect adversarial samples based on the decision threshold: perform wavelet transform on the adversarial samples as well, then input the samples before and after the wavelet transform into the voiceprint recognition model to obtain the output probability vector, obtain the cosine similarity value, compare the value with the decision threshold, and determine the value that is smaller than the decision threshold as an adversarial sample.
[0080] Denoising neural network defense against adversarial samples: We first train a denoising neural network, and then input the adversarial samples that were not detected in the previous step into the denoising network. We finally get a denoised sample, so that a large number of adversarial samples will lose their adversarial nature, thus achieving further defense.
[0081] The adversarial samples of the present invention are different from normal samples. The adversarial samples themselves carry more noise. After denoising, they will change more compared to clean samples. This is reflected in the model as a large change in the model output probability before and after denoising of the adversarial samples. This change can be measured by the cosine similarity value between vectors. If the output vector changes less, the cosine similarity value is large, and vice versa. This feature can be used to detect the output adversarial samples. In order to achieve further defense, further denoising of the adversarial samples that are not detected after the detection step will enhance the defense effect. The present invention strengthens the security of the voiceprint recognition model.
[0082] The advantages of the present invention are that it can accurately detect adversarial samples in the data and perform further label purification defense on undetected adversarial samples, effectively reducing the risks brought by adversarial samples and enhancing the security of the voiceprint recognition model. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] Figure 1 It is a basic flow chart of the method of the present invention.
[0084] Figure 2 It is the wavelet transform denoising flow chart of the present invention.
[0085] Figure 3 This is a DCCRN network structure diagram of the present invention. DETAILED DESCRIPTION
[0086] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0087] Example 1
[0088] A method for defending against adversarial samples of voiceprint recognition based on cosine similarity and speech denoising, comprising the following steps:
[0089] (1) Perform data preprocessing on the speaker's speech signal:
[0090] First, we extract data from the existing voice file (.WAV format). We use the librosa audio processing Python toolkit to extract voice data from the speaker's voice as follows:
[0091] X i ,sr=librosa(T i ,sr=None),i=1,2,...n+m (1)
[0092] Where X i is the speech data of the i-th speaker’s speech file extracted, sr is the sampling rate of the speech data, T i is the speech file of the i-th speaker.
[0093] Normalize the speech signal data set and divide the data set D into training set D train With the test set D test , where the dataset
[0094] D train ={(X1,Y1),(X2,Y2),...,(X n ,Y n )}
[0095] D test ={(X n+1 ,Y n+1 ),(X n+2 ,Y n+2 ),...,(X n+m ,Y n+m )}
[0096] X i =(X i1 ,X i2 ,...,X id ),d represents X i The data length, Represents the c-class label. The normalization formula is:
[0097]
[0098] in represents the normalized sample, max(X i ) represents the maximum value among the d sampling points in the sample, after normalization
[0099] (2) Steps of building a voiceprint recognition model: pre-specify the structure and parameters of the classification model, and do not change. The classification model structure used in the present invention mainly includes a 1D convolution layer, a maximum pooling layer, a batch normalization layer, and a fully connected layer. The specific structure is shown in Table 1. The training data set is used for training, and the voiceprint recognition classification model is as follows:
[0100] Target model:
[0101]
[0102] F target (·) represents the probability vector of the model output.
[0103] (3) Design adversarial sample steps based on voiceprint recognition classifier:
[0104] Adversarial examples are defined as:
[0105]
[0106] Among them, δ i is the perturbation added to the original sample.
[0107] Here, we take the optimization-based adversarial attack method as an example to illustrate the generation of adversarial samples. The optimization-based attack method is essentially a gradient-based adversarial sample generation method.
[0108] The optimization function is defined as:
[0109]
[0110]
[0111] (4) Calculate the cosine similarity in the clean samples according to the wavelet transform and select the decision threshold step:
[0112] Suppress disturbances i (t), and get the real signal f i (t), after the signal is transformed by wavelet, the signal f can be removed to the greatest extent i (t), concentrating most of the energy on a few wavelet coefficients with larger amplitudes. i (t) will be distributed on all time axes at all scales after wavelet transformation, and the amplitude is not very large. , the purpose of speech signal noise reduction can be achieved by threshold filtering.
[0113] The steps of wavelet threshold denoising are as follows:
[0114] I. For speech signal f i (t) Perform wavelet transform. Select an orthogonal wavelet and the number of decomposition layers N, and transform the signal f i(t) Perform N-layer wavelet decomposition.
[0115] II. For the sample signal f i The wavelet transform coefficients of (t) are processed by linear threshold. The high-frequency coefficients of each layer from the first layer to the Nth layer are processed by the threshold function, and the low-frequency coefficients of each layer are not processed. The threshold formula is as follows:
[0116]
[0117] Where w is the wavelet coefficient and λ is the selected threshold.
[0118] III. Reconstruct the processed wavelet coefficients. Reconstruct the speech signal based on the low-frequency coefficients of the Nth layer of wavelet decomposition and the high-frequency coefficients of the first to Nth layers after processing, so as to obtain the denoised speech signal f i de (t). Then we have:
[0119]
[0120] The original clean sample and the clean samples after wavelet transformation Input the voiceprint recognition model to get the corresponding output probability vector and Then the cosine similarity between the two vectors is calculated as
[0121]
[0122] Then, a decision threshold is selected from the cosine similarity value set C of the clean samples. For example, if the false detection rate is set to b%, the decision threshold T is the b% value in sort(C) (sort(C) means sorting C from small to large).
[0123] (5) Detect adversarial samples according to the decision threshold: Perform wavelet transform and then get the cosine similarity value C′ i , compare the cosine similarity value of the adversarial sample with the decision threshold T, if C′ is satisfied i <T then judge For adversarial samples.
[0124] (6) De-noising steps for undetected adversarial samples using a denoising neural network:
[0125] First, we need to train a denoising neural network. The denoising neural network used in the present invention is DCCRN. The DCCRN network first transforms the noisy speech through short-time Fourier transform to obtain the speech complex spectrum with real and imaginary parts, and then defines the input complex matrix I=Ir +jI i , complex convolution filter W = W r +jW i , where the matrix W r and W i represents the real and imaginary parts of the complex convolution kernel, and the output feature F of the complex layer out for:
[0126] F out =(I r *W r -I i *W i )+j(I r *W i +I i *W r ) (10)
[0127] Similar to complex convolution, given a complex input X r and X i The real and imaginary parts of the complex LSTM output L out Can be defined as:
[0128] L rr =LSTM r (X r );L ii =LSTM i (X i ) (11)
[0129] L ri =LSTM i (X r );L ir =LSTM r (X i ) (12)
[0130] L out =(L rr -L ii )+(L ri +L ir ) (13)
[0131] Among them, LSTM r and LSTM i are the real and imaginary parts of two regular LSTM modules.
[0132] In the process of training the network, the purpose of DCCRN is to optimize a complex mask matrix M = M r +jM i , the noisy speech can be obtained after the mask matrix is passed through to obtain the denoised speech
[0133] S=M*Y (14)
[0134] Where S = S r +jS i , which is the complex spectrum of clean speech, Y = Y r +jY i is the complex spectrum of noisy speech.
[0135] Given S and Y, M can be calculated as follows:
[0136]
[0137] M is expressed in polar coordinates as:
[0138]
[0139]
[0140] M phase =arctan2(M i ,M r ) (18)
[0141] The noisy speech can be estimated as:
[0142]
[0143] The loss function of DCCRN is SI-SNR, and the calculation formula is as follows:
[0144]
[0145]
[0146]
[0147] Where <·,·> represents the dot product between two vectors. When it is very close to the clean speech S, e n ≈0.
[0148] After the DCCRN denoising network is trained, we input the undetected adversarial samples into the denoising network to obtain the denoised speech, which will be restored to the original correct label.
[0149] Implementation Case 2: Data from Actual Experiments
[0150] (1) Select experimental data.
[0151] The dataset used in the experiment is the AISHELL-1 speech dataset, which collects speech recorded by speakers of different ages, genders, and regions in a quiet environment, with a sampling rate of 16,000. We selected the speech of 20 people as the dataset for the voiceprint recognition model. For each sentence, the length of the original waveform time domain data we extracted is 60,000. The data preprocessing is saved as a dataset of an array of (bathsize, 60000, 1) and the corresponding label data is generated. The processed datasets are all saved as .npy files. The input data of the denoising network DCCRN are the generated adversarial samples of FGSM, BIM, PGD, DeepFool, and CW, and the output is the corresponding clean samples.
[0152] (2) Parameter determination.
[0153] The threshold λ in the threshold processing of wavelet transform is 0.02, and the decision threshold T used to detect adversarial samples is 0.91955.
[0154] (3) Experimental results
[0155] This paper uses five attack algorithms (FGSM, BIM, PGD, DeepFool, CW) to generate five adversarial samples, and uses the defense method proposed in this paper to defend against the five adversarial samples. The defense success rate ACC and the false detection rate FPR are used as the defense effect of the defense method, and are compared with other defense (detection) methods. The experimental results are shown in Table 1.
[0156] Table 1 Defense effect
[0157]
[0158] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.
Claims
1. A voiceprint recognition adversarial sample defense method based on cosine similarity and speech denoising, characterized in that: The following steps are involved: Step 1: Perform data preprocessing on the speaker's speech signal; Step 2: Build a voiceprint recognition model; Step 3: Design adversarial samples based on the voiceprint model; Step 4: Obtain the detection threshold based on the clean data and wavelet transform reconstruction method; specifically including: Suppress disturbances i (t), and get the real signal f i (t), after the signal is transformed by wavelet, the signal f can be removed to the greatest extent i (t), most of the energy is concentrated on a few wavelet coefficients with larger amplitudes; while the perturbation e i (t) After wavelet transformation, it will be distributed on all time axes at all scales, and the amplitude is not very large; the purpose of reducing the noise of the speech signal can be achieved by threshold filtering; The steps of wavelet threshold denoising are as follows: I. For speech signal f i (t) Perform wavelet transform; select an orthogonal wavelet and the number of decomposition layers N, and transform the signal f i (t) Perform N-layer wavelet decomposition; II. For the sample signal f i The wavelet transform coefficients of (t) are subjected to linear threshold processing; the high-frequency coefficients of each layer from the first layer to the Nth layer are processed by the threshold function, and the low-frequency coefficients of each layer are not processed; the threshold formula is as follows: Where w is the wavelet coefficient and λ is the selected threshold; III. Reconstruct the processed wavelet coefficients; reconstruct the speech signal based on the low-frequency coefficients of the Nth layer of wavelet decomposition and the high-frequency coefficients of the first to Nth layers after processing, so as to obtain the denoised speech signal Then we have: The original clean sample and the clean samples after wavelet transformation Input the voiceprint recognition model to get the corresponding output probability vector and Then the cosine similarity between the two vectors is calculated as Then, a detection threshold is selected from the cosine similarity value set C of the clean samples. For example, if the false detection rate is set to b%, the detection threshold T is the value of the b% in sort(C), where sort(C) means sorting C from small to large. Step 5: Detect adversarial samples based on the detection threshold; i Perform wavelet transform and then get the cosine similarity value C i ′, compare the cosine similarity value of the adversarial sample with the detection threshold T. If C i '<T then determine X i For adversarial examples; Step 6: Denoising the neural network to defend against adversarial examples; specifically includes: First, a denoising neural network is trained. The denoising neural network used is DCCRN. The DCCRN network first transforms the noisy speech through short-time Fourier transform to obtain the speech complex spectrum with real and imaginary parts, and then defines the input complex matrix I = I r +jI i , complex convolution filter W = W r +jW i , where the matrix W r and W i represents the real and imaginary parts of the complex convolution kernel, and the output feature F of the complex layer out for: F out =(I r *W r -I i *W i )+j(I r *W i +I i *W r ) (10) Similar to complex convolution, given a complex input X r and X i The real and imaginary parts of the complex LSTM output L out Can be defined as: L rr =LSTM r (X r );L ii =LSTM i (X i ) (11) L ri =LSTM i (X r );L ir =LSTM r (X i ) (12) L out =(L rr -L ii )+(L ri +L ir ) (13) Among them, LSTM r and LSTM i are the real and imaginary parts of two regular LSTM modules; In the process of training the network, the purpose of DCCRN is to optimize a complex mask matrix M = M r +jM i , the noisy speech can be obtained after the mask matrix is passed through the denoised speech S=M*Y (14) Where S = S r +jS i , which is the complex spectrum of clean speech, Y = Y r +jY i is the complex spectrum of noisy speech; Given S and Y, M can be calculated as follows: M is expressed in polar coordinates as: M phase =arctan2(M i ,M r ) (18) The noisy speech can be estimated as: The loss function of DCCRN is SI-SNR, and the calculation formula is as follows: Where <·,·> represents the dot product between two vectors. When it is very close to the clean speech S, e n ≈0; When the DCCRN denoising network is trained, the undetected adversarial samples are input into the denoising network to obtain the denoised speech, which will be restored to the original correct label.
2. The voiceprint recognition adversarial sample defense method based on cosine similarity and speech denoising as claimed in claim 1, characterized in that: Step 1 specifically includes: First, extract data from the existing voice file, where the existing voice file refers to a voice file in .WAV format; use the librosa audio processing python toolkit to extract voice data from the speaker's voice, as follows: X i ,sr=librosa(T i ,sr=None),i=1,2,...n+m (1) Where X i is the speech data of the i-th speaker’s speech file extracted, sr is the sampling rate of the speech data, T i is the speech file of the i-th speaker; Normalize the speech signal data set and divide the data set D into training set D train With the test set D test , where the dataset D train ={(X1,Y1),(X2,Y2),...,(X n ,Y n )} D test ={(X n+1 ,AND n+1 ),(X n+2 ,AND n+2 ),...,(X n+m ,AND n+m )} X i =(X i1 ,X i2 ,...,X id ),d represents X i The data length, Represents the c-class label; the normalization formula is: in represents the normalized sample, max(X i ) represents the maximum value among the d sampling points in the sample, after normalization 3. The voiceprint recognition adversarial sample defense method based on cosine similarity and speech denoising as claimed in claim 1, characterized in that: Step 2 specifically includes: The structure and parameters of the classification model are pre-specified and remain unchanged; the classification model structure used mainly includes 1D convolution layer, maximum pooling layer, batch normalization layer, and fully connected layer; the voiceprint recognition classification model is trained using the training data set as follows: Target model: F target (·) represents the probability vector of the model output.
4. The voiceprint recognition adversarial sample defense method based on cosine similarity and speech denoising as claimed in claim 1, characterized in that: Step 3 specifically includes: Adversarial examples are defined as: Among them, δ i is the perturbation added to the original sample; Here, we take the optimization-based adversarial attack method as an example to illustrate the generation of adversarial samples. The optimization-based attack method is essentially a gradient-based adversarial sample generation method. The optimization function is defined as:
Citation Information
Cited By
Defense method and system for confrontation sample of modulation signal
CN120639484A
