A lightweight voice spoofing detection algorithm based on one-class classification

Abstract

The present invention discloses a lightweight voice deception detection algorithm based on one-class classification, and designs a new loss function DOC‑Softmax according to the characteristics of real voice and deceptive voice, that is, a dispersion loss function is introduced into the deceptive voice space of the one-class classification loss function OC‑Softmax to alleviate the problem of feature distribution mismatch between training data and test data, thereby improving the accuracy and generalization ability of the voice deception detection model. At the same time, the voice deception detection algorithm is designed as a lightweight voice deception detection algorithm using the knowledge distillation framework, which reduces the number of parameters of the model and makes it easy to deploy it to mobile terminals or embedded devices. This model has better generalization ability than the model obtained using exactly the same model structure, training data and only using hard label training methods.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a lightweight voice deception detection algorithm based on one-class classification. Background Art

[0002] Voiceprint recognition refers to a biometric recognition technology that identifies the speaker's identity based on the speaker information in the voice signal. It has the characteristics of non-concentration, non-contact, not afraid of occlusion, and requires subjective consciousness to cooperate. It has been widely used in scenarios such as finance, social security, government and enterprises, and the Internet of Things.

[0003] However, there are many uncertainties in the actual application environment, especially malicious deception attacks, which cause the performance of existing voiceprint recognition systems to drop sharply. Voice deception refers to the process of "modifying and counterfeiting" an illegal voice that has not been authenticated by the automatic speaker verification system through recording, speech synthesis, speech conversion, etc. in order to pass the detection of the automatic speaker verification system. Currently, there are three main deception attacks: (1) deliberate imitation of other speakers; (2) realistic speech obtained through speech synthesis or speech conversion technology; (3) playback or splicing of recordings from high-fidelity recording equipment. Among the above three deception attacks, the deliberate imitation deception attack method can be identified as true or false by mainstream voiceprint recognition systems.

[0004] However, the improvement of recording equipment quality and the rapid development of speech processing technologies such as speech synthesis and speech conversion have brought increasingly severe challenges to the security of voice spoofing detection and voiceprint recognition systems. Voice spoofing detection refers to the use of deep learning or machine learning methods to input manual features or original speech into deep learning or machine learning models for learning, ultimately achieving the purpose of voice authentication. Next, we will introduce residual networks, knowledge distillation, Softmax loss function, and AM-Softmax loss function.

[0005] (1) Residual Network

[0006] In 2015, He Kaiming and others proposed the residual network (Residual network, ResNet) to alleviate the gradient vanishing problem caused by increasing the network depth in deep neural networks. It is widely used in image classification, target detection, speech recognition and other fields. The residual network solves the degradation problem by introducing a deep residual learning framework. The main idea is to remove the same main part and highlight the small changes. It uses some stacked nonlinear layers to fit a residual mapping F(x): = H(x)-x instead of directly fitting an underlying mapping H(x). In this way, the original mapping becomes F(x)+x, and optimizing the residual mapping is easier than optimizing the original mapping. This residual learning structure can be implemented through a forward neural network + shortcut connection, such as Figure 1As shown in Figure 3, the shortcut connection is equivalent to performing an equivalent mapping, which does not generate additional parameters or increase the computational complexity, and the entire network can still be trained through end-to-end back-propagation.

[0007] (2) Knowledge Distillation

[0008] Deep learning has achieved incredible performance in many fields such as computer vision, speech recognition, and natural language processing. However, most deep learning models are too computationally expensive to run on mobile or embedded devices. Therefore, the model needs to be compressed, and knowledge distillation is one of the important technologies in model compression. Knowledge distillation was first proposed by Hinton et al., and mainly includes three distillation settings: one is model compression, which distills the knowledge of complex models into small models; the second is cross-modal transfer knowledge, which transfers knowledge from one modality to another; and the third is integrated distillation, which distills the knowledge of multiple models into a single model. The present invention uses the knowledge distillation framework to compress a class of classified voice deception detection models. The core idea is to first train a complex network model, and then use the output of this complex network and the real label of the data to train a smaller network. Therefore, the knowledge distillation framework usually contains a complex model (called Teacher model) and a small model (called Student model). The complex model is generally a single complex network or a collection of several networks, which has good performance and generalization ability, while the small model has limited expression ability due to its small network size. The knowledge distillation framework uses the knowledge learned by the large model to guide the training of the small model, that is, using the Soft-target predicted by the Teacher model to assist the Hard-target training, so that the Student model has the same performance as the Teacher model, greatly reducing the number of model parameters, thereby achieving model compression and reducing model inference latency.

[0009] (3) Softmax loss function and AM-Softmax loss function

[0010] The formula for the original Softmax loss function for binary classification is as follows:

[0011]

[0012] in, is the embedding vector, y i ∈{0,1} is the label of the i-th sample, y i =0 means that the sample is the target class, y i =1 means that the sample is a non-target class. is the weight vector, and N is the number of samples in a batch.

[0013] AM-Softmax improves the Softmax function by introducing an angular interval to make the embedding distribution of the two classes more compact. The specific formula is as follows:

[0014]

[0015] Where α is a scaling factor, m is the cosine additive interval, and are the normalized representations of w0, w1 and x respectively.

[0016] For these two loss functions, the embedding vectors of the target class and the non-target class tend to converge to two opposite directions, which are w0-w1 and w1-w0 respectively. For the AM-softmax function, the target class and the non-target class have the same compact margin. The larger the value of m, the more compact the embedding.

[0017] In voice spoofing detection, it is reasonable to train a compact embedding space for real speech, but if a compact space is also trained for spoofing attacks, known attacks may be overfitted. In summary, although voice spoofing detection algorithms have achieved a lot of results, these algorithms generally have the problem of insufficient generalization for unknown spoofing attacks. This is because most methods formulate the voice spoofing problem as a binary classification problem of real speech and spoofed speech, essentially assuming that the feature distribution between training data and test data is the same or similar. Although this assumption is reasonable for real speech, it is not so correct for spoofed speech. Due to the development of voice spoofing technologies such as voice conversion and speech synthesis, the spoofing attacks in the training set may never catch up with the expansion of the distribution of actual spoofing attacks. In addition, most of the existing voice spoofing detection algorithms have problems such as complex network structure, large amount of calculation, slow speed, and are difficult to port to mobile or embedded devices. Summary of the invention

[0018] The present invention aims to solve the problem of feature distribution mismatch between training data and test data, and proposes a lightweight voice deception detection algorithm based on one-class classification in order to improve the accuracy and generalization ability of the voice deception detection algorithm and reduce the model inference delay.

[0019] In order to achieve the above object, the present invention provides the following technical solutions:

[0020] A lightweight speech spoofing detection algorithm based on one-class classification uses the knowledge distillation framework to learn a feature space through a one-class classification loss function DOC-Softmax based on dispersion loss. In this feature space, the real speech is embedded with a tight boundary, and there is a certain distance between the deceptive speech and the real speech. At the same time, dispersion loss is introduced in the deceptive speech feature space to maximize the distance from each deceptive speech sample to its center, so that the deceptive speech covers the entire deceptive speech space.

[0021] Furthermore, the total loss function L of the one-class classification loss function DOC-Softmax based on the dispersion loss DOCS is the one-class classification loss L OCS and dispersion loss L D The weighted sum of is λ, and the specific formula is as follows:

[0022]

[0023] in, is the weight vector, is the standardized representation of w0, which refers to the optimization direction of the real speech, α is a scaling factor, and two intervals m0 and m1 are introduced to limit the angle between the real speech, the deceptive speech and the real speech weight vector, that is, and The angle θ i , m0,m1∈[-1,1],m0>m1;

[0024] The formula for the one-class classification loss function OC-Softmax is as follows:

[0025]

[0026] Two intervals m0 and m1, m0, m1∈[-1,1], m0>m1, are introduced to limit the real class and deception class samples respectively. and The angle θ i , when y i = 0, m0 is used to make θ i Less than arccosm0; when y i =1, m1 is used to make θ i is larger than arccosm1, a small arccosm0 makes the target class gather on the weight vector w0, and a relatively large arccosm1 makes the non-target class stay away from the weight vector w0;

[0027] The formula for introducing dispersion loss is as follows:

[0028]

[0029] in, is the embedding vector, is the normalized representation of x, y i ∈{0,1} is the label of the i-th sample, y i =0 means the sample is real speech, y i =1 indicates that the sample is a fraudulent speech, N is the number of samples in a batch, M is the number of fraudulent samples in a batch, and ε is a very small constant to avoid the situation where the denominator is 0. For the center of the deceptive samples in each batch, the dispersion loss L D To maximize the deception of speech samples The distance from their center μ makes the deceptive speech cover the entire deceptive area as much as possible.

[0030] Further, α=20, m0=0.9, m1=0.2.

[0031] Furthermore, when the speech sample is real speech, that is, y i = 0, m0 is used to make θ i is smaller than arccosm0. A small arccosm0 makes the real speech gather near the weight vector w0. When the speech sample is a deceptive speech, that is, y i =1, m1 is used to make θ i Greater than arccosm1, a relatively large arccosm1 makes the spoofed speech far away from the weight vector w0.

[0032] Furthermore, the teacher model adopts a network structure based on the deep residual network ResNet-18, and uses attention pooling instead of global average pooling.

[0033] Furthermore, the teacher model takes the extracted LFCC features as input and the output of the fully connected layer as the embedding of the input speech.

[0034] Furthermore, the embedding is fed into the DOC-Softmax loss function to calculate the confidence score This confidence score represents the probability that the input speech is a real speech or a spoofed speech, and thus the classification result of the input speech is obtained.

[0035] Furthermore, the model architecture of the student model is basically the same as that of the teacher model, except that the three residual modules are removed. The soft labels predicted by the teacher model are used to assist the hard label training of the student model. The loss function of the student model consists of two parts: one is a classification loss L based on the dispersion loss DOCS , one is the confidence score output by the student model Confidence scores with teacher model output The mean square error loss LMSE , the specific formula is as follows:

[0036]

[0037] The total loss function of the student model is to minimize L DOCS and L MSE The weighted sum of is β, and the specific formula is as follows:

[0038]

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The lightweight voice deception detection algorithm based on one-class classification of the present invention designs a new loss function DOC-Softmax for the characteristics of real voice and deceptive voice, that is, a dispersion loss function is introduced into the deceptive voice space of the one-class classification loss function OC-Softmax to alleviate the problem of feature distribution mismatch between training data and test data, thereby improving the accuracy and generalization ability of the voice deception detection model. At the same time, the voice deception detection algorithm is designed as a lightweight voice deception detection algorithm using the knowledge distillation framework, which reduces the number of parameters of the model and makes it easy to deploy it to mobile terminals or embedded devices. This model has better generalization ability than the model obtained using exactly the same model structure, training data and only using hard label training methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0042] Figure 1 Schematic diagram of the residual structure.

[0043] Figure 2 A model architecture diagram provided for an embodiment of the present invention.

[0044] Figure 3 A schematic diagram of a residual module provided in an embodiment of the present invention.

[0045] Figure 4 is a comparison of four loss functions: Softmax, AM-Softmax, OC-Softmax, and DOC-Softmax. Figure 4(a) is Softmax, Figure 4(b) is AM-Softmax, Figure 4(c) is OC-Softmax, and Figure 4(d) is DOC-Softmax. DETAILED DESCRIPTION

[0046] The present invention designs a lightweight voice spoofing detection algorithm based on one-class classification, which can well alleviate the problem of feature distribution mismatch between training data and test data. In the one-class classification method, the target class does not have this distribution mismatch problem, while for the non-target class, the samples in the training set either do not exist or are not statistically representative. The key idea of ​​the one-class classification method is to capture the target class distribution and set a strict classification boundary around it so that all non-target data are placed outside the boundary.

[0047] The definition of one-class classification loss function OC-Softmax is as follows:

[0048]

[0049] This loss function has only one weight vector It is the optimization direction of the target class embedding. and Two intervals m0 and m1 (m0, m1∈[-1,1], m0>m1) are introduced to limit the real class and the deception class samples respectively. and The angle θ i When y i = 0, m0 is used to make θ i Less than arccosm0; when y i =1, m1 is used to make θ i Larger than arccosm1, a small arccosm0 can make the target class gather in the weight vector w0, and a relatively large arccosm1 can make the non-target class stay away from the weight vector w0.

[0050] However, it can be seen from formula (3) that when the target class and the non-target class are closely around the weight vector w0 and the reverse direction of the weight vector w0′, respectively, the one-class classification loss function L OCS Therefore, the final direction of non-target class optimization is still closely around the opposite direction of the weight vector w0′. When the OC-Softmax function is optimized well enough, this loss function is essentially no different from the AM-Softmax function.

[0051] The present invention aims to improve the accuracy and generalization ability of the voice deception detection algorithm and reduce the model inference delay. It proposes a lightweight voice deception detection algorithm based on one-class classification. Specifically, a new loss function DOC-Softmax is designed for the characteristics of real voice and deceptive voice, that is, a dispersion loss function is introduced in the deceptive voice space of the above-mentioned one-class classification loss function OC-Softmax to alleviate the problem of feature distribution mismatch between training data and test data, thereby improving the accuracy and generalization ability of the voice deception detection model. At the same time, the above-mentioned voice deception detection algorithm is designed as a lightweight voice deception detection algorithm using the knowledge distillation framework, which reduces the number of parameters of the model and makes it easy to deploy on mobile terminals or embedded devices.

[0052] The present invention proposes a lightweight voice deception detection algorithm based on one-class classification, which is a single-feature, single-system voice deception detection algorithm. This algorithm defines the voice deception problem as a type of feature learning problem to improve the generalization ability of the model.

[0053] The method of the present invention mainly includes two aspects: one is the design of a class classification loss function based on dispersion loss, and the other is the design of a lightweight voice deception detection model based on knowledge distillation. A feature space is learned by using a class classification loss function based on dispersion loss, in which the real voice is embedded with a tight boundary, and there is a certain distance between the deceptive voice and the real voice. At the same time, dispersion loss is used to maximize the distance from each deceptive voice sample to its center, so that the deceptive voice covers the entire deceptive voice space. This method improves the generalization ability of the model by increasing the probability of unknown attacks entering the deceptive space.

[0054] In an embodiment of the present invention, 60-dimensional linear frequency cepstral coefficient LFCC features are extracted from the speech, a network structure based on the deep residual network ResNet-18 is adopted, and attention pooling is used instead of global average pooling. The network structure takes the extracted LFCC features as input, and the output confidence score represents the classification result, which improves the performance of the voice deception detection algorithm without using any data enhancement, feature fusion and model integration methods. In addition, based on the knowledge distillation framework, we designed a lightweight voice deception detection model StudentNet with the above network as TeacherNet, which greatly reduced the model parameters (reduced the number of parameters by about 30 times), making it easier to deploy the model on mobile terminals or embedded devices.

[0055] 1. Design of a Classification Loss Function Based on Dispersion Loss

[0056] The comparison of the four loss functions of Softmax, AM-Softmax, OC-Softmax, and DOC-Softmax is shown in Figure 4, where Figure 4(a) is Softmax, Figure 4(b) is AM-Softmax, Figure 4(c) is OC-Softmax, and Figure 4(d) is DOC-Softmax.

[0057] For the two loss functions, Softmax and AM-Softmax, the embedding vectors of the target class and the non-target class tend to converge to two opposite directions, which are w0-w1 and w1-w0, respectively, as shown in Figure 4(ab). In the AM-softmax loss function, the target class and the non-target class have the same compact edge. The larger the value of m, the more compact the embedding. In speech spoofing detection, it is reasonable to train a compact embedding space for real speech, but if a compact space is also trained for spoofing attacks, it may overfit the known attacks. The one-class classification loss function OC-Softmax can solve this problem well by introducing two different intervals to better compact the real speech and isolate the spoofed speech. However, it can be seen from formula (3) that when the target class and the non-target class are closely around the weight vector w0 and the reverse direction of the weight vector w0′, respectively, the one-class classification loss function OC-Softmax reaches the minimum. Therefore, the final direction of the non-target class optimization is still closely around the reverse direction of the weight vector w0′, as shown in Figure 4(c). When the OC-Softmax function is optimized well enough, this loss function is essentially no different from the AM-Softmax function. Therefore, we introduce a dispersion loss for non-target class samples, the formula is as follows:

[0058]

[0059] in, is the embedding vector, is the normalized representation of x, y i ∈{0,1} is the label of the i-th sample, y i =0 means the sample is real speech, y i =1 indicates that the sample is a fraudulent speech, N is the number of samples in a batch, and M is the number of fraudulent samples in a batch. ε is a very small constant used to avoid the situation where the denominator is 0. is the center of the deceptive samples in each batch. Dispersion loss L D To maximize the deception of speech samples The distance from their center μ makes the deceptive speech cover the entire deceptive area as much as possible, that is, the deceptive speech covers θ as much as possible. iis larger than the entire area of ​​arccosm1, as shown in Figure 4(d), thereby increasing the probability that the deceptive speech enters the deceptive area and improving the generalization performance of the model. Therefore, the total loss function L DOCS is the one-class classification loss L OCS and dispersion loss L D The weighted sum of is λ, and the specific formula is as follows:

[0060]

[0061] in, is the weight vector, is the normalized representation of w0, which is the optimization direction of the real speech. α is a scaling factor, and α=20 in the present invention. Two intervals m0 and m1 (m0, m1∈[-1,1], m0>m1) are introduced to define the angles between the real speech, the deceptive speech and the real speech weight vector, i.e. and The angle θ i , in the present invention, m0=0.9, m1=0.2. When the speech sample is real speech, that is, y i = 0, m0 is used to make θ i is smaller than arccosm0. A small arccosm0 can make the real speech gather near the weight vector w0. When the speech sample is a deceptive speech, that is, y i =1, m1 is used to make θ i Greater than arccosm1, a relatively large arccosm1 can make the spoofed speech far away from the weight vector w0.

[0062] 2. Design of lightweight voice deception detection model based on knowledge distillation

[0063] The overall structure of the present invention is shown in FIG. Figure 2 As shown in, the residual module is as follows Figure 3 As shown in the figure. The teacher model TeacherNet is designed based on the deep residual network ResNet-18, in which the global average pooling layer is replaced by the attention time pooling layer. The input of the teacher model is the extracted 60-dimensional linear frequency cepstral coefficient LFCC feature, and the output of the fully connected layer is used as the embedding of the input speech, with a dimension of 256. This embedding is sent to the DOC-Softmax loss function to calculate the confidence score This confidence score represents the probability of the input speech being a real speech or a spoofed speech, thereby obtaining a classification result of the input speech.

[0064] The model architecture of the student model StudentNet is basically the same as that of the teacher model, except that the three residual modules are removed. Soft labels predicted by the teacher model TeacherNet To assist the hard label training of the student model StudentNet, this soft label carries a lot of information for TeacherNet's inductive reasoning. The loss function of the student model consists of two parts: one is a classification loss L based on the dispersion loss DOCS , one is the confidence score output by the student model Confidence scores with teacher model output The mean square error loss L MSE , the specific formula is as follows:

[0065]

[0066] The total loss function of the student model is to minimize L DOCS and L MSE The weighted sum of is β, and the specific formula is as follows:

[0067]

[0068] StudentNet trained based on the knowledge distillation framework has better generalization ability than the model obtained using the same model structure, training data and only hard label training method. The model parameters of this StudentNet are about 402K, which is about 30 times less than that of TeacherNet. The model size of StudentNet is 1590KB, which is convenient for deployment on mobile terminals or embedded devices. The voice deception detection algorithm proposed in the present invention is a single-system, single-feature algorithm. Without using any data enhancement, feature fusion and model integration methods, the accuracy of the algorithm is improved and the inference delay of the model is greatly reduced.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight voice spoofing detection algorithm based on one-class classification, It is characterized in that Using the knowledge distillation framework, a feature space is learned through a first-class classification loss function DOC-Softmax based on dispersion loss. In this feature space, the real speech embedding has a tight boundary, and there is a certain distance between the spoofed speech and the real speech. At the same time, a dispersion loss is introduced in the spoofed speech feature space to maximize the distance from each spoofed speech sample to its center, so that the spoofed speech covers the entire spoofed speech space. One-class classification loss function based on dispersion loss DOC-Softmax total loss function L DOCS is the one-class classification loss L OCS and dispersion loss L D The weighted sum of is λ, and the specific formula is as follows: in, is the weight vector, w 0 The standardized representation is the optimization direction of the real speech, α is a scaling factor, and the two intervals m 0 and m 1 are introduced to define the angles between the real speech, the deceptive speech and the real speech weight vector, namely and The angle θ i , m 0 ,m 1 ∈[-1,1],m 0 >m 1 ;m yi is the cosine additive interval of the label of the i-th sample; The formula for the one-class classification loss function OC-Softmax is as follows: Two intervals m 0 and m 1 , m 0 ,m 1 ∈[-1,1],m 0 >m 1 , are introduced to limit the real and deceptive samples respectively and The angle θ i , when y i =0,m 0 To make θ i Smaller than arccosm 0 When y i =1,m 1 To make θ i Bigger than arccosm 1 , a small arccosm 0 Make the target class clustered in the weight vector w 0 , a relatively large arccosm 1 Keep non-target classes away from the weight vector w 0 ; The formula for introducing dispersion loss is as follows: in, is the embedding vector, is the normalized representation of x, y i ∈{0,1} is the label of the i-th sample, y i =0 means the sample is real speech, y i =1 indicates that the sample is a fraudulent speech, N is the number of samples in a batch, M is the number of fraudulent samples in a batch, and ε is a very small constant to avoid the situation where the denominator is 0. For the center of the deceptive samples in each batch, the dispersion loss L D To maximize the deception of speech samples The distance from their center μ makes the deceptive speech cover the entire deceptive area as much as possible.

2. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that α=20,m 0 =0.9,m 1 = 0.

2.

3. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that When the speech sample is real speech, that is, y i = 0, m 0 To make θ i Smaller than arccosm 0 , a small arccosm 0 Make the real speech gather on the weight vector w 0 nearby; When the speech sample is a deceptive speech, y i =1, m 1 To make θ i Bigger than arccosm 1 , a relatively large arccosm 1 Keep the spoof speech away from the weight vector w 0 .

4. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that The teacher model in the knowledge distillation framework adopts a network structure based on the deep residual network ResNet-18, and uses attention pooling instead of global average pooling.

5. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that The teacher model in the knowledge distillation framework takes the extracted LFCC features as input and the output of the fully connected layer as the embedding of the input speech.

6. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that The embedding is fed into the DOC-Softmax loss function to calculate the confidence score This confidence score represents the probability of the input speech being a real speech or a spoofed speech, thereby obtaining a classification result of the input speech.

7. The lightweight voice deception detection algorithm based on one-class classification according to claim 1, It is characterized in that The model architecture of the student model in the knowledge distillation framework is basically the same as that of the teacher model, except that the three residual modules are removed. The soft labels predicted by the teacher model are used to assist the hard label training of the student model. The loss function of the student model consists of two parts: one is a classification loss L based on the dispersion loss. DOCS , one is the confidence score output by the student model Confidence scores with teacher model output The mean square error loss L MSE , the specific formula is as follows: The total loss function of the student model is to minimize L DOCS and L MSE The weighted sum of is β, and the specific formula is as follows:

Images