Voiceprint recognition and voice deception detection integration method

Abstract

A voiceprint recognition and voice cheating detection integration method belongs to the voiceprint recognition field, and comprises the following steps: simultaneously inputting a registration voice and a test voice into a voiceprint recognition subsystem, inputting the test voice into a voice cheating detection subsystem, outputting registration voice voiceprint embedding and test voice voiceprint embedding by the voiceprint recognition subsystem, the voice spoofing detection subsystem outputs test voice spoofing embedding; inputting the inserts into a back-end fusion classifier based on a cross attention mechanism, fusing the inserts by using the cross attention mechanism to simulate global feature interaction, and capturing complementary information of the two tasks; and outputting a classification result after classification of an integrated model based on model-independent element learning. According to the method, a cross attention fusion mechanism and a model irrelevant element learning technology are combined, so that the model can judge the identity of a speaker according to voiceprint information in voice, malicious attacks such as voice synthesis are resisted, the model can quickly and flexibly adapt to a task scene, and the adaptability and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of voiceprint recognition, and in particular relates to an integrated method of voiceprint recognition and voice deception detection. Background Art

[0002] In today's era of rapid digitalization and intelligent development, biometric authentication technology has received increasing attention, and voiceprint recognition is one of the core research areas with great potential and value. The main goal of this technology is to verify whether the individual characteristics in the voice signal match the claimed identity by analyzing and comparing them, thereby confirming the identity of the speaker. Voiceprint recognition systems have been widely used in security verification, access control, forensic monitoring and other fields, providing an efficient and convenient solution for identity authentication.

[0003] However, with the continuous development of technology, voiceprint recognition systems are also facing increasingly complex deceptive attacks, such as text to speech (TTS), voice conversion (VC) and replay attacks. Attackers simulate or copy real voice signals to deceive. Unprotected systems find it difficult to distinguish between real and fake voices, which greatly reduces the reliability of the system. In order to meet these challenges, protecting voiceprint recognition systems from deception interference has become a hot topic of current research. Voice deception detection systems have emerged, which can keenly detect those fake or tampered false deceptive voices from input voice samples. In actual application scenarios, voice deception detection systems can work closely with voiceprint recognition systems to form an organic whole. Through this double insurance working mechanism, a more stable, reliable and secure authentication system is jointly built to provide solid protection for users' identity authentication needs.

[0004] Although voiceprint recognition and voice spoofing detection systems have made significant progress in independent research, the integration of the two still requires more in-depth research. At present, most research focuses on the separate speaker recognition and voice spoofing detection tasks, and the integration of the two is still in its infancy. The related work on the integration of speaker recognition and voice spoofing detection can be roughly divided into two types. One method is a single integrated model trained based on multi-task learning, and the other method is the back-end fusion method. In the back-end fusion method, the independent voiceprint recognition subsystem and voice spoofing detection subsystem will extract different embeddings respectively, and then make decisions together. The current common integration model usually consists of three independent subsystems, namely the voiceprint recognition subsystem, the voice spoofing detection subsystem, and the back-end classifier. In actual operation, the voiceprint recognition subsystem and the voice spoofing detection subsystem will extract the embeddings of the registration voice and the test voice respectively, and then these embeddings will be transmitted to the back-end classifier, which will make the final judgment based on them. Specifically, in the back-end classifier, the three embeddings are first fused, and a simple feature stacking method is generally used, that is, the voiceprint embedding and the spoofing embedding are directly combined, followed by further calculation and processing through the convolution layer, and finally the classification result is output.

[0005] There are still some incomplete aspects in the existing research on the integration of speaker recognition and voice spoofing detection. For example, in practical applications, the lack of training data has become one of the key factors limiting the performance improvement of the integrated solution. A small amount of training data cannot fully characterize the diversity of speaker characteristics and the complexity of deception methods. This may cause the model to overfit during the learning process. The model is overly dependent on the information contained in the limited training samples and cannot be well generalized to new voice data. Another prominent problem is the weak ability to respond to unknown attacks.

[0006] As technology continues to develop, the deception methods used by attackers are becoming increasingly complex and diverse, and new unknown attack types continue to emerge. However, existing integrated models are usually designed and trained based on known attack patterns and limited training data. The model has not been exposed to these new attack types during training and cannot learn the corresponding recognition patterns and features. Therefore, when faced with unknown voice deception attacks, the model often cannot accurately detect anomalies. Summary of the invention

[0007] In order to solve the problems of poor adaptability and generalization of the existing integrated model of voiceprint recognition and voice deception detection, the present invention provides an integrated method of voiceprint recognition and voice deception detection to enhance the fusion performance of the voiceprint recognition task and the voice deception detection task in the model. The present invention realizes the effective fusion of embedding by combining the cross-attention fusion mechanism and the model-independent meta-learning technology, so that the model can quickly and flexibly adapt to the task scenario when facing a small amount of training data, significantly improving the adaptability and generalization ability of the model, thereby providing a more reliable and practical solution for voiceprint recognition and voice deception detection tasks.

[0008] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0009] The present invention provides an integrated method for voiceprint recognition and voice deception detection, which specifically includes the following steps:

[0010] Firstly, a back-end fusion classifier based on the cross-attention mechanism and an integrated model based on model-independent meta-learning are designed; then, a pair of registration speech and test speech of a speaker are input, and the registration speech and test speech are simultaneously input into the voiceprint recognition subsystem, and the test speech is input into the voice deception detection subsystem, the voiceprint recognition subsystem outputs the registration speech voiceprint embedding and the test speech voiceprint embedding, and the voice deception detection subsystem outputs the test speech deception embedding; then, the registration speech voiceprint embedding, the test speech voiceprint embedding and the test speech deception embedding are input into the back-end fusion classifier based on the cross-attention mechanism, and the cross-attention mechanism is used to fuse the registration speech voiceprint embedding and the test speech voiceprint embedding, as well as the test speech voiceprint embedding and the test speech deception embedding, respectively, to simulate the global feature interaction and capture the complementary information of the two tasks at the same time; finally, the classification result is output after classification by the integrated model based on model-independent meta-learning.

[0011] Furthermore, the back-end fusion classifier based on the cross-attention mechanism includes: an embedding processing layer, an embedding fusion layer and a classification layer; the embedding processing layer is composed of a linear layer and a LeakyReLU activation function, which is used to map the registered voiceprint embedding, the test voice voiceprint embedding and the test voice deception embedding to the same dimension; the embedding fusion layer includes two cross-attention modules, which are used to fuse and splice the registered voice voiceprint embedding, the test voice voiceprint embedding and the test voice deception embedding; the classification layer is provided with a splicing layer and a fully connected layer, and the input features are gradually reduced in dimension by two linear plus ReLU activation functions before being output.

[0012] Furthermore, the voiceprint recognition subsystem extracts the registration voiceprint embedding from the registration voice and extracts the test voiceprint embedding from the test voice; and the voice deception detection subsystem extracts the test voice deception embedding from the test voice.

[0013] Furthermore, the specific operation steps of the back-end fusion classifier based on the cross attention mechanism are:

[0014] The registered voiceprint embedding, the test voiceprint embedding and the test voice spoof embedding are mapped to the same dimension; the registered voiceprint embedding and the test voiceprint embedding are grouped together to perform feature fusion for the voiceprint recognition task, the registered voiceprint embedding is used as the query vector, and the test voiceprint embedding is used as the key vector and value vector; the weight is obtained by calculating the similarity score between the query and the key, and then the weight is normalized; the registered voiceprint embedding and the test voiceprint embedding are cross-attention fused to obtain the cross-attention feature F ASV ; Cross-attention fusion is performed on the test voiceprint embedding and the test voice spoofing embedding, the test voiceprint embedding is used as the query vector, and the test voice spoofing embedding is used as the key vector and value vector to obtain the cross-attention feature F CM ; The fused cross-attention feature F ASV and the cross attention feature F CM Through the splicing layer, the final comprehensive feature F is obtained fusion ; The comprehensive feature F fusion Input into the classification layer to determine whether the test voice is from a real user.

[0015] Furthermore, the specific calculation formula for the similarity score between the query and the key is:

[0016]

[0017] Among them, d k It represents the dimension size of the key K, which is used for scaling to avoid the similarity score being too large; Q represents query, K represents key, V represents value, and the superscript T represents the transpose of the matrix.

[0018] Furthermore, the cross-attention feature F ASV The mathematical expression is:

[0019]

[0020] The cross attention feature F CM The mathematical expression is:

[0021]

[0022] The comprehensive feature F fusion The mathematical expression is:

[0023] F fusion =concat(F ASV ,FCM ) (4)

[0024] in, Embed the registered voiceprint; To test voiceprint embedding; Embeddings for testing voice spoofing.

[0025] Furthermore, the comprehensive feature F fusion The mathematical expression input into the classification layer to determine whether the test speech is a real user process is:

[0026] Output = OCSoftmax(ReLU(W f F fusion +b f )) (5)

[0027]

[0028] Among them, x i represents the embedding vector; y i ∈{0,1} represents the label of the i-th sample, y i =1 indicates the target class, y i =0 indicates a non-target class; ω0 indicates a unique weight vector, which is the optimization direction for embedding the target class; N indicates the number of mini-batch samples; α indicates a scaling factor; W f represents the weight vector of the fully connected layer; b f represents the bias vector; m is the residual of cosine similarity, which represents the angle between the test speech and the real speech weight vector; It is used to constrain the range of the angle between ω0 and the ith sample eigenvector, m0(y i =0) and m1(y i =1) are two boundary values, m0, m1∈[-1,1], to achieve the effect of concentrating the target category around ω0 and pushing the non-target data away from ω0.

[0029] Furthermore, the specific operation steps of the integrated model based on model-independent meta-learning are:

[0030] S2.1: Initialization parameters of the model;

[0031] Decompose the total task T into m small tasks T1, T2, T m-1 ,,……T m, M is the total number of tasks, each small task corresponds to a specific speaker or deception attack type; a specific data set is assigned to each small task, and the data set is divided into a training set and a validation set. The training set is used for the inner loop optimization of the model, and the validation set is used for the global optimization of the outer loop; at the same time, the weights of the integrated model based on model-independent meta-learning are initialized;

[0032] S2.2: Inner loop updates temporary parameters;

[0033] For each small task, the integrated model is updated to temporary parameters adapted to the current specific task by performing several gradient descent updates on the training data of the current task;

[0034] S2.3: Outer loop updates model parameters;

[0035] Based on the validation set of each small task, the validation loss of the model is calculated and the meta-gradient is extracted from it; the meta-gradient is used to adjust the initialization parameters of the global model and update the global parameters: by calculating the difference between the parameters adapted to the specific task and the initialization parameter φ, the meta-gradients of all small tasks are merged, and the global parameters of the model are updated through the meta-gradient.

[0036] Furthermore, the calculation formula for updating the temporary parameters of the inner loop is:

[0037]

[0038] Among them, loss(θ) represents the loss value for parameter θ, and the loss function is the above-mentioned OCSoftmax function. represents the gradient of the loss value with respect to the parameter θ, and β represents the learning rate at the task level.

[0039] Furthermore, the outer loop parameter update formula is:

[0040]

[0041] Among them, φ represents the initialization parameter of the model, and L(φ) represents the parameter θ that is suitable for all small tasks. i The sum of the loss values ​​of represents the gradient of the loss value L(φ), η represents the meta-learning rate, and θ m Represents the parameters learned in the inner loop of the mth small task.

[0042] The beneficial effects of the present invention are:

[0043] At present, the existing integrated models of voiceprint recognition and voice deception detection often find it difficult to balance the accurate extraction and efficient fusion of voiceprints and deception embeddings, resulting in limited model recognition accuracy, poor adaptability to new speakers or unknown attack types, and easy overfitting of training to specific data sets. Therefore, the present invention focuses on the field of voiceprint recognition and voice deception detection, and proposes an integrated method for voiceprint recognition and voice deception detection. The innovation of the present invention lies in the use of a back-end fusion classifier based on a cross-attention mechanism and an integrated model based on model-independent meta-learning to achieve better performance and stronger adaptability of the integrated model. The present invention adds a cross-attention mechanism to the back-end fusion classifier and uses model-independent meta-learning technology to learn the optimal parameters. By combining the cross-attention fusion mechanism and the model-independent meta-learning technology, the integrated model can determine the identity of the speaker based on the voiceprint information in the speech, while resisting malicious attacks such as speech synthesis, thereby achieving effective fusion of the two, so that the model can quickly and flexibly adapt to task scenarios when faced with a small amount of training data, significantly improving the adaptability and generalization ability of the model, and can still extract common features under limited data, effectively solving the overfitting problem, and effectively filling the gaps in the existing technology, thereby bringing more reliable and practical solutions to voiceprint recognition and voice deception detection tasks, and has high practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of an integrated method of voiceprint recognition and voice deception detection provided by the present invention.

[0045] Figure 2 It is a flow chart of the back-end fusion classifier based on the cross-attention mechanism designed by the present invention.

[0046] Figure 3 It is a schematic diagram of the training process of the integrated model based on model-independent meta-learning designed by the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0048] The present invention provides an integrated method for voiceprint recognition and voice deception detection, which mainly includes two aspects: one is the design of a back-end fusion classifier based on a cross-attention mechanism, and the other is the design of an integrated model based on model-independent meta-learning. The combination of these two technologies aims to improve the overall performance and adaptability of voiceprint recognition and voice deception detection tasks. Among them, the back-end fusion classifier based on the cross-attention mechanism designed by the present invention effectively captures the deep correlation between voiceprint and deception embedding by optimizing the embedding fusion process; and the integrated model based on model-independent meta-learning designed by the present invention enhances the adaptability and generalization ability of the integrated model, laying a solid foundation for improving overall performance and practicality.

[0049] like Figure 1 As shown, the present invention provides an integrated method for voiceprint recognition and voice deception detection, and its specific implementation process is as follows:

[0050] Step S1: Design of backend fusion classifier based on cross attention mechanism;

[0051] The cross-attention mechanism allows the model to process two different sequences, usually one sequence as a query and the other sequence as a key and value, thereby focusing on relevant information. Figure 2 As shown in the figure, the back-end fusion classifier is mainly composed of the following parts: embedding processing layer, embedding fusion layer and classification layer; among them, the embedding processing layer is mainly composed of a linear layer and a LeakyReLU activation function, which is used to process the registered voice print embedding, the test voice print embedding and the test voice deception embedding, and map them to the same dimension. The embedding fusion layer mainly includes two cross attention modules, which are used to process the fusion of three embeddings (registered voice print embedding, test voice print embedding and test voice deception embedding) respectively, and then concatenate the two fused embeddings. The classification layer is provided with a Concat concatenation layer and a fully connected layer, and two linear plus ReLU (Rectified Linear Unit) activation functions are used to gradually reduce the dimension of the input features before outputting them.

[0052] The specific operation steps of the back-end fusion classifier based on the cross attention mechanism are as follows:

[0053] S1.1: First, three features are extracted from the registered voice and the test voice through the voiceprint recognition subsystem and the voice deception detection subsystem: Test voiceprint embedding Testing Voice Spoofing Embeddings

[0054] S1.2: In order to fuse these three features, a processing layer is needed to map them to the same dimension.

[0055] S1.3: Then the three embeddings are divided into two groups, the registered voiceprint embedding and the test voiceprint embedding are one group, which performs feature fusion of the voiceprint recognition task, takes the registered voiceprint embedding as the query vector, and takes the test voiceprint embedding as the key vector and value vector.

[0056] S1.4: By calculating the correlation between the query and the key, the weight is obtained, and then the weight is normalized similar to softmax to indicate the importance of different keys to the query. In other words, the larger the weight, the more attention the network will pay to its corresponding value.

[0057] Specifically, the present invention uses a scaled dot product attention mechanism to calculate the similarity score between the query Q and the key K, and the specific calculation formula is:

[0058]

[0059] Among them, d k It represents the dimension size of the key K, which is used for scaling to avoid the similarity score being too large; Q represents query, K represents key, V represents value, and the superscript T represents the transpose of the matrix.

[0060] S1.5: Cross-attention fusion;

[0061] Embedding the registered voiceprint And test voiceprint embedding Perform cross attention fusion to obtain the cross attention feature F ASV , its specific mathematical expression is as follows:

[0062]

[0063] Similarly, the test voiceprint is embedded Testing Voice Spoofing Embeddings Perform cross-attention fusion to embed the test speech voiceprint into As query vector, test speech spoof embedding As the key vector and value vector, we get the cross-attention feature F CM , its specific mathematical expression is as follows:

[0064]

[0065] Finally, the fused cross-attention feature F ASV and the cross attention feature F CM Through the Concat splicing layer, the final comprehensive feature F is obtained. fusion , its specific mathematical expression is as follows:

[0066] F fusion =concat(F ASV ,F CM ) (4)

[0067] S1.6: The comprehensive feature F fusion Input into the classification layer to determine whether the test voice is from a real user.

[0068] Specifically, the comprehensive feature F is firstly processed through a multi-layer fully connected network, that is, a fully connected layer fusionThe activation function is ReLU, which introduces nonlinear expression capability. The OCSoftmax layer is used to calculate the final classification probability to determine the authenticity of the test speech.

[0069] The OCSoftmax layer in the present invention specifically uses One-Class (OC) Softmax Loss, which is a variant of the soft maximum loss function and is mainly used for single-class classification tasks. Different from the traditional multi-class classification Softmax loss, it focuses on distinguishing the target class from the non-target class (considering all non-target classes as a whole), which conforms to the characteristics of the integrated task of voiceprint recognition and voice deception detection, that is, only the real speaker's voice is the target class, and the rest of the non-speakers and deceptive voices are non-target classes. The model mainly learns the feature representation of the target class in order to distinguish the samples belonging to the target class from other samples. Its specific mathematical expression is as follows;

[0070] Output = OCSoftmax(ReLU(W f F fusion +b f )) (5)

[0071]

[0072] Among them, x i represents the embedding vector; y i ∈{0,1} represents the label of the i-th sample, y i =1 indicates the target class, y i =0 indicates a non-target class; ω0 indicates a unique weight vector, which is the optimization direction for embedding the target class; N indicates the number of small batch samples; α indicates a scaling factor, and α=20 in the present invention; W f represents the weight vector of the fully connected layer; b f represents the bias vector; m is the residual of cosine similarity, which represents the angle between the test speech and the real speech weight vector; It is used to constrain the range of the angle between ω0 and the ith sample eigenvector, m0(y i =0) and m1(y i =1) are two boundary values, m0, m1∈[-1,1], to achieve the effect of concentrating the target category around ω0 and pushing the non-target data away from ω0.

[0073] In the embodiment of the present invention, the embedding dimension output by the voiceprint recognition subsystem is 512 dimensions, that is, the dimensions of the registered voiceprint embedding and the test voiceprint embedding are both 512 dimensions, and the dimension of the test voice deception embedding output by the voice deception detection subsystem is 160 dimensions. Then, the registered voice voiceprint embedding, the test voice voiceprint embedding, and the test voice deception embedding are mapped to the same dimension through the embedding processing layer, that is, the 160-dimensional test voice deception embedding is mapped to 512 dimensions; then, the 512-dimensional registered voice voiceprint embedding, the test voice voiceprint embedding, and the test voice deception embedding are fused through two cross-attention modules, and the two fused embeddings are spliced ​​again, and the dimension becomes 1024; finally, two linear plus ReLU activation functions are used to gradually reduce the input features from 1024 dimensions to 256 dimensions, and then the 256-dimensional features are mapped to 1-dimensional output.

[0074] In the present invention, by introducing the cross-attention mechanism, the weights of each input embedding can be dynamically adjusted during the embedding fusion process. This method allows the model to effectively capture the relationship and complementary information between the voiceprint and the deception embedding, which helps to improve the model's ability to distinguish different speakers and attack types. The introduction of the cross-attention mechanism makes the model more interpretable in the decision-making process. By analyzing the attention weights, we can intuitively understand how the model makes classification decisions based on different features, providing valuable information for subsequent model optimization and troubleshooting.

[0075] Step S2: Design of ensemble model based on model-independent meta-learning;

[0076] The training framework of the integrated model based on model-independent meta-learning has the following structure: Figure 3 As shown in the figure, T represents the prediction tasks corresponding to different speakers or deception algorithms, and M represents the total number of tasks. The training framework of the integrated model based on model-independent meta-learning is divided into two parts: inner update and outer update.

[0077] The specific operation steps of the integrated model based on model-independent meta-learning are as follows:

[0078] S2.1: Initialization parameters of the model;

[0079] First, decompose the total task T into m small tasks T1, T2, T m-1 ,,……T m Each small task corresponds to a specific speaker or deception attack type. A specific dataset is assigned to each small task. The dataset is divided into a training set and a validation set. The training set is used for the inner loop optimization of the model, and the validation set is used for the global optimization of the outer loop. At the same time, the weights of the integrated model based on model-independent meta-learning are initialized.

[0080] S2.2: Inner loop updates temporary parameters;

[0081] Then, the inner loop is used to update the temporary parameters: for each small task, the integrated model is updated to the temporary parameters that adapt to the current specific task by performing several gradient descent updates on the training data of the current task. The temporary parameters are only used for the training of the current task and do not affect the global model. The specific calculation formula for the inner loop to update the temporary parameters is as follows:

[0082]

[0083] Among them, loss(θ) represents the loss value for parameter θ, and the loss function is the above-mentioned OCSoftmax function. represents the gradient of the loss value with respect to the parameter θ, and β represents the learning rate at the task level.

[0084] S2.3: Outer loop updates model parameters;

[0085] After the inner loop optimization of all small tasks is completed, the validation set of each small task is used for the outer loop optimization: based on the validation set of each small task, the validation loss of the model is calculated, and the "meta-gradient" is extracted from it. These meta-gradients are used to adjust the initialization parameters of the global model, and finally the global parameters are updated. Specifically, by calculating the difference between the parameters adapted to a specific task and the initialization parameter φ, the meta-gradients of all small tasks are merged, and the global parameters of the model are updated through the meta-gradient, so that the model can also have good effects when encountering new tasks in the future. Among them, the specific calculation formula for updating the model parameters in the outer loop is as follows:

[0086]

[0087] Among them, φ represents the initialization parameter of the model, and L(φ) represents the parameter θ that is suitable for all small tasks. i The sum of the loss values ​​of represents the gradient of the loss value L(φ), η represents the meta-learning rate, and θ m Represents the parameters learned in the inner loop of the mth small task.

[0088] Meta-training is completed and the model parameters θ are obtained best This parameter enables the integrated model to quickly achieve significant performance on new tasks in the future and provide stronger generalization capabilities when dealing with new speakers and unknown attack types.

[0089] The present invention adopts model-independent meta-learning technology, so that the training of the integrated model can extract efficient learning strategies and more general parameters when the training data is limited, so that the model can have excellent adaptability when facing new speakers and deception attacks, ensuring the robustness and high performance of the system in different environments.

[0090] The present invention applies model-independent meta-learning technology to construct a model training framework that can quickly adapt to new tasks. The framework improves the initial parameters of the model through meta-learning, so that the model can be quickly adjusted through a small number of updates when facing new speakers or attacks, significantly improving the adaptability of the system. In the case of limited training data, model-independent meta-learning technology can make more effective use of data. By learning on multiple related tasks, the model can extract more general features and learning strategies, reducing dependence on a large amount of task-specific data, which is of great significance for solving the problem of insufficient training data, effectively solving the overfitting phenomenon of the current model to the training set, and realizing a stronger generalization ability of the integrated system on different speakers and attack types.

[0091] Step S3: the input is a pair of registration voice and test voice of a speaker, the registration voice and the test voice are input into the voiceprint recognition subsystem at the same time, and the voice deception detection subsystem only needs to input the test voice; wherein, the voiceprint recognition subsystem outputs the registration voice voiceprint embedding and the test voice voiceprint embedding, and the voice deception detection subsystem outputs the test voice deception embedding, and the registration voice voiceprint embedding, the test voice voiceprint embedding, and the test voice deception embedding output by the subsystem are input into the back-end fusion classifier based on the cross-attention mechanism designed by the present invention: first, the cross-attention mechanism is used to fuse between the registration voiceprint embedding and the test voice voiceprint embedding, and between the test voiceprint embedding and the test voice deception embedding, respectively, to simulate the global feature interaction and capture the complementary information of the two tasks at the same time; finally, after classification by the neural network classification layer in the integrated model based on model-independent meta-learning, the final classification result is output.

[0092] In an embodiment of the present invention, the voiceprint recognition subsystem uses a pre-trained ECAPA-TDNN model, which is an efficient, robust, and state-of-the-art speaker embedding extractor consisting of a Res2net backbone architecture, a compression excitation module, a channel, a context-dependent statistical pooling layer, and multi-layer feature aggregation.

[0093] In an embodiment of the present invention, the speech spoofing detection subsystem uses a pre-trained AASIST model, which is an end-to-end, state-of-the-art deception adversarial system based on a RawNet2 encoder and a spectral-temporal graph attention network (RawGAT-ST).

[0094] The present invention provides an integrated method for voiceprint recognition and voice deception detection. The method uses model-independent meta-learning technology to fuse the voiceprint embedding and deception embedding output by the voiceprint recognition subsystem and the voice deception detection subsystem through a back-end fusion classifier based on a cross-attention mechanism, thereby greatly improving the overall performance of the integrated system of voiceprint recognition and voice deception detection. By guiding the learning of the integrated model through the model-independent meta-learning method, the problem of insufficient training data in the integrated system is solved, and unknown voice deception attacks can be effectively dealt with.

[0095] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An integrated method for voiceprint recognition and voice deception detection, characterized in that: The following steps are involved: First, a back-end fusion classifier based on the cross-attention mechanism and an integrated model based on model-independent meta-learning are designed; then a pair of registration speech and test speech of a speaker are input, and the registration speech and test speech are simultaneously input into the voiceprint recognition subsystem, and the test speech is input into the voice spoofing detection subsystem, and the voiceprint recognition subsystem outputs the registration speech voiceprint embedding and the test speech voiceprint embedding, and the voice spoofing detection subsystem outputs the test speech spoofing embedding; The registered voiceprint embedding, the test voiceprint embedding and the test voice spoofing embedding are then input into the back-end fusion classifier based on the cross-attention mechanism. The cross-attention mechanism is used to fuse the registered voiceprint embedding and the test voiceprint embedding, as well as the test voiceprint embedding and the test voice spoofing embedding, to simulate the global feature interaction and capture the complementary information of the two tasks at the same time. Finally, the classification result is output after classification by an integrated model based on model-independent meta-learning.

2. The integrated method of voiceprint recognition and voice deception detection according to claim 1, characterized in that: The back-end fusion classifier based on the cross-attention mechanism includes: an embedding processing layer, an embedding fusion layer and a classification layer; the embedding processing layer is composed of a linear layer and a LeakyReLU activation function, which is used to map the registered voice print embedding, the test voice print embedding and the test voice deception embedding to the same dimension; the embedding fusion layer includes two cross-attention modules, which are used to fuse and splice the registered voice print embedding, the test voice print embedding and the test voice deception embedding; the classification layer is provided with a splicing layer and a fully connected layer, and the input features are gradually reduced in dimension by two linear plus ReLU activation functions before being output.

3. The integrated method of voiceprint recognition and voice deception detection according to claim 1, characterized in that: The voiceprint recognition subsystem extracts the registration voiceprint embedding from the registration voice and extracts the test voice voiceprint embedding from the test voice; the voice deception detection subsystem extracts the test voice deception embedding from the test voice.

4. The integrated method of voiceprint recognition and voice deception detection according to claim 1, characterized in that: The specific operation steps of the back-end fusion classifier based on the cross attention mechanism are: The registered voiceprint embedding, the test voiceprint embedding and the test voice spoof embedding are mapped to the same dimension; the registered voiceprint embedding and the test voiceprint embedding are grouped together to perform feature fusion for the voiceprint recognition task, the registered voiceprint embedding is used as the query vector, and the test voiceprint embedding is used as the key vector and value vector; the weight is obtained by calculating the similarity score between the query and the key, and then the weight is normalized; the registered voiceprint embedding and the test voiceprint embedding are cross-attention fused to obtain the cross-attention feature F ASV ; Cross-attention fusion is performed on the test voiceprint embedding and the test voice spoofing embedding, the test voiceprint embedding is used as the query vector, and the test voice spoofing embedding is used as the key vector and value vector to obtain the cross-attention feature F CM ; The fused cross-attention feature F ASV and the cross attention feature F CM Through the splicing layer, the final comprehensive feature F is obtained fusion ; The comprehensive feature F fusion Input into the classification layer to determine whether the test voice is from a real user.

5. The integrated method of voiceprint recognition and voice deception detection according to claim 4, characterized in that: The specific calculation formula for the similarity score between the query and the key is: Among them, d k It represents the dimension size of the key K, which is used for scaling to avoid the similarity score being too large; Q represents query, K represents key, V represents value, and the superscript T represents the transpose of the matrix.

6. The integrated method of voiceprint recognition and voice deception detection according to claim 4, characterized in that: The cross attention feature F ASV The mathematical expression is: The cross-attention feature F CM The mathematical expression is: The comprehensive feature F fusion The mathematical expression is: F fusion =concat(F ASV ,F CM ) (4) in, Embed the registered voiceprint; To test voiceprint embedding; Embeddings for testing voice spoofing.

7. The integrated method of voiceprint recognition and voice deception detection according to claim 4, characterized in that: The comprehensive feature F fusion The mathematical expression input into the classification layer to determine whether the test speech is a real user process is: Output=OCSoftmax(ReLU(W f F fusion +b f )) (5) Among them, x i represents the embedding vector; y i ∈{0, 1} represents the label of the i-th sample, y i =1 indicates the target class, y i =0 indicates a non-target class; ω0 indicates a unique weight vector, which is the optimization direction for embedding the target class; N indicates the number of mini-batch samples; α indicates a scaling factor; W f represents the weight vector of the fully connected layer; b f represents the bias vector; m is the residual of cosine similarity, which represents the angle between the test speech and the real speech weight vector; It is used to constrain the range of the angle between ω0 and the ith sample eigenvector, m0(y i =0) and m1(y i =1) are two boundary values, m0, m1∈[-1, 1], to achieve the effect of concentrating the target category around ω0 and pushing the non-target data away from ω0.

8. The integrated method of voiceprint recognition and voice deception detection according to claim 1, characterized in that: The specific operation steps of the integrated model based on model-independent meta-learning are: S2.1: Initialization parameters of the model; Decompose the total task T into m small tasks T1, T2, T m-1 ,,...T m , M is the total number of tasks, each small task corresponds to a specific speaker or deception attack type; a specific data set is assigned to each small task, and the data set is divided into a training set and a validation set. The training set is used for the inner loop optimization of the model, and the validation set is used for the global optimization of the outer loop; at the same time, the weights of the integrated model based on model-independent meta-learning are initialized; S2.2: Inner loop updates temporary parameters; For each small task, the integrated model is updated to temporary parameters adapted to the current specific task by performing several gradient descent updates on the training data of the current task; S2.3: Outer loop updates model parameters; Based on the validation set of each small task, the validation loss of the model is calculated and the meta-gradient is extracted from it; the meta-gradient is used to adjust the initialization parameters of the global model and update the global parameters: by calculating the difference between the parameters adapted to the specific task and the initialization parameter φ, the meta-gradients of all small tasks are merged, and the global parameters of the model are updated through the meta-gradient.

9. The integrated method of voiceprint recognition and voice deception detection according to claim 8, characterized in that: The calculation formula for the inner loop updating temporary parameters is: Among them, loss(θ) represents the loss value for parameter θ, and the loss function is the above-mentioned OCSoftmax function. represents the gradient of the loss value with respect to the parameter θ, and β represents the learning rate at the task level.

10. The integrated method of voiceprint recognition and voice deception detection according to claim 8, characterized in that: The outer loop parameter update formula is: Among them, φ represents the initialization parameter of the model, and L(φ) represents the parameter θ that is suitable for all small tasks. i The sum of the loss values ​​of represents the gradient of the loss value L(φ), η represents the meta-learning rate, and θ m Represents the parameters learned in the inner loop of the mth small task.

Images