A voiceprint spoofing defense method, device and computer readable storage medium

Abstract

A method, apparatus, and computer-readable storage medium for voiceprint spoofing defense include: acquiring a speech signal to be verified; extracting multi-dimensional features from the speech signal to obtain a multi-dimensional feature vector, wherein the multi-dimensional feature vector includes at least speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial perturbation features; fusing the multi-dimensional feature vector to obtain a joint feature vector; inputting the joint feature vector into a pre-trained joint risk decision model to obtain a spoofing risk score; and determining the verification result of the speech signal to be verified based on the comparison result of the spoofing risk score and a preset threshold. This application can effectively detect multiple attack modes such as replay attacks, speech synthesis attacks, and adversarial example attacks simultaneously; and accurately detect high-quality spoofing attacks.

Description

Technical Field

[0001] This application relates to the field of voice recognition security technology, specifically to a voiceprint spoofing defense method, device, and computer-readable storage medium. Background Technology

[0002] Voiceprint recognition technology, with its advantages of being contactless and convenient, has been widely used in scenarios such as smart device wake-up, financial payment authentication, security access control, and vehicle system login. With the rapid development of artificial intelligence and the Internet of Things (IoT) technologies, users are placing higher demands on the security and ease of use of identity authentication. As an important branch of biometric identification, voiceprint recognition's overall technical requirements are evolving from simple identity verification to high reliability, interference resistance, and strong anti-spoofing capabilities. How to effectively improve the robustness of voiceprint recognition systems against various deception attacks while ensuring user experience is a key problem that urgently needs to be solved in this field.

[0003] Currently, spoofing attacks against voiceprint recognition systems mainly include replay attacks, speech synthesis (speech cloning) attacks, and adversarial example attacks. To counter these attacks, existing technologies have proposed several defense schemes. For example, some employ liveness detection (such as detecting breathing sounds, channel noise, or phase distortion in audio) to determine if the voice is genuine; others require users to read random numbers or specific phrases to increase the difficulty for attackers to obtain valid speech; and still others use feature extraction and binary classifiers (such as those based on Gaussian mixture models or convolutional neural networks) to distinguish between genuine and fake voices based on specific acoustic features (such as constant Q-transform and linear prediction residuals).

[0004] However, existing defense solutions have significant shortcomings in practical applications. Most current technologies are designed for single attack types, such as liveness detection primarily targeting replay attacks. However, their protective performance drops sharply when facing other types or mixed attacks. Secondly, with the development of voiceprint spoofing technology, single-type protection methods are insufficient to defend against high-quality attack samples and struggle to detect them accurately. Therefore, there is an urgent need for a unified decision-making mechanism that can effectively resist multiple spoofing methods such as replay, synthetic, and adversarial samples, improving the overall defense capability against voiceprint spoofing and the detection accuracy of high-quality attack samples. Summary of the Invention

[0005] This application provides a voiceprint spoofing defense method, device, and computer-readable storage medium, which can simultaneously and effectively detect multiple types of voiceprint spoofing, and has strong robustness when detecting high-quality attack samples. It can identify high-quality attack samples and improve protection performance.

[0006] In a first aspect, embodiments of this application provide a method for preventing voiceprint spoofing, including: Acquire the voice signal to be verified; Multi-dimensional feature extraction is performed on the speech signal to be verified to obtain a multi-dimensional feature vector. The multi-dimensional feature vector includes at least the speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial perturbation features. Multidimensional feature vectors are fused to obtain joint feature vectors. The joint feature vectors are then input into a pre-trained joint risk decision model to obtain a deception risk score. The verification result of the voice signal to be verified is determined based on the comparison between the deception risk score and the preset threshold.

[0007] In conjunction with the first aspect, in one implementation method, the process of extracting identity features includes: Acoustic feature extraction is performed on the speech signal to be verified to obtain an acoustic feature sequence; The acoustic feature sequence is input into the pre-trained voiceprint recognition model, and the acoustic feature sequence is pooled to aggregate the features, resulting in a fixed-dimensional identity embedding vector, which is used as the output identity feature. Pooling operations include statistical pooling or self-attention pooling.

[0008] In conjunction with the first aspect, in one implementation method, the process of extracting liveness features includes: Analyze and extract the sound source spatial data and / or physiological feature signals of the speech signal to be verified; Live body characteristics are obtained based on spatial data of the sound source and / or physiological characteristic signals.

[0009] In conjunction with the first aspect, in one implementation, the spatial attributes of the sound source include the direction of arrival and distance of the sound source determined based on a multi-microphone array, and / or chest vibration features extracted based on the micro-Doppler effect. Physiological characteristic signals include respiratory rhythm pauses, and / or physiological structural vibration harmonics, and / or physiological modulation characteristics.

[0010] In conjunction with the first aspect, in one implementation, the process of extracting anti-perturbation features includes: The speech signal to be verified is subjected to time-frequency transformation processing to obtain a time-frequency spectrum; The time-frequency spectrum is input into a pre-trained adversarial perturbation detection network, and the probability of adversarial perturbation existence is output as an adversarial perturbation feature. Among them, the pre-trained adversarial perturbation detection network is a binary classification model built on a convolutional neural network, and is trained on a training set containing real-person audio samples and adversarial sample speech.

[0011] In conjunction with the first aspect, in one implementation, the multidimensional feature vectors are fused to obtain a joint feature vector, including: The importance weights of each dimension of the feature vector in the multidimensional feature vector are determined based on the attention mechanism. The multidimensional feature vectors are weighted and fused according to their importance to obtain a joint feature vector.

[0012] In conjunction with the first aspect, in one implementation, the verification result of the voice signal to be verified is determined based on a comparison between a deception risk score and a preset threshold, including: Obtain the security level of the current application scenario and determine the corresponding preset threshold based on the security level; If the deception risk score is greater than or equal to the preset threshold, the voice signal to be verified will be identified as a deception attack.

[0013] In conjunction with the first aspect, in one implementation, the voiceprint spoofing defense method further includes: If the verification results indicate that the obtained speech signal to be verified is a deception attack, then the speech signal to be verified will be anonymized and used as a negative sample to train and optimize the joint risk decision model.

[0014] Secondly, embodiments of this application provide a voiceprint spoofing defense device, comprising: The audio acquisition module is used to acquire the voice signal to be verified. The feature extraction module is used to extract multi-dimensional features from the speech signal to be verified, and obtain a multi-dimensional feature vector. The multi-dimensional feature vector includes at least the speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial features. The risk decision module is used to fuse multi-dimensional feature vectors to obtain a joint feature vector. The joint feature vector is then input into a pre-trained joint risk decision model to obtain a deception risk score. The decision execution module is used to determine the verification result of the voice signal to be verified based on the comparison between the deception risk score and the preset threshold.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a voiceprint spoofing defense program, wherein when the voiceprint spoofing defense program is executed by a processor, it implements the steps of the voiceprint spoofing defense method described above.

[0016] The beneficial effects of the technical solutions provided in this application include: This application analyzes multi-dimensional features, including content features, identity features, audio quality features, liveness features, and adversarial perturbation features, and uses a joint decision model for comprehensive risk assessment to effectively detect multiple attack modes such as replay attacks, speech synthesis attacks, and adversarial sample attacks. At the same time, the joint decision method of multi-dimensional features can capture forgery traces that are difficult to distinguish with a single feature, thereby improving the detection accuracy of high-quality deception attacks. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the voiceprint spoofing defense method of this application; Figure 2 This is a schematic diagram of the process for extracting identity features according to an embodiment of this application; Figure 3 This is a schematic diagram of the process for extracting live features according to an embodiment of this application; Figure 4 This is a schematic diagram of the process for extracting anti-disturbance features according to an embodiment of this application; Figure 5 This is a schematic diagram of the multidimensional feature vector fusion process in an embodiment of this application; Figure 6 This is a flowchart illustrating the process of determining the verification result in an embodiment of this application; Figure 7 This is a schematic diagram of the functional modules of an embodiment of the voiceprint deception defense device of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0020] In a first aspect, embodiments of this application provide a method for preventing voiceprint spoofing.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a schematic flowchart illustrating an embodiment of the voiceprint spoofing defense method of this application. Figure 1 As shown, voiceprint spoofing defense methods include: S101. Obtain the voice signal to be verified; S102. Perform multi-dimensional feature extraction on the speech signal to be verified to obtain a multi-dimensional feature vector. The multi-dimensional feature vector includes at least the speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial perturbation features. S103. The multi-dimensional feature vectors are fused to obtain a joint feature vector. The joint feature vector is then input into a pre-trained joint risk decision model to obtain a deception risk score. S104. Based on the comparison between the deception risk score and the preset threshold, determine the verification result of the voice signal to be verified.

[0022] Specifically, to achieve multi-dimensional decision analysis against voiceprint spoofing attacks, the embodiment extracts voice text content features, identity features, sound quality features, liveness features, and adversarial perturbation features for the voice signal to be verified. Voice text content features are used to identify the text content in the voice for matching verification against expected commands (such as fixed wake words or random data), verifying whether the voice content meets expectations. Identity features are used to extract embedding vectors representing the speaker's identity through a voiceprint recognition model to verify identity. Sound quality features analyze and extract spectral characteristics, harmonic structure, background noise patterns, and encoding compression traces from the voice signal. These are used to detect device noise and environmental reverberation differences caused by recording and playback, as well as unnatural formants or spectral discontinuities that may be introduced by speech synthesis. Liveness features are extracted by extracting features related to the physiological processes of human speech, such as: analyzing the direction of arrival (DOA) and distance of the sound source through a multi-microphone array to determine whether the sound source is a near-field human; and analyzing whether there are natural breathing pauses or weak laryngeal or nasal vibration harmonics in the voice. Adversarial perturbation features are achieved by using a pre-trained "adversarial perturbation detection network," which learns from a large number of normal and adversarial speech samples and focuses on detecting whether there are specific spectral or temporal perturbation patterns in the speech signal that are artificially added to mislead the speaker model.

[0023] After extracting the above five types of features, the embodiment fuses the extracted feature vectors (e.g., by concatenation or weighted fusion based on an attention mechanism) and inputs them into a joint risk decision model. This model is a classifier based on deep neural networks or ensemble learning, which has been trained on a dataset containing massive amounts of positive samples (real human speech) and negative samples (various types of replay, synthetic, and adversarial deception speech). The model outputs a comprehensive deception risk score.

[0024] Then, based on the obtained deception risk score, the embodiment determines the verification result of the voice signal to be verified according to the preset dynamic threshold corresponding to the security level of the current application scenario (e.g., device wake-up is low risk, payment transfer is high risk).

[0025] In this embodiment, multi-dimensional features such as content features, identity features, sound quality features, liveness features, and adversarial perturbation features are extracted and analyzed. A joint decision model is used for comprehensive risk assessment, which enables effective detection of multiple attack modes such as replay attacks, speech synthesis attacks, and adversarial sample attacks. At the same time, the joint decision method of multi-dimensional features can capture forgery traces that are difficult to distinguish with a single feature, thereby improving the detection accuracy of high-quality deception attacks.

[0026] Furthermore, in one embodiment, Figure 2 This is a schematic diagram of the process for extracting identity features according to an embodiment of this application, such as... Figure 2 As shown, the process of extracting identity features includes: S201. Perform acoustic feature extraction on the speech signal to be verified to obtain an acoustic feature sequence; S202. Input the acoustic feature sequence into the pre-trained voiceprint recognition model, perform pooling operation on the acoustic feature sequence to aggregate features, and obtain a fixed-dimensional identity embedding vector as the output identity feature. Pooling operations include statistical pooling or self-attention pooling.

[0027] Specifically, for extracting identity features from the speech signal to be verified, the embodiment first preprocesses the speech and extracts acoustic features (such as MFCC Mel-frequency cepstral coefficients or FBank Mel filter bank coefficients) to obtain an acoustic feature sequence. Then, the acoustic feature sequence is input into a pre-trained voiceprint recognition model (such as a model based on x-vector, d-vector, ECAPA-TDNN architecture). Through pooling layers, the variable-length feature sequence is aggregated into a fixed-dimensional embedding vector, which serves as the output identity feature. The pooling layer can employ statistical pooling or self-attention pooling. This embedding vector, as the speaker's identity feature, can be used to compare its similarity with the voiceprint embedding of a registered target user (e.g., calculating cosine similarity) to verify identity.

[0028] Furthermore, in one embodiment, Figure 3 This is a schematic diagram of the process for extracting live features according to an embodiment of this application, as shown below. Figure 3 As shown, the process of extracting liveness features includes: S301. Analyze and extract the sound source spatial data and / or physiological feature signals of the speech signal to be verified; S302. Obtain living characteristics based on sound source spatial data and / or physiological characteristic signals.

[0029] Furthermore, in one embodiment, the spatial attributes of the sound source include the direction of arrival and distance of the sound source determined based on a multi-microphone array, and / or chest vibration features extracted based on the micro-Doppler effect; Physiological characteristic signals include respiratory rhythm pauses, and / or physiological structural vibration harmonics, and / or physiological modulation characteristics.

[0030] Specifically, liveness features refer to features related to the physiological processes and physical presence of authentic human vocalization. In this embodiment, liveness features are obtained by extracting the sound source spatial data and / or physiological feature signals of the speech signal to be verified.

[0031] Among them, the sound source spatial data calculates the direction of arrival (DOA) and distance of the sound source by using the phase difference and intensity difference of the signals received by multiple microphones, analyzes whether these parameters change dynamically during the pronunciation process (such as micro-movement of a real person's head), or whether they are a fixed-point sound source (such as a loudspeaker); and detects whether there is a micro-Doppler effect caused by chest cavity vibration when a real person speaks.

[0032] Physiological feature signals are used to analyze whether there are natural pauses in the speech signal that are related to the breathing rhythm, and to detect whether there are weak harmonics or specific modulation features generated by the vibration of physiological structures such as the larynx and nasal cavity.

[0033] The embodiment can analyze and determine whether the voice signal to be verified comes from a living person in a real physical space who is breathing and using vocal organs, rather than a recorded or synthesized voice source, by identifying and extracting the sound source spatial data and / or physiological feature signals of the voice signal to be verified.

[0034] Furthermore, in one embodiment, Figure 4 This is a schematic diagram of the process for extracting anti-disturbance features according to an embodiment of this application, as shown below. Figure 4 As shown, the process of extracting adversarial features includes: S401. Perform time-frequency transformation processing on the speech signal to be verified to obtain a time-frequency spectrum; S402. Input the time-frequency spectrum into the pre-trained adversarial perturbation detection network and output the probability of adversarial perturbation existence as an adversarial perturbation feature. Among them, the pre-trained adversarial perturbation detection network is a binary classification model built on a convolutional neural network, and is trained on a training set containing real-person audio samples and adversarial sample speech.

[0035] Specifically, for adversarial perturbation features, the implementation uses a pre-trained adversarial perturbation detection network to identify and extract them. The core idea is to distinguish between "clean speech" and "speech subjected to adversarial perturbations." These perturbations are imperceptible to the human ear but are specifically designed to deceive voiceprint recognition models.

[0036] In voiceprint spoofing attacks, adversarial perturbations are not random noise, but subtle signals with specific "direction" and "patterns". Therefore, adversarial perturbation detection networks can learn the statistical regularities or feature representations of this "adversarial pattern" and thus identify whether adversarial perturbations have been applied to the speech signal.

[0037] In the training of the adversarial perturbation detection network, positive samples consist of a large number of clean human recordings, while negative samples are generated by attacking the positive samples using adversarial perturbation algorithms. In addition, to enhance generalization ability, the training data includes adversarial samples generated using various attack algorithms and different perturbation budgets, and may also include hybrid attack samples that further perturb the replayed or synthesized speech.

[0038] Adversarial perturbation detection networks are typically binary classification deep neural networks. Their architecture is designed to capture subtle traces of adversarial perturbations in the time-frequency domain. In this embodiment, the model architecture consists of an input layer, a feature extraction backbone network, and a classification head.

[0039] The input layer receives a time-frequency representation of the speech, typically a Mel spectrogram or a short-time Fourier transform amplitude spectrum. This is because adversarial perturbations are often easier for the model to capture in the spectral domain than in the original waveform. Therefore, the speech signal to be verified is processed by time-frequency transformation before being input into the adversarial perturbation detection network.

[0040] The feature extraction backbone network consists of multiple stacked convolutional and pooling layers. Its shallow convolutional kernels learn low-level features such as edges and textures (corresponding to local spectral discontinuities introduced by perturbations); deep convolutional kernels learn more abstract, global patterns (corresponding to specific perturbation distributions generated by a certain attack algorithm). In addition, to utilize the relationship between the time and frequency domains simultaneously, 2D convolutions (sliding in both time and frequency dimensions) can be used, or an attention mechanism can be introduced to focus the network on the regions most significantly affected by perturbations.

[0041] The classification head flattens the feature maps extracted by the convolutional layers and passes them through one or more fully connected layers. Finally, it passes through a sigmoid or softmax output layer to produce a scalar value.

[0042] Finally, the adversarial perturbation detection network outputs a probability value or confidence score between 0 and 1. This output value can be directly used as the numerical value of the "adversarial perturbation detection feature" and input into the subsequent joint risk decision model.

[0043] In addition, for the characteristics of speech text content, the embodiment uses an automatic speech recognition model (such as a model based on CTC, RNN-T or Transformer) to identify the text content in the speech signal to be verified, and matches the identified text with the system's expected instructions (such as a preset wake word or a random string of numbers issued by the system) to verify whether the speech content meets expectations.

[0044] Regarding sound quality characteristics, the embodiment uses signal processing and analysis to detect physical traces and distortions introduced into the recording, transmission, and playback of speech, specifically including: Spectral characteristics and harmonic structure: Analyze the Mel spectrum and linear spectrum to check the continuity, naturalness, and harmonic structure of formants. Synthesized speech may produce unnatural formant transitions or spectral discontinuities.

[0045] Background noise pattern: Analyze the noise spectrum of non-speech segments to detect the presence of inherent electronic noise of specific devices (such as speaker noise floor) and whether the environmental reverberation characteristics are abnormal or mismatched.

[0046] Decoding compression traces: Analyze whether there are specific spectral or temporal traces in the speech caused by audio encoding and decoding (such as MP3, AAC), such as quantization distortion, pre-echo.

[0047] In the process of extracting sound quality features, the above methods can be used to distinguish between the original human voice and the voice processed by recording and playback equipment, speech synthesizers or network transmission channels, and non-natural human voice features or equipment / environment traces can be identified.

[0048] In this embodiment, by analyzing and extracting the above five types of features, the characteristics of voiceprint spoofing attacks are captured at each level, achieving effective detection of multiple attack modes. At the same time, it can capture forgery traces that are difficult to distinguish with a single feature, thereby improving the detection accuracy of high-quality spoofing attacks.

[0049] Furthermore, in one embodiment, Figure 5 This is a schematic diagram of the multi-dimensional feature vector fusion process in an embodiment of this application, as shown below. Figure 5 As shown, the multi-dimensional feature vectors are fused to obtain a joint feature vector, including: S501. Determine the importance weights of each dimension of the feature vector in the multidimensional feature vector based on the attention mechanism; S502. The multi-dimensional feature vectors are weighted and fused according to their importance to obtain a joint feature vector.

[0050] Specifically, after extracting multi-dimensional features through various methods, feature fusion is required to vectorize the features of each dimension. Common feature fusion methods include feature concatenation and weighted fusion based on attention mechanisms.

[0051] Feature concatenation involves aligning feature vectors and then linking them end-to-end along the feature dimension. Its advantages are simplicity and directness, and the ability to retain all information. However, its disadvantages include treating all features as equally important, and the potential for "curse of dimensionality" and redundancy in high-dimensional concatenation, requiring subsequent models to have sufficient capacity to handle these issues.

[0052] Weighted fusion based on an attention mechanism can dynamically evaluate the importance of different features to the current sample decision and perform weighted combination. Considering that the importance of each feature to the voiceprint spoofing decision varies under different attack modes, this embodiment adopts weighted fusion based on an attention mechanism.

[0053] First, the embodiment maps five heterogeneous feature vectors to a shared semantic space through their respective independent fully connected layers (or linear projection layers), giving them the same dimensionality. Then, by calculating a context vector (which can be the average of all features or a learnable global context), a relevance score to the context is calculated for each feature. The softmax function is then used to transform the scores into normalized attention weights, which represent the importance of that type of feature for the final fraud risk assessment in the current sample. Finally, the embodiment uses the calculated attention weights to perform a weighted summation of the projected features to obtain the final fused feature vector.

[0054] Next, the embodiment inputs the obtained fused feature vector into the joint risk decision model. The joint risk decision model is a binary classifier that performs linear transformation, batch normalization and activation function through several fully connected layers. Finally, it is mapped to a scalar through the output layer and compressed into the (0,1) interval as the deception risk score of the prediction output.

[0055] In this embodiment, by using attention-based weighted fusion, the weights of each feature can be automatically adjusted based on the possible attack types of the sample, thereby improving the accuracy of risk judgment.

[0056] Furthermore, in one embodiment, Figure 6 This is a schematic diagram of the process for determining the verification result in an embodiment of this application, as shown below. Figure 6 As shown, based on the comparison between the deception risk score and the preset threshold, the verification result of the voice signal to be verified is determined, including: S601. Obtain the security level of the current application scenario and determine the corresponding preset threshold based on the security level; S602. If the deception risk score is greater than or equal to the preset threshold, the voice signal to be verified will be determined as a deception attack.

[0057] Specifically, when determining the final verification result based on the deception risk score, the embodiment sets up an adaptive decision-making mechanism based on security level. That is, by obtaining the security level of the current application scenario, different preset risk thresholds are adaptively set according to the current scenario's security level. For example, in a low-risk scenario like device wake-up, the threshold is appropriately increased to avoid false positives; in a high-risk scenario like payment transfers, the threshold is moderately decreased to improve the sensitivity to risk response. Based on the determined dynamic preset thresholds, when the deception risk score is less than the threshold, it is determined to be genuine human voice, and verification is passed; when the deception score is greater than or equal to the threshold, it is determined to be a deception attack, and the verification is rejected.

[0058] In this embodiment, by using a dynamic threshold based on the scene security level, voiceprint spoofing verification can achieve sensitive identification of spoofing attacks in high-risk scenarios to ensure security, while also avoiding misjudgments that could affect user experience in low-risk scenarios.

[0059] Furthermore, in one embodiment, the voiceprint spoofing defense method further includes: If the verification results indicate that the obtained speech signal to be verified is a deception attack, then the speech signal to be verified will be anonymized and used as a negative sample to train and optimize the joint risk decision model.

[0060] Specifically, to achieve continuous optimization of the model, in this embodiment, with user authorization and privacy protection, voice samples determined to be high-risk can be anonymized and uploaded to the cloud analysis center. The cloud analysis center regularly trains and optimizes the joint risk judgment model and adversarial disturbance detection network based on new samples, and distributes the updated model parameters to terminal devices, forming a dynamically evolving defense capability.

[0061] In summary, this application analyzes multiple dimensions of features, including content features, identity features, audio quality features, liveness features, and adversarial perturbation features, and uses a joint decision model for comprehensive risk assessment to effectively detect multiple attack modes such as replay attacks, speech synthesis attacks, and adversarial sample attacks. Furthermore, the joint decision method based on multiple dimensions of features can capture forgery traces that are difficult to distinguish with a single feature, thereby improving the detection accuracy of high-quality deception attacks.

[0062] Secondly, embodiments of this application also provide a voiceprint deception defense device.

[0063] In one embodiment, reference is made to Figure 7 , Figure 7 This is a schematic diagram of the functional modules of an embodiment of the voiceprint spoofing defense device of this application. Figure 7 As shown, the voiceprint spoofing defense device includes: The audio acquisition module 701 is used to acquire the voice signal to be verified. The feature extraction module 702 is used to extract multi-dimensional features from the speech signal to be verified to obtain a multi-dimensional feature vector. The multi-dimensional feature vector includes at least speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial features. The risk decision module 703 is used to fuse multi-dimensional feature vectors to obtain joint feature vectors. The joint feature vectors are then input into a pre-trained joint risk decision model to obtain a deception risk score. The decision execution module 704 is used to determine the verification result of the voice signal to be verified based on the comparison result between the deception risk score and the preset threshold.

[0064] The functions of each module in the aforementioned voiceprint deception defense device correspond to the steps in the aforementioned voiceprint deception defense method embodiment, and their functions and implementation processes will not be described in detail here.

[0065] Thirdly, embodiments of this application also provide a computer-readable storage medium.

[0066] The present application provides a computer-readable storage medium storing a voiceprint spoofing defense program, wherein when the voiceprint spoofing defense program is executed by a processor, it implements the steps of the voiceprint spoofing defense method described above.

[0067] The method implemented when the voiceprint spoofing defense program is executed can be referred to in various embodiments of the voiceprint spoofing defense method of this application, and will not be repeated here.

[0068] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0069] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0070] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0071] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0072] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods of the various embodiments of this application.

[0074] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for preventing voiceprint spoofing, characterized in that, include: Acquire the voice signal to be verified; Multi-dimensional feature extraction is performed on the speech signal to be verified to obtain a multi-dimensional feature vector, wherein the multi-dimensional feature vector includes at least speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and adversarial features. The multidimensional feature vectors are fused to obtain a joint feature vector, which is then input into a pre-trained joint risk decision model to obtain a deception risk score. The verification result of the voice signal to be verified is determined based on the comparison between the deception risk score and the preset threshold.

2. The voiceprint spoofing defense method according to claim 1, characterized in that, The process of extracting the identity features includes: Acoustic feature extraction is performed on the speech signal to be verified to obtain an acoustic feature sequence; The acoustic feature sequence is input into a pre-trained voiceprint recognition model, and the acoustic feature sequence is pooled to aggregate features, resulting in a fixed-dimensional identity embedding vector, which is used as the output identity feature. The pooling operation includes statistical pooling or self-attention pooling.

3. The voiceprint spoofing defense method according to claim 1, characterized in that, The extraction process of the liveness feature includes: Analyze and extract the sound source spatial data and / or physiological feature signals of the speech signal to be verified; Live body characteristics are obtained based on the spatial data of the sound source and / or the physiological characteristic signals.

4. The voiceprint spoofing defense method according to claim 3, characterized in that, The spatial attributes of the sound source include the direction of arrival and distance of the sound source determined based on a multi-microphone array, and / or the thoracic vibration characteristics extracted based on the micro-Doppler effect; The physiological characteristic signals include respiratory rhythm pauses, and / or physiological structural vibration harmonics, and / or physiological modulation characteristics.

5. The voiceprint spoofing defense method according to claim 1, characterized in that, The extraction process of the anti-perturbation features includes: The speech signal to be verified is subjected to time-frequency transformation processing to obtain a time-frequency spectrum; The time-frequency spectrum is input into a pre-trained adversarial perturbation detection network, and the probability of adversarial perturbation existence is output as an adversarial perturbation feature. The pre-trained adversarial perturbation detection network is a binary classification model built on a convolutional neural network and trained on a training set containing real-person audio samples and adversarial sample speech.

6. The voiceprint spoofing defense method according to claim 1, characterized in that, The process of fusing the multidimensional feature vectors to obtain a joint feature vector includes: The importance weights of each dimension of the feature vector in the multidimensional feature vector are determined based on the attention mechanism. The multidimensional feature vectors are weighted and fused according to the importance weights to obtain a joint feature vector.

7. The voiceprint spoofing defense method according to claim 1, characterized in that, The step of determining the verification result of the voice signal to be verified based on the comparison result of the deception risk score and the preset threshold includes: Obtain the security level of the current application scenario, and determine the corresponding preset threshold based on the security level; If the deception risk score is greater than or equal to the preset threshold, the voice signal to be verified is determined to be a deception attack.

8. The voiceprint spoofing defense method according to claim 1, characterized in that, The method further includes: If the verification result indicates that the obtained voice signal to be verified is a deception attack, then the voice signal to be verified is anonymized and used as a negative sample to train and optimize the joint risk decision model.

9. A voiceprint deception defense device, characterized in that, include: The audio acquisition module is used to acquire the voice signal to be verified. The feature extraction module is used to extract multi-dimensional features from the speech signal to be verified to obtain a multi-dimensional feature vector. The multi-dimensional feature vector includes at least speech text content features, identity features, sound quality features representing recording or synthesis traces, liveness features representing the physiological characteristics of the sound source, and anti-perturbation features. The risk decision module is used to fuse the multi-dimensional feature vectors to obtain a joint feature vector, and input the joint feature vector into a pre-trained joint risk decision model to obtain a deception risk score. The decision execution module is used to determine the verification result of the voice signal to be verified based on the comparison result between the deception risk score and the preset threshold.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a voiceprint spoofing defense program, wherein when the voiceprint spoofing defense program is executed by a processor, it implements the steps of the voiceprint spoofing defense method as described in any one of claims 1 to 8.

Images