Method and system for detecting forged voice based on voiceprint driving
Through the joint learning of the voiceprint recognition model and the forged speech detection model, the multi-head attention mechanism and joint optimization algorithm are used to improve the recognition accuracy and robustness of the forged speech detection system, and solve the detection accuracy and security problems of the forged speech detection method in the existing technology in complex environments.
Patent Information
- Application Number
- CN202510827712.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-19
- Publication Date
- 2025-08-08
AI Technical Summary
The existing forged speech detection methods are not robust enough when facing complex background noise and unknown forged models, and fail to effectively use the speaker's identity characteristics for discrimination, resulting in a decrease in detection accuracy and security.
By constructing a joint learning between the voiceprint recognition model and the forged speech detection model, the speaker's identity characteristics are extracted and fused with the spectrum characteristics, the voiceprint embedding vector is generated using the multi-head attention mechanism, and GE2ELoss and OC-Softmax Loss are used for joint optimization.
It improves the recognition accuracy and robustness of the forged voice detection system in complex environments, can effectively distinguish between real and forged voice, and improves the system's security and detection capabilities.
Smart Images
Figure CN120452455A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence voice security technology, and more particularly to a voiceprint-driven forged voice detection method and system. Background Art
[0002] Today, speech recognition and speech synthesis systems are widely used in a variety of real-world scenarios, such as voice assistants, identity authentication, and remote command interaction, becoming key modules in artificial intelligence systems. At the same time, speech forgery technology has rapidly developed in recent years. Forged speech synthesized using deep learning models has achieved a high degree of realism, effectively deceiving speech recognition systems and the human auditory system, posing a serious threat to the security of speech systems in real-world applications.
[0003] Spoofed speech is typically generated using text-to-speech or voice conversion methods. These forged samples have the spectral distribution and auditory characteristics of real speech, making them difficult for standard detection systems to distinguish using the human ear or simple algorithms. Attackers can use synthesis techniques to simulate the voice of a specific speaker, achieving cross-speaker impersonation without the need for original audio material, allowing them to conduct illegal activities such as impersonation attacks or identity fraud.
[0004] Most existing forged speech detection methods rely on spectral domain statistical features, such as Linear Frequency Cepstral Coefficients (LFCC) and Constant Q Cepstral Coefficients (CQCC). These methods use deep neural network classifiers to perform binary classification between real and forged speech. However, these methods commonly suffer from the following issues: First, spectral statistical features are significantly affected by non-speaker factors such as background noise, playback equipment, and compression formats, resulting in limited generalization of the detection model. Second, some detection models become less robust when attackers add real-world backgrounds or use unknown forged models, making them susceptible to circumvention.
[0005] In addition, existing methods generally ignore the role of speaker identity features in distinguishing the authenticity of speech and fail to establish a forgery detection mechanism from the speaker dimension. The voiceprint recognition model has the ability to extract speaker identity embedded features, can effectively distinguish the differences in speech expressions of different individuals, and has a high ability to distinguish the structure of speech spectrum. If the voiceprint model and the forgery detection model can be jointly constructed so that the speaker identity information is involved in the authenticity judgment, it will be expected to improve the detection system's discrimination effect and system robustness under complex speech synthesis attacks. Therefore, proposing a voiceprint-driven forgery voice detection method and system to improve the overall recognition accuracy and security robustness of the forgery voice detection system is a problem that those skilled in the art urgently need to solve. Summary of the Invention
[0006] In view of this, the present invention provides a voiceprint-driven forged speech detection method and system, which improves the detection system's recognition ability and robustness for forged speech samples by extracting speaker identity features and fusing them with spectral features for modeling.
[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting forged voice based on voiceprint drive, comprising:
[0008] Step 1: Input voice information, pre-process the voice information, and extract the LFCC features of the voice information;
[0009] Step 2: Build a voiceprint recognition model, extract spectrum distribution features of the LFCC features, and generate a voiceprint embedding vector;
[0010] Step 3: Construct a forged voice detection model, fuse the LFCC features with the voiceprint embedding vector according to a preset ratio, input the features into the forged voice detection model for recognition, and perform true and false voice classification;
[0011] Step 4: Optimize the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.
[0012] Preferably, the process of pre-processing the voice information in step 1 includes:
[0013] All speech information is unified to the same length through padding operation, and the speech information is frame divided and windowed to divide the continuous speech signal into overlapping short frames, and LFCC features are extracted for each short frame speech signal.
[0014] Preferably, the voiceprint recognition model is trained using real voice information, and the forged voice detection model is trained using both real and forged voice information, and the two model parameters are jointly optimized through an algorithmic process.
[0015] Preferably, the voiceprint recognition model includes a basic network structure, a multi-head attention mechanism and feature aggregation pooling;
[0016] The basic network structure is used to extract spectral distribution features of the LFCC features and output the spectral features of each frame. The attention weights are generated through a multi-head attention mechanism and combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector.
[0017] Preferably, the method for calculating the voiceprint embedding vector includes:
[0018] First calculate the attention weight a corresponding to the nth frame n , the formula is as follows:
[0019]
[0020] in, is the voiceprint feature extracted by the voiceprint recognition model, W1 and W2 are the parameter matrices of the multi-head attention mechanism, g() is the ReLU activation function, softmax is the normalization function, and the attention weight a n ∈[0,1];
[0021] The attention weight is combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector, which is L2 normalized to obtain the voiceprint embedding vector e, as shown in the following formula:
[0022]
[0023] Among them, ||·||2 is the L2 normalization processing function.
[0024] Preferably, the LFCC feature is fused with the voiceprint embedding vector according to a preset ratio, and the formula is as follows:
[0025] h l =concat(h n ,h d )
[0026] Among them, h l is the multi-layer feature representation after fusion, h n is the spectrum feature after being processed by the voiceprint recognition model, h d It is the output of LFCC features after passing through the basic network structure.
[0027] Preferably, the multi-objective joint learning method uses GE2ELoss and OC-Softmax Loss for comprehensive training, and the joint loss function formula is as follows:
[0028] L=αL spk +(1-α)L spoof
[0029] Among them, α is the weight coefficient, L spk The voiceprint recognition loss calculated by GE2ELoss loss, L spoof Loss calculated using OC-Softmax loss for the fake speech detection model.
[0030] Preferably, the L spoof The calculation formula is as follows:
[0031]
[0032] Among them, x i ∈R D is the embedding vector of the i-th sample, R Dis the D-dimensional real vector space where the voiceprint embedding vector e is located, y i ∈{0,1} is its category label, w0 is the weight vector in the target category direction, and a and m are the hyperparameters in OC-Softmax.
[0033] Preferably, a voiceprint-driven forged voice detection system is characterized by comprising:
[0034] Extraction module: inputs voice information, preprocesses the voice information, and extracts the LFCC features of the voice information;
[0035] Voiceprint recognition module: builds a voiceprint recognition model, extracts spectral distribution features of the LFCC features, and generates a voiceprint embedding vector;
[0036] Forged voice detection module: Constructs a forged voice detection model, fuses the LFCC features with the voiceprint embedding vector according to a preset ratio, and inputs the LFCC features into the forged voice detection model for recognition and classification of true and false voices;
[0037] Optimization module: optimizes the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.
[0038] The above technical solution demonstrates that, compared to existing technologies, the present invention provides a voiceprint-driven forged speech detection method and system. This system leverages a multi-head attention mechanism to enhance embedded representation capabilities, fuses features to inject discriminative information into a multi-layer network structure, and utilizes a joint training strategy to enhance the collaborative optimization of the performance of the two sub-models. The system maintains good detection accuracy in both real-world and forged mixed samples and in multi-source noise environments, demonstrating its practical deployment value and promising engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0040] Figure 1 This is a flow chart of a forged voice detection method based on voiceprint drive provided by the present invention. DETAILED DESCRIPTION
[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0042] See also Figure 1 The embodiment of the present invention discloses a method for detecting forged voice based on voiceprint drive, comprising:
[0043] Step 1: Input voice information, pre-process the voice information, and extract the LFCC features of the voice information;
[0044] Step 2: Build a voiceprint recognition model, extract spectrum distribution features of the LFCC features, and generate a voiceprint embedding vector;
[0045] Step 3: Construct a forged voice detection model, fuse the LFCC features with the voiceprint embedding vector according to a preset ratio, input the features into the forged voice detection model for recognition, and perform true and false voice classification;
[0046] Step 4: Optimize the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.
[0047] Specifically, the process of preprocessing the voice information in step 1 includes:
[0048] All speech information is unified to the same length through padding operation, and the speech information is frame divided and windowed to divide the continuous speech signal into overlapping short frames, and LFCC features are extracted for each short frame speech signal.
[0049] In one specific embodiment of the present invention, the input speech information is preprocessed to extract LFCC features. The input speech is sampled with a 20ms frame length and a 10ms frame shift. A linear filter bank is used to extract 60-dimensional Linear Frequency Cepstral Coefficient (LFCC) features as the spectral representation of the input speech. All audio samples are uniformly padded to a fixed length to accommodate the batch processing requirements of the neural network.
[0050] Specifically, the voiceprint recognition model is trained using real voice information, and the forged voice detection model is trained using both real and forged voice information, and the two model parameters are jointly optimized through an algorithm process.
[0051] In a specific embodiment of the present invention, the voiceprint recognition model uses the Adam optimizer during training, setting parameters β1=0.9, β2=0.999, and the initial learning rate is 0.001; the classification module in the forgery detection model uses the SGD optimizer, with an initial learning rate of 0.0003, and the cosine annealing algorithm is used to dynamically adjust the learning rate during training.
[0052] Specifically, the voiceprint recognition model includes a basic network structure, a multi-head attention mechanism, and feature aggregation pooling;
[0053] The basic network structure is used to extract spectral distribution features of the LFCC features and output the spectral features of each frame. The attention weights are generated through a multi-head attention mechanism and combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector.
[0054] Specifically, the method for calculating the voiceprint embedding vector includes:
[0055] First calculate the attention weight a corresponding to the nth frame n , the formula is as follows:
[0056]
[0057] in, is the voiceprint feature extracted by the voiceprint recognition model, W1 and W2 are the parameter matrices of the multi-head attention mechanism, g() is the ReLU activation function, softmax is the normalization function, and the attention weight a n ∈[0,1];
[0058] The attention weight is combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector, which is L2 normalized to obtain the voiceprint embedding vector e, as shown in the following formula:
[0059]
[0060] Among them, ||·||2 is the L2 normalization processing function.
[0061] In a specific embodiment of the present invention, the LFCC features extracted in step 1 are input into the voiceprint recognition model. The voiceprint recognition model uses a three-layer LSTM structure or a ResNet18 structure to extract the voiceprint features of the Nth frame, which is recorded as Calculate the attention weight a through the multi-head attention mechanism n , and use weighted average method to generate voiceprint embedding representation e,
[0062] Specifically, the LFCC feature is fused with the voiceprint embedding vector according to a preset ratio, and the formula is as follows:
[0063] h l=concat(h n ,h d )
[0064] Among them, h l is the multi-layer feature representation after fusion, h n is the spectrum feature after being processed by the voiceprint recognition model, h d It is the output of LFCC features after passing through the basic network structure.
[0065] Specifically, the multi-objective joint learning method uses GE2ELoss and OC-Softmax Loss for comprehensive training, and the joint loss function formula is as follows:
[0066] L=αL spk +(1-α)L spoof
[0067] Among them, α is the weight coefficient, L spk The voiceprint recognition loss calculated by GE2ELoss loss, L spoof Loss calculated using OC-Softmax loss for the fake speech detection model.
[0068] Specifically, the L spoof The calculation formula is as follows:
[0069]
[0070] Among them, x i ∈R D is the embedding vector of the i-th sample, R D is the D-dimensional real vector space where the voiceprint embedding vector e is located, y i ∈{0,1} is its category label, w0 is the weight vector in the target category direction, and a and m are the hyperparameters in OC-Softmax.
[0071] Furthermore, taking α = 0.5, L spk GE2E loss is used for speaker representation clustering optimization, L spoof is the OC-Softmax loss for true and false classification. Based on experience, the hyperparameters m0 and m1 are set to 0.9 and 0.2, respectively.
[0072] Furthermore, the value of D is determined by the dimension of the embedding vector e in the voiceprint recognition model or the forged speech detection model. For example, when the voiceprint recognition model is a three-layer LSTM, each LSTM layer contains 512 hidden units. The dimension of the embedding vector e obtained after attention pooling is 512, and D = 512.
[0073] Specifically, a voiceprint-driven forged voice detection system is characterized by including:
[0074] Extraction module: inputs voice information, preprocesses the voice information, and extracts the LFCC features of the voice information;
[0075] Voiceprint recognition module: builds a voiceprint recognition model, extracts spectral distribution features of the LFCC features, and generates a voiceprint embedding vector;
[0076] Forged voice detection module: Constructs a forged voice detection model, fuses the LFCC features with the voiceprint embedding vector according to a preset ratio, and inputs the LFCC features into the forged voice detection model for recognition and classification of true and false voices;
[0077] Optimization module: optimizes the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.
[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.
[0079] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting forged voice based on voiceprint drive, characterized in that: include: Step 1: Input voice information, pre-process the voice information, and extract the LFCC features of the voice information; Step 2: Build a voiceprint recognition model, extract spectrum distribution features of the LFCC features, and generate a voiceprint embedding vector; Step 3: Construct a forged voice detection model, fuse the LFCC features with the voiceprint embedding vector according to a preset ratio, input the features into the forged voice detection model for recognition, and perform true and false voice classification; Step 4: Optimize the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.
2. The method for detecting forged voice based on voiceprint drive according to claim 1, characterized in that: The process of pre-processing the voice information in step 1 includes: All speech information is unified to the same length through padding operation, and the speech information is frame divided and windowed to divide the continuous speech signal into overlapping short frames, and LFCC features are extracted for each short frame speech signal.
3. The method for detecting forged voice based on voiceprint drive according to claim 1, characterized in that: The voiceprint recognition model is trained using real voice information, and the forged voice detection model is trained using both real and forged voice information, with the two model parameters jointly optimized through an algorithmic process.
4. The method for detecting forged voice based on voiceprint drive according to claim 1, characterized in that: The voiceprint recognition model includes a basic network structure, a multi-head attention mechanism, and feature aggregation pooling; The basic network structure is used to extract spectral distribution features of the LFCC features and output the spectral features of each frame. The attention weights are generated through a multi-head attention mechanism and combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector.
5. The method for detecting forged voice based on voiceprint drive according to claim 4, characterized in that: The method for calculating the voiceprint embedding vector includes: First calculate the attention weight a corresponding to the nth frame n , the formula is as follows: in, is the voiceprint feature extracted by the voiceprint recognition model, W1 and W2 are the parameter matrices of the multi-head attention mechanism, g() is the ReLU activation function, softmax is the normalization function, and the attention weight a n ∈[0,1]; The attention weight is combined with the spectral features of each frame to generate a fixed-dimensional voiceprint embedding vector, which is L2 normalized to obtain the voiceprint embedding vector e, as shown in the following formula: Among them, ||·||2 is the L2 normalization processing function.
6. The method for detecting forged voice based on voiceprint drive according to claim 4, characterized in that: The LFCC feature is fused with the voiceprint embedding vector according to a preset ratio. The formula is as follows: h l =concat(h n ,h d ) Among them, h l is the multi-layer feature representation after fusion, h n is the spectrum feature after being processed by the voiceprint recognition model, h d It is the output of LFCC features after passing through the basic network structure.
7. The method for detecting forged voice based on voiceprint drive according to claim 1, characterized in that: The multi-objective joint learning method adopts GE2ELoss and OC-Softmax Loss for comprehensive training, and the joint loss function formula is as follows: L=αL spk +(1-α)L spoof Among them, α is the weight coefficient, L spk The voiceprint recognition loss calculated by GE2ELoss loss, L spoof Loss calculated using OC-Softmax loss for the fake speech detection model.
8. The method for detecting forged voice based on voiceprint drive according to claim 7, characterized in that: The L spoof The calculation formula is as follows: Among them, x i ∈R D is the embedding vector of the i-th sample, R D is the D-dimensional real vector space where the voiceprint embedding vector e is located, y i ∈{0,1} is its category label, w0 is the weight vector in the target category direction, and a and m are the hyperparameters in OC-Softmax.
9. A voiceprint-driven forged voice detection system, characterized in that: include: Extraction module: inputs voice information, preprocesses the voice information, and extracts the LFCC features of the voice information; Voiceprint recognition module: builds a voiceprint recognition model, extracts spectral distribution features of the LFCC features, and generates a voiceprint embedding vector; Forged voice detection module: Constructs a forged voice detection model, fuses the LFCC features with the voiceprint embedding vector according to a preset ratio, and inputs the LFCC features into the forged voice detection model for recognition and classification of true and false voices; Optimization module: optimizes the voiceprint recognition model and the forged voice detection model through multi-objective joint learning.