Extensible forged voice detection method and system

Abstract

The invention provides an extensible forged voice detection method and system, and aims to solve the problem of insufficient forged voice detection capability in the prior art. According to the method, a known content boundary is constructed based on multi-modal features of standardized voice by using a single-classification learning model, an input voice signal is judged, and if the signal exceeds the boundary, an extended feature is extracted and a new detector is trained as a subtask; and a plurality of subtasks generate a comprehensive detection result through a serialized decision. The system comprises a preprocessing module, a feature extraction module, a detection module, a judgment module and a decision module, supports real-time updating of a forged voice sample library, and realizes the expansion capability of unknown forged features. And through modular design of multiple sub-detectors and dynamic threshold optimization, the adaptability and efficiency of the detection model are improved. The method has the characteristics of strong expansibility and high detection precision, can be widely applied to the fields of voice identity verification, intelligent voice interaction and the like, and has important practical application value and popularization significance.

Description

Technical Field

[0001] The present invention relates to the field of voice detection technology, and in particular to an expandable forged voice detection method and a corresponding detection system, which are used to identify whether the voice is forged and ensure the security and authenticity of voice communication and related applications. Background Art

[0002] With the rapid development of voice technology, voice has been widely used in many fields such as communications, financial transaction authentication, and intelligent voice assistants. However, at the same time, forged voice technology has become increasingly rampant, which poses a serious threat to personal privacy, information security, and social stability. Traditional voice detection methods often target specific types of forged voices or detect based on limited voice features, lacking comprehensive adaptability and scalability to multiple forms of forgery. For example, some early detection methods can only identify simple audio clip splicing forgeries, and it is difficult to effectively detect forged voices generated by complex speech synthesis technology or deep forgery. Moreover, when new types of forged voices appear, traditional methods are difficult to quickly adjust and update the detection mechanism and cannot meet the ever-changing security needs. Therefore, there is an urgent need for a detection method and system that can effectively deal with a variety of forged voice situations and has good scalability.

[0003] Publication No. CN111564163A discloses a method for detecting multiple forged voice operations based on RNN. The method comprises: a. performing M kinds of forgery processing on the original voice sample to obtain M voices after forgery processing and 1 unprocessed original voice, extracting features from the above voices to obtain the LFCC matrix of the training voice sample, sending it to the RNN classifier network for training, and obtaining a multi-classification training model; b. obtaining a test voice, extracting features from the test voice, obtaining the LFCC matrix of the test voice data, sending it to the RNN classifier trained by step a for classification, obtaining an output probability for each test voice, and merging all output probabilities as the final prediction result: if the prediction result is the original voice, the test voice is identified as the original voice; if the prediction result is the voice after a certain forgery operation, the test voice is identified as a forged voice with the corresponding forgery operation. Summary of the invention

[0004] In order to solve the mismatch problem that the existing forged voice detection system will have performance degradation when facing data forged by an unknown synthesis algorithm, the present invention provides an expandable forged voice detection method and system, which can construct a known feature boundary through single classification learning, perform extended feature analysis on data beyond the boundary and independently train the forged detector, thereby realizing dynamic expansion and comprehensive detection, and improving the robustness and adaptability of forged voice detection.

[0005] A scalable forged voice detection method comprises the following steps:

[0006] S1, the judgment module uses a single-class learning model to construct the boundary of known features;

[0007] S2, the judgment module determines whether the speech signal is outside the boundary of the known feature. If it is determined to be yes, the feature extraction module extracts the extended feature. If it is determined to be no, the calculation of the known feature is set as the known feature subtask for result calculation;

[0008] S3, the judgment module sets the calculation of the extended feature as the extended feature subtask and independently trains the corresponding detector;

[0009] S4. Based on the serialized decision of multiple subtasks, the decision module generates comprehensive detection results.

[0010] The scalable forged speech detection method described in the present invention, as a preferred embodiment, before step S1, also includes a feature extraction module to extract known features from speech sampling points, wherein the known features are used to train the detector respectively; before step S3, also includes a preprocessing module to perform standardization processing on the speech signal.

[0011] The scalable forged speech detection method described in the present invention is, as a preferred embodiment, a subtask includes a detector calculating a forged feature; each forged feature corresponds to an independent detector, and the forged features include known features and extended features.

[0012] The scalable forged speech detection method described in the present invention is, as a preferred embodiment, the detection result of the detector is generated by comparing the calculated score with the activation threshold; wherein the activation threshold is determined from the detector training set or the detector development set according to the maximum detection accuracy.

[0013] A scalable forged voice detection method comprises the following steps:

[0014] (1) Construct known feature boundaries:

[0015] A single-class learning model is used to extract different features from speech sampling points, including frequency features, cepstral domain features, phase domain features, rhythm-related features, speech recognition pre-training model features, etc.

[0016] Model selection:

[0017] One-Class SVM: By maximizing the distance between the hyperplane and the normal samples, a known feature boundary is established to separate normal samples from abnormal samples.

[0018] Input: extracted speech feature vector.

[0019] Output: Classification result based on feature boundaries (belongs to normal or exceeds the boundary).

[0020] Advantages: It performs well on small-scale, high-dimensional feature sets and is suitable for preliminary boundary construction for speech forgery detection.

[0021] Gaussian Mixture Model (GMM): Use multiple Gaussian distribution functions to fit the characteristic distribution of normal speech samples to form a probability density function.

[0022] Input: Speech feature vector sequence.

[0023] Output: The probability density value of the input feature in the model. Samples outside the low probability interval are judged as abnormal.

[0024] Advantages: Good at processing complex continuous feature distribution in speech signals.

[0025] Autoencoder (AE): Use a neural network to reconstruct normal speech features and calculate the error between the input features and the reconstructed features.

[0026] Input: Initial speech feature vector.

[0027] Output: Reconstruction error value. Samples exceeding the preset threshold are marked as abnormal.

[0028] Advantages: It can effectively capture the nonlinear relationship of speech features and is suitable for complex data sets.

[0029] Model training:

[0030] Single-class support vector machine: Use the feature vector of the known speech sample to train the One-Class SVM model, find the optimal hyperplane, adjust the penalty parameter ν to control the sensitivity of the model (that is, the proportion of abnormal samples allowed), validation set optimization: Use the development set to adjust the boundaries of the model so that it can better separate the extended samples.

[0031] Mixed Gaussian model: Input known speech features into the GMM model, initialize distribution parameters (mean vector, covariance matrix, weight), use the expectation maximization (EM) algorithm to fit the feature distribution, iteratively optimize the parameters of the Gaussian distribution, verify the model's ability to distinguish the boundaries of extended samples, and adjust the distribution quantity or distribution weight.

[0032] Autoencoder: construct input layer, hidden layer and output layer, the hidden layer represents the compressed speech features; train with known speech samples to minimize reconstruction error (such as mean square error or cross entropy loss); set threshold: calculate the reconstruction error distribution of known samples through the development set, and select a suitable threshold (such as the 90% quantile) to distinguish between known and extended.

[0033] (2) Speech Signal Boundary Determination

[0034] After receiving the speech signal to be detected, preprocessing is performed, including noise reduction, signal enhancement, frame segmentation and feature normalization to ensure the availability and consistency of the input signal.

[0035] The features of the input speech are mapped to a known feature space and compared with the constructed feature boundary. If it is determined that the speech signal is outside the boundary, the corresponding extended features are extracted.

[0036] (3) Extended feature extraction

[0037] First, anomaly localization is performed: based on the difference between the features of the input speech signal and the known boundaries, potential forgery areas are marked. Then, deep feature extraction is performed: high-order feature extraction techniques (such as wavelet transform and short-time spectral entropy) are used to mine potential forgery characteristics: fine-grained analysis of spectral features is performed to capture abnormal frequency distribution; abnormal cepstral coefficients are extracted to identify the unnaturalness of forged signals in acoustic space; discontinuities or mutation areas are located through phase domain analysis; and dynamic characteristics of speech signals, such as energy changes and rhythmic rhythm anomalies, are extracted. These extended features will serve as key information for further in-depth detection.

[0038] (4) Independent subtask construction

[0039] If it is within the boundary, it means there is no extended feature, and the calculation task is set based on the known features and the trained detector; if it is outside the boundary, the calculation of the extracted extended features is set as a new subtask, and a forgery detector is independently trained for each subtask to learn the distribution law of specific extended features. Different forgery features correspond to different subtasks and special forgery detectors.

[0040] For the subtask corresponding to the abnormal spectral feature, its forgery detector has been trained with a large number of spectral abnormality samples and can accurately identify forged speech caused by spectral tampering. Each forgery detector calculates a score through its own detector model and compares it with the activation threshold calculated in advance from the training set or development set based on the maximum detection accuracy to obtain the detector result.

[0041] (5) Detector result generation and comprehensive decision making

[0042] After the counterfeit detectors of each subtask complete the detection, the final calculation result is achieved by serializing the detection results of multiple subtasks in sequence to generate a comprehensive detection result:

[0043] For example, a preliminary evaluation is first made on the detection results of speech quality features and phase domain features, and then a comprehensive judgment is made based on the rhythmic feature detection results. Similarly, through a multi-step serialized decision-making process, a comprehensive and accurate judgment can be made as to whether the speech is forged.

[0044] When integrating the detection results, a weighted fusion strategy is used to assign different weights according to the reliability of the detection results of each subtask. Subtasks with higher detection accuracy are given larger weights to obtain more accurate final detection results, effectively improving the accuracy of the overall detection.

[0045] A forged voice detection system, comprising:

[0046] Preprocessing module: This module is mainly responsible for standardizing the input voice signal. For example, the sampling rate, volume and other parameters of the voice signal are uniformly adjusted to meet the required format and parameter standards of the subsequent detection module, laying the foundation for subsequent accurate detection. In the specific implementation, hardware acceleration circuits can be used to greatly improve the speed of voice signal standardization processing, effectively reduce detection delays, and meet the needs of application scenarios with high real-time requirements.

[0047] Feature extraction module: It can use a variety of advanced signal processing technologies, such as using the fast Fourier transform (FFT) algorithm to obtain spectrum information for spectrum features, and using the cepstrum analysis algorithm for cepstrum domain features, etc., to provide a rich and accurate data basis for subsequent forgery feature detection. In addition, this module has multi-channel parallel processing capabilities and can extract multimodal features of multiple voice signal channels at the same time, significantly improving processing efficiency and being suitable for the detection needs of multi-channel voice data.

[0048] Detection module: It consists of multiple detectors targeting different forgery features, and each detector can independently complete its own training and learning process.

[0049] Judgment module: When new data is calculated with the boundary model and determined to be outside the boundary, the relevant calculation of the new data expansion is started. Otherwise, the calculation of the known features is set as the known feature subtask. This module assumes the function of extending the condition calculation.

[0050] Decision module: used to integrate the detection results of multiple subtasks to obtain the final detection result. The decision module adopts a serializable input structure to ensure the accuracy and efficiency of the integration of multiple subtask results. When integrating the detection results, a weighted fusion strategy can be used to assign different weights according to the reliability of the detection results of each subtask, and assign larger weights to subtasks with higher detection accuracy to obtain more accurate final detection results, effectively improving the accuracy of the overall detection.

[0051] Updateable forged voice database: used to store different types of forged voice samples and real voice samples, and can support real-time model update operations based on new sample data. A distributed storage architecture is used to increase data storage capacity and read and write speeds to ensure the efficiency of real-time model updates. When a new type of forged voice is discovered, its sample data can be stored in the database in a timely manner, and the detection device can automatically update the model parameters, thereby continuously improving the accuracy and adaptability of the entire system to forged voice detection, and always maintaining effective detection capabilities for forged voices.

[0052] The advantages of the present invention compared with the prior art are:

[0053] (1) The boundary detection mechanism based on the single-classification learning model can quickly identify unknown forged speech and trigger the extraction and training of extended features. Compared with the traditional fixed detection model, the present invention can dynamically adapt to new forged features and has stronger adaptability and scalability.

[0054] (2) It avoids the problem of false detection caused by overfitting or insufficient generalization ability of a single model. Each detector focuses on specific features, resulting in higher detection accuracy.

[0055] (3) Compared with a single decision model, the comprehensive decision-making mechanism can significantly reduce the false detection rate and missed detection rate and improve the overall robustness of the system.

[0056] (4) Introducing an updateable forged speech database, supporting real-time addition of new forged speech samples and updating of the model. Traditional methods require retraining the entire model to adapt to new forgery types, while the present invention supports local model updates and quickly enhances the ability to detect new forged speech.

[0057] (5) The modular design can flexibly adapt to a variety of application requirements and is suitable for multiple practical scenarios such as telephone fraud detection, identity authentication, content review, etc., and has high practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of the steps of the method of the present invention;

[0059] Figure 2 It is a system structure block diagram of the present invention;

[0060] Figure 3 It is a structural block diagram of the detection module of the present invention;

[0061] Figure 4 Schematic diagram of the extension of the new detector for OR sequence calculation;

[0062] Figure 5 Schematic diagram of the extension of the new detector to compute serializable inputs. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0064] Example 1

[0065] A scalable forged voice detection system, the process of which is shown in Figure 1 , specifically including the following steps:

[0066] (1) The feature extraction module extracts the features required by different detectors from the speech sampling points:

[0067] Extraction of spectral features:

[0068] Perform short-time Fourier transform (STFT) on the signal of the speech sampling point to obtain the amplitude and phase information in the frequency domain.

[0069] Calculate the spectrum energy distribution and extract the amplitude spectrum, power spectrum, etc. as features.

[0070] Combined with the detector requirements, key frequency bands are selected for feature dimension reduction or aggregation processing.

[0071] Extraction of cepstrum domain features:

[0072] Extract Mel-frequency cepstral coefficients (MFCC) from speech signals using Python, or obtain cepstral features by taking the logarithm of the magnitude spectrum and performing discrete cosine transform (DCT).

[0073] Extract first-order and second-order dynamic coefficients to enhance the ability to distinguish forged signals.

[0074] Extraction of phase domain features:

[0075] Extract phase information from the STFT results and calculate envelope phase differences or instantaneous frequency characteristics.

[0076] Phase statistics (such as phase divergence or mean) are used as detection features.

[0077] Direct calculation via the phase component of the Fourier transform, or using advanced signal processing methods such as group delay analysis.

[0078] Extraction of speech quality features:

[0079] Use Praat or OpenSMILE to extract speech quality indicators, extract the pitch period, jitter, shimmer and other features of the speech signal, and quantify the speech quality.

[0080] Analyze the signal-to-noise ratio (SNR) and harmonic-to-noise ratio (HNR) to capture the unnatural quality characteristics in forged speech.

[0081] Extraction of rhythmic features:

[0082] The fundamental frequency (F0) and energy contour of the speech signal are calculated by the pitch detection algorithm PyPitch to analyze the prosodic information.

[0083] Features such as speaking rate, stress patterns, and intonation changes are extracted to determine the unnaturalness of the forged speech.

[0084] (2) Construct known feature boundaries:

[0085] Select Gaussian Mixture Model (GMM): Use multiple Gaussian distribution functions to fit the characteristic distribution of known speech samples to form a probability density function.

[0086] Input: a sequence of known speech feature vectors.

[0087] Output: The probability density value of the input feature in the model. Samples outside the low probability interval are judged as abnormal.

[0088] Estimate mean and covariance matrix: Use maximum likelihood estimation (MLE) to calculate the mean and covariance:

[0089] Code implementation:

[0090]

[0091] Use SciPy or other libraries to build a multivariate Gaussian model:

[0092]

[0093] (3) Standardized speech signal

[0094] The input speech is unified in length. The sampling rate of the unified speech is 16000Hz. Speech less than 4 seconds is supplemented by the previous content to 4 seconds, and speech longer than 4 seconds is cut off by the first 4 seconds.

[0095] (4) Determination inside and outside the boundary:

[0096] For the input signal feature vector x, calculate its probability density in the Gaussian distribution:

[0097]

[0098] Use code to calculate the probability density:

[0099] scores = gaussian_model.pdf(input_features) # Calculate the probability density

[0100] Decision rule:

[0101] Set the activation threshold T and make a decision according to the probability density P(x):

[0102] If P(x) ≥ T, it is determined as a known feature.

[0103] If P(x) < T, it is determined as an unknown feature.

[0104] Code implementation:

[0105]

[0106] (5) Extraction of extended features and training of a new detector:

[0107] If it is determined that the speech signal is outside the boundary, it indicates that the speech may have extended features. At this time, extract the un-covered features, such as special forged pitch characteristics, and reconstruct a multivariate Gaussian distribution model for the extended features. The new extended features are added to the known features, and the mean and covariance are re-estimated.

[0108] An incremental update formula can be adopted to avoid full-scale retraining:

[0109] Update the mean:

[0110]

[0111] Update the covariance:

[0112]

[0113] Incremental update code:

[0114]

[0115] Dynamically adjust the threshold, and recalculate the optimal activation threshold according to the new extended features to maximize the detection accuracy: Use the development set or cross-validation technology to search for the optimal threshold.

[0116] (6) Generation of detector results and comprehensive decision-making:

[0117] Each detector calculates a score through its own detector model and compares it with the activation threshold calculated in advance from the training set or development set based on the maximum detection accuracy to obtain the detector result.

[0118] First, make a preliminary evaluation of the speech quality feature and phase domain feature detection results, and then make a comprehensive judgment based on the prosodic feature detection results. Figure 4 , and so on, through a multi-step serialized decision-making process, it is possible to comprehensively and accurately determine whether the voice is forged.

[0119] When integrating the detection results, a weighted fusion strategy is used, see Figure 5 ,Different weights are assigned according to the reliability of the detection results of each subtask, and a larger weight is assigned to the subtask with higher detection accuracy to obtain a more accurate final detection result, effectively improving the accuracy of the overall detection.

[0120] Example 2

[0121] A scalable forged speech detection system, the structural block diagram is as follows Figure 2 As shown, including:

[0122] Preprocessing module: This module is mainly responsible for standardizing the input voice signal. The sampling rate, volume and other parameters of the voice signal are uniformly adjusted to meet the required format and parameter standards of the subsequent detection module, laying the foundation for subsequent accurate detection. In the specific implementation, hardware acceleration circuits can be used to greatly improve the speed of voice signal standardization processing, effectively reduce detection delays, and meet the needs of application scenarios with high real-time requirements.

[0123] Feature extraction module: It is specially used to extract multimodal features from speech signals. It can use a variety of advanced signal processing technologies, such as using the fast Fourier transform (FFT) algorithm to obtain spectrum information for spectrum features, and using the cepstrum analysis algorithm for cepstrum domain features, etc., to provide a rich and accurate data basis for subsequent forgery feature detection. In addition, this module has multi-channel parallel processing capabilities, and can extract multimodal features of multiple speech signal channels at the same time, significantly improving processing efficiency, and is suitable for the detection needs of multi-channel speech data.

[0124] Detection module: See Figure 3, which consists of multiple forgery detectors targeting different forgery features. There are detectors specifically for detecting spectral anomalies, detectors for voice quality degradation, etc., and each forgery detector can independently complete its own training and learning process. During the training process, a large number of forged voice samples and real voice samples can be used to enable the detector to accurately distinguish between normal and forged voices. It adopts a modular design, which is convenient for flexible combination and replacement according to different application scenarios and detection requirements, enhancing the adaptability and maintainability of the system.

[0125] Judgment module: Use single-class Gaussian mixture model (GMM) or single-class support vector machine (SVM) to carry out single-class learning, thereby building a boundary model of known knowledge. When new data is calculated with the boundary model and is determined to be outside the boundary, the relevant calculation of the new data expansion is started, and this module assumes the function of expanding the condition calculation.

[0126] Decision module: It is used to integrate the detection results of multiple subtasks to obtain the final detection result. The decision module adopts a serializable input structure to ensure the accuracy and efficiency of the integration of multiple subtask results.

[0127] Updateable forged voice database: used to store different types of forged voice samples and real voice samples, and can support real-time model update operations based on new sample data. A distributed storage architecture is used to increase data storage capacity and read and write speeds to ensure the efficiency of real-time model updates. When a new type of forged voice is discovered, its sample data can be stored in the database in a timely manner, and the detection device can automatically update the model parameters, thereby continuously improving the accuracy and adaptability of the entire system to forged voice detection, and always maintaining effective detection capabilities for forged voices.

[0128] The operation process of the forged voice detection system:

[0129] (1) Preprocessing module. Standardize the input speech signal, including denoising, sampling rate conversion and volume normalization. Output the processed speech signal for use by subsequent modules.

[0130] (2) The feature extraction module extracts the multimodal features of the speech signal and saves them as high-dimensional feature vectors:

[0131] The spectral features are extracted by STFT method;

[0132] The cepstral domain features are generated by the MFCC algorithm;

[0133] Phase domain features obtain phase information through Hilbert transform.

[0134] (3) Detection module

[0135] The detection module includes a plurality of forgery detectors corresponding to different forgery features.

[0136] Each forgery detector is trained independently using a single-classification learning method. After the model training is completed, it is stored as a weight file for online inference.

[0137] (4) The judgment module uses a single-classification learning model to determine whether the speech signal is outside the known boundary:

[0138] If it is within the boundary, output "not forged";

[0139] If it is outside the boundary, extended feature extraction is triggered and the forgery feature detection process begins.

[0140] (5) Decision-making module

[0141] Aggregate the results of all forgery detectors and generate the final detection result by serializing decisions; the decision rules include a score weighting strategy to ensure the robustness and accuracy of detection.

[0142] Dynamic model updates:

[0143] (1) Update of the forged speech database. Forged speech samples are captured through manual annotation or other detection models and stored in the forged speech database. Forged samples are classified and cleaned regularly to ensure the accuracy and diversity of the database.

[0144] (2) Model update and optimization. The forgery detector in the detection module regularly loads new samples from the forged speech database for retraining. After the model is updated, its performance is tested using the development set to ensure the model's adaptability to new forgery features.

[0145] (3) Real-time adaptation to new features. After the extended features of new types of forged speech are analyzed, the model expansion and forgery detector training are directly triggered; the weight file of the forgery detector is updated in real time and loaded into the detection module to achieve dynamic expansion capabilities.

[0146] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A scalable forged speech detection method, characterized in that: The following steps are involved: S1, the judgment module uses a single-class learning model to construct the boundary of known features; S2, the judgment module determines whether the speech signal is outside the boundary of the known feature. If it is determined to be yes, the feature extraction module extracts the extended feature. If it is determined to be no, the calculation of the known feature is set as a known feature subtask for result calculation; S3, the judgment module sets the calculation of the extended feature as an extended feature subtask, and independently trains the corresponding detector; S4. Based on the serialized decision of multiple subtasks, the decision module generates comprehensive detection results.

2. A scalable forged speech detection method according to claim 1, characterized in that: Before the step S1, the feature extraction module is also included to extract the known features from the speech sampling points, wherein the known features are respectively used to train the detector; before the step S3, a preprocessing module is also included to perform standardization processing on the speech signal.

3. A scalable forged speech detection method according to claim 2, characterized in that: A subtask includes calculation of a forgery feature by said detector; Each of the forged features corresponds to an independent detector, and the forged features include the known features and the extended features.

4. A scalable forged speech detection method according to any one of claims 1 to 3, characterized in that: The detection result of the detector is generated by comparing the calculated score with the activation threshold; The activation threshold is determined from a training set or a development set of the detector according to a maximum detection accuracy.

5. A forged voice detection system, characterized in that: include: Preprocessing module: used to standardize the input speech signal; Detection module: It consists of multiple detectors, each used to detect different counterfeit features; Feature extraction module: used to extract known features from speech sampling points and extended features from speech signals; Judgment module: Use the single-classification learning model to build the boundary of known features; judge whether the input signal is inside or outside the boundary by comparing with the known feature boundary. Data inside the boundary triggers the calculation of known feature tasks, and data outside the boundary triggers the calculation of extended feature tasks. Decision module: used to integrate the detection results of multiple subtasks and generate comprehensive detection results.

6. The forged voice detection system according to claim 5, characterized in that: The system includes an updateable forged voice database for storing different types of forged voice samples and supports real-time model updating to enhance detection capabilities.

Images