Counterfeit voice detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a forged voice detection method and device, electronic equipment and a storage medium, and relates to the technical field of voice recognition, and the method comprises the steps: collecting a voice sample of a user, extracting physiological acoustic features, and constructing an initial credible voice feature model according to physiological features; obtaining to-be-detected voice, and performing preprocessing operation on the to-be-detected voice to generate voice data of the to-be-detected voice; performing multi-dimensional joint detection on the voice data based on the forged voice detection model and the initial credible voice feature model to generate a multi-dimensional detection result; and calculating a comprehensive risk score according to the multi-dimensional detection result, calling different preset processing strategies based on the comprehensive risk score, and performing differentiation processing on the to-be-detected voice according to the different preset processing strategies. The method has the technical effects that the accuracy and pertinence of forged voice detection are improved, the detection robustness is enhanced, and risk grading response is realized to more accurately guarantee the safety and credibility of a voice interaction system.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a method, apparatus, electronic device and storage medium for detecting fake speech. Background Technology

[0002] With the rapid development of deep learning and speech synthesis technologies, especially the widespread application of generative adversarial networks, variational autoencoders, and end-to-end speech synthesis models, the quality of forged speech has reached a level that is indistinguishable from real speech. This has brought about security threats such as identity impersonation and targeted fraud. Therefore, detection technologies that analyze subtle anomalies in speech to distinguish between genuine and fake speech have emerged, which are crucial for ensuring financial authentication, content review, and protecting the safety of smart products for the elderly and children.

[0003] Among related technologies, end-to-end forged speech detection methods based on deep learning can directly learn features from raw audio, but they still face serious challenges in practical applications. At the system architecture level, most related technical solutions rely on a single AI detection model, lacking coordination and integration with multi-dimensional information such as device liveness detection, user voiceprint physiological characteristics, semantic consistency, and communication behavior analysis. This results in insufficient generalization ability and overall security robustness when dealing with complex hybrid attacks. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for detecting spoofed voice. It improves the accuracy and targeting of spoofed voice detection, enhances detection robustness, and simultaneously enables risk-level response to more accurately ensure the security and reliability of voice interaction systems.

[0005] According to a first aspect of this application, a method for detecting fake voice is provided, comprising: Collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features; Acquire the speech to be detected, and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected; A multi-dimensional joint detection of speech data is performed based on a fake speech detection model and an initial reliable speech feature model to generate multi-dimensional detection results. A comprehensive risk score is calculated based on the multi-dimensional detection results. Different preset processing strategies are invoked based on the comprehensive risk score, and differentiated processing is performed on the speech to be detected according to the different preset processing strategies.

[0006] According to a second aspect of this application, a forged voice detection device is provided, comprising: The module is configured to collect user speech samples, extract physiological acoustic features, and construct an initial reliable speech feature model based on the physiological features. The generation module is configured to acquire the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected. The detection module is configured to perform multi-dimensional joint detection of speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results. The calling module is configured to calculate a comprehensive risk score based on multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

[0007] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the forged speech detection method of the first aspect described above.

[0008] According to a fourth aspect of this application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the forged speech detection method of the first aspect described above.

[0009] According to a fifth aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the spoofing voice detection method as described in the first aspect above.

[0010] This application provides a method, apparatus, electronic device, and storage medium for detecting spoofed speech. By collecting user speech samples to extract physiological acoustic features to construct an initial credible speech feature model, preprocessing the acquired speech to be detected, and performing multi-dimensional joint detection on the preprocessed speech data based on the spoofed speech detection model and the initial credible speech feature model, and then calculating a comprehensive risk score based on the multi-dimensional detection results and calling different preset processing strategies to perform differentiated processing, this application can solve the problems in related technologies where a single inspection method lacks a user-specific credible speech benchmark, fails to combine user physiological acoustic features with the spoofing detection model to achieve multi-dimensional collaborative detection, and lacks a risk-level-based differentiated processing mechanism. This results in insufficient detection generalization ability, weak targeting, and a lack of graded risk response. The application achieves the technical effects of improving the accuracy and targeting of spoofed speech detection, enhancing detection robustness, and simultaneously realizing risk-level response to more accurately ensure the security and credibility of the voice interaction system.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a method for detecting fake voice provided in an embodiment of this application; Figure 2 A flowchart illustrating another method for detecting fake voice provided in an embodiment of this application; Figure 3 A flowchart illustrating another method for detecting fake voice provided in an embodiment of this application; Figure 4 A flowchart illustrating another method for detecting fake voice provided in an embodiment of this application; Figure 5 This is a structural diagram of a fake voice detection device provided in an embodiment of this application. Detailed Implementation

[0014] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] The following description, with reference to the accompanying drawings, outlines a method, apparatus, electronic device, and storage medium for detecting fake voice messages according to embodiments of this application.

[0016] Figure 1 This is a flowchart illustrating a method for detecting fake voice provided in an embodiment of this application.

[0017] like Figure 1 As shown, the method includes the following steps: Step 101: Collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features.

[0018] In some embodiments, the device guides the user to complete voice sample collection through a trusted audio acquisition component. During the collection process, the user is prompted to read a preset, personalized command to obtain voice data containing the user's stable acoustic characteristics, avoiding incomplete feature extraction due to random pronunciation. Next, physiological acoustic features are extracted from the collected voice samples. These features originate from the user's unique physiological structure and vocal habits, exhibiting strong individual differences, specifically including voiceprint features, intonation patterns, and pronunciation habits. Subsequently, an initial trusted voice feature model is constructed based on the extracted physiological acoustic features. This model is not a simple superposition of single features, but rather integrates multi-dimensional features such as voiceprint, intonation, and pronunciation habits into a dual template of voiceprint and physiological features through a feature fusion algorithm, ensuring that the model comprehensively reflects the unique attributes of the user's voice. Simultaneously, the model is processed using an encryption algorithm and then either stored securely locally or uploaded to a dedicated cloud database via an encrypted communication channel to prevent the feature data from being illegally stolen or tampered with, providing a reliable personalized benchmark for subsequent verification of the authenticity of the voice to be detected. It solves the problems of lack of user-specific trusted speech benchmarks and insufficient targeting due to reliance on general detection standards. By constructing a personalized initial trusted speech feature model, it provides accurate reference for subsequent detection, improves the ability to distinguish forged speech, and lays the foundation for detection accuracy.

[0019] Step 102: Obtain the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected.

[0020] In some embodiments, the voice to be detected is acquired through a liveness detection microphone array deployed on the device. During the acquisition process, the voice data itself, the acquisition timestamp, and the current environmental information are recorded simultaneously. At the same time, the characteristics of the microphone array are used to analyze the time difference and intensity difference of the voice signal arriving at each microphone. Combined with environmental noise spectrum analysis, it is determined whether the voice originates from the near field range. This prevents external devices from playing fake audio for injection attacks and ensures that the source of the voice to be detected has preliminary authenticity. The acquired speech data is then preprocessed: first, noise reduction is performed using acoustic noise reduction algorithms to filter redundant noise in the environment and reduce the impact of irrelevant interference on speech features; next, frame segmentation is performed, dividing the continuous speech signal into discrete speech frames at fixed time intervals, balancing the temporal continuity and local stability of the speech signal, laying the foundation for subsequent feature processing; then, preliminary feature extraction is carried out, extracting basic acoustic features such as short-time energy and zero-crossing rate to complete the initial screening of core speech information; simultaneously, user responses are guided by preset dynamic challenge commands, and by verifying the real-time correspondence between user speech and challenge commands, non-real-time generated speech such as recorded playback is excluded, ultimately generating speech data that has undergone source verification, noise filtering, structural optimization, and ensures real-time performance, providing high-quality input material for subsequent detection stages. Through liveness detection and multi-stage preprocessing, the authenticity and quality of the speech data are ensured, providing reliable data support for subsequent multi-dimensional joint detection and improving the overall accuracy of forged speech detection.

[0021] Step 103: Perform multi-dimensional joint detection on the speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results.

[0022] In some embodiments, the forged speech detection model is a model built based on an optimized end-to-end architecture, capable of accurately capturing forgery artifacts and long-term prosodic anomalies in speech. It can provide a basis for authenticity judgment by analyzing the spectral-temporal features of speech data. The initial reliable speech feature model is a user-specific "voiceprint + physiological feature" dual template constructed in step 101, used to lock the individual matching degree between the speech to be detected and the user's real speech. Multi-dimensional joint detection is carried out based on these two types of models: The first dimension is physiological acoustic feature matching detection, which extracts physiological acoustic features such as voiceprint features, intonation patterns, and pronunciation habits from the speech data to be detected, calculates similarity with the user-specific template in the initial reliable speech feature model, and generates feature matching results containing specific similarity values ​​and "match / non-match" judgments to exclude speech forgery cases of non-target users; The second dimension is forgery acoustic artifact detection, which inputs the preprocessed speech data into the forged speech detection model. The model extracts features at multiple scales to analyze whether there are features corresponding to forgery behaviors such as AI synthesis, recording playback, and speech splicing in the speech, and outputs a 0-1 range. The system performs several tests, including: 1) Detecting the authenticity of the voice and making a preliminary judgment of whether it is "suspected real" or "suspected fake"; 2) Detecting semantic and real-time correlation, combining the preset dynamic challenge instructions that guide user response in step 102 to verify the semantic matching degree between the voice content to be detected and the challenge instructions, while analyzing the temporal rationality of the voice response to generate semantic consistency results and real-time judgments; 3) Detecting communication behavior, collecting the communication frequency, IP address, request mode, interaction timestamp, device geographical location, operation behavior type, and interaction duration of the voice device's context information, and identifying whether there are any abnormal behaviors by comparing them with the user's historical normal interaction patterns, generating communication behavior detection results of "normal behavior," "minor abnormality," or "serious abnormality." Finally, the detection sub-results of the four dimensions are integrated to form a complete multi-dimensional detection result, providing a comprehensive basis for subsequent risk assessment. Through multi-dimensional detection using the collaboration of two types of models, it covers both the common artifact features of fake voice and anchors the individual voice attributes of users, effectively distinguishing real voice from various types of fake voice, and improving the accuracy and robustness of detection.

[0023] Step 104: Calculate the comprehensive risk score based on the multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

[0024] In some embodiments, based on the multi-dimensional detection results generated in step 103, a comprehensive risk score is calculated using a fusion algorithm (such as weighted voting, Bayesian inference, or a lightweight neural network). During the calculation process, differentiated weights are assigned according to the core importance of each detection dimension. For example, the probability value of voice authenticity has the highest weight due to its direct association with forgery attributes (e.g., 40%), the physiological acoustic feature matching degree (from the initial credible voice feature model) has the second highest weight due to its anchoring to individual user attributes (e.g., 30%), and semantic consistency and real-time judgment each account for 15% due to their auxiliary exclusion of attacks such as recording playback. After weighted summation and normalization, a comprehensive risk score between 0 and 100 is obtained, and three risk levels are divided: low risk (0-30 points, all multi-dimensional sub-results conform to real voice features), medium risk (31-70 points, some sub-results are questionable, such as feature matching degree meets the standard but the authenticity probability is slightly low), and high risk (71-100 points, core sub-results are abnormal, such as authenticity probability close to 0 or feature matching degree far below the threshold). Subsequently, based on the risk level, a preset processing strategy is invoked: In low-risk cases, the detected voice is deemed genuine, allowing subsequent operations; in medium-risk cases, secondary verification is triggered, a new dynamic challenge command is pushed to the user, and voice data is re-collected for supplementary detection; in high-risk cases, interception and alarms are activated, the current operation is immediately rejected, and a reminder containing the risk score and operation type is sent to preset emergency contacts, while sensitive device functions are frozen. Ultimately, differentiated processing is applied to the operations associated with the detected voice based on different strategies, achieving precise matching of risk and response. This step, through quantitative scoring and differentiated processing, effectively intercepts high-risk forged voice while avoiding misjudgment interference from low-risk genuine voice, balancing voice interaction security and user experience.

[0025] Compared with related technologies, this embodiment collects user voice samples, extracts physiological acoustic features, and constructs an initial reliable voice feature model based on these features. It then acquires the voice to be detected, performs preprocessing operations to generate voice data, and performs multi-dimensional joint detection on the voice data based on the forged voice detection model and the initial reliable voice feature model to generate multi-dimensional detection results. A comprehensive risk score is calculated based on the multi-dimensional detection results, and different preset processing strategies are invoked based on the comprehensive risk score, resulting in differentiated processing of the voice to be detected according to the different preset processing strategies. This solves the problem in related technologies where the lack of coordination and integration with multi-dimensional information such as device-side liveness detection, user voiceprint physiological features, semantic consistency, and communication behavior analysis leads to insufficient generalization ability and overall security robustness in the face of complex hybrid attacks. It improves the accuracy and targeting of forged voice detection, enhances detection robustness, and simultaneously achieves risk-level response to more accurately ensure the security and reliability of the voice interaction system.

[0026] Figure 2A flowchart of a forged voice detection method provided in this application embodiment is further shown, including the following steps: Step 201: Based on wavelet transform convolution and frequency adaptive dilated convolution, optimize and adapt the end-to-end fake speech detection model to generate a fake speech detection model.

[0027] Table 1 In some embodiments, the RawNet2 model applied to forged speech detection consists of Fixed Sincefilters, Res block, GRU (Gated Recurrent Unit), FC (Fully Connected) layer, and Output layer modules. Its model structure is shown in Table 1. The Fixed Sinc filters module includes: Conv (convolutional layer), MaxPool (maximum pooling), Batch Normalization (BN), and LeakyReLU (activation function). The Conv module, instead of the original learnable cutoff frequency filter, uses three fixed filters, effectively reducing the probability of overfitting. However, the fixed filters have limited spectral coverage, making it difficult to capture complex and varied non-stationary artifacts in forged speech (such as the mixed features of instantaneous click noise and spectral distortion in A17 attacks), and they also suffer from insufficient adaptability to spectral differences in different speakers.

[0028] Therefore, wavelet transform convolution (WTConv) is introduced before processing to effectively decompose the original speech waveform through wavelet basis functions. While preserving the time-frequency localization characteristics, it can directly separate and enhance non-stationary artifacts (such as the short-time high-frequency energy of click noise), supplement the high-frequency details and transient features not covered by the fixed filter, and improve the robustness of the model to diverse forgery attacks.

[0029] Note: A17 attack refers to the use of a generalized direct waveform modification method to generate waveforms that introduce difficult-to-detect transient click noise and spectral distortion mixed features into speech signals.

[0030] The Res block consists of Batch Normalization (BN), LeakyReLU (activation function), Convolutional Variation (CVA), Maxpooling, and Filter-Level Feature Scaling (FMS) modules. Spoofing artifacts in fake speech often exhibit cross-frame temporal correlations. Traditional convolutional kernels have a fixed receptive field size, failing to capture artifact features such as variable-length temporal correlations and non-local temporal anomalies. While increasing the receptive field by adding convolutional or pooling layers can lead to gradient vanishing, adding pooling layers can compress the temporal dimension, resulting in temporal information loss. Therefore, dilated convolutions are used to increase the receptive field. However, traditional dilated convolutions, while expanding the receptive field, also reduce the frequency response, leading to high-frequency information loss and aliasing, thus affecting model performance. Therefore, Frequency Adaptive Dilated Convolution (FADConv) is employed.

[0031] Frequency Adaptive Dilated Convolution (FADConv) achieves targeted extraction of speech features from three perspectives: Dynamic dilation rate, which dynamically adjusts the dilation rate at each temporal position based on local spectral characteristics; a smaller dilation rate is used in high-frequency spectral regions (such as the 2-8kHz speech formant region) to preserve fine spectral details (such as high-frequency noise artifacts in fake speech); and a larger dilation rate is used in low-frequency spectral regions (such as the 80-500Hz fundamental frequency region) to capture long-term prosodic features (such as the fundamental frequency periodic anomalies in synthesized speech); and an adaptive kernel, which uses different kernel weights for high and low frequency regions of the speech signal. Narrow-bandwidth kernels are designed for high-frequency regions to enhance sensitivity to fake artifacts (such as click noise from A17 attacks). A wide-bandwidth convolutional kernel is designed for the low-frequency region to capture the fundamental frequency variation patterns of real speech, thereby improving the frequency response specificity of the convolutional kernel to deception features in different frequency bands. Frequency selection employs a time-spectrum variable weighting strategy to balance high and low frequency components in the feature map, suppressing stable background noise (such as low-frequency ambient noise) and redundant high-frequency components in the human voice center band (such as normal harmonics in real speech). This encourages FADConv to use a larger dilation rate in frequency bands where forgery artifacts may occur (such as the 1-3kHz transition speech anomaly region), thereby expanding the receptive field to cross-temporal artifacts (such as synthesis traces with periodic intervals). This adaptive modification enables frequency-adaptive dilated convolution to better capture complex spectral-temporal correlation features in forged speech, preserving subtle artifact information in key frequency bands while covering long-term prosodic anomaly patterns, providing a more discriminative feature representation for deception detection.

[0032] The modified model structure is shown in Table 2:

[0033] Table 2 The implementation process of wavelet transform convolution is as follows: Wavelet decomposition: The Daubechies wavelet transform is selected to decompose the input speech signal into low-frequency and high-frequency components, which respectively reflect the main energy components of the audio and the transient noise information of the audio.

[0034] Multi-frequency convolution: Depth-separable convolution is performed on components of different frequencies, with low-frequency components using large convolution kernels and high-frequency components using small convolution kernels.

[0035] Wavelet reconstruction: The convolution results of different frequency components are weighted and summed, and then upsampled to obtain the final output.

[0036] The implementation process of frequency-adaptive convolution FADConv is as follows: Speech Feature Decomposition: FreqSelect decomposes the input speech features into multiple frequency channels based on speech frequency features (such as time-domain-frequency domain hybrid features learned by the Sinc Conv filter). These channels are divided according to key frequency bands of human speech: low frequency 80-500Hz, mid-low frequency 500-2000Hz, mid-high frequency 2000-8000Hz, and high frequency 8000-16000Hz, to adapt to the frequency domain distortion of forged speech.

[0037] Dynamic weight adjustment: FreqSelect generates a frequency attention vector for each frequency channel and dynamically enhances the weight of fake sensitive frequency bands based on supervised learning (or unsupervised mining of frequency band differences) based on speech authenticity labels (e.g., the weight of the high-frequency noise channel of fake speech is increased, while the weight of the low-frequency stable channel of natural speech is retained).

[0038] Frequency band feature fusion: The multi-frequency channel features after weight adjustment are merged according to the speech time-frequency domain correlation (such as through 1D convolution or attention fusion) to output a speech feature sequence with balanced frequency and abnormal enhancement.

[0039] Dynamic expansion rate prediction: AdaDR predicts the expansion rate for each time step in the speech temporal dimension based on the frequency equalization feature map. Temporal vulnerabilities in forged speech (such as fundamental frequency jumps in synthesized speech and discontinuous temporal sequences in spliced ​​speech) can be addressed by using dynamic expansion rate, allowing the convolutional kernel to flexibly capture long / short-term context (e.g., using a large expansion rate to cover long temporal sequences when detecting fundamental frequency jumps, and using a small expansion rate to focus on short-term details when capturing transient noise).

[0040] Adaptive kernel: AdaKern decomposes the convolution kernel into a low-frequency kernel (capturing stable features such as fundamental frequency and harmonics) and a high-frequency kernel (capturing noise and transient forgery features), and dynamically adjusts the ratio according to the forgery risk of the current speech frame (feedback from the frequency channel weights) (e.g., when a high-frequency anomaly is detected, the weight of the high-frequency kernel is increased).

[0041] Convolution operation: Using the temporal dilation rate predicted by AdaDR and the high and low frequency kernels adjusted by AdaKern, 1D convolution is performed on the speech feature sequence (adapting to the speech temporal structure) to output a deep feature sequence that fuses frequency domain anomalies and temporal vulnerabilities.

[0042] In this embodiment, by optimizing and adapting the two types of convolutions, the model's detection accuracy and robustness against diverse forged speech such as AI synthesis, spectrum tampering, and speech splicing are significantly improved.

[0043] Step 202: Collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features.

[0044] Step 203: Obtain the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected.

[0045] Step 204: Perform multi-dimensional joint detection on the speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results.

[0046] Step 205: Calculate the comprehensive risk score based on the multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

[0047] For a description of steps 202-205, please refer to the description of steps 101-104 in the above embodiment. This embodiment will not repeat these steps in detail.

[0048] Figure 3 A flowchart of a forged voice detection method provided in this application embodiment is further shown, including the following steps: Step 301: During the initial registration on the voice device, collect voice samples under the user's preset password and extract the physiological acoustic features of the voice samples.

[0049] In some embodiments, during initial registration on the voice device, the device guides the user to read a preset command. This command must encompass different combinations of syllables and tones to avoid one-sided speech sample features due to a single pronunciation, ensuring complete capture of the user's normal vocalization. Subsequently, the device acquires speech samples of the user reading the command through its built-in microphone component, simultaneously recording the raw speech waveform data to reduce interference from external environmental noise. After acquisition, physiological acoustic features are extracted from the speech samples. These features originate from the user's unique physiological structure and vocal habits, specifically including voiceprint features, intonation patterns, and pronunciation habits, ensuring that the extracted features accurately correspond to the user's unique voice attributes. This provides personalized initial features for subsequent construction of a reliable speech model, addressing the issues of lacking user-specific voice benchmarks and insufficient targeting in subsequent detection in related technologies.

[0050] Step 302: Construct an initial reliable speech feature model based on physiological characteristics, including voiceprint features, intonation patterns, and pronunciation habits.

[0051] In some embodiments, when constructing an initial reliable speech feature model, the core roles of each physiological acoustic feature are first clarified: voiceprint features are determined by the user's vocal tract length, vocal cord vibration frequency, and other physiological structures, presenting a unique formant distribution pattern in the speech spectrum, which is the core identifier distinguishing different users; intonation patterns reflect the dynamic changes in pitch when a user speaks, such as the falling intonation of declarative sentences and the rising intonation of interrogative sentences in daily expression, which has individual habits; pronunciation habits are manifested in the differences in how users process specific syllables, such as some users pronouncing the "sh" sound with slightly longer duration and slightly higher intensity. Subsequently, through a feature fusion mechanism, the key information of voiceprint features, intonation patterns, and pronunciation habits are integrated, redundant interference data is removed, and a multi-dimensional feature template covering the user's unique speech attributes is formed. This template is then encrypted and securely stored locally on the voice device or uploaded to a designated storage unit in an encrypted manner, ultimately constructing the initial reliable speech feature model. This provides an accurate user-specific benchmark for subsequent speech detection.

[0052] Step 303: Acquire the speech to be detected and record the acquisition timestamp and environmental information of the speech to be detected.

[0053] In some embodiments, the voice to be detected is acquired through a voice acquisition component mounted on the voice device, ensuring the continuity of the voice signal during acquisition and reducing signal gaps or distortion caused by hardware compatibility issues. Simultaneously, the acquisition timestamp of the voice to be detected is accurately recorded, covering the entire time period from the start to the end of acquisition, accurate to the millisecond level, facilitating subsequent preliminary verification based on the regular time patterns of users' daily voice interactions. Furthermore, environmental information is recorded simultaneously, including environmental noise characteristics during acquisition and the basic location of the device, completely preserving the background data of the acquisition scenario for the voice to be detected. This provides supplementary time and environmental references for subsequent judgment of the authenticity of the voice to be detected, solving the problem of incomplete detection caused by relying solely on the voice itself and lacking supporting scene information in related technologies.

[0054] Step 304: The voice device performs preprocessing operations on the voice to be detected to generate voice data. The preprocessing operations include noise reduction, frame segmentation, and feature extraction of the voice to be detected.

[0055] In some embodiments, when the voice device performs preprocessing on the acquired speech to be detected, it first performs noise reduction processing. This involves filtering out environmental noise (such as appliance noise in a home setting or traffic noise in an outdoor setting) mixed in during the acquisition process using acoustic processing methods adapted to the speech scene, reducing irrelevant interference from obscuring the core speech information. Next, it performs frame segmentation processing, dividing the continuous speech to be detected into discrete speech frames at fixed time intervals of 20-30ms. This preserves the temporal continuity of the speech while ensuring the local stability of the speech features in each frame, facilitating subsequent accurate analysis. Finally, it performs feature extraction, extracting basic acoustic features such as short-time energy and zero-crossing rate from the segmented speech frames, filtering out key information that reflects the essential attributes of the speech, and ultimately generating speech data suitable for subsequent detection needs. This improves the purity and feature effectiveness of the speech to be detected, providing high-quality, highly available data support for subsequent forged speech detection.

[0056] Step 305: Verify the semantic matching degree between the speech content to be detected and the challenge command, and generate a semantic consistency detection result; extract the voiceprint features of the speech to be detected, and generate a voiceprint verification result based on the initial reliable speech feature model and voiceprint features; use the forged speech detection model to determine whether it is forged speech, and generate a forged speech detection result; combine the context information of the voice device communication frequency, IP address, and request mode to identify abnormal behavior, and generate a communication behavior detection result.

[0057] In some embodiments, the semantic matching degree between the voice content to be detected and the challenge instruction is first verified. After the voice to be detected is converted into text, it is compared with the semantic core of the preset challenge instruction (such as key information such as "family dinner time" or "last three digits of the device password") to determine whether the answer fits the instruction requirements, avoid recording playback or irrelevant voice interference, and generate a semantic consistency detection result. Then, voiceprint features are extracted from the voice to be detected and compared with the user voiceprint features stored in the initial trusted voice feature model to confirm whether they conform to the user's exclusive voiceprint attributes and generate a voiceprint verification result. Subsequently, the preprocessed voice data is input into the fake voice detection model. The model analyzes the spectral artifacts, temporal anomalies and other features in the voice to determine whether the voice is AI synthesized, spliced ​​or tampered with, and generates a fake voice detection result. At the same time, the communication frequency of the voice device (such as whether it deviates from the daily interaction frequency), IP address (whether it is an unfamiliar address), request pattern (such as whether there are high-frequency abnormal requests) and other contextual information are collected to analyze whether it conforms to normal usage rules and generate a communication behavior detection result. By obtaining results from multiple dimensions such as semantics, voiceprints, model detection, and communication behavior, comprehensive evidence is provided for subsequent integrated risk assessment, thereby improving the comprehensiveness and accuracy of voice forgery detection.

[0058] Step 306: Generate multi-dimensional detection results based on semantic consistency detection results, voiceprint verification results, forged voice detection results, and communication behavior detection results.

[0059] In some embodiments, the core information of semantic consistency detection results, voiceprint verification results, forged speech detection results, and communication behavior detection results is summarized: The matching degree judgment of semantic consistency (e.g., "complete match," "partial match," "no match"), the similarity score of voiceprint verification (e.g., a value in the range of 0-100), the authenticity probability value of forged speech detection (e.g., a value in the range of 0-1), and the abnormality indicators of communication behavior detection (e.g., "normal," "slightly abnormal," "severely abnormal") are retained. Then, through a structured integration method, the four types of results are presented in a logical order of "semantics-voiceprint-authenticity-behavior," ensuring that key data of each type of result is not omitted or redundant. Finally, a multi-dimensional detection result that comprehensively reflects the judgment status of each dimension of the detected speech is generated. Integrating multi-dimensional key detection information provides a complete and accurate basis for subsequent calculation of the comprehensive risk score, ensuring the reliability of the risk assessment.

[0060] Step 307: Based on the multi-dimensional detection results, a comprehensive risk score is generated through a fusion algorithm.

[0061] In some embodiments, the sub-results in the multi-dimensional detection results are first quantified: semantic consistency detection is converted to "100 points for complete match, 60 points for partial match, and 0 points for non-match"; voiceprint verification directly uses the similarity score (0-100 points); forged voice detection converts the authenticity probability (0-1) into a 100-point scale (probability × 100); and communication behavior detection is quantified as "100 points for normal, 50 points for slight abnormality, and 0 points for severe abnormality". Then, a fusion algorithm (such as weighted summation) is used to allocate dimensional weights, where forged voice detection (associated with core authenticity determination) accounts for 40%, voiceprint verification (anchoring user identity) accounts for 30%, and semantic consistency and communication behavior detection (auxiliary verification) each account for 15%. Finally, the quantified results are multiplied by the weights and summed to generate a comprehensive risk score of 0-100 (the higher the score, the higher the authenticity of the voice and the lower the risk). In this embodiment, a fusion algorithm is used to achieve quantitative risk assessment, avoiding the one-sidedness of single-dimensional judgment and providing a reliable basis for subsequent accurate execution of processing strategies.

[0062] Step 308: If the overall risk score exceeds the preset threshold, trigger an alarm and refuse authentication or start a secondary verification process.

[0063] In some embodiments, a preset threshold is first determined based on the application scenario of the voice device. For example, in scenarios such as financial identity authentication and sensitive operation authorization, the preset threshold needs to be set to a higher value (e.g., 70 points) to meet high security requirements. In low-sensitivity scenarios such as daily voice communication, the threshold can be appropriately lowered (e.g., 50 points) to ensure that the threshold matches the risk tolerance of the scenario. When the comprehensive risk score exceeds the preset threshold, it indicates that the voice to be detected has a high risk of forgery, and the system will immediately trigger an alarm: on the one hand, it will issue a prompt tone or text pop-up locally through the voice device to inform that the current voice verification is abnormal; on the other hand, it will perform subsequent processing according to the operation type: if it involves critical authentication scenarios such as transfers or account logins, it will directly reject the current authentication request and block the operation; if it is a normal identity verification scenario, it will initiate a secondary verification process, guide the user to read aloud a randomly generated new instruction (e.g., a temporary short sentence), re-collect voice data for supplementary detection, and further confirm the authenticity of the voice. By quickly intercepting high-risk forged voices, security problems caused by false authentication are avoided. At the same time, by balancing security and user experience through secondary verification, the security and reliability of voice interaction are improved.

[0064] Step 309: If the overall risk score does not exceed the preset threshold, no processing operation will be performed.

[0065] In some embodiments, for scenarios such as daily voice communication and low-sensitivity information queries, the preset threshold is typically set at a low level (e.g., 50 points) to meet the basic security requirements of the scenario. For scenarios such as financial inquiries and general identity verification, the threshold is also reasonably set based on the scenario's risk tolerance (e.g., 70 points). When the comprehensive risk score does not exceed the preset threshold, it indicates that the authenticity of the voice being detected meets the requirements of the current scenario. The system does not perform any additional processing operations, allowing subsequent processes associated with the voice to proceed normally, such as direct execution of voice commands, automatic identity authentication, and smooth voice communication, without triggering alarms, blocking operations, or initiating secondary verification, thus avoiding interrupting the normal user's workflow. By ensuring the authenticity of the voice, unnecessary process intervention is reduced, ensuring the smoothness of normal voice interaction, and improving user experience and operational efficiency.

[0066] Figure 4 A flowchart of a forged voice detection method provided in this application embodiment is further shown, including the following steps: Step 401: Collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features.

[0067] Step 402: Obtain the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected.

[0068] Step 403: Perform multi-dimensional joint detection on the speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results.

[0069] Step 404: Calculate the comprehensive risk score based on the multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

[0070] For a description of steps 401-404, please refer to the description of steps 101-104 in the above embodiment. This embodiment will not repeat them in detail.

[0071] Step 405: Construct a fake speech feature database based on the speech data and comprehensive risk score, and optimize the fake speech recognition model based on the fake speech database.

[0072] In some embodiments, the voice data is first preprocessed to remove sensitive information such as user names and unique device identifiers, retaining only acoustic data related to forgery features. Then, a comprehensive risk score is used to label the forgery attributes of the voice data: a score ≤30 is labeled "high forgery risk," and 31-50 is labeled "medium forgery risk," ensuring that the data labeling matches the actual risk. Subsequently, the processed voice data is categorized and stored according to forgery technology type (e.g., AI synthesis, voice splicing, recording playback) and application scenario (e.g., financial authorization, daily communication), constructing a clearly structured forgery voice feature database. Next, using this database as a training data source, labeled samples are input into the forgery voice recognition model. By fine-tuning the model's feature extraction layer (e.g., convolutional kernel parameters) and decision layer parameters, the model's ability to recognize various forgery features in the database is enhanced, achieving model performance optimization. This step continuously supplements forgery feature samples, enabling the model to adapt to more forgery scenarios and improving the accuracy and generalization ability of forgery voice detection.

[0073] Figure 5 This is a schematic diagram of a forged voice detection device provided in an embodiment of this application, as shown below. Figure 5 As shown, it includes: a construction module 501, a generation module 502, a detection module 503, and a calling module 504.

[0074] Module 501 is configured to collect user speech samples, extract physiological acoustic features, and construct an initial reliable speech feature model based on the physiological features. The generation module 502 is configured to acquire the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected. The detection module 503 is configured to perform multi-dimensional joint detection of speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results. Module 504 is called and configured to calculate a comprehensive risk score based on the multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

[0075] In some examples of this embodiment, the construction module 501 is specifically configured to collect voice samples under the user's preset password and extract the physiological acoustic features of the voice samples when registering for the first time on the voice device. An initial reliable speech feature model is constructed based on physiological characteristics, including voiceprint features, intonation patterns, and pronunciation habits.

[0076] In some examples of this embodiment, the generation module 502 is specifically configured to acquire the speech to be detected, record the acquisition timestamp and environmental information of the speech to be detected; the speech device performs preprocessing operations on the speech to be detected to generate speech data, the preprocessing operations include noise reduction, framing, and feature extraction of the speech to be detected.

[0077] In some examples of this embodiment, the detection module 503 is specifically configured to: verify the semantic matching degree between the speech content to be detected and the challenge instruction, and generate a semantic consistency detection result; extract the voiceprint features of the speech to be detected, generate a voiceprint verification result based on the initial trusted speech feature model and the voiceprint features; use a forged speech detection model to determine whether it is forged speech, and generate a forged speech detection result; identify abnormal behavior by combining the context information of the voice device's communication frequency, IP address, and request mode, and generate a communication behavior detection result; and generate a multi-dimensional detection result based on the semantic consistency detection result, the voiceprint verification result, the forged speech detection result, and the communication behavior detection result.

[0078] In some examples of this embodiment, module 504 is specifically configured to generate a comprehensive risk score based on multi-dimensional detection results using a fusion algorithm; if the comprehensive risk score exceeds a preset threshold, an alarm is triggered and authentication is rejected or a secondary verification process is initiated; if the comprehensive risk score does not exceed the preset threshold, no processing operation is performed.

[0079] It should be noted that other corresponding descriptions of the functional units involved in the fake voice detection device provided in this embodiment can be found in [reference]. Figure 1 , Figure 2 , Figure 3 and Figure 4 The corresponding descriptions in [the document] will not be repeated here.

[0080] Based on the above, Figure 1 , Figure 2 , Figure 3 and Figure 4The embodiment illustrates a method for detecting spoofed speech. Correspondingly, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method. Figure 1 , Figure 2 , Figure 3 and Figure 4 This illustrates a method for detecting fake voice messages.

[0081] Based on the above, Figure 1 , Figure 2 , Figure 3 and Figure 4 The embodiment illustrates a method for detecting spoofed speech. Correspondingly, this embodiment also provides a computer program product on which a computer program is stored. When executed by a processor, this computer program implements the above-described method. Figure 1 , Figure 2 , Figure 3 and Figure 4 This illustrates a method for detecting fake voice messages.

[0082] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0083] Based on the above, Figure 1 , Figure 2 , Figure 3 and Figure 4 A method for detecting fake voice is shown, and Figure 5 To achieve the above objectives, the present application also provides an electronic device, such as a personal computer or a server, in the illustrated virtual device embodiment. This device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to implement the above-described virtual device. Figure 1 , Figure 2 , Figure 3 and Figure 4 This illustrates a method for detecting fake voice messages.

[0084] In some embodiments, the aforementioned physical device may further include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, an input unit such as a keyboard, etc., and optionally, a USB interface, a card reader interface, etc. In some embodiments, the network interface may include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0085] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0087] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for detecting forged voice, characterized in that, include: Collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features; Acquire the speech to be detected, and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected; The speech data is jointly detected in multiple dimensions based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results. A comprehensive risk score is calculated based on the multi-dimensional detection results. Different preset processing strategies are invoked based on the comprehensive risk score, and differentiated processing is performed on the speech to be detected according to the different preset processing strategies.

2. The method for detecting fake voice according to claim 1, characterized in that, Also includes: Based on wavelet transform convolution and frequency adaptive dilated convolution, the end-to-end fake speech detection model is optimized and adapted to generate the fake speech detection model.

3. The method for detecting fake voice according to claim 1, characterized in that, The process of collecting user voice samples, extracting physiological acoustic features, and constructing an initial reliable voice feature model based on the physiological features includes: During the initial registration on the voice device, voice samples under the user's preset password are collected, and the physiological acoustic features of the voice samples are extracted. The initial reliable speech feature model is constructed based on the physiological features, which include voiceprint features, intonation patterns, and pronunciation habits.

4. The method for detecting fake voice according to claim 1, characterized in that, The step of acquiring the speech to be detected and performing preprocessing operations on the speech to be detected to generate speech data includes: Acquire the speech to be detected, and record the acquisition timestamp and environmental information of the speech to be detected; The voice device performs preprocessing operations on the voice to be detected to generate the voice data. The preprocessing operations include noise reduction, frame segmentation, and feature extraction of the voice to be detected.

5. The method for detecting fake voice according to claim 1, characterized in that, The step of performing multi-dimensional joint detection on the speech data based on the fake speech detection model and the initial reliable speech feature model to generate multi-dimensional detection results includes: Verify the semantic matching degree between the speech content to be detected and the challenge instruction, and generate a semantic consistency detection result; Extract the voiceprint features of the speech to be detected, and generate a voiceprint verification result based on the initial reliable speech feature model and the voiceprint features; The forged speech detection model is used to determine whether the speech is forged, and a forged speech detection result is generated. By combining the contextual information of the voice device's communication frequency, IP address, and request pattern, abnormal behavior is identified, and communication behavior detection results are generated. The multi-dimensional detection results are generated based on the semantic consistency detection results, voiceprint verification results, forged voice detection results, and communication behavior detection results.

6. The method for detecting fake voice according to claim 1, characterized in that, The step of calculating a comprehensive risk score based on the multi-dimensional detection results, invoking different preset processing strategies based on the comprehensive risk score, and performing differentiated processing on the detected speech according to the different preset processing strategies includes: Based on the multi-dimensional detection results, a comprehensive risk score is generated through a fusion algorithm; If the comprehensive risk score exceeds a preset threshold, an alarm will be triggered and authentication will be rejected or a secondary verification process will be initiated. If the overall risk score does not exceed the preset threshold, no processing operation will be performed.

7. The method for detecting fake voice according to claim 1, characterized in that, Also includes: A fake speech feature database is constructed based on the speech data and the comprehensive risk score, and the fake speech recognition model is optimized based on the fake speech database.

8. A device for detecting fake voice, characterized in that, include: The construction module is configured to collect user voice samples, extract physiological acoustic features, and construct an initial reliable voice feature model based on the physiological features; The generation module is configured to acquire the speech to be detected and perform preprocessing operations on the speech to be detected to generate speech data of the speech to be detected. The detection module is configured to perform multi-dimensional joint detection on the speech data based on the fake speech detection model and the initial trusted speech feature model to generate multi-dimensional detection results; The calling module is configured to calculate a comprehensive risk score based on the multi-dimensional detection results, call different preset processing strategies based on the comprehensive risk score, and perform differentiated processing on the speech to be detected according to the different preset processing strategies.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the fake speech detection method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the fake voice detection method according to any one of claims 1-7.

Images