Content-assisted synthetic audio detection method

Abstract

The invention discloses a content-assisted synthetic audio detection method, which comprises the following steps of: extracting acoustic and natural language semantic features from each section of voice in parallel, determining the level of a collaborative alarm according to the two types of features, and on the basis, adjusting the level of the collaborative alarm according to a text intention and an acoustic style while detecting a synthetic voice. A natural language processing method is deeply introduced into a synthetic audio detection process, a semantic acoustic double-path conjoint analysis architecture is constructed, and the limitation that a traditional detection scheme only depends on acoustic fingerprints, spectrum features or model generation traces is broken through.

Description

Technical Field

[0001] This invention relates to the field of audio detection technology, and more specifically to a content-assisted synthetic audio detection method. Background Technology

[0002] Existing synthetic audio detection methods are mostly based on acoustic features (such as MFCC and Mel spectrum) and deep models (such as CNN and TDNN). These existing detection models are designed only for single-speaker speech, and both training and testing assume a single speaker. They lack modeling and processing mechanisms for multi-speaker aliasing speech commonly found in real-world scenarios. Therefore, when faced with complex audio such as multi-speaker aliasing, cross-dialogue, and background narration, they suffer from insufficient separation and feature aliasing, leading to a surge in false positives and false negatives. Furthermore, since the detection itself does not involve the speech content, it is difficult to further classify the synthesized speech in a refined manner, only outputting a binary result of true or false. It cannot further determine the synthesizer's intent category (such as promotion, fraud, customer service, malicious imitation, etc.) based on the content, making subsequent processing lack targeted approaches.

[0003] Therefore, the lack of existing methods in handling multi-speaker scenarios that are common in real-world situations leads to a sharp decline in performance in complex environments such as conferences, live streaming, and social voice communication, which severely limits their practicality and scalability. Summary of the Invention

[0004] In view of this, the present invention provides a content-assisted synthetic audio detection method, which integrates the characteristics of acoustics and natural language to construct a semantic-acoustic dual-path joint analysis architecture, thereby achieving more accurate synthetic audio detection.

[0005] This invention provides a content-assisted synthesized audio detection method, which specifically includes the following steps:

[0006] Step 1: Extract text data from the input audio segment, process it to obtain a text segment queue, record the timestamp of the corresponding speech stream, and use natural language processing to obtain the judgment type, timestamp, judgment confidence and judgment reason from the text segment queue to form the first judgment result; at the same time, preprocess the input audio segment to obtain speech segment, record the timestamp of the corresponding speech stream, and use the synthesized audio detection model to detect the speech segment to obtain the abnormal probability score and timestamp to form the second judgment result.

[0007] Step 2: Determine the speech segments that are in the same time window as the text segments based on the timestamps, establish a one-to-many mapping relationship between the first and second judgment results, and complete the time alignment of the first and second judgment results;

[0008] Step 3: Within the time window, calculate the total risk base score, continuous gain score, and high score density based on the second judgment result, and then calculate the joint risk score for that time window; based on the first judgment result and the joint risk score, determine the level of triggering the coordinated alarm.

[0009] Step 4: The natural language processing model determines the intent of the text segment that generates the collaborative alarm based on the judgment type of the first judgment result; determines the acoustic synthesis ratio of the audio segment, and increases the priority of the collaborative alarm when the judgment type, confidence level and acoustic synthesis ratio all meet the conditions.

[0010] Step 5: Generate a structured analysis report, including the time range, risk assessment results, purpose classification labels, key judgment criteria, and recommended handling level.

[0011] Further, the method for calculating the total risk base score, continuous gain score, and high score density within the time window based on the second judgment result is as follows: accumulate the abnormal probability scores in all second judgment results to obtain the total risk base score for the time window; select speech segments that are consecutive, exceed a set number, and have abnormal probability scores higher than the first threshold as high score segment groups, and determine the continuous gain score for the time window based on the number of high score segment groups contained within the time window; if the maximum abnormal probability score within the time window is greater than the second threshold, and the abnormal probability scores of adjacent speech segments are all greater than the third threshold, then set the high score density of the time window to 1.

[0012] Furthermore, the continuous gain score is calculated as follows: when the time window contains N high-resolution segment groups, the continuous gain score is N.

[0013] Furthermore, the method for calculating the joint risk score for this time window is as follows: the sum of the total risk base score, the continuous gain score, and the high score density is used as the joint risk score.

[0014] Furthermore, the method for determining the level of triggering the collaborative alarm based on the first judgment result and the joint risk score is as follows: if the judgment type of the first judgment result is fraud and the confidence level is higher than the fourth threshold, and the joint risk score is greater than the dynamic threshold, then a severe level collaborative alarm is triggered; if the judgment type is promotion and marketing, and both the joint risk score and the high score density are greater than the dynamic threshold, then a severe level collaborative alarm is triggered; if the judgment type is normal, but the joint risk score is greater than the dynamic threshold, then a lower level collaborative alarm is triggered.

[0015] Furthermore, the synthetic audio detection model takes the original audio data as input, uses the Wav2Vec 2.0 pre-trained model to extract deep feature representations of the audio, and then uses CNN as a back-end classifier to classify the features extracted by Wav2Vec 2.0, finally outputting a floating-point score as the basis for judgment.

[0016] Furthermore, a multi-speaker aliased speech dataset for synthetic audio detection is constructed based on a single-speaker speech dataset, and the synthetic audio detection model is trained using the multi-speaker aliased speech dataset.

[0017] Furthermore, the construction method of the multi-speaker aliasing speech dataset is as follows: first, two audio recordings from different speakers are randomly selected; the long audio recording is kept intact, and the short audio recording is first subjected to random volume attenuation within a set ratio range; then, the attenuated short audio recording is randomly superimposed onto any consecutive time period of the long audio recording; finally, highly realistic aliasing samples are generated through linear mixing.

[0018] Furthermore, the stated intentions include the intention to induce money transfer fraud, the intention to impersonate an authority fraud, the intention to promote limited-time sales and marketing, and the intention to guide clicks and marketing.

[0019] Furthermore, the risk assessment result is a risk type, with risk levels ranging from high to low including high-risk synthetic fraud, marketing promotion robots, and suspected synthetic audio and normal audio.

[0020] Beneficial effects:

[0021] This invention extracts acoustic and natural language semantic features in parallel for each speech segment, determines the level of collaborative alarm based on the two types of features, and adjusts the level of collaborative alarm according to the text intent and acoustic style while detecting synthesized speech, ultimately forming a structured analysis report. By deeply introducing natural language processing methods into the synthesized audio detection process, a semantic-acoustic dual-path joint analysis architecture is constructed, breaking through the limitations of traditional detection schemes that rely solely on acoustic fingerprints, spectral features, or model generation traces.

[0022] Furthermore, to address the common multi-speaker aliasing problem in real-world speech scenarios, a dedicated data augmentation and model adaptation mechanism was designed. Without relying on real multi-speaker recordings, highly realistic multi-speaker aliasing training samples were generated based on existing single-speaker speech datasets. Subsequently, the basic detection model was fine-tuned using this synthetic dataset, enabling the model to retain its single-speaker speech detection capabilities while significantly improving its adaptability to complex acoustic environments such as background noise, interruptions, and overlapping sound sources. This effectively reduces the false alarm rate and false negative rate in multi-speaker scenarios, ensuring that the system maintains stable detection performance in uncontrolled scenarios such as real-world calls, conference recordings, and public environment data collection. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the processing flow of a content-assisted synthesized audio detection method provided by the present invention.

[0024] Figure 2 This diagram illustrates the process of constructing a multi-speaker aliased speech dataset and optimizing a synthetic audio detection model in a content-assisted synthetic audio detection method provided by the present invention. Detailed Implementation

[0025] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0026] The present invention provides a content-assisted synthetic audio detection method, the core idea of ​​which is to extract acoustic and natural language semantic features in parallel for each speech segment, determine the level of collaborative alarm based on the two types of features, and adjust the level of collaborative alarm according to the text intent and acoustic style while detecting synthetic speech, and finally form a structured analysis report.

[0027] This invention provides a content-assisted synthesized audio detection method, the processing flow of which is as follows: Figure 1 As shown, the specific steps include:

[0028] Step 1: Extract text data from the input audio segment using Automatic Speech Recognition (ASR), then clean and standardize the text data to obtain a structured text segment queue. Simultaneously, record the timestamp of the corresponding speech stream for each text segment. Then, input the text segment queue into a Natural Language Processing (NLP) model to obtain the judgment type, timestamp, judgment confidence, and judgment reason for each text segment, forming the first judgment result. At the same time, remove interference noise from the input audio segment through preprocessing to obtain speech segments. Record the timestamp of the corresponding speech stream for each speech segment. Then, use a synthetic audio detection model to detect the speech segments to obtain their abnormality probability score and corresponding timestamp, forming the second judgment result.

[0029] Existing methods, neglecting semantic content, fail to identify anomalies in speech context, intent coherence, or phrasing. They also struggle to segment detection results into risk scenarios, such as distinguishing between fraud, marketing, and legitimate content. Furthermore, they cannot dynamically adjust judgment thresholds based on semantic confidence, resulting in insufficient generalization ability in complex dialogues or adversarial samples. Relying solely on NLP for judgment lacks acoustic forgery evidence, making it difficult to confirm the authenticity of speech, especially in cases of semantic ambiguity, neutral sentence structure, or ambiguous expressions, leading to misjudgments. This invention simultaneously extracts semantic features and acoustic anomaly scores from audio, achieving information complementarity and cross-validation at the decision-making level: the semantic path provides intent classification and confidence guidance, while the acoustic path provides forgery evidence and density distribution. The combined approach can identify not only deeply forged samples that are "semantically plausible but acoustically forged," but also high-risk phrasing that is "acoustically plausible but semantically contradictory." This allows for finer-grained risk classification, more adaptive decision-making strategies, and stronger anti-interference and false alarm suppression capabilities without relying on a single modality, significantly improving the system's accuracy and robustness in real-world complex scenarios.

[0030] Specifically, the process of generating the first determination result includes the following steps:

[0031] Step 1.1.1: Data input and preprocessing.

[0032] Text data is extracted from the input audio segments using automatic speech recognition, and then cleaned and standardized. The timestamp information for each text segment is also recorded.

[0033] Step 1.1.2, Semantic Intent Reasoning.

[0034] The speech text information transcribed by ASR is input into the NLP model for behavior analysis to obtain the judgment result, confidence level and reason for judgment for each text segment.

[0035] Furthermore, to ensure semantic coherence and contextual integrity, this invention adopts an incremental text input mechanism, specifically: each time input occurs, the transcribed text of the current speech segment is automatically concatenated with all previous segment texts, and then used as the new input for the NLP model.

[0036] Furthermore, since most mobile terminals currently carry a large local language model, in order to minimize system size and complexity, this invention will prioritize using the large language model that is already installed on the mobile terminal.

[0037] Step 1.1.3, Output the results.

[0038] The model predicts text data and outputs the judgment type, corresponding timestamp, judgment confidence level and judgment reason for each text segment, thus forming the first judgment result.

[0039] The process of generating the second determination result includes the following steps:

[0040] Step 1.2.1: Audio preprocessing.

[0041] The original audio clips are standardized to remove interference noise that may seriously affect the detection.

[0042] Step 1.2.2: Synthesized audio detection.

[0043] The synthetic audio detection model constructed in this invention is used to detect each input speech segment. The synthetic audio detection model constructed in this invention takes raw audio data as input, uses a Wav2Vec 2.0 pre-trained model to extract deep feature representations of the audio, then uses a CNN as a backend classifier to classify the features extracted by Wav2Vec 2.0, and finally outputs a floating-point score as the decision criterion.

[0044] Among them, the Wav2Vec 2.0 pre-trained model can effectively capture semantic information and acoustic features in audio.

[0045] Because real-world speech scenarios are noisy and varied, with multiple people speaking simultaneously and voices overlapping, current mainstream synthetic audio detection solutions are all designed based on the assumption of "single speaker" clean speech, lacking the ability to model overlapping speech. Faced with such complex inputs, the system is prone to misjudgment. Experiments show that some detection models have a false positive rate of over 40% in multi-speaker scenarios, which is seriously deviates from actual needs and is basically not feasible for real-world deployment.

[0046] Furthermore, to enhance the model's ability to detect multi-speaker scenarios, this invention constructs a multi-speaker aliased speech dataset for synthetic audio detection based on existing single-speaker speech datasets. This dataset uses high-quality single-speaker speech as a foundation and employs a unified and controllable mixing strategy to generate two types of multi-speaker aliased audio: real-to-real and real-to-synthetic. The dataset construction and model optimization process is as follows: Figure 2 As shown, the specific steps include: first, randomly selecting two audio clips from different speakers; keeping the long audio clip intact, applying a random volume attenuation within a set range to the short audio clip to simulate spatial distance or differences in sound source intensity; then, randomly superimposing the attenuated short audio clip onto any consecutive time segment of the long audio clip, without truncating or stretching, to recreate the natural occurrence of interruptions or background voices in a real dialogue; finally, generating highly realistic aliasing samples through linear mixing; using the dataset constructed using the above process, the synthetic audio detection model constructed in this invention is fine-tuned, significantly improving the detection accuracy for multi-speaker aliased speech without affecting its basic performance, thus optimizing the synthetic audio detection model.

[0047] Steps 1, 2, and 3: Output the results.

[0048] Output the anomaly probability score for each audio segment and its corresponding timestamp information.

[0049] This invention improves detection accuracy while enhancing the traceability of detection results through structured score recording and spatiotemporal indexing mechanisms. During inference, the system independently calculates and fully retains the anomaly score for each audio segment, binding it with a precise timestamp to form a time-stamped score sequence. When an audio segment is deemed high-risk, the system automatically traces back the score distribution of all sub-segments within that window, pinpointing the specific time period of the anomaly peak and associating it with the corresponding speech content and NLP semantic tags. This allows users to quickly focus on problematic segments, improving review efficiency and decision-making transparency. In terms of classification, the system overcomes the limitations of traditional binary true / false output, achieving closed-loop management from fine-grained detection to precise response based on confirmed audio anomalies.

[0050] Step 2: Since the time range of the audio segment corresponding to the text segment is larger than that of the speech segment, the speech segment that is in the same time window as the text segment is determined based on the timestamps of the audio segments corresponding to the text segment and the speech segment recorded in Step 1. Thus, a one-to-many mapping relationship is established between the first judgment result and the second judgment result, and the time alignment of the first judgment result and the second judgment result is completed.

[0051] Step 3: Obtain the abnormal probability score from the second judgment result within the time window, and sum all abnormal probability scores to obtain the total risk base score for the time window; select speech segments that are consecutive, exceed a set number, and have abnormal probability scores higher than the first threshold as high-scoring segment groups, and determine the continuous gain score for the time window based on the number of high-scoring segment groups contained within the time window; if the maximum abnormal probability score within the time window is greater than the second threshold, and the abnormal probability scores of adjacent speech segments are all greater than the third threshold, then set the high-scoring density of the time window to 1; calculate the joint risk score for the time window based on the total risk base score, continuous gain score, and high-scoring density.

[0052] The continuous gain score can be calculated as follows: if the time window contains N high-scoring segment groups, then the continuous gain score is N. The joint risk score can be calculated as the sum of the total risk base score, the continuous gain score, and the high-scoring density, or it can be a weighted sum based on the impact factors of these three factors on the risk.

[0053] Step 4: Within the time window, if the first judgment result is classified as fraud and the confidence level is higher than the fourth threshold, and the joint risk score is greater than the dynamic threshold, a severe level collaborative alarm is triggered; if the judgment type is promotion and marketing, and both the joint risk score and the high score density are greater than the dynamic threshold, a severe level collaborative alarm is triggered; if the judgment type is normal, but the joint risk score is greater than the dynamic threshold, a lower level collaborative alarm is triggered.

[0054] Step 5: The natural language processing model performs end-to-end intent reasoning on the text segment that generates the collaborative alarm based on the judgment type of the first judgment result to determine the intent of the text segment; acoustic detection is performed on the audio segment to determine the acoustic synthesis ratio. When the judgment type is fraud and the confidence level is greater than the fifth threshold, and the acoustic synthesis ratio is greater than the threshold, the priority of the collaborative alarm is increased.

[0055] Step 6: Generate a structured analysis report, including the time range, risk assessment results, purpose classification labels, key judgment criteria, and recommended handling level.

[0056] The structured analysis report generated by this invention supports standard data interface output and source tracing of original voice / text fragments, facilitating integration with subsequent risk control processes. The risk assessment result is categorized by risk type, from high to low, including high-risk synthetic fraud, marketing bots, suspected synthetic audio, and normal audio.

[0057] Implementation Case:

[0058] This embodiment employs a content-assisted synthesized audio detection method provided by the present invention, which realizes a dual-path joint analysis based on semantics and acoustics, and then provides a comprehensive judgment result through a joint decision module. Specifically, it includes the following steps:

[0059] S1, NLP detection path.

[0060] S1.1 Data Input and Preprocessing.

[0061] Text data is extracted from input audio segments using automatic speech recognition, and then cleaned and standardized. This includes removing irrelevant characters, word segmentation, stop word filtering, and stemming. The preprocessed text data is then converted into a structured format to facilitate subsequent feature extraction. Timestamp information for each text segment is also recorded.

[0062] S1.2 Semantic Intent Reasoning.

[0063] The speech text information transcribed by ASR is input into the NLP model for behavioral analysis, obtaining the judgment result, confidence level, and reasoning for each text segment. Each time input occurs, the transcribed text of the current speech segment is automatically concatenated with all previous segment texts as new input to the NLP model. The first input is segment 1 text, the second input is segment 1 and segment 2 texts, and so on, ensuring that the model always performs semantic analysis and risk assessment based on the complete dialogue context.

[0064] S1.3, Output the results.

[0065] The model makes predictions on text data and outputs the classification type of each text segment, its corresponding timestamp information, the confidence level of the classification, and the reason for the classification.

[0066] S2, Synthetic Audio Detection Path.

[0067] S2.1 Audio preprocessing.

[0068] The original audio signal is standardized to remove interference noise that may seriously affect the detection, including fixed sampling rate resampling and cutoff of silent segments before and after the speech.

[0069] S2.2 Synthetic audio detection.

[0070] The synthetic audio detection model constructed using this invention detects each input speech segment. The model takes the original audio data as input and outputs a floating-point score as the basis for judgment.

[0071] S2.3, Output the results.

[0072] Output the anomaly probability score for each audio segment and its corresponding timestamp information.

[0073] S3. Joint decision-making: The final detection result is determined jointly based on the output results of the two paths (including the anomaly score with timestamps).

[0074] S3.1 Time Alignment: To address the time granularity mismatch between NLP semantic detection (long audio segments) and synthesized audio detection (short audio segments), the system dynamically establishes a mapping relationship between semantic units and acoustic detection results based on the millisecond-level timestamps of the original speech stream. The system records the specific start and end timestamps of each detection segment in the original speech. Since the lengths of NLP detection segments and synthesized speech detection segments are different, the system associates the time-corresponding NLP segments with the synthesized audio segments to ensure that the audio judgment associated with each text segment comes from the corresponding segment within its physical time coverage, avoiding misassociations caused by cutting offsets or boundary misalignments, and ensuring the spatiotemporal consistency of subsequent joint analysis.

[0075] S3.2 Collaborative Detection: Based on time alignment, the system performs risk density modeling on multiple synthesized audio anomaly score sequences covered within an NLP segment window, comprehensively calculating the total risk, high-score density, and continuity gain. Specifically, firstly, the sum of all segment anomaly scores is accumulated as the base score for the total risk; secondly, the number of consecutive segments with scores higher than the anomaly judgment threshold is counted, and each group of consecutive high-score segments is additionally weighted with a continuity gain score; simultaneously, if the peak anomaly score of an audio segment within the window is higher than a specific value and the scores of adjacent segments are both higher than the judgment base threshold, the value of the high-score density index is incremented by 1. Afterward, the system comprehensively calculates the joint risk score corresponding to the window based on the total risk, high-score density, and continuity gain. The score is conditionally fused with the scenario classification confidence level output by the NLP: if the NLP classification is "fraud" and the confidence level is >0.7, the joint risk score must exceed a dynamic threshold to trigger an alarm; if it is "promotion and marketing," both the joint risk score and the high score density must meet the requirements; if it is "normal" but the joint risk score is abnormal, a lower level warning is given. Risk types, from highest to lowest, include: high-risk synthetic fraud, marketing bots, suspected synthetic audio, and normal audio.

[0076] S3.3 Purpose Inference: After a collaborative alarm is triggered, the system automatically initiates a semantic intent classification process, where the NLP model directly performs end-to-end intent inference on the speech-to-text content. The model integrates contextual keywords, sentence structure features, and typical speech patterns to autonomously determine semantic risk tendencies and outputs three intent labels: "fraud," "promotion / marketing," or "normal." For example, it identifies fraudulent intents such as "money transfer inducement" or "impersonating authority," or marketing intents such as "limited-time promotion" or "click-guided marketing." Acoustic detection results (such as the proportion of synthesized audio) do not participate in the model's inference process but are used as a posterior confidence enhancement factor: when the model outputs "fraud" with high confidence and the acoustic synthesis proportion is significantly high, the system increases the alarm priority.

[0077] S3.4 Final Output: The system generates a structured analysis report, including the time interval, risk assessment results, purpose classification labels, key judgment criteria (such as "7 / 8 audio segment is identified as fake + semantics include 'verification code' 'transfer'") and suggested handling level. It supports standard data interface output and original voice / text segment tracing, which facilitates integration with risk control systems.

[0078] Finally, it should be noted that those skilled in the art can obviously make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0079] The above description is merely one embodiment of the present invention, and should not be construed as limiting the scope of the invention. Any structural changes made based on the present invention, as long as they do not depart from the essence of the invention, should be considered as falling within the protection scope of the present invention and subject to its restrictions. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the method described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0080] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / method.

[0081] The technical solutions of the present invention have been described above with further embodiments listed. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0082] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A content-assisted synthetic audio detection method, characterized in that, Specifically, the following steps are included: Step 1: Extract text data from the input audio segment, process it to obtain a text segment queue, record the timestamp of the corresponding speech stream, and use natural language processing to obtain the judgment type, timestamp, judgment confidence and judgment reason from the text segment queue to form the first judgment result; at the same time, preprocess the input audio segment to obtain speech segment, record the timestamp of the corresponding speech stream, and use the synthesized audio detection model to detect the speech segment to obtain the abnormal probability score and timestamp to form the second judgment result. Step 2: Determine the speech segments that are in the same time window as the text segments based on the timestamps, establish a one-to-many mapping relationship between the first and second judgment results, and complete the time alignment of the first and second judgment results; Step 3: Calculate the total risk base score, continuous gain score, and high score density based on the second judgment result within the time window, and then calculate the joint risk score for that time window. Based on the initial judgment result and the joint risk score, determine the level that triggers the collaborative alarm; Step 4: The natural language processing model determines the intent of the text segment that generates the collaborative alarm based on the judgment type of the first judgment result; Determine the acoustic synthesis ratio of the audio segment, and increase the priority of the collaborative alarm when the judgment type, confidence level and acoustic synthesis ratio all meet the conditions. Step 5: Generate a structured analysis report, including the time range, risk assessment results, purpose classification labels, key judgment criteria, and recommended handling level.

2. The synthetic audio detection method according to claim 1, characterized in that, The method for calculating the total risk base score, continuous gain score, and high-score density within a time window based on the second judgment result is as follows: accumulate the abnormal probability scores in all second judgment results to obtain the total risk base score for that time window; select speech segments that are consecutive, exceed a set number, and have abnormal probability scores higher than the first threshold as high-score segment groups, and determine the continuous gain score for that time window based on the number of high-score segment groups contained within that time window; if the maximum abnormal probability score within that time window is greater than the second threshold, and the abnormal probability scores of adjacent speech segments are all greater than the third threshold, then set the high-score density of that time window to 1.

3. The synthetic audio detection method according to claim 2, characterized in that, The continuous gain score is calculated as follows: when the time window contains N high-resolution segment groups, the continuous gain score is N.

4. The synthetic audio detection method according to claim 1, characterized in that, The method for calculating the joint risk score for this time window is as follows: the sum of the total risk base score, the continuous gain score, and the high score density is used as the joint risk score.

5. The synthetic audio detection method according to claim 1, characterized in that, The method for determining the level of triggering a collaborative alarm based on the first judgment result and the joint risk score is as follows: if the judgment type of the first judgment result is fraud and the confidence level is higher than the fourth threshold, and the joint risk score is greater than the dynamic threshold, then a severe level collaborative alarm is triggered; if the judgment type is promotion and marketing, and both the joint risk score and the high score density are greater than the dynamic threshold, then a severe level collaborative alarm is triggered; if the judgment type is normal, but the joint risk score is greater than the dynamic threshold, then a lower level collaborative alarm is triggered.

6. The synthetic audio detection method according to claim 1, characterized in that, The synthetic audio detection model takes the original audio data as input, uses the Wav2Vec 2.0 pre-trained model to extract deep feature representations of the audio, and then uses CNN as a back-end classifier to classify the features extracted by Wav2Vec 2.0, finally outputting a floating-point score as the basis for judgment.

7. The synthetic audio detection method according to claim 1, characterized in that, A multi-speaker aliased speech dataset for synthetic audio detection is constructed based on a single-speaker speech dataset, and the synthetic audio detection model is trained using the multi-speaker aliased speech dataset.

8. The synthetic audio detection method according to claim 7, characterized in that, The multi-speaker aliasing speech dataset is constructed as follows: First, two audio recordings from different speakers are randomly selected; the long audio recording is kept intact, and the short audio recording is first subjected to random volume attenuation within a set ratio range; then, the attenuated short audio recording is randomly superimposed onto any consecutive time period of the long audio recording; finally, highly realistic aliasing samples are generated through linear mixing.

9. The method for detecting synthesized audio according to claim 1, characterized in that, The stated intentions include the intention to induce money transfer fraud, the intention to impersonate an authority fraud, the intention to promote limited-time sales and marketing, and the intention to guide clicks and marketing.

10. The method for detecting synthesized audio according to claim 1, characterized in that, The risk assessment result is the risk type, and the risk level from high to low includes high-risk synthetic fraud, marketing promotion robots, suspected synthetic audio and normal audio.

Images