Real-time fake audio detection method and system based on complexity and dynamic routing

Abstract

The application discloses a real-time fake audio detection method and system based on complexity and dynamic routing. The method first divides the network audio stream into discrete audio frames for feature extraction, obtaining pre-check features, and encapsulating the audio frames to obtain standardized detection packages. Second, the standardized detection packages are received, effective audio frames are obtained, the spectral centroid sequence is extracted and the acoustic information density is calculated to obtain the task priority score. Then, based on the priority score, the dynamic computing power allocation and linkage scheduling are combined with the discriminant link. Finally, after the dynamic computing power allocation and linkage scheduling, the detection package is sent to the discriminant link for fake audio detection, and the detection package that meets the preset review condition is executed to obtain the final detection result. The application can maintain the reliability of fake audio detection while improving the computing power utilization rate and detection throughput capacity in high-concurrency real-time detection scenarios, and taking into account the detection efficiency and detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of fake audio detection technology, and specifically relates to a real-time fake audio detection method and system based on complexity and dynamic routing. Background Technology

[0002] With the development of audio generation technologies such as text-to-speech synthesis, speech conversion, neural vocoders, and diffusion models, generated audio is increasingly approaching the real human voice in terms of timbre, rhythm, emotion, and naturalness. Such forged audio can be applied to real-time voice interaction scenarios such as live streaming, voice calls, online meetings, and content broadcasting, posing a strong deceptive and propagation risk. Therefore, rapid and accurate detection of forged audio in real-time voice interaction scenarios is of great significance.

[0003] Existing methods for detecting fake audio typically extract acoustic features, deep semantic features, or discrete speech representations from the audio, and then use machine learning models or deep neural network models to determine the authenticity of the audio. For example, some methods identify artifacts in synthetic audio based on acoustic features such as spectrum, phase, and cepstral coefficients; some methods utilize self-supervised speech models to extract deep continuous features to improve the detection capability of complex fake audio; and some methods are based on discrete speech word sequence modeling to analyze anomalous features in the sequence distribution of generated audio. These methods can improve the accuracy of fake audio detection to some extent.

[0004] However, in high-concurrency real-time speech detection scenarios, existing technologies still have the following shortcomings:

[0005] Existing audio detection methods typically use a uniform model to process different input audio, failing to adequately consider the varying detection difficulty of different audio segments. Low-complexity audio segments that are easily identifiable, such as silent segments, noisy segments, or poorly synthesized audio, may still be fed into detection models with high GPU memory usage, leading to unnecessary computational consumption and inference latency. Current one-size-fits-all detection methods struggle to balance detection efficiency and accuracy.

[0006] Existing high-concurrency audio detection architectures typically rely on static load balancing or fixed distribution strategies, lacking a comprehensive understanding of the complexity of the detection task and the real-time load status of computing nodes. When a large number of audio streams enter the system simultaneously, some computing nodes are prone to task backlog and resource contention, leading to increased response tail latency and affecting the stability of real-time detection. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a real-time spoofed audio detection method and system based on complexity and dynamic routing. Firstly, this invention employs an adaptive routing strategy based on task complexity, specifically designed for high-concurrency real-time spoofed audio detection. By pre-evaluating the detection difficulty of audio segments and assigning them to matching computational links, wasted computational resources can be effectively avoided, ensuring high detection performance even under high load. Furthermore, a dynamic computing power and coordinated scheduling mechanism is adopted, combining real-time load status to allocate computing resources, thereby reducing network congestion and suppressing response tail latency. This method can improve overall throughput while maintaining the accuracy and generalization ability of spoofed audio recognition.

[0008] The basic content of the technical solution of this invention:

[0009] In one aspect, this invention provides a real-time fake audio detection method based on complexity and dynamic routing, which consists of four parts: audio standardization, complexity assessment, dynamic computing power allocation and linkage scheduling, model detection and result output.

[0010] (1) Audio standardization: In order to adapt to the processing requirements of high-concurrency real-time audio streams, the present invention divides the continuous network audio stream into discrete audio frames and extracts features from the audio frames to obtain pre-detection features for complexity evaluation; then the audio frames and their pre-detection features are encapsulated to obtain a standardized detection package, which serves as the input for subsequent complexity evaluation.

[0011] (1.1) Streaming slicing: For a continuously input audio data stream, a streaming sliding window mechanism is used to extract continuous audio data to obtain a set of audio frames. The data in the current audio frame is equivalently divided into multiple continuous subframes. The part that overlaps with the previous audio frame is used as a historical subframe, and the part that enters the current audio frame as the sliding window advances is used as a new subframe.

[0012] (1.2) Incremental Feature Extraction: A circular buffer mechanism is introduced to perform incremental feature calculation. The repeated calculation of overlapping data is skipped, and the signal-to-noise ratio is calculated only for the newly added subframe and defined as the incremental feature. Then, the incremental feature is concatenated with the historical features read from the circular buffer to form the pre-detection features of the current audio frame, and the circular buffer is updated synchronously.

[0013] (1.3) Encapsulation: The current audio frame and its pre-detection features are serialized and encapsulated, and a globally unique task tracking identifier (Task ID) is added to obtain a standardized detection packet. Among them, the pre-detection features are used to calculate the task priority score, and the task tracking identifier is used for subsequent task routing and preemptive reflow.

[0014] (2) Complexity assessment: This step receives the standardized detection package output by the audio standardization stage, reads the pre-detection features in it, and determines whether the current audio frame is a valid audio frame based on the pre-detection features; for audio frames that are determined to be valid, their spectral centroid sequence is further extracted and the acoustic information density is calculated, thereby quantifying the detection value of the current audio frame and obtaining the task priority score.

[0015] (2.1) Information density quantization: Based on the signal-to-noise ratio in the pre-detection features, determine whether the current audio frame is a valid audio frame. If the signal-to-noise ratio of the current audio frame is within the preset normal range, it is determined to be a valid audio frame. Extract the spectral centroid sequence of the valid audio frame, calculate the variance of the spectral centroid sequence, and define the acoustic information density of the current audio frame accordingly.

[0016] (2.2) Initial Priority Calculation: The task priority score is calculated based on the signal-to-noise ratio and acoustic information density, and this score is written into the corresponding standardized detection packet to form a detection packet carrying the task priority score, which awaits dynamic computing power allocation and coordinated scheduling. This task priority score is used to accurately distinguish between high-value core speech and low-value stable background noise, and serves as the basis for subsequent dynamic routing.

[0017] (3) Dynamic computing power allocation and linkage scheduling: After receiving the detection packet carrying the task priority score, the detection packet is dynamically routed to the shallow discrimination link or the deep discrimination link for detection by the model instance according to the current load of the discrimination link and the task priority score of the detection packet.

[0018] (3.1) Backend load quantization: The number of queued tasks and GPU memory usage ratio of model instances in the shallow and deep discrimination links are obtained in real time, and the current detection load rate is quantitatively calculated. The detection load rate is used for subsequent dynamic routing and elastic scaling of model instances, and the model instances are used for audio authenticity detection.

[0019] (3.2) Dynamic Routing: A dynamic interception threshold is generated in real time based on the deep detection load rate, and the task priority score is obtained from the detection packet. The task priority score is compared with the dynamic interception threshold. If the task priority score is greater than or equal to the dynamic interception threshold, the detection packet is routed to the deep discrimination link for high-precision detection. If the task priority score is less than the dynamic interception threshold, the detection packet is determined to be below the current deep discrimination link admission requirements, and the detection packet is downgraded and diverted to the shallow discrimination link for fast detection.

[0020] (3.3) Elastic scaling and concurrency control of model instances: For model instances, the number of target model instances is scaled up or down horizontally based on the detection load rate. At the micro level, this invention strictly limits the dynamic concurrency of model instances based on the remaining video memory of the host GPU to achieve task backpressure, and forcibly cleans up the underlying tensor video memory after inference, forming a dual resource protection system.

[0021] (4) Model Detection and Result Output: After dynamic computing power allocation and linkage scheduling, the detection packets are sent to model instances in the shallow or deep discrimination links for spoofed audio detection, obtaining preliminary detection results for the audio segments corresponding to each detection packet. When the preliminary detection result output by the detection packet in the shallow discrimination link is within a preset uncertainty range, the corresponding detection packet is returned to the deep discrimination link as a detection packet to be verified for detection. Finally, the detection results for the audio segments corresponding to each detection packet are output.

[0022] (4.1) Detection model deployment: A shallow discrimination model is deployed in the shallow discrimination link and a deep discrimination model is deployed in the deep discrimination link; both the shallow discrimination model and the deep discrimination model are used for fake audio detection, and both are configured to receive audio input with a uniform format and output the classification result of whether the audio is real audio or fake audio.

[0023] (4.2) Detection packet decapsulation and input processing: After receiving the detection packet, the model instance decapsulates the detection packet to obtain the audio segment to be detected, and performs input normalization processing on the audio segment to be detected to obtain the audio that meets the input requirements of the corresponding discrimination model.

[0024] (4.3) Model detection: The processed audio sequence is input into the corresponding discrimination model. The discrimination model extracts and fuses the acoustic features, spectral change features and temporal change features in the audio, and outputs the confidence level of the audio segment corresponding to the current detection package. The confidence level is used to characterize the probability that the audio segment belongs to fake audio.

[0025] (4.4) Preemptive backflow mechanism: When the confidence of a task that has been downgraded to a shallow discrimination link is in an uncertain range, the task status is marked as pending preemptive backflow. The task will bypass the ordinary queue by using its globally unique Task ID in its detection packet and will be inserted into the head of the deep discrimination link for detection.

[0026] (4.5) Authenticity judgment: Based on the confidence level output by the discrimination model and the preset authenticity judgment rules, generate the detection results of the audio segment corresponding to the current detection package.

[0027] In another aspect, the present invention also provides a real-time fake audio detection system based on complexity and dynamic routing, comprising the following modules:

[0028] The standardized detection package module is used to segment a continuous network audio stream into discrete audio frames for feature extraction to obtain pre-detection features; then, the audio frames and their pre-detection features are encapsulated to obtain a standardized detection package.

[0029] The task priority module is used to receive standardized detection packets, read pre-detection features to obtain valid audio frames, extract spectral centroid sequences and calculate acoustic information density to obtain task priority scores.

[0030] The link discrimination module, based on priority scores, performs dynamic computing power allocation and coordinated scheduling in conjunction with the discriminative links.

[0031] The fake audio detection output module is used to send the detection packets to the discrimination link after dynamic computing power allocation and linkage scheduling to detect fake audio and obtain the detection result of the corresponding audio segment for each detection packet.

[0032] Beneficial effects of this invention:

[0033] (1) By using streaming feature incremental extraction based on sliding window and circular buffer, this invention eliminates redundant calculations in feature extraction of adjacent audio frames, reduces preprocessing delay, and can meet the timeliness requirements of high-concurrency real-time detection more quickly.

[0034] (2) This invention quantifies task priority through acoustic information density and performs dynamic routing in conjunction with backend load. This mechanism avoids the waste of computing power caused by blind detection in a one-size-fits-all manner, and makes the computing resources of deep discrimination links preferentially allocated to high-priority, high-detection-value detection packets, thereby improving resource utilization.

[0035] (3) For high-traffic scenarios, this invention introduces a dynamic computing power allocation mechanism that combines elastic scaling of model instances with concurrency control. This mechanism not only provides the ability to expand horizontally on demand, but also constrains the dynamic concurrency within the model instance by the remaining GPU memory and cleans up the GPU memory after inference, thereby reducing the risk of GPU memory overflow and ensuring availability under high concurrency impact.

[0036] (4) The present invention designs shallow and deep discrimination links with preemptive backflow. This structure can prioritize backflowing tasks with uncertain confidence in the shallow discrimination link to the deep discrimination link for high-precision verification during high-load degraded operation, thereby reducing the risk of missed detections, achieving a balance between throughput and detection recall, and taking into account both detection efficiency and detection accuracy while maintaining detection reliability. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the basic process of the present invention;

[0038] Figure 2 A flowchart for the standardization of streaming audio;

[0039] Figure 3 This refers to the dynamic computing power allocation and coordinated scheduling process. Detailed Implementation

[0040] The basic process of this invention is as follows: Figure 1 As shown, this invention provides a real-time fake audio detection method based on complexity and dynamic routing. The overall framework consists of four parts: audio standardization, complexity assessment, dynamic computing power allocation and linkage scheduling, model detection and result output.

[0041] (1) Audio standardization: In order to adapt to the processing requirements of high-concurrency real-time audio streams, the present invention divides the continuous network audio stream into discrete audio frames and extracts features from the audio frames to obtain pre-detection features for complexity evaluation; then the audio frames and their pre-detection features are encapsulated to obtain a standardized detection package, which serves as the input for subsequent complexity evaluation.

[0042] (1.1) Streaming slicing: For continuously input audio data streams A streaming sliding window mechanism is used to extract continuous audio data to obtain the current audio frame F. The duration of the sliding window is set to... The sliding step size is This allows for the preservation of space between adjacent frames. Durational overlap data. The current audio frame F is divided into multiple consecutive subframes, denoted as... n is the number of subframes into which the current audio frame F is divided. To represent a set of audio frames The first in One audio subframe, To facilitate feature alignment at the microscale, the portion overlapping with the previous audio frame is designated as a historical subframe, and the portion newly entering the current audio frame as the sliding window advances is designated as a new subframe. (That is, the overlapping data includes subframes 1 to i-1, and the new data corresponds to subframe i).

[0043] (1.2) Incremental Feature Extraction: A circular buffer is introduced for feature increment calculation. For newly added subframes... Calculate the signal-to-noise ratio of the newly added subframe. :

[0044]

[0045] in and These represent the extracted speech signal power and the background noise power, respectively. The calculated... Incremental features and buffer lengths The historical features corresponding to the overlapping data are concatenated to form the pre-detection features of the current frame. The buffer is updated synchronously after the calculation is completed.

[0046] (1.3) Encapsulation: The current audio frame and its pre-detection features are serialized and encapsulated, and a globally unique task tracking identifier (Task ID) is added to obtain a standardized detection package. Among them, the pre-detection features are used to calculate the task priority score, and the task tracking identifier is used for subsequent task routing and preemptive reflow.

[0047] (2) Complexity assessment: This step receives the standardized detection package output by the audio standardization stage, reads the pre-detection features in it, and determines whether the current audio frame is a valid audio frame based on the pre-detection features; for audio frames that are determined to be valid, their spectral centroid sequence is further extracted and the acoustic information density is calculated, thereby quantifying the detection value of the current audio frame and obtaining the task priority score.

[0048] (2.1) Information density quantization: for signal-to-noise ratio For valid audio frames within the normal range, read the pre-detection features and adjust the signal-to-noise ratio. Audio frames within a defined range are considered valid audio frames; for each valid audio frame, the variance of its spectral centroid sequence is further extracted. And define acoustic information density :

[0049]

[0050] in Indicates the first Acoustic information density of each audio frame Indicates the first The spectral centroid sequence corresponding to each audio frame This represents the normalization function. Through the above processing, the degree of spectral variation of the current audio frame is converted into a comparable acoustic information density value, which is then used as input for subsequent calculations of task priority scores.

[0051] (2.2) Initial priority calculation: The task priority score is calculated by combining the signal-to-noise ratio and acoustic information density. :

[0052]

[0053] in For the minimum normalization function, and These are the preset signal-to-noise ratio weighting coefficient and information density weighting coefficient, respectively. The priority score is then calculated. The data is appended to the standardized detection package of the current task to form a detection package carrying the task priority score, which awaits subsequent dynamic computing power allocation and coordinated scheduling. This score is used to distinguish between high-value core speech streams and low-value stable background sounds, and serves as the input basis for subsequent dynamic routing.

[0054] (3) Dynamic computing power allocation and linkage scheduling: After receiving the detection packet carrying the task priority score, the detection packet is dynamically routed to the shallow discrimination link or the deep discrimination link for detection by the model instance according to the current load of the discrimination link and the task priority score of the detection packet.

[0055] It should be noted that the model instance refers to a running instance loaded with a shallow discriminant model or a deep discriminant model and used to perform model inference tasks.

[0056] (3.1) Backend load quantization: Construct shallow discrimination links and deep discrimination links. The shallow discrimination links include multiple shallow discrimination model instances, and the deep discrimination links include multiple deep discrimination model instances. The number of network structure layers of the shallow discrimination model is less than that of the deep discrimination model. Both are used to classify audio segments.

[0057] Real-time acquisition of the number of queued tasks for model instances in shallow and deep discrimination paths. GPU memory usage ratio ,in, Used to characterize the Task backlog for each model instance Used to characterize the The computational resource constraints of each model instance. Among these, the shallow detection load rate... Deep detection load rate Used to characterize the task backlog and resource consumption of each discrimination link and its corresponding model instance, for any detection branch Its detection load rate The calculation method is as follows:

[0058]

[0059] in, This indicates a shallow discriminative branch. Indicates a deep discriminant branch; The total number of model instances. The maximum capacity threshold for a single model instance. and These are the preset task queuing weights and video memory usage weights, respectively. , For the first The GPU memory usage ratio of each model instance, its value range is: .

[0060] (3.2) Dynamic routing: based on deep load rate detection Real-time generation of dynamic interception thresholds The dynamic interception threshold is used to represent the minimum priority requirement for a task to enter the current deep discrimination link, and the deep detection load rate. The higher the threshold, the busier the model instances in the deep discrimination chain. In this case, the dynamic interception threshold is increased accordingly to reduce the number of low-priority tasks entering the deep discrimination chain. The dynamic interception threshold is calculated as follows:

[0061]

[0062] in, Based on the interception threshold, To penalize the scaling factor, This is the load sensitivity coefficient. When the load rate is detected in depth... As the threshold increases, it grows exponentially. Task priority scores are read from the detection packet and compared in real-time with their initial priorities. With dynamic threshold ,like The detection packet is then sent to the deep discrimination link for high-precision detection; if If the task priority score of the detection packet is lower than the current deep discrimination link admission requirement, the elastic degradation mechanism is triggered, and the detection packet is sent to the shallow discrimination link for fast detection.

[0063] (3.3) Elastic scaling and concurrency control of model instances: For model instances, this invention adopts a dynamic computing power allocation mechanism that combines macro-scale expansion and micro-scale control. Among them, macro-scale expansion is used to adjust the number of model instances according to the load of the discrimination link, and micro-scale control is used to limit the number of concurrent instances within a model instance according to the remaining video memory of a single host GPU, thereby improving throughput while avoiding video memory overflow.

[0064] At the macro-level model scaling, the task backlog of the shallow and deep discrimination links is monitored separately, and the detection load rate of model instances in the links is obtained. When the detection load rate exceeds the preset safe load level, it indicates that the number of currently active model instances in the discrimination link is insufficient to handle the task pressure, and the corresponding target model instance count should be increased. Define the target model instance count. The dynamic update strategy is as follows:

[0065]

[0066] in, Indicates a shallow detection branch. Indicates a deep detection branch. This represents the number of currently active model instances. The preset safe load water level, For system expansion smoothing coefficient, This is the floor function. When When the detection load rate exceeds the safe load level, the number of target model instances is increased proportionally to distribute the task peak in the discrimination link; when Furthermore, when resources are continuously idle, redundant model instances are smoothly recycled according to a preset recycling strategy, thereby achieving horizontal elastic scaling of backend computing resources.

[0067] At the micro level, to ensure that no single model instance experiences memory overflow under high-concurrency peak scenarios, the remaining video memory of the host machine's GPU is obtained in real time. Peak memory usage predicted by the detection model inference and the deep discrimination link During model runtime, the number of concurrent worker processes within a model instance is dynamically limited based on the remaining video memory and peak video memory usage. .when At this time, the dynamic concurrency of worker processes within a model instance is limited to the following safety boundaries:

[0068]

[0069] in To reserve a safety memory redundancy, This is a floor function. Through this joint control mechanism, the model instance internally implements natural backpressure for the task flow. Simultaneously, after the lifecycle of each batch of inference in the asynchronous worker node ends, the model runtime performs underlying tensor memory cleanup, releasing unused memory and reducing fragmented memory usage, thus forming a dual resource protection system covering both the model instance layer and the hardware resource layer.

[0070] (4) Model Detection and Result Output: After dynamic computing power allocation and linkage scheduling, the detection packets are sent to model instances in the shallow or deep discrimination links for spoofed audio detection, obtaining preliminary detection results for the audio segments corresponding to each detection packet. When the preliminary detection result output by the detection packet in the shallow discrimination link is within a preset uncertainty range, the corresponding detection packet is returned to the deep discrimination link as a detection packet to be verified for detection. Finally, the detection results for the audio segments corresponding to each detection packet are output.

[0071] (4.1) Detection Model Deployment: A shallow discrimination model is deployed in the shallow discrimination chain, and a deep discrimination model is deployed in the deep discrimination chain. Both models are used for detecting fake audio and are configured to receive audio input in a uniform format and output a classification result indicating whether the audio is real or fake. Before deployment, the discrimination model is trained on a dataset labeled with real and fake audio, where real audio is labeled as 0 and fake audio as 1. In one implementation, 80% of the audio is used as the training set, 10% as the validation set, and 10% as the test set. When training on the training set, the model predicts the input audio and outputs a confidence score representing the probability of fake audio.

[0072] (4.2) Detection Packet Decapsulation and Input Processing: After receiving the detection packet, the model decapsulates the detection packet to obtain the audio segment to be detected. Subsequently, the audio segment to be detected is subjected to input normalization processing, including sampling rate unification, channel unification, and fixed-length pruning or padding processing, to obtain an audio sequence that meets the model's input requirements.

[0073] (4.3) Model detection: The processed audio is input into the model, which extracts and fuses the acoustic features, spectral variation features and temporal variation features in the audio, and outputs the confidence score of the audio segment corresponding to the current detection package. .

[0074] (4.4) Preemptive backflow: When the detection packet assigned to the shallow discrimination link obtains the confidence level... Preset uncertain interval ,in If confidence level If the value is not within the preset uncertainty range, then the task status is marked as detection complete; if the confidence level is... If the task is in a preset uncertainty range, the task status is marked as a preemptive backflow, and based on the globally unique task tracking identifier in the detection packet corresponding to the task, the task is preferentially inserted into the deep discrimination link header for detection.

[0075] (4.5) Authenticity judgment: Preset authenticity judgment threshold ,when When, the audio segment corresponding to the current detection packet is determined to be real audio; when When the current detection packet corresponds to an audio segment, it is determined that the audio is fake.

[0076] In another aspect, the present invention also provides a real-time fake audio detection system based on complexity and dynamic routing, for implementing the aforementioned real-time fake audio detection method, including a standardized detection packet module, a task priority module, a discrimination link module, and a fake audio detection output module:

[0077] The standardized detection package module is used to segment the continuous network audio stream into discrete audio frames for feature extraction to obtain pre-detection features; then, the audio frames and their pre-detection features are encapsulated to obtain a standardized detection package.

[0078] The task priority module is used to receive standardized detection packets, read pre-detection features to obtain valid audio frames, extract spectral centroid sequences and calculate acoustic information density to obtain task priority scores.

[0079] The discrimination link module performs dynamic computing power allocation and coordinated scheduling based on priority scores and discrimination links.

[0080] The forged audio detection output module is used to send the detection packets to the discrimination link after dynamic computing power allocation and linkage scheduling to perform forged audio detection and obtain the detection result of the audio segment corresponding to each detection packet.

[0081] This invention addresses the characteristic of overlapping sliding windows and step sizes in streaming audio analysis. The method limits the extraction of audio features to newly added step segments and concatenates these incremental features with historical features read from a circular buffer to form complete audio frame pre-detection features. This mechanism avoids redundant feature extraction from overlapping data between adjacent sliding windows at the computational logic level. This invention also designs a task priority quantification mechanism based on acoustic information density. Acoustic information density is quantified by extracting the spectral centroid variance of audio frames and combined with the signal-to-noise ratio to evaluate the detection value of audio segments. This mechanism achieves accurate differentiation between "high-value speech" and "low-value background noise / silence" in the audio stream, providing a quantification criterion for resource scheduling in high-concurrency scenarios.

[0082] This invention proposes a dynamic routing method. It collects the number of queued tasks and GPU memory usage ratios of model instances in the shallow and deep discrimination links in real time, quantifies the shallow and deep detection load rates, and generates an exponentially changing dynamic interception threshold based on the deep detection load rate. The task priority score in the detection packet is compared with the dynamic interception threshold in real time, and when the task priority score is lower than the dynamic interception threshold, the detection packet is automatically downgraded and diverted to the shallow discrimination link. A two-layer computing power allocation scheme combining elastic scaling of model instances and concurrency control is proposed. At the macro level, the number of backend model instances is dynamically adjusted according to the detection load rate to horizontally expand concurrent processing capacity; at the micro level, the dynamic concurrency within a model instance is calculated in real time based on the remaining GPU memory of the host machine and strictly limited, and forced memory cleanup is performed after each inference cycle. This invention constructs shallow and deep discrimination links that support preemptive backflow. Confidence monitoring is introduced in the shallow discrimination link, with a preset uncertainty interval. When the confidence level of the shallow discrimination model falls into the uncertainty range, a preemptive backflow mechanism is triggered, allowing the task to bypass the ordinary queue and be sent to the deep discrimination link for high-precision verification and detection by using a globally unique task tracking identifier (Task ID).

[0083] Figure 1 This invention describes its basic process, which mainly includes four core stages: audio standardization, complexity assessment, dynamic computing power allocation and coordinated scheduling, model detection, and result output. First, continuous real-time streaming audio is input, segmented into discrete audio frames by the audio standardization module, and pre-detection features are extracted and packaged into detection packets with globally unique task tracking identifiers (Task IDs). The complexity assessment step calculates the task priority score of each audio frame and appends this score to the detection packet. The dynamic computing power allocation and coordinated scheduling step dynamically routes tasks based on their priority scores and real-time load status. Low-value tasks or those experiencing computing power constraints are downgraded to the shallow discrimination link, while high-value tasks are routed to the deep discrimination link for high-precision detection. For tasks diverted to the shallow link with low confidence, a preemptive backflow mechanism re-enters the deep discrimination link for verification. After dynamic computing power allocation and coordinated scheduling, the detection packets are sent to model instances in the shallow or deep discrimination links for spoofed audio detection, ultimately yielding the detection results for each audio segment corresponding to each detection packet.

[0084] Figure 2The streaming audio standardization process is described: continuously input audio data is segmented into discrete audio frames and encapsulated into detection packets of the same format. First, a streaming sliding window mechanism is used to extract audio data, preserving overlapping data between adjacent frames (i.e., subframes 1 to i-1), with new data starting from subframe i. Then, a circular buffer is introduced for incremental feature extraction, skipping redundant calculations of overlapping data and calculating the signal-to-noise ratio only for newly added audio subframes, defining it as an incremental feature. Next, this incremental feature is concatenated with historical features read from the circular buffer to form the pre-detection features of the current audio frame, and the circular buffer is updated synchronously. Finally, the current audio frame, pre-detection features, and Task ID are serialized and encapsulated to output a standardized detection packet.

[0085] Figure 3 The dynamic computing power allocation and coordinated scheduling process is described: After receiving a detection packet carrying pre-detection features, audio frames, Task ID, and task priority score, a dynamic interception threshold is generated based on the current load, and the task priority score is compared with this threshold. If the task priority score is greater than or equal to the dynamic interception threshold, the detection packet is routed to the deep discrimination link for high-precision detection; if the task priority score is less than the dynamic interception threshold, the detection packet is downgraded and diverted to the shallow discrimination link for fast detection. Tasks entering each discrimination link trigger elastic scaling and concurrency control of model instances based on the detection load rate. When the confidence level output by the shallow discrimination link is in the uncertain range, a preemptive backflow mechanism is triggered. This task bypasses the ordinary queue by using its Task ID and is preferentially inserted into the head of the deep discrimination link for high-precision verification detection.

[0086] To verify the effectiveness of the real-time fake audio detection method based on complexity and dynamic routing proposed in this invention, comparative experiments were conducted. The experiments mainly verified the method in three aspects: detection accuracy, response latency, and system throughput.

[0087] (1) Introduction to the dataset

[0088] The experiment used three public datasets: ASVspoof2019 LA, FoR (Fake-or-Real), and In-The-Wild. The composition of the experimental data is shown in Table 1 below.

[0089] Table 1

[0090]

[0091] (2) Experimental setup and evaluation indicators

[0092] The experiment compares the traditional unified detection scheme with the scheme of this invention. The traditional unified detection scheme only performs authenticity detection on audio; the scheme of this invention, based on task priority scores and dynamic interception thresholds, diverts detection packets to shallow or deep discrimination links, and performs preemptive reflow on detection packets with uncertain confidence in the shallow discrimination link.

[0093] In the experiment, the shallow discrimination model used a lightweight convolutional neural network (CNN) classifier, while the deep discrimination model used a fake audio detection model based on self-supervised speech features and a Transformer encoder.

[0094] The evaluation metrics are shown in Table 2, including accuracy, equal error rate, average response time, P99 latency, and throughput.

[0095] Table 2

[0096]

[0097] (3) Experimental results

[0098] (3.1) Results of detection accuracy

[0099] Table 3

[0100]

[0101] Specifically, the traditional unified detection scheme in the ASVspoof2019 LA dataset uses the experimental results reproduced on the same dataset in the SSL paper; the traditional unified detection scheme in the FoR dataset uses the deep learning classifier results in the FoR published paper; and the traditional unified detection scheme in the In-The-Wild dataset uses the cross-domain test results of RawGAT-ST in the published literature.

[0102] As shown in Table 3, the experimental results demonstrate that on the ASVspoof2019 LA dataset, the EER of this invention's approach is 1.07%, close to the 0.94% of the traditional unified detection approach. On the FoR dataset, the accuracy of this invention's approach is 99.99%, maintaining a high accuracy rate for synthesized speech detection. The In-The-Wild dataset, derived from real-world network scenarios, is suitable for verifying the generalization ability of forged audio detection methods in open environments. Therefore, this dataset was introduced as real-world generalization test data. On this dataset, the proposed approach achieves an accuracy of 95.86% and an EER of 3.08%, indicating that it still exhibits good detection performance in real-world network scenarios.

[0103] (3.2) Real-time processing performance results

[0104] To further verify the processing performance of the present invention in real-time detection scenarios, response latency and throughput were statistically analyzed under different numbers of model instances and concurrent requests. The results are shown in Table 4 below.

[0105] Table 4

[0106]

[0107] Experimental results show that, under a single-model, single-concurrent detection configuration, the average response latency of the proposed solution is 135ms, and the P99 latency is 186ms. Under a single-model instance, multi-concurrent detection configuration, the throughput increases to 20.6 packets / s. Under a multi-model instance detection configuration, the throughput further increases to 62.0 packets / s, while the P99 latency is controlled at 729ms. These results demonstrate that increasing the number of backend detection model instances can improve real-time detection throughput while maintaining a low response tail latency.

[0108] (4) Results Explanation

[0109] The above experimental results are used to verify the feasibility and effectiveness of the technical solution of this invention. The detection accuracy results show that the solution of this invention can distinguish between real and fake audio on the ASVspoof2019 LA, FoR, and In-The-Wild datasets, indicating that the detection process consisting of audio standardization, complexity assessment, dynamic computing power allocation and coordinated scheduling, model detection, and result output is feasible. Real-time processing performance results show that after increasing the number of model instances from 1 to 4, the throughput increased from 20.6 packets / s to 62.0 packets / s, with a P99 latency of 729ms. This result demonstrates that the technique of adjusting the number of model instances based on the backend load status can improve the detection packet processing capability in high-concurrency scenarios.

[0110] The above experimental results demonstrate that the present invention can achieve task diversion, computing resource scheduling, and preemptive return of uncertain tasks in real-time fake audio detection scenarios through techniques such as task priority scoring, dynamic interception threshold, elastic scaling of model instances, and preemptive return. This results in reducing task backlog, improving throughput, and maintaining detection reliability.

Claims

1. A real-time fake audio detection method based on complexity and dynamic routing, characterized in that, Includes the following steps: Step 1: Divide the continuous network audio stream into discrete audio frames for feature extraction to obtain pre-detection features; The audio frames and their pre-detection features are then encapsulated to obtain a standardized detection package; Step 2: Receive the standardized detection packet, read the pre-detection features to obtain valid audio frames, extract the spectral centroid sequence and calculate the acoustic information density to obtain the task priority score. The specific implementation process is as follows: Step 2.1: Read the pre-detection features and adjust the signal-to-noise ratio. Audio frames within a defined range are considered valid audio frames. For each valid audio frame, the variance of the spectral centroid sequence is extracted. and variance Acoustic information density is defined by normalization. ; Step 2.2: Combining the signal-to-noise ratio (SNR) and acoustic information density, perform a weighted sum of the normalized SNR and acoustic information density to calculate the task priority score. ; Score the priority The standardized detection package is attached to the current task to form a detection package carrying the task priority score, which awaits subsequent dynamic computing power allocation and linkage scheduling. Step 3: Based on priority scores and in conjunction with the discrimination links, perform dynamic computing power allocation and coordinated scheduling. The specific implementation process is as follows: Step 3.1: Construct shallow discrimination links and deep discrimination links. The shallow discrimination link includes multiple shallow discrimination model instances, and the deep discrimination link includes multiple deep discrimination model instances. The network structure of the shallow discrimination model has fewer layers than that of the deep discrimination model. All of them are used for discrimination and classification. Real-time acquisition of the number of queued tasks for model instances in shallow and deep discrimination paths. GPU memory usage ratio ,in, Characterizing the first Task backlog for each model instance Characterizing the first The computational resource constraints of each model instance are quantified to determine the shallow detection load rate. and deep detection load rate Among them, shallow detection load rate Characterizing the task backlog and resource consumption of shallow discrimination links and their corresponding model instances, and deep detection load rate. Characterizing the task backlog and resource consumption of the deep discriminative chain and its corresponding model instances, for any detection branch The percentage of GPU memory usage With the standardized number of queuing tasks The load value is calculated by weighted summation. The average load value of all model instances that identify link branches is then calculated to obtain the detection load rate. ; Step 3.2: Based on deep detection load rate Real-time generation of dynamic interception thresholds The dynamic interception threshold represents the minimum priority requirement for a task to enter the current deep discrimination link. The interception threshold exponent is calculated. , The load sensitivity coefficient is used to sum the interception threshold exponent with the base interception threshold through penalty scaling, resulting in the dynamic interception threshold. ; Read the task priority score from the detection package and compare it with the task priority score in real time. With dynamic interception threshold ,like The detection packet is then sent to the deep discrimination link for detection; if If the task priority score of the detection packet is lower than the current deep discrimination link admission requirement, the elastic degradation mechanism is triggered, and the detection packet is sent to the shallow discrimination link for detection. Step 3.3: For model instances, a dynamic computing power allocation mechanism combining macro-scale expansion and micro-scale control is adopted; macro-scale expansion is used to adjust the number of model instances according to the load of the identification link, and micro-scale control is used to limit the number of concurrent connections within a model instance according to the remaining video memory of a single host GPU. Step 4: After dynamic computing power allocation and linkage scheduling, the detection packets are sent to the discrimination link for spoofed audio detection, and the preliminary detection results of the audio segments corresponding to each detection packet are obtained. For the detection packets that meet the preset verification conditions, preemptive backflow is performed to obtain the final detection results.

2. The real-time fake audio detection method based on complexity and dynamic routing according to claim 1, characterized in that, The specific implementation process of step 1 is as follows: Step 1.1: For continuously input audio data streams A streaming sliding window mechanism is used to extract continuous audio data to obtain the current audio frame F; the duration of the sliding window is set to... The sliding step size is , This allows for the preservation of space between adjacent frames. Durational overlap data; the current audio frame F is divided into multiple consecutive subframes, denoted as... , This represents the number of subframes into which the current audio frame F is divided. To represent a set of audio frames The first in One audio subframe, The portion that overlaps with the previous audio frame is designated as a historical subframe, and the portion that newly enters the current audio frame as the sliding window advances is designated as a new subframe. Step 1.2: Introduce a circular buffer for feature increment calculation for newly added subframes. Calculate the signal-to-noise ratio of the newly added subframe. ,Will With the length of the buffer The historical features corresponding to the overlapping data are spliced ​​together to form the pre-detection features of the current frame; Step 1.3: Sequentially encapsulate the current audio frame and its corresponding pre-detection features, add a globally unique task tracking identifier, and obtain a standardized detection package. Among them, the pre-detection features are used to calculate the task priority score, and the task tracking identifier is used for subsequent task routing and preemptive reflow.

3. The real-time fake audio detection method based on complexity and dynamic routing according to claim 2, characterized in that, Number of model instances The dynamic update strategy is as follows: First calculate , To preset the safe load level, the number of currently active model instances is corrected by multiplying it with the smoothing coefficient, and then summed with the number of currently active model instances to obtain the updated number of model instances. At the micro-model concurrency level, the remaining video memory of the current host machine's GPU is obtained in real time. Peak memory usage predicted by the detection model inference and the deep discrimination link ; During model instance runtime, the number of concurrent worker processes within the model instance is dynamically limited based on remaining video memory and peak video memory usage. , recorded as ,in To reserve a safety memory redundancy, This is the floor function.

4. The real-time fake audio detection method based on complexity and dynamic routing according to claim 3, characterized in that, In step 3.3, at the macro-level scaling, the task backlog of the shallow discrimination link and the deep discrimination link are monitored respectively, and the detection load rate of the model instances in the link is obtained. When it detects the load rate Exceeding the preset safe load water level At that time, increase the number of model instances corresponding to the discrimination link.

5. The real-time fake audio detection method based on complexity and dynamic routing according to claim 4, characterized in that, In step 3.3, at the micro level, the remaining video memory of the host machine's GPU is obtained in real time. Peak memory usage compared to single-instance model instance inference prediction ; During model instance runtime, the number of concurrent worker processes within the model instance is dynamically limited based on remaining video memory and peak video memory usage. ;when At that time, the dynamic concurrency of worker processes within a model instance is limited to [number]. ,in This is a safety memory redundancy.

6. The real-time fake audio detection method based on complexity and dynamic routing according to claim 5, characterized in that, In step 4, the preset verification conditions include: when the detection packet assigned to the shallow discrimination link obtains a confidence level. Preset uncertain interval ,in If confidence level If the value is not within the preset uncertainty range, then the task status is marked as detection complete; if the confidence level is... If the task is in a preset uncertainty range, the task status is marked as a preemptive backflow, and based on the globally unique task tracking identifier in the detection packet corresponding to the task, the task is preferentially inserted into the deep discrimination link header for detection.

7. A real-time fake audio detection system based on complexity and dynamic routing, used to implement the real-time fake audio detection method according to any one of claims 1 to 6, characterized in that, Includes the following modules: The standardized detection package module is used to segment a continuous network audio stream into discrete audio frames for feature extraction to obtain pre-detection features; The audio frames and their pre-detection features are then encapsulated to obtain a standardized detection package; The task priority module is used to receive standardized detection packets, read pre-detection features to obtain valid audio frames, extract spectral centroid sequences and calculate acoustic information density to obtain task priority scores. The link discrimination module, based on priority scores, performs dynamic computing power allocation and coordinated scheduling in conjunction with the discriminated links; The forged audio detection output module is used to send the detection packets to the discrimination link for forged audio detection after dynamic computing power allocation and linkage scheduling. It obtains the preliminary detection results of the audio segment corresponding to each detection packet, and performs preemptive backflow on the detection packets that meet the preset verification conditions to obtain the final detection results.

Images