An AI synthetic video detection method and system based on multi-agent collaboration

Abstract

This invention relates to the field of computer vision technology and discloses an AI-synthesized video detection method and system based on multi-agent collaboration. The method includes: acquiring the video to be detected and sampling keyframes; understanding the keyframes based on a visual language model and generating a forgery hypothesis for the current round based on the generated global semantic context; generating a spatial-temporal routing query; performing target localization and segmentation on the video to be detected based on the spatial-temporal routing query; extracting structured physical evidence from the target region video; performing evidence-constrained reasoning to obtain a preliminary judgment result and confidence level; performing consistency checks on the execution trajectory to obtain a reliability score and feedback information, and outputting the final authenticity judgment result. This invention improves the reliability, generalization ability, and traceability of AI-synthesized video detection by combining contextual modeling, hypothesis-driven routing, multi-dimensional physical evidence collection, evidence-constrained reasoning, and closed-loop reflective verification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to an AI-synthesized video detection method and system based on multi-agent collaboration. Background Technology

[0002] In recent years, with the rapid development of generative artificial intelligence technology, deep learning-based video generation models have made significant progress in visual realism and content consistency. Diffusion models, autoregressive models, and other technologies have driven the development of content creation, but have also brought security risks such as the spread of misinformation. Existing research shows that artificially synthesized videos have been used to fabricate false events and spread misleading content, posing a high risk and placing higher demands on video authenticity detection technology. Among existing technologies, one type of method mainly relies on low-level visual features such as pixel distribution, frequency domain features, optical flow changes, or geometric consistency to model and identify synthetic traces; another type attempts to introduce semantic-level analysis, using visual language models or multimodal large models to achieve a holistic understanding of video content. However, these methods typically depend on specific video generation methods or training data distributions, and their detection performance and generalization ability remain limited when faced with unknown video generation methods or highly realistic synthetic videos. Furthermore, existing detection results are mostly based on simple classification outputs, lacking clear explanatory evidence, making it difficult to meet the reliability and interpretability requirements of high-risk application scenarios such as social security. Summary of the Invention

[0003] This invention aims to provide an AI-synthesized video detection method and system based on multi-agent collaboration. It addresses the shortcomings of existing technologies, such as reliance on a single model, fixed processes, and insufficient generalization to high-fidelity synthesized videos and unknown video generation methods, thereby improving the reliability, generalization, and interpretability of detection. Specifically, using the video to be detected as input, it comprehensively utilizes fine-grained physical clues (temporal drift, motion / geometric anomalies, etc.) and high-level semantic logic clues (common sense conflicts, unreasonable interactions, etc.) to construct verifiable evidence support and output authenticity judgment results. Unlike fixed reasoning models, this invention introduces a multi-agent collaboration mechanism. Based on video content features and verifiable clues, it uses forensic tools from a tool pool to obtain fine-grained forgery scores, extracts evidence from key areas from coarse to fine, and performs consistency checks on semantic inference, physical evidence, and conclusions through a self-reflective verification module, avoiding erroneous judgments when evidence is insufficient or based solely on contextual guesses.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides an AI-synthesized video detection method based on multi-agent collaboration, comprising the following steps: S1. Acquire the video to be detected and sample key frames; understand the key frames based on the visual language model to generate a global semantic context; generate the forgery hypothesis for the current round based on the global semantic context; S2: Based on the video to be detected, the global semantic context, and the forgery hypothesis, generate a spatial-temporal routing query; perform target localization and segmentation on the video to be detected based on the spatial-temporal routing query to obtain the target area video; call multiple dedicated evidence collection tools to extract structured physical evidence from the target area video; and perform evidence-constrained reasoning based on the global semantic context, the forgery hypothesis, and the structured physical evidence to obtain a preliminary judgment result and confidence level. S3 performs a full-link consistency check on the execution trajectory of the preliminary judgment results to obtain a reliability score and feedback information; S4. When the reliability score is lower than the threshold, update the falsified hypothesis based on the feedback information and repeat steps S2 to S3 until the reliability score requirement is met or the maximum number of iterations is reached; output the final authenticity judgment result.

[0005] In one embodiment, the global semantic context includes one or more of the following: scene type, main entity, interaction relationship between entities, action pattern, dynamic intensity, scene risk level, and scene anomaly prompt.

[0006] In one embodiment, generating the forgery hypothesis for the current round based on the global semantic context specifically includes: Based on global semantic context Using the first visual language model Generate fake hypotheses for the current iteration round This is used to represent the most likely forgery mechanism and the corresponding expected anomaly in the current context of the video to be detected: ; The forgery hypothesis includes the category of suspected forgery object, forgery mechanism, expected abnormal behavior, and key areas of focus.

[0007] In one embodiment, generating a space-time routing query based on the video to be detected, the global semantic context, and the forgery hypothesis specifically includes: According to the video to be tested Global semantic context and fabricated assumptions Using a second visual language model Generate spatial-temporal route query It can convert fake hypotheses into detection hints targeting local regions: ; The spatial-temporal routing query can be described in natural language, structured field, or a combination of natural language and structured field descriptions. The query content in spatial-temporal routing queries includes one or more of the following: target entity description, part description, appearance features, location information, and time segment information.

[0008] In one embodiment, the step of performing target localization and segmentation on the video to be detected based on spatial-temporal routing query to obtain the target region video specifically includes: Space-time routing query The GroundingDINO model was used to detect the video. Perform target localization frame by frame to obtain the corresponding bounding box set. Based on the bounding box, the SAM2 model is used for the video to be detected. Generate pixel-level masks The target region video is then cropped from the original video to be detected based on the mask. : ; ; ; in, This represents the target localization function based on the GroundingDINO model. This represents the mask generation function based on the SAM2 model. This indicates a mask clipping operation.

[0009] In one embodiment, the step of invoking multiple dedicated forensic tools to extract structured physical evidence from the target area video specifically includes: Specialized evidence collection tools include: Motion consistency evidence collection tool It is used to detect optical flow anomalies, temporal tearing, and motion jumps; Depth or geometric consistency forensics tools It is used to detect unreasonable three-dimensional structures and abnormal spatial relationships; Appearance or identity consistency evidence tools It is used to detect cross-frame identity drift, texture instability, and abnormal part attributes; Pixel-level trace evidence collection tool It is used to detect boundary noise, local manipulation traces, and high-frequency anomalies; The extraction process is represented as follows: ; For the target area video, For structured physical evidence.

[0010] In one embodiment, the step of performing evidence-constrained reasoning based on global semantic context, forgery assumptions, and structured physical evidence to obtain preliminary judgment results and confidence levels specifically includes: Based on global semantic context The current round of falsification assumptions and structured physical evidence Using a third visual language model As a reasoning model, the expected abnormal behavior in the forgery hypothesis is cross-validated with structured physical evidence; When structured physical evidence supports the expected anomalous behavior of the forgery hypothesis, the preliminary judgment result is... To determine if it is forged, confidence levels are calculated based on the salience characteristics of structured physical evidence. ; When the structured physical evidence contradicts the expected anomalous behavior of the forgery hypothesis, the constrained inference model abandons the current forgery hypothesis and makes a preliminary judgment. Assess whether the value is true or unknown, and output the corresponding confidence level. ; The reasoning process of the reasoning model is represented as follows: .

[0011] In one embodiment, the step of performing a full-link consistency check on the execution trajectory of the preliminary judgment result to obtain a reliability score and feedback information specifically includes: The self-reflective verifier performs a full-link consistency check on the execution trajectory of the preliminary judgment result obtained in step S2, and outputs a reliability score. and feedback information : ; For global semantic context, To fabricate the hypothesis, For spatial-temporal routing lookup, This is the mask used when segmenting the video to be tested. This is a collection of specialized forensic tools used when extracting structured physical evidence. For structured physical evidence, For the preliminary judgment results, for The corresponding confidence level; For self-reflective validators; The end-to-end consistency check includes at least one or more of the following: Does the fabricated hypothesis match the context of the video to be detected? Check if the spatial-temporal routing query hits the expected target area; Does the mask used when segmenting the video to be tested accurately cover the target? Whether the dedicated evidence collection tools are suitable for the current scenario of the video to be tested; Whether the structured physical evidence is consistent with the preliminary judgment; Does the evidence-constrained reasoning process contain logical leaps or illusions?

[0012] In one embodiment, when the reliability score is lower than a threshold, the falsification hypothesis is updated based on feedback information, and steps S2 to S3 are repeated until the reliability score requirement is met or the maximum number of iterations is reached; the final authenticity determination result is output, specifically including: Based on global semantic context The current round of falsification assumptions and feedback information Using the fourth visual language model Update the falsification hypothesis for the current round to obtain the falsification hypothesis for the next round. : ; Strategies for updating the fabricated assumptions for the next round include at least one of the following: Shifting the focus area: When feedback indicates that no anomalies were found in the current target area, the focus area is shifted to other undetected local areas or alternative target areas in the fake hypothesis. Modify forgery mechanism: When feedback indicates that the structured physical evidence does not support the current expected forgery mechanism, replace it with an alternative forgery mechanism in the forgery hypothesis; Based on the assumption of the next round of forgery Repeat steps S2 to S3; When either scenario one or scenario two occurs, the iteration is terminated, and the preliminary judgment result, confidence level, and structured physical evidence corresponding to the optimal round are summarized to generate the final authenticity judgment result. Case 1 is when the new reliability score obtained after repeated execution reaches or exceeds the threshold. The round in which situation 1 occurs is the optimal round. Scenario 2 is when the number of iterations reaches the preset maximum number of iterations. In scenario two, the round with the highest reliability score is the optimal round. The final authenticity determination result specifically includes: authenticity determination of the video to be detected, confidence level, target area, structured physical evidence, end-to-end consistency check conclusion, and structured analysis report; The structured analysis report is used to record and present the complete reasoning chain of the multi-agent system in the optimal iteration round. The complete reasoning chain includes the causal deduction process from global semantic context parsing, falsification hypothesis proposal, extraction of corresponding structured physical evidence to obtaining the authenticity judgment.

[0013] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0014] Compared with the prior art, the beneficial technical effects of the present invention are: This invention achieves deep collaboration between high-level semantic understanding and low-level physical evidence collection, constructing a bidirectional interactive framework of semantically guided evidence collection and evidence-constrained reasoning. This breaks through the limitations of traditional single detection models and significantly improves the detection stability of the method in complex scenarios. By utilizing hypothesis-driven spatial-temporal routing queries to focus on expected key target areas (such as entity interaction boundaries), it effectively filters redundant background information in full-frame videos, significantly improving detection efficiency while reducing computational interference. By invoking multi-dimensional dedicated evidence collection tools, it comprehensively covers temporal jumps, 3D geometric anomalies, identity drift, and pixel-level micro-traces, greatly enhancing the generalization detection capability for highly realistic unknown AI-generated videos. Through a self-reflective verifier, it achieves end-to-end consistency checks and feedback iterative error correction, effectively suppressing the "illusions" that may arise from visual language models and misjudgments caused by insufficient evidence in a single round, ensuring the rigor of the reasoning process. Finally, it outputs a structured analysis report with a complete and traceable causal reasoning chain, breaking through the black-box limitations of traditional AI detection models and making its judgment results highly interpretable, providing solid technical support for scenarios such as forensic identification, news verification, and digital content supervision. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is an overall framework diagram of the present invention, illustrating the interaction between the hierarchical perceptual solver and the self-reflective verifier; Figure 3 This is a logic flowchart of the present invention, illustrating the overall flow from video input to final report output; Figure 4 The flowchart of the hierarchical perceptual solver of the present invention shows the execution order of context modeling, fake hypothesis generation, routing, segmentation, evidence collection and reasoning; Figure 5 The flowchart of the self-reflective verifier of the present invention illustrates the end-to-end consistency check, feedback generation, and closed-loop reflection and correction mechanism. Figure 6 The schematic diagram of the final authenticity determination result of the present invention shows the output fields such as authenticity determination, confidence level, target area, structured physical evidence, and end-to-end consistency check conclusion. Detailed Implementation

[0016] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] like Figure 1 As shown, an AI-synthesized video detection method based on multi-agent collaboration includes the following steps: S1. Acquire the video to be detected and sample key frames; understand the key frames based on the visual language model to generate a global semantic context; generate the forgery hypothesis for the current round based on the global semantic context; S2: Based on the video to be detected, the global semantic context, and the forgery hypothesis, generate a spatial-temporal routing query; perform target localization and segmentation on the video to be detected based on the spatial-temporal routing query to obtain the target area video; call multiple dedicated evidence collection tools to extract structured physical evidence from the target area video; and perform evidence-constrained reasoning based on the global semantic context, the forgery hypothesis, and the structured physical evidence to obtain a preliminary judgment result and confidence level. S3 performs a full-link consistency check on the execution trajectory of the preliminary judgment results to obtain a reliability score and feedback information; S4. When the reliability score is lower than the threshold, update the falsified hypothesis based on the feedback information and repeat steps S2 to S3 until the reliability score requirement is met or the maximum number of iterations is reached; output the final authenticity judgment result.

[0018] This invention proposes a multi-agent collaborative AI-synthesized video detection framework, which, for input videos to be detected... By working together with a hierarchical perceptual solver and a self-reflective verifier, the final judgment result, confidence level, and structured analysis report are generated.

[0019] The hierarchical perceptual solver is used to complete the process from global semantic understanding to local physical evidence extraction and then to preliminary judgment generation; The self-reflective verifier is used to perform consistency checks on the execution trajectory of the hierarchical perceptual solver and trigger feedback-driven closed-loop correction when reliability is insufficient.

[0020] The present invention will be described in detail below in several parts.

[0021] 1. Sampling keyframes.

[0022] Obtain the video to be tested Multiple video frames are uniformly extracted from the video to be detected at a preset sampling interval as key frames to form the input for subsequent multimodal inference. The sampling method is uniform sampling.

[0023] 2. Obtain the global semantic context.

[0024] Perform global semantic understanding on keyframes using a visual language model to generate a global semantic context. In a preferred embodiment, the visual language model in this invention can be a large model with multimodal understanding capabilities, such as GPT-4o, Gemini-2.5-Pro, or Qwen3-VL.

[0025] The global semantic context includes at least one or more of the following information: scene type, main entity, interaction relationship between entities, action mode, dynamic intensity, scene risk level, and scene anomaly prompt.

[0026] It is important to specify that the main entities are target objects with independent physical meaning or semantic reference in the video keyframes, including people, objects, animals, or background structures; people include the whole human body or its parts (such as the face, hands, etc.); the interaction relationships between entities are the spatial and temporal relationships or physical contact relationships between the main entities. Extracting entity interaction relationships can effectively locate high-frequency error areas in AI-synthesized videos that are prone to clipping or temporal tearing.

[0027] 3. Constructing a fabricated hypothesis.

[0028] Based on global semantic context Using the first visual language model Generate the current iteration round falsified assumptions It is used to represent the most likely forgery mechanism and its expected abnormal form in the current video context.

[0029] The above process can be represented as: ; The forgery hypothesis can include the category of suspected forgery object, possible forgery mechanism, expected abnormal behavior, and key areas of focus. Among them, the forgery mechanism refers to the way in which the underlying physical laws are violated due to the limitations of the AI ​​video generation model (such as the diffusion model). For example: (1) Three-dimensional spatiotemporal discontinuity mechanism: the expected abnormal behavior is cross-frame structural distortion or temporal tearing; (2) Multi-entity interaction clipping mechanism: the expected abnormal behavior is texture fusion or physical penetration at the boundary of entities; (3) Identity feature forgetting mechanism: the expected abnormal behavior is the change of clothing or facial features of the same person in the previous and next frames.

[0030] 4. Generate a spatial-temporal routing query.

[0031] According to the video to be tested Global semantic context and fabricated assumptions Using a second visual language model Generate spatial-temporal route query This transforms high-level assumptions into detection cues oriented towards local regions.

[0032] The above process can be represented as: ; Spatial-temporal routing queries can be described using natural language, structured fields, or a combination of both. Query content can include target entity descriptions, part descriptions, appearance features, location information, and time segment information.

[0033] 5. Crop the target area of ​​the video.

[0034] Space-time routing query The GroundingDINO model is used to perform target localization frame by frame in the video to obtain the corresponding bounding box set. Furthermore, based on the bounding box, pixel-level masks are generated using the SAM2 model. And based on the mask, the target area video is cropped from the original video. .

[0035] The above process can be represented as: ; ; ; in, This represents the target localization function based on the GroundingDINO model. This represents a mask generation function based on SAM2. This indicates a mask clipping operation.

[0036] 6. Extract structured physical evidence.

[0037] Video targeting the area Multiple specialized forensic tools were used to extract structured physical evidence. .

[0038] Specialized forensic tools include at least the following four categories: Motion consistency evidence collection tool It is used to detect optical flow anomalies, temporal tearing, and motion jumps; Depth or geometric consistency forensics tools It is used to detect unreasonable three-dimensional structures and abnormal spatial relationships; Appearance or identity consistency evidence tools It is used to detect cross-frame identity drift, texture instability, and abnormal part attributes; Pixel-level trace evidence collection tool It is used to detect boundary noise, local manipulation traces, and high-frequency anomalies.

[0039] The extraction process can be represented as follows: ; Each tool outputs either a spoofing probability or a predicted label.

[0040] 7. Preliminary judgment results and confidence level.

[0041] Based on global semantic context , fabricated assumptions and structured physical evidence Using a third visual language model As a reasoning model, it performs evidence-constrained reasoning and outputs a preliminary judgment result. and confidence level .

[0042] The above process can be represented as: ; In this step, reasoning no longer relies on simple contextual guessing from a large model, but instead applies strict evidentiary constraints: the anticipated anomalous behavior in the fabricated hypothesis is cross-validated with the actually extracted structured physical evidence. When the structured physical evidence supports the anticipated anomalous behavior, a preliminary judgment is made. To prevent forgery, when physical evidence contradicts expected behavior (i.e., evidence shows normal physical characteristics), the forced constraint inference model abandons the current forgery hypothesis to avoid the large model developing "illusions," thereby outputting a preliminary judgment result as true or unknown, and calculating the corresponding confidence level. .

[0043] 8. Self-reflective validator.

[0044] By self-reflective validator (A visual language model can be used) Perform a full-link consistency check on the execution trajectory and output a reliability score. and feedback information .

[0045] The above process can be represented as: .

[0046] in, A collection of specialized forensic tools invoked when extracting structured physical evidence.

[0047] The consistency check includes at least one or more of the following: Does the fabricated hypothesis match the context of the video to be detected? Check if the spatial-temporal routing query hits the expected target area; Does the mask used when segmenting the video to be tested accurately cover the target? Whether the dedicated evidence collection tools are suitable for the current scenario of the video to be tested; Whether the structured physical evidence is consistent with the preliminary judgment; Does the evidence-constrained reasoning process contain logical leaps or illusions?

[0048] 9. Output results.

[0049] When reliability score Below the reliability threshold At that time, based on the global semantic context The previous round of fabricated assumptions And the feedback information output by the self-reflective validator Using the fourth visual language model Update the fabrication hypothesis for the next round The update strategy includes: shifting the focus to key areas (such as moving them to other undetected local areas of interaction) or changing the forgery mechanism (replacing it with an alternative forgery mechanism). Subsequently, Re-enter step S2 as a new hypothesis and execute the loop iteration.

[0050] When reliability score Reaching or exceeding the threshold Or the number of iterations reaches the upper limit. At this point, the system terminates the iteration, summarizes the data from the optimal round, and outputs the final truthfulness determination result. The rule for updating the falsified hypothesis can be expressed as: ; in, For the falsified hypothesis in the t-th iteration, This is the index of the t-th iteration round.

[0051] The final authenticity determination results include: authenticity determination of the video to be detected, confidence level, target area, structured physical evidence, end-to-end consistency check conclusions, and structured analysis report.

[0052] Example 1: In this embodiment, the video to be detected is first uniformly sampled to obtain multiple keyframes. The visual language model understands the keyframes, outputs scene summaries, entity relationships, and action patterns, and forms a global semantic context. The system then generates a fabricated hypothesis. Examples include "topological anomalies exist at the boundaries of interactive limbs" or "the identity of the target person drifts across frames." Based on this hypothesis, a spatial-temporal routing query is generated. Examples include "the arm area of ​​the person waving", "the boundary area of ​​contact between the two people", and "the facial area of ​​the target person".

[0053] The system then uses the language-guided localization model GroundingDINO to obtain candidate bounding boxes. Then, pixel-level masks are generated using the segmentation model SAM2. The target area of ​​the video is cropped. Evidence was extracted from the optical flow, depth, appearance, and pixel levels to form structured physical evidence. .based on , and The preliminary judgment result is given by applying evidence-bound reasoning. With confidence level Subsequently, the self-reflective validator examines the execution trajectory; if it determines that the physical evidence output by the current forensic tool does not meet the consistency requirements with the hypothesis to be verified—for example, if the extracted physical evidence cannot support the hypothesis that "there is a topological anomaly at the boundary of the interacting limbs"—it generates feedback information. This drives the model to update its hypothesis or tool list, entering a new round of iterations until the threshold requirement is met or the maximum number of rounds is reached.

[0054] Example 2: The visual language model of the hierarchical perception solver adopts the GPT-4o model; The self-reflective validator uses the Gemini-2.5-Pro ​​model to reduce the bias of the homology model; Eight keyframes are uniformly sampled from the video input and used as the multimodal inference input. The target localization uses the grounding-dino-base model; The SAM 2 model is used to generate pixel-level masks, with the sam2.1_hiera_base_plus pre-trained weights being preferred. The RAFT model was used as a tool for collecting evidence of motion consistency. Acquisition of depth or geometrical consistency evidence The forensic tools use the DepthAnythingV2 model; Collect evidence of appearance or identity consistency The forensic tools use the DINOv2 model; The forensic tool for collecting pixel-level trace evidence adopts the D3 class pixel forensics model.

[0055] The modules communicate with each other via structured JSON, including context, forgery assumptions, space-time routing queries, masks, reliability scores, preliminary judgment results, and feedback information.

[0056] In a preferred configuration, to reduce generation randomness, the inference parameters are set to... , , This is a temperature coefficient used to adjust the probability distribution of the model output; the lower the value, the more deterministic the output results. This is a kernel sampling threshold based on cumulative probability, used to select a set of candidate lexical units from the probability distribution whose cumulative probability does not exceed this threshold for sampling; the number of iterations reaches the upper limit. Reliability threshold .

[0057] Example 3: In one embodiment, the hierarchical awareness solver uses at least three types of cue words: Context and hypothesis generation prompts are used to output global semantic context. and fabricated assumptions ; Routing query prompts are used to translate high-level assumptions into space-time routing queries. ; Evidence-constrained reasoning prompts are used to input structured physical evidence and output... , And the chain of reasoning.

[0058] The self-reflective validator uses consistency check prompts, takes the complete execution trajectory as input, and outputs end-to-end consistency check results, reliability scores, and feedback information.

[0059] Example 4: In one embodiment, the final authenticity determination result includes at least: the authenticity determination of the video to be detected, confidence level, target area, structured physical evidence, end-to-end consistency check conclusion, and structured analysis report. If a conflict exists, an exclusion reason or correction feedback is output.

[0060] The following experiments are used to illustrate the effectiveness and feasibility of the method of the present invention.

[0061] 1. Public benchmark generalization verification experiments: The method of this invention was validated using publicly available benchmarks GenVideo, DVF, LOKI, and GenVidBench. In the training configuration, the training set partition of the publicly available GenVideo dataset was used to prepare the required forensic tools. No additional customized parameter tuning was performed on the target benchmarks during the testing phase.

[0062] Based on publicly available benchmarks, the method of this invention is evaluated under different combinations of solvers and self-reflective verifiers, wherein: One configuration is to use the GPT-4o model as a hierarchical perceptual solver and Gemini-2.5-Pro ​​as a self-reflective verifier; Another configuration is to use the Qwen3-VL-32B model as both a hierarchical perceptual solver and a self-reflective verifier.

[0063] The corresponding experimental results are shown in Table 1.

[0064] Table 1

[0065] The above results show that the present invention does not rely on a single closed-source model and can maintain strong cross-benchmark generalization ability under both closed-source and open-source models.

[0066] 2. Comparison experiment with the general large model baseline: Results on public benchmarks show that when relying solely on large models to directly determine the authenticity of videos, their detection performance and generalization ability remain significantly insufficient.

[0067] Specifically: (1) When the GPT-4o model is used as the hierarchical sensing solver, the average accuracy (Acc) / F1 score (F1) of direct judgment is 67.4 / 74.2; after adopting the complete framework of this invention, it is improved to 90.3 / 87.9 under the condition of using the same large model configuration; (2) When the Qwen3-VL-32B model is used as the hierarchical sensing solver, the average accuracy (Acc) / F1 score (F1) of direct judgment is 68.8 / 71.1; after adopting the complete framework of this invention, it is improved to 89.7 / 87.7 under the condition of using the same large model configuration.

[0068] The above results show that relying solely on large models for semantic-level authenticity judgment is insufficient to meet the requirements of high-reliability forensic tasks. However, this invention, by introducing structured physical evidence and a full-link consistency check mechanism, can significantly improve detection performance under different large model configurations, demonstrating strong model independence and generalization ability.

[0069] 3. Module ablation experiment: To verify the contribution of each module of this invention to the overall performance, incremental ablation experiments were conducted on publicly available benchmarks. The publicly available benchmarks used included GenVideo, DVF, LOKI, and GenVidBench. The experimental results are shown in Table 2.

[0070] Table 2

[0071] The results above show that: (1) After introducing Social Context-Aware Hypothesis Generation and Routing (SHGR) on the basis of the baseline large model (GPT-4o), the average accuracy (Acc) increased from 67.4% to 69.1%, and the average F1 score increased from 74.2% to 77.3%, indicating that the social context-aware hypothesis generation and routing mechanism can improve the localization ability of high-risk targets; (2) After further introducing video cropping (VC) on the basis of SHGR, the average accuracy increased to 73.7% and the average F1 score increased to 79.6%, indicating that video cropping helps to improve the analyzability of local areas; (3) The performance was significantly improved after introducing the self-reflective verifier (SRV) or physical evidence (FTE) on the basis of SHGR and VC, respectively, indicating that self-reflective verification and physical evidence both play an important role in improving the results. (4) When SHGR, VC, FTE and SRV are all combined to form a complete closed loop, the performance reaches the optimal level, indicating that the advantage of the method of the present invention comes from the synergy of multiple modules rather than the independent action of a single module.

[0072] 4. Ablation Experiment in the Evidence Collection Toolkit To verify the complementarity among the multidimensional forensics tools, a stepwise accumulation experiment was conducted. The publicly available benchmarks used included GenVideo, DVF, LOKI, and GenVidBench. The experimental results are shown in Table 3.

[0073] Table 3

[0074] Based on publicly available benchmark experiments, module ablation experiments, and toolbox ablation experiments, it is evident that the core advantage of this invention lies not in the direct judgment capability of a single large model, but rather in: first, identifying high-risk targets through contextual modeling and fabrication hypotheses; second, extracting quantifiable physical evidence from local targets; and finally, performing consistency checks and closed-loop error correction through an independent self-reflective verifier. This three-layer mechanism constitutes the key technical approach that distinguishes this invention from existing technologies.

[0075] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0076] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0077] In one embodiment, the present invention also provides a computer system, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor of the computer system provides computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer system stores data used in the methods described above. The network interface of the computer system is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0080] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting AI-synthesized videos based on multi-agent collaboration, characterized in that, For the input video to be detected, a hierarchical perceptual solver and a self-reflective verifier work together. The hierarchical perceptual solver completes the process from global semantic understanding to local physical evidence extraction and then to preliminary judgment generation. The self-reflective verifier checks the consistency of the execution trajectory of the hierarchical perceptual solver and triggers feedback-driven closed-loop correction when the reliability is insufficient. The method includes the following steps: S1. Acquire the video to be detected and sample key frames; understand the key frames based on the visual language model to generate a global semantic context; generate the forgery hypothesis for the current round based on the global semantic context; S2 generates a spatial-temporal routing query based on the video to be detected, the global semantic context, and the forgery hypothesis; performs target localization and segmentation on the video to be detected based on the spatial-temporal routing query to obtain the target region video; and invokes multiple dedicated forensic tools to extract structured physical evidence from the target region video, including: dedicated forensic tools such as motion consistency forensics tools. Used to detect optical flow anomalies, temporal tears, and motion jumps; depth or geometric consistency forensics tools. Used to detect unreasonable 3D structures and abnormal spatial relationships; a tool for obtaining evidence of appearance or identity consistency. Used to detect cross-frame identity shift, texture instability, and abnormal part attributes; pixel-level trace evidence collection tool. It is used to detect boundary noise, local manipulation traces, and high-frequency anomalies; the extraction process is represented as: ; For the target area video, For structured physical evidence; Based on global semantic context, forgery assumptions, and structured physical evidence, evidence-constrained reasoning is performed to obtain preliminary judgment results and confidence levels; S3 performs a full-link consistency check on the execution trajectory of the preliminary judgment results to obtain a reliability score and feedback information; S4. When the reliability score is lower than the threshold, update the falsified hypothesis based on the feedback information and repeat steps S2 to S3 until the reliability score requirement is met or the maximum number of iterations is reached; output the final authenticity judgment result.

2. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The global semantic context includes one or more of the following: scene type, main entity, interaction relationship between entities, action mode, dynamic intensity, scene risk level, and scene anomaly prompt.

3. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The process of generating forgery hypotheses for the current round based on the global semantic context specifically includes: Based on global semantic context Using the first visual language model Generate fake hypotheses for the current iteration round This is used to represent the most likely forgery mechanism and the corresponding expected anomaly in the current context of the video to be detected: ； The forgery hypothesis includes the category of suspected forgery object, forgery mechanism, expected abnormal behavior, and key areas of focus.

4. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The step of generating a space-time routing query based on the video to be detected, the global semantic context, and the forgery hypothesis specifically includes: According to the video to be tested Global semantic context and fabricated assumptions Using a second visual language model Generate spatial-temporal route query It can convert fake hypotheses into detection hints targeting local regions: ； The spatial-temporal routing query can be described in natural language, structured field, or a combination of natural language and structured field descriptions. The query content in spatial-temporal routing queries includes one or more of the following: target entity description, part description, appearance features, location information, and time segment information.

5. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The step of performing target localization and segmentation on the video to be detected based on spatial-temporal routing query to obtain the target region video specifically includes: Space-time routing query The GroundingDINO model was used to detect the video. Perform target localization frame by frame to obtain the corresponding bounding box set. Based on the bounding box, the SAM2 model is used for the video to be detected. Generate pixel-level masks The target region video is then cropped from the original video to be detected based on the mask. : ； ； ； in, This represents the target localization function based on the GroundingDINO model. This represents the mask generation function based on the SAM2 model. This indicates a mask clipping operation.

6. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The process of performing evidence-constrained reasoning based on global semantic context, forgery assumptions, and structured physical evidence to obtain preliminary judgment results and confidence levels specifically includes: Based on global semantic context The current round of falsification assumptions and structured physical evidence Using the third visual language model As a reasoning model, the expected abnormal behavior in the forgery hypothesis is cross-validated with structured physical evidence; When structured physical evidence supports the expected anomalous behavior of the forgery hypothesis, the preliminary judgment result is... To determine if it is forged, confidence levels are calculated based on the salience characteristics of structured physical evidence. ; When the structured physical evidence contradicts the expected anomalous behavior of the forgery hypothesis, the constrained inference model abandons the current forgery hypothesis and makes a preliminary judgment. Assess whether the value is true or unknown, and output the corresponding confidence level. ; The reasoning process of the reasoning model is represented as follows: 。 7. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, The process of performing a full-link consistency check on the execution trajectory of the preliminary judgment result to obtain a reliability score and feedback information specifically includes: The self-reflective verifier performs a full-link consistency check on the execution trajectory of the preliminary judgment result obtained in step S2, and outputs a reliability score. and feedback information : ； For global semantic context, To fabricate the hypothesis, For spatial-temporal routing lookup, This is the mask used when segmenting the video to be tested. This is a collection of specialized forensic tools used when extracting structured physical evidence. For structured physical evidence, For the preliminary judgment results, for The corresponding confidence level; For self-reflective validators; The end-to-end consistency check includes at least one or more of the following: Does the fabricated hypothesis match the context of the video to be detected? Check if the spatial-temporal routing query hits the expected target area; Does the mask used when segmenting the video to be tested accurately cover the target? Whether the dedicated evidence collection tools are suitable for the current scenario of the video to be tested; Whether the structured physical evidence is consistent with the preliminary judgment; Does the evidence-constrained reasoning process contain logical leaps or illusions? 8. The AI-synthesized video detection method based on multi-agent collaboration according to claim 1, characterized in that, When the reliability score is lower than the threshold, the falsification hypothesis is updated according to the feedback information and steps S2 to S3 are repeated until the reliability score requirement is met or the maximum number of iterations is reached. The final authenticity determination result is output, including: Based on global semantic context The current round of falsification assumptions and feedback information Using the fourth visual language model Update the falsification hypothesis for the current round to obtain the falsification hypothesis for the next round. : ； Strategies for updating the fabricated assumptions for the next round include at least one of the following: Shifting the focus area: When feedback indicates that no anomalies were found in the current target area, the focus area is shifted to other undetected local areas or alternative target areas in the fake hypothesis. Modify forgery mechanism: When feedback indicates that the structured physical evidence does not support the current expected forgery mechanism, replace it with an alternative forgery mechanism in the forgery hypothesis; Based on the assumption of the next round of forgery Repeat steps S2 to S3; When either scenario one or scenario two occurs, the iteration is terminated, and the preliminary judgment result, confidence level, and structured physical evidence corresponding to the optimal round are summarized to generate the final authenticity judgment result. Case 1 is when the new reliability score obtained after repeated execution reaches or exceeds the threshold. The round in which situation 1 occurs is the optimal round. Scenario 2 is when the number of iterations reaches the preset maximum number of iterations. In scenario two, the round with the highest reliability score is the optimal round. The final authenticity determination result specifically includes: authenticity determination of the video to be detected, confidence level, target area, structured physical evidence, end-to-end consistency check conclusion, and structured analysis report; The structured analysis report is used to record and present the complete reasoning chain of the multi-agent system in the optimal iteration round. The complete reasoning chain includes the causal deduction process from global semantic context parsing, falsification hypothesis proposal, extraction of corresponding structured physical evidence to obtaining the authenticity judgment.

9. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images