False information detection method and device based on adaptive routing

Abstract

The disclosure provides a false information detection method and device based on adaptive routing, belonging to the technical field of artificial intelligence. The method comprises: inputting a to-be-detected multi-modal statement into a trained multi-modal screening model for processing to output multi-modal fusion features; inputting the multi-modal fusion features into a first verification layer of a trained adaptive routing model for processing to output an initial prediction value; determining respective selection probabilities of the first verification layer, a second verification layer and a third verification layer according to the initial prediction value and the multi-modal fusion features; determining a target verification layer from the first verification layer, the second verification layer and the third verification layer according to the respective selection probabilities of the first verification layer, the second verification layer and the third verification layer and a harm score; inputting the multi-modal fusion features into the target verification layer for processing to output a target prediction value; and generating a detection result according to the target prediction value and verification information of the target verification layer.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a method and apparatus for detecting false information based on adaptive routing. Background Technology

[0002] With the development of social media and mobile internet, various carriers such as text, images, video screenshots, and links are combined for information dissemination. False information is often constructed by combining persuasive text with selected, edited, or context-detached images, making it difficult for single-modal detection to accurately determine its authenticity. Summary of the Invention

[0003] In view of this, the present disclosure provides a method and apparatus for detecting false information based on adaptive routing, which can at least partially solve the above-mentioned technical problems.

[0004] One aspect of this disclosure provides a method for detecting false information based on adaptive routing, comprising: inputting a multimodal claim to be detected into a pre-trained multimodal screening model for processing and outputting multimodal fusion features; the multimodal claim to be detected includes text content and visual evidence, the pre-trained multimodal screening model is used to measure the consistency relationship between the text content and the visual evidence and to generate multimodal fusion features based on the consistency relationship; inputting the multimodal fusion features into a first verification layer in a pre-trained adaptive routing model for processing to output an initial prediction value; the pre-trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer, the first verification layer being configured to lightweight screen the claim to be detected. The process involves a multimodal claim, a second verification layer configured to enhance the verification of the multimodal claim to be detected based on external evidence, and a third verification layer configured to infer the multimodal claim to be detected using multiple agents. The selection probabilities of the first, second, and third verification layers are determined based on initial predicted values ​​and multimodal fusion features. A target verification layer is determined from the first, second, and third verification layers based on their respective selection probabilities and hazard scores. The hazard score represents the social risk level of the multimodal claim after propagation. The multimodal fusion features are input into the target verification layer for processing to output a target predicted value. Finally, a detection text is generated based on the target predicted value and the verification information from the target verification layer.

[0005] According to embodiments of this disclosure, the pre-trained multimodal screening model includes a text encoder, a visual encoder, and a gating unit. Inputting the multimodal declaration to be detected into the pre-trained multimodal screening model for processing to output multimodal fusion features includes: processing the text content in the multimodal declaration to be detected using the text encoder to output a text semantic representation; processing the visual evidence in the multimodal declaration to be detected using the visual encoder to output a visual semantic representation; measuring the consistency relationship between the text semantic representation and the visual semantic representation using the gating unit; and outputting multimodal fusion features based on the consistency relationship, the text semantic representation, and the visual semantic representation.

[0006] According to embodiments of this disclosure, determining the selection probability of each of the first verification layer, the second verification layer, and the third verification layer based on the initial predicted value and the multimodal fusion features includes: determining a complexity feature based on the initial predicted value and the consistency relationship, wherein the complexity feature characterizes the verification and recognition difficulty of the multimodal statement to be detected; and determining the selection probability of each of the first verification layer, the second verification layer, and the third verification layer based on the complexity feature and the multimodal fusion features.

[0007] According to embodiments of this disclosure, the initial predicted values ​​include true category predicted values ​​and false category predicted values; determining complexity features based on the initial predicted values ​​and consistency relationships includes: determining maximum class confidence, prediction information entropy, and class prediction interval based on true category predicted values ​​and false category predicted values; maximum class confidence represents the maximum predicted value between true category predicted values ​​and false category predicted values, prediction information entropy represents the degree of ambiguity in the determination of the truth or falsehood of the multimodal declaration to be detected, and class prediction interval represents the gap between true category predicted values ​​and false category predicted values; and determining complexity features based on maximum class confidence, prediction information entropy, class prediction interval, and consistency relationships.

[0008] According to embodiments of this disclosure, determining a target verification layer from the first verification layer, the second verification layer, and the third verification layer based on their respective selection probabilities and hazard scores includes: sorting the selection probabilities of the first verification layer, the second verification layer, and the third verification layer by size; selecting the verification layer with the highest selection probability as a candidate verification layer; selecting the third verification layer as the target verification layer when the hazard score is greater than or equal to a first preset threshold and the candidate verification layer is either the first verification layer or the second verification layer; or selecting the candidate verification layer as the target verification layer when the hazard score is less than the first preset threshold.

[0009] According to embodiments of this disclosure, the multimodal claim to be detected further includes metadata; the hazard score is determined based on the metadata, text content, and visual evidence.

[0010] According to embodiments of this disclosure, the second verification layer includes: a query generation unit, an evidence retrieval unit, an evidence reordering unit, and an evidence enhancement verification unit; the multimodal claim to be detected also includes metadata; the query generation unit generates a retrieval query from the text content, visual evidence, and metadata; the evidence retrieval unit determines multiple candidate external evidence from a pre-stored database based on the retrieval query; the evidence reordering unit determines the target external evidence from the multiple candidate external evidence; and the evidence enhancement verification unit outputs the predicted value of the second verification layer based on the target external evidence and multimodal fusion features.

[0011] According to embodiments of this disclosure, the third verification layer includes: a first intelligent agent, a second intelligent agent, a third intelligent agent, and a fourth intelligent agent; the first intelligent agent processes visual evidence to generate analysis results, which characterize the risk of video image modification; the second intelligent agent determines the credibility of external evidence of the target; the third intelligent agent processes the multimodal claim to be detected, external evidence of the target, and metadata to generate inference results, which characterize the factual consistency of the multimodal claim to be detected; and the fourth intelligent agent processes the analysis results, credibility, inference results, and multimodal fusion features to output the predicted value of the third verification layer.

[0012] According to embodiments of this disclosure, generating detection text based on target predicted values ​​and verification information from the target verification layer includes: when the target verification layer is a first verification layer, the detection text includes a hazard score, complexity features, and the predicted value of the first verification layer; when the target verification layer is a second verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, and the predicted value of the second verification layer; or, when the target verification layer is a third verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, analysis results output by the first agent, credibility output by the second agent, reasoning results output by the third agent, and the predicted value of the third verification layer output by the fourth agent.

[0013] A second aspect of this disclosure provides an adaptive routing-based information detection apparatus, comprising: a first processing module for inputting a multimodal claim to be detected into a pre-trained multimodal screening model for processing and outputting multimodal fusion features; the multimodal claim to be detected includes text content and visual evidence, and the pre-trained multimodal screening model is used to measure the consistency relationship between the text content and the visual evidence and to generate multimodal fusion features based on the consistency relationship; a second processing module for inputting the multimodal fusion features into a first verification layer in a pre-trained adaptive routing model for processing and outputting an initial prediction value; the pre-trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer, wherein the first verification layer is configured to lightweight screen the multimodal claims to be detected. The second verification layer is configured to enhance the verification of the multimodal statement to be detected based on external evidence, and the third verification layer is configured to use multiple agents to reason about the multimodal statement to be detected. A first determination module is used to determine the selection probabilities of the first, second, and third verification layers based on initial predicted values ​​and multimodal fusion features. A second determination module is used to determine the target verification layer from the first, second, and third verification layers based on the selection probabilities and hazard scores of the first, second, and third verification layers. The hazard score represents the social risk level after the multimodal statement to be detected is propagated. A first generation module is used to input the multimodal fusion features into the target verification layer for processing to output a target predicted value. A second generation module is used to generate detection text based on the target predicted value.

[0014] The method and apparatus for detecting false information based on adaptive routing disclosed herein have at least the following technical effects:

[0015] By dynamically allocating verification computing resources based on the verification complexity and potential social harm of the multimodal claims to be detected, a lightweight screening process is directly adopted for simple samples with low complexity and low harm, without the need to perform redundant complex reasoning, effectively reducing the overall average computing overhead and detection response latency of the system. At the same time, a mandatory upgrade mechanism for high-harm information is set up, and multi-agent collaborative deep verification is uniformly enabled for high-risk multimodal claims, which significantly improves the reliability, robustness and interpretability of false information detection results in complex samples and high-harm rumor scenarios. Attached Figure Description

[0016] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 A flowchart illustrating an adaptive routing-based misinformation detection method according to an embodiment of the present disclosure is shown schematically.

[0018] Figure 2A flowchart illustrating the training process of an adaptive routing model according to an embodiment of the present disclosure is shown.

[0019] Figure 3 A flowchart illustrating the generation of multimodal fusion features according to embodiments of the present disclosure is shown schematically.

[0020] Figure 4 A flowchart illustrating the generation of selection probabilities according to embodiments of the present disclosure is shown schematically.

[0021] Figure 5 A flowchart illustrating the determination of the target verification layer according to an embodiment of the present disclosure is shown schematically;

[0022] Figure 6 A flowchart illustrating the second verification layer according to an embodiment of the present disclosure is shown schematically;

[0023] Figure 7 A flowchart illustrating the third verification layer according to an embodiment of the present disclosure is shown schematically;

[0024] Figure 8 A flowchart illustrating the generation of detection results according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0025] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] Existing multimodal fake information detection methods typically extract features through text encoders and visual encoders, and use attention, contrastive learning, or feature fusion networks to complete the classification.

[0029] In recent years, large language models, retrieval-enhanced generation, and multi-agent collaborative reasoning have been used for fact-checking. These methods can leverage external knowledge, long-chain reasoning, and role division to handle complex claims, but their computational overhead, invocation latency, and lexical consumption are significantly higher than lightweight detection models. Existing solutions typically employ a fixed verification process for all information to be detected, failing to differentiate between simple samples, knowledge-intensive samples, and high-risk samples, which easily leads to the following problems: First, complex reasoning modules are still invoked for simple information that can be directly identified by lightweight models, resulting in wasted computational resources; second, over-reasoning by complex models on simple information may introduce unstable judgments; third, a unified reasoning process dominated by text may weaken image evidence that conflicts with the text; and fourth, for information involving public safety, healthcare, disasters, or large-scale transmission risks, conventional confidence routing may fail to provide sufficiently rigorous verification.

[0030] Furthermore, traditional model selection or cascading methods often upgrade based on a single confidence threshold, making it difficult to simultaneously utilize signals such as cross-modal consistency, prediction uncertainty, external knowledge requirements, and potential social harm. The manual hierarchical labels used to train routers are costly and cannot directly reflect the actual differences in accuracy and computational cost between each verification layer. Therefore, a technical solution is needed that can adaptively allocate verification resources based on the verification complexity and potential harm of each multimodal message, and prioritize security upgrades for high-risk information.

[0031] Based on this, this disclosure provides an adaptive routing information detection method, which outputs the selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer through a normalized exponential function; the selection probabilities of the three layers are sorted to obtain candidate verification layers, and a high-risk forced escalation priority rule is set: if the risk score reaches a preset threshold, regardless of whether the candidate verification layer is the first or the second layer, the third verification layer is forcibly determined as the target verification layer; in low-risk scenarios, the candidate verification layer with the highest probability is directly used as the target verification layer.

[0032] This solution utilizes adaptive hierarchical scheduling to avoid using complex large models and multi-agent retrieval inference for simple samples, saving computing power, eliminating excessive inference interference, and solving the problems of resource waste and unstable judgment. For high-social-harm multimodal claims such as medical, disaster, public policy, and widespread dissemination claims, a hard constraint of a harm score threshold is set, with priority higher than the hierarchical selection probability output by the routing model. Even if a lightweight model can clearly identify the claim and the routing model recommends a lightweight verification process, high-harm content will still be forced into a third verification layer involving multi-agent collaboration. This layer conducts in-depth cross-verification from multiple dimensions, including image authenticity, the credibility of evidence sources, and long-chain factual logic, avoiding the security risks of missed or incorrect judgments due to simple screening of high-risk rumors. This meets the strict control requirements for online public opinion and public safety content. Multi-dimensional indicators are jointly used to quantify sample complexity, and independent harm score hierarchies are superimposed, overcoming the limitations of traditional single confidence threshold hierarchies.

[0033] Figure 1 A flowchart illustrating an adaptive routing-based information detection method according to an embodiment of the present disclosure is shown.

[0034] like Figure 1 As shown, this embodiment provides a method for detecting false information based on adaptive routing, including steps S110 to S160.

[0035] In step S110, the multimodal statement to be detected is input into the pre-trained multimodal screening model for processing and output of multimodal fusion features. The multimodal statement to be detected includes text content and visual evidence. The pre-trained multimodal screening model is used to measure the consistency relationship between text content and visual evidence and to generate multimodal fusion features based on the consistency relationship between text content and visual evidence.

[0036] In step S120, the multimodal fusion features are input into the first verification layer of the pre-trained adaptive routing model for processing to output an initial prediction value. The pre-trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer. The first verification layer is configured to lightweight filter the multimodal claims to be detected, the second verification layer is configured to enhance the verification of the multimodal claims to be detected based on external evidence, and the third verification layer is configured to use multiple agents to reason about the multimodal claims to be detected.

[0037] In step S130, the selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer are determined based on the initial predicted values ​​and the multimodal fusion features.

[0038] In step S140, the target verification layer is determined from the first verification layer, the second verification layer, and the third verification layer based on the selection probability and hazard score corresponding to each of the three verification layers; the hazard score represents the social risk level after the multimodal statement to be detected is propagated.

[0039] In step S150, the multimodal fusion features are input into the target verification layer for processing to output the target prediction value.

[0040] In step S160, detection text is generated based on the target predicted value and the verification information of the target verification layer.

[0041] In the embodiments of this disclosure, the text content in the multimodal claim to be detected is first subjected to denoising, format normalization, special character filtering, lexicalization, and length truncation or padding to obtain standardized text content. The visual evidence in the multimodal claim to be detected is then subjected to anomalous image filtering, size adjustment, center cropping, pixel normalization, and visual encoding adaptation to obtain standardized visual evidence.

[0042] The pre-trained multimodal screening model processes standardized text content and standardized visual evidence separately, generating corresponding textual and visual semantic representations for each. Based on these representations, the consistency relationship between them is determined. Finally, the textual and visual semantic representations are fused according to the consistency relationship to generate multimodal fusion features.

[0043] The pre-trained adaptive routing model comprises a first verification layer, a second verification layer, and a third verification layer, with the pre-trained multimodal filtering model set within the first verification layer. Specifically, the first verification layer directly outputs lightweight detection results based on the input multimodal fusion features; the second verification layer outputs evidence-enhanced detection results based on the input multimodal fusion features combined with external evidence; and the third verification layer evaluates the source credibility of the visual evidence based on the aggregated visual evidence output by multiple agents, performs fact-consistency reasoning by combining text content, visual descriptions, external evidence, and event context, and processes the multimodal fusion features to achieve collaborative verification by multiple agents. The visual descriptions are information obtained from visual evidence, consisting of readable structured descriptions obtained through image captioning models, OCR, object detection, or manual annotation, such as "a building / person / time sign appears in the image," used for textual alignment with external evidence to avoid relying solely on uninterpretable visual vectors.

[0044] The multimodal fusion features are input into the first verification layer for processing to output an initial predicted value. The complexity of the multimodal claim to be detected is determined based on this initial predicted value and the consistency relationship. Based on this complexity and the multimodal fusion features, the selection probabilities of the first, second, and third verification layers are output. The verification layer with the highest selection probability is determined as the candidate verification layer, and then the target verification layer is determined by combining this with the hazard score.

[0045] According to embodiments of this disclosure, a multimodal screening model built into the first verification layer quantifies the consistency relationship between text and images, and generates multimodal fusion features. Combining initial predicted values ​​with a harm score representing the social risk of information dissemination, an adaptive routing model outputs the selection probabilities corresponding to the three verification layers, and automatically matches the target verification layer based on the selection probabilities. The three verification layers respectively adopt differentiated architectures: lightweight screening, external evidence-enhanced verification, and multi-agent collaborative deep reasoning. This solution can dynamically allocate verification computing resources based on the recognition difficulty and potential social risk of multimodal claims, eliminating complex retrieval and multi-agent reasoning for simple samples, reducing the average system computation cost and detection latency; simultaneously, relying on external evidence and multi-dimensional agent-based reasoning improves the reliability and robustness of detecting complex text-image conflicts and high-social-risk false information.

[0046] Furthermore, the pre-trained adaptive routing model is trained using a preference optimization approach. Specifically, the first, second, and third verification layers are run on the training samples, and the prediction accuracy and computational cost of each verification layer are recorded. For training samples with a hazard score greater than or equal to a preset hazard threshold, the third verification layer is designated as the preferred layer. For training samples with a hazard score less than the preset hazard threshold, the verification layer with the correct prediction is selected first. When multiple verification layers predict correctly, the verification layer with the lower computational cost is selected as the preferred layer. Preference triples are constructed based on the training samples, preferred layers, and non-preferred layers, and the adaptive routing model is trained using a direct preference optimization method, enabling it to learn a routing strategy that balances detection accuracy and computational cost.

[0047] The computational cost includes at least one of the following: number of model calls, number of input and output tokens, processing latency, GPU memory usage, computing power consumption, or energy consumption. The average computational cost of the first verification layer, the second verification layer, and the third verification layer increases sequentially. The first verification layer is suitable for rapid screening, the second verification layer is suitable for evidence enhancement verification, and the third verification layer is suitable for deep verification of highly complex and high-risk information.

[0048] The first verification layer can be trained using at least one of the following: classification loss, cross-modal contrastive loss, and cross-modal conflict perception loss. The classification loss is used for authenticity prediction, the cross-modal contrastive loss is used to enhance the semantic alignment between text and visual evidence, and the cross-modal conflict perception loss is used to improve the model's sensitivity to text-visual contradictory samples. Specifically, the classification loss L... cls Used for training real / fake binary classification; cross-modal contrastive loss L contrast For example, InfoNCE makes the textual and visual representations of the same sample close to each other, and different samples far apart; cross-modal conflict perception loss L conflictIt can perform binary classification on text-visual consistent / conflicting labels, or improve the conflict head output for contradictory samples. The total loss is L. T1 =L cls +λ con L contrast +λ cfl L conflict . λ con and λ cfl These represent the weights corresponding to the cross-modal contrast loss and the cross-modal conflict perception loss, respectively.

[0049] Figure 2 A flowchart illustrating the training process of an adaptive routing model according to an embodiment of the present disclosure is shown.

[0050] like Figure 2 As shown, the first, second, and third verification layers are run for each sample X in the training set, yielding three prediction results and corresponding costs. Az(X) is defined as the prediction accuracy of the z-th verification layer for sample X, taking the value 1 for correct prediction and 0 for incorrect prediction. Cz is defined as the normalized cost of the z-th verification layer.

[0051] For samples whose hazard scores reach the threshold, the third verification layer is designated as the preferred layer. For other samples, when Az... a (X)>Az b When (X), level z a Superior to level z b When Az a (X)=Az b When (X)=1, if Cz a <Cz b Then the level z a Superior to level z b Therefore, a preference triple (X, z) is constructed. w , z l ), where z w For optimal selection of levels, z l This is a non-preferred level.

[0052] The goal of direct preference optimization is to increase the relative probability of the routing policy on the preferred level. Its loss function is used to increase the preferred level z. w With non-preferred level z l The relative probability difference between the two, where the probability ratio between the routing policy to be trained and the reference routing policy is used to constrain the optimization direction, and the preference separation strength parameter is used to control the degree of distinction between the preferred level and the non-preferred level.

[0053] The reference routing strategy can be a routing model before training, a routing model pre-trained using supervised learning, or a fixed-rule routing model. Through the above training, the router prioritizes the low-cost layer when multiple verification layers can correctly process samples, selects a deeper verification layer when the low-cost layer fails, and prioritizes safety for high-risk samples.

[0054] Figure 3 A flowchart illustrating the generation of multimodal fusion features according to an embodiment of the present disclosure is shown schematically.

[0055] like Figure 3 As shown, the pre-trained multimodal screening model in this embodiment includes a text encoder, a visual encoder, and a gating unit.

[0056] The process of inputting the multimodal claims to be detected into a pre-trained multimodal screening model to output multimodal fusion features includes: processing the text content of the multimodal claims to be detected using a text encoder to output a text semantic representation; processing the visual evidence of the multimodal claims to be detected using a visual encoder to output a visual semantic representation; measuring the consistency between the text semantic representation and the visual semantic representation using a gating unit; and outputting multimodal fusion features based on the consistency relationship, the text semantic representation, and the visual semantic representation.

[0057] In embodiments of this disclosure, the text encoder can be RoBERTa, BERT, or other Transformer encoders, and the visual encoder can be a CLIP visual encoder, a Vision Transformer, or a convolutional neural network. A consistency gating unit is used to measure the consistency relationship between text content and visual evidence. This consistency relationship can be determined by the cosine similarity between the text semantic representation and the visual semantic representation, the cross-attention matching score, the conflict detection head output, or the gating network output.

[0058] For example, project the text semantic representation t and the visual semantic representation v onto the same dimension and normalize them to obtain t' and v' respectively; calculate the cosine similarity s0=cos(t',v'); simultaneously input [t';v';|t'-v'|;t'⊙v'] into the conflict detection head to obtain the conflict probability sc; finally, the consistency parameter s=λ can be taken. s0 +(1-λ)(1-sc), or directly output s=sigmoid(W) from the gated network. s [t';v';|t'-v'|;t'⊙v']+b s ), where W s b represents the science department weight parameters of the gated network. s is the learnable bias term of the gated network; ⊙ represents element-wise multiplication; λ is the weight coefficient, which can take values ​​from 0 to 1.

[0059] Furthermore, in order to control computational costs, some or all of the pre-trained encoder parameters can be frozen, and only the projection layer, consistency gating unit, and classification head can be trained.

[0060] According to embodiments of this disclosure, the first verification layer processes low-complexity multimodal claims that can be identified without external evidence, reducing computational resources.

[0061] Figure 4 A flowchart illustrating the generation of selection probabilities according to an embodiment of the present disclosure is shown schematically.

[0062] See also Figure 3 and Figure 4 Based on the above embodiments, this embodiment determines the selection probabilities of the first verification layer, the second verification layer, and the third verification layer according to the initial predicted value and the multimodal fusion features, including: determining the complexity features based on the initial predicted value and the consistency relationship, wherein the complexity features characterize the verification and recognition difficulty of the multimodal declaration to be detected; and determining the selection probabilities of the first verification layer, the second verification layer, and the third verification layer according to the complexity features and the multimodal fusion features.

[0063] Furthermore, a temperature scaling method can be used to calibrate the initial predictions, and subsequent complexity features can be determined based on the calibrated initial predictions. For example, assuming the output logits of the first verification layer is 'a', the temperature T>0 is learned on the validation set to minimize the negative log-likelihood (NLL) or expected calibration error (ECE); during inference, p1=softmax(a / T) is used as the calibrated probability. T>1 reduces overconfidence, while T<1 strengthens the confidence distribution.

[0064] The initial predicted values ​​include true class predicted values ​​and false class predicted values. Complexity features are determined based on the initial predicted values ​​and consistency relationships, including: determining the maximum class confidence, prediction information entropy, and class prediction interval based on the true class predicted values ​​and false class predicted values; the maximum class confidence represents the maximum predicted value between the true class predicted value and the false class predicted value, the prediction information entropy represents the degree of ambiguity in the determination of the true or false nature of the multimodal declaration to be detected, and the class prediction interval represents the gap between the true class predicted value and the false class predicted value; complexity features are determined based on the maximum class confidence, prediction information entropy, class prediction interval, and consistency relationships.

[0065] In the embodiments of this disclosure, the category probability output by the first verification layer is a binary classification probability p1, including the true category prediction value p. 1，real and false category prediction value p 1，fake Among them, p 1，real+p 1，fake =1.

[0066] The maximum class confidence is c max =max(p1) can intuitively reflect the model's confidence in the true or false results.

[0067] Predicted information entropy is P 1,i It represents the predicted probability of the first verification layer for the i-th class, characterizing the overall disorder of the true and false probability distribution, and making up for the deficiency that the single-point maximum confidence cannot identify boundary samples where the probabilities of the two classes are close and the model is oscillating.

[0068] The class probability interval is m=|p 1,real -p 1,fake | Quantify the difference between the probability of truth and falsehood.

[0069] Complexity feature u=[1-c max H(p1), 1-m, 1-s], where 1-c max Representing the low confidence level of the first layer of verification for the multimodal claim to be detected, 1-c reflects whether the claim requires a higher level of verification. max The larger the value, the lower the confidence level; the larger the value of H(p1), the more ambiguous the model's judgment; 1-m reflects difficult scenarios with a narrow gap between true and false information, and the larger the value of 1-m, the closer the probabilities of true and false information are; the higher the value of s, the more consistent the text and image are, and the higher the value of 1-s, the stronger the cross-modal conflict. By unifying feature trends, the learning difficulty of the adaptive routing model is reduced, the model training and fitting cost is decreased, the routing model's ability to distinguish the difficulty of samples is improved, and it is easier to stably output reliable level selection probabilities.

[0070] By concatenating or weighting the complexity features and multimodal fusion features, and inputting them into the trained adaptive routing model, the selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer are obtained after processing by a multilayer perceptron and a normalized exponential function.

[0071] According to embodiments of this disclosure, based on the initial binary classification prediction value output by the first verification layer, three indicators characterizing the degree of ambiguity in the model's prediction are extracted to quantify the degree of internal judgment oscillation within the model. Based on the image-text consistency coefficient s output by the consistency gating unit of the multimodal screening model, a cross-modal conflict component 1-s is constructed to address multimodal image-text forgery scenarios. Finally, these components are concatenated into a one-dimensional feature vector that can identify boundary samples where the model's prediction is ambiguous, and accurately capture false information such as reusing old images or altering accompanying images. The complex quantification results closely match the actual verification difficulty; the larger the u component value, the higher the difficulty of verifying and identifying the corresponding sample. By jointly quantifying sample complexity using multi-dimensional indicators, the one-sidedness of single-indicator judgment is avoided, and by adopting a unified positive standardized feature vector, the difficulty of fitting the routing model and the accuracy of hierarchical differentiation are reduced.

[0072] Furthermore, complexity features may also include time sensitivity, title persuasiveness, or estimates of external knowledge requirements.

[0073] Figure 5 A flowchart illustrating the determination of the target verification layer according to an embodiment of the present disclosure is shown schematically.

[0074] like Figure 5 As shown, based on the above embodiments, determining the target verification layer from the first verification layer, second verification layer, and third verification layer according to their respective selection probabilities and hazard scores includes: sorting the selection probabilities of the first verification layer, second verification layer, and third verification layer by size; selecting the verification layer with the highest selection probability as a candidate verification layer; and selecting the third verification layer as the target verification layer if the hazard score is greater than or equal to a first preset threshold and the candidate verification layer is either the first or second verification layer. Alternatively, selecting the candidate verification layer as the target verification layer if the hazard score is less than the first preset threshold.

[0075] Furthermore, the hazard score, complexity features, and multimodal fusion features are concatenated and fused to generate routing features. These routing features are then input into the adaptive routing target verification layer of the pre-trained adaptive routing model.

[0076] In the embodiments of this disclosure, the trained adaptive routing model outputs three sets of selection probabilities P1 (selection probability of the first verification layer), P2 (selection probability of the second verification layer), and P3 (selection probability of the third verification layer) through a normalized exponential function. The values ​​of the three probabilities are all between 0 and 1, and the sum is 1. The larger the probability value, the higher the degree to which the routing model determines that the layer is adapted to the current multimodal declaration to be detected.

[0077] If the hazard score h is greater than or equal to the first preset threshold τ, regardless of whether the candidate verification layer is the first or second verification layer, a forced replacement is made, and the third verification layer is directly determined as the final target verification layer. Even if the routing model calculates that P1 or P2 is much greater than P3, as long as the information society risk meets the standard, multi-agent collaborative deep verification with the highest computing power and the strongest verification intensity will be used.

[0078] Alternatively, if the hazard score h is less than the first preset threshold τ, and P1 is at its maximum: the target verification layer is the first verification layer, and lightweight rapid screening is performed; if P2 is at its maximum: the target verification layer is the second verification layer, and enhanced evidence retrieval verification is performed; if P3 is at its maximum: the target verification layer is the third verification layer, and multi-agent deep verification is performed. That is, the original candidate verification layer z... * As a target verification layer.

[0079] According to embodiments of this disclosure, candidate verification layers for probabilistically matched samples are first selected based on a three-layer verification layer, balancing the difficulty of sample verification with computational overhead. Simultaneously, a hard constraint is set to force upgrades for high-risk information, with this constraint having higher priority than the routing probability recommendation results. Regardless of the model's judgment difficulty, high-risk multimodal declarations uniformly enter the multi-agent deep verification process. This mechanism, while achieving adaptive allocation of computing resources and reducing the system's average computational cost and detection latency, specifically builds a safety net for high-risk information security. It addresses the vulnerability of traditional pure probabilistic routing mechanisms where high-risk rumors are simplified due to the high confidence of lightweight models, simultaneously balancing the efficiency of massive information review with the rigor of public opinion and social security management.

[0080] Furthermore, the multimodal claims to be detected also include metadata, which represents auxiliary content that assists in the detection of textual content and visual evidence; a hazard score is determined based on the metadata, textual content, and visual evidence.

[0081] Metadata can include the event's theme, involved entities, dissemination scope, timeliness, and potential social impact. The event's theme is obtained through text topic classification or keyword clustering, and involved entities are identified using NER (Network Entity Recognition). Location or time is obtained through text parsing or image EXIF / posting information. Dissemination scope is obtained through platform statistics such as the number of reposts, comments, and the coverage of posting accounts. Timeliness is obtained from the difference between the event's occurrence time and the detection time. Potential social impacts, such as the affected population and behavioral inducement, can be identified using rule templates or classifiers. The hazard score can be output by the hazard assessment model: h = sigmoid(W h z h +b h ), where zh is the concatenated / encoded representation of the above metadata features, and W h b is a learnable weight parameter. h The bias term can be learned; a weighted sum h=Σw can also be used. i f j Normalize to [0,1].

[0082] Hazard assessment models can be implemented using classification models, regression models, or a combination of rule-based and learning models. Their inputs can include characteristics such as subject area, entity type, affected population, scope of spread, timeliness, action inducement, and health or safety consequences.

[0083] For example, information related to public safety, healthcare, disaster warnings, major events, and information that may trigger real-world behavior receives a higher harm weight; while general entertainment or low-impact information receives a lower harm weight.

[0084] According to embodiments of this disclosure, multi-dimensional metadata extraction covers all social harm inducements of false information. Harm scores calculated based on metadata such as dissemination, theme, and audience targeting quantify the negative social impact of information on public safety, people's livelihoods, and public opinion. The fusion of harm scores and complexity features enables dual-dimensional routing judgment based on both difficulty and risk.

[0085] Figure 6 A flowchart illustrating a second verification layer according to an embodiment of the present disclosure is shown schematically.

[0086] like Figure 6 As shown, the second verification layer of this embodiment includes: a query generation unit, an evidence retrieval unit, an evidence reordering unit, and an evidence enhancement verification unit; the query generation unit generates a retrieval query from text content, visual evidence, and metadata; the evidence retrieval unit determines multiple candidate external evidence from a pre-stored database based on the retrieval query; the evidence reordering unit determines the target external evidence from the multiple candidate external evidence; and the evidence enhancement verification unit outputs the predicted value of the second verification layer based on the target external evidence and multimodal fusion features.

[0087] Furthermore, metadata also includes visual descriptions.

[0088] In the embodiments of this disclosure, the second verification layer is a retrieval-enhanced verification module, which internally connects a query generation unit, an evidence retrieval unit, an evidence reordering unit, and an evidence enhancement verification unit. It is adapted to medium-complexity multimodal claims that cannot be verified by relying solely on the features of images and texts themselves and require external objective factual support.

[0089] Its input includes four types of information: the text content of the multimodal claim to be detected, the original visual evidence, general metadata, and structured visual description. Among them, the visual description transforms abstract image vectors into readable text information, making up for the deficiency that the original visual evidence cannot be directly used for matching in text retrieval databases, and incorporating visual entities such as people, buildings, signs, scenes, time and location in the image into the retrieval dimension.

[0090] The query generation unit extracts key entities, event keywords, spatiotemporal elements, scene objects, and publishing background from all input information and integrates them to generate one or more standardized text retrieval query statements.

[0091] The evidence retrieval unit receives the retrieval query output by the query generation unit, connects to pre-stored authoritative databases (such as fact-checking databases, official news databases, policy knowledge bases, and historical event material databases), performs text / vector retrieval, and batch retrieves multiple relevant materials to form a set of unfiltered candidate external evidence.

[0092] The evidence re-ranking unit processes all candidate external evidence, and scores and ranks them based on the semantic relevance of the evidence to the multimodal statement, the timeliness of publication, the authority and credibility of the information source, and the redundancy of the content. It eliminates invalid materials that are low in relevance, outdated, have low credibility, or are duplicated, and selects and retains the top-k high-quality materials as target external evidence.

[0093] The evidence enhancement verification unit combines the multimodal fusion features output from the first verification layer upstream with the selected target external evidence to perform feature fusion, jointly conduct cross-modal fact consistency reasoning, output the exclusive true / false prediction value of the second verification layer, and complete the enhanced verification with external objective fact support.

[0094] According to embodiments of this disclosure, the second verification layer employs a four-unit pipeline to complete enhanced verification. When constructing the search query, it integrates text, visual evidence, metadata, and structured visual descriptions. It relies on database retrieval to obtain external objective facts and performs credibility filtering. It then integrates external target evidence with multimodal fusion features to complete truthfulness reasoning. On one hand, by jointly generating search queries from multiple sources, it compensates for the deficiency of pure text retrieval in missing key image clues, improving the accuracy of external evidence retrieval. On the other hand, it relies on evidence reordering to filter low-quality materials, reducing the interference of irrelevant evidence on the verification results. It also relies on authoritative external facts to compensate for the lack of information in a single multimodal statement, accurately solving the problem of verifying moderately complex false information related to knowledge gaps.

[0095] Furthermore, raw visual evidence consists only of pixel features and cannot be directly used in text retrieval. This solution transforms images into readable, structured visual descriptions and incorporates them into the retrieval query, converting all hidden factual clues such as scenes, entities, and time markers within the image into search keywords. For multimodal rumors involving deliberately fabricated text and images with realistic backgrounds, this solution can retrieve supporting materials that cannot be matched by a single text search, preventing the loss of crucial factual evidence due to missing key image information and significantly improving the recall coverage of relevant external evidence.

[0096] Figure 7 A flowchart illustrating the third verification layer according to an embodiment of the present disclosure is shown schematically.

[0097] like Figure 7 As shown, the third verification layer of this embodiment includes: a first intelligent agent, a second intelligent agent, a third intelligent agent, and a fourth intelligent agent.

[0098] Specifically, the first intelligent agent processes visual evidence to generate analysis results, which characterize the risk of video image modification; the second intelligent agent determines the credibility of external evidence of the target; the third intelligent agent processes the multimodal claim to be detected, external evidence of the target, and metadata to generate inference results, which characterize the factual consistency of the multimodal claim to be detected; and the fourth intelligent agent processes the analysis results, credibility, inference results, and multimodal fusion features to output the prediction value of the third verification layer.

[0099] In the embodiments of this disclosure, a first intelligent agent analyzes the risks of image tampering, image generation, reuse of old images, or inconsistencies between images and text in visual evidence. A second intelligent agent evaluates the sources of external evidence, including dimensions such as evidence release channels (e.g., official media, social media, personal accounts, forums), release time, timeliness, and completeness of the original evidence text (whether it is taken out of context). A third intelligent agent performs consistency reasoning on text content, visual description, external evidence, and event background to reduce text dominance bias and multi-agent correlation errors. A fourth intelligent agent integrates the outputs of the above agents, resolves conflicts, and aggregates results.

[0100] According to embodiments of this disclosure, generating detection text based on target prediction values ​​includes: when the target verification layer is a first verification layer, the detection text includes a hazard score, complexity features, and the prediction value of the first verification layer; when the target verification layer is a second verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, and the prediction value of the second verification layer; or, when the target verification layer is a third verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, analysis results output by a first agent, credibility output by a second agent, reasoning results output by a third agent, and the prediction value of the third verification layer output by a fourth agent.

[0101] In the embodiments of this disclosure, the output detection results include at least one of the following: complexity features, hazard score, predicted value of the target verification layer, external evidence and source, opinions of each agent, reasons for the decision, processing delay and lexical consumption, which can form a traceable false information verification record.

[0102] Figure 8 A flowchart illustrating the generation of detection results according to an embodiment of the present disclosure is shown schematically.

[0103] like Figure 8As shown, the text content T and visual evidence V from the multimodal claims to be detected are input into a pre-trained multimodal screening model for processing. The text encoder processes the text content T, outputting a text semantic representation t. The visual encoder processes the visual evidence V, outputting a visual semantic representation v. A gating unit measures the consistency relationship s between the text semantic representation and the visual semantic representation, and outputs a multimodal fusion feature based on the consistency relationship s, the text semantic representation t, and the visual semantic representation v. The metadata from the multimodal claims to be detected is input into a hazard scoring model to output a hazard score h, used to assess potential social risks. An initial prediction value is determined based on this multimodal fusion feature, and then a complexity feature is determined. The complexity feature, hazard score, and multimodal fusion feature are fused to generate a routing feature. Based on this routing feature, the selection probabilities of the first verification layer (Tier1), the second verification layer (Tier2), and the third verification layer (Tier3) are determined, and the verification layer with the highest selection probability is selected as the candidate verification layer. If the hazard score h is greater than or equal to the first preset threshold τ, regardless of whether the candidate verification layer is the first or second verification layer, a forced replacement is performed, and the third verification layer is directly determined as the final target verification layer. Alternatively, if the hazard score h is less than the first preset threshold τ, the candidate verification layer is used as the target verification layer, and the detection result is output.

[0104] A second aspect of this disclosure provides an information detection device based on adaptive routing, comprising:

[0105] The first processing module is used to input the multimodal claims to be detected into a pre-trained multimodal screening model for processing and outputting multimodal fusion features. The multimodal claims to be detected include text content and visual evidence. The pre-trained multimodal screening model is used to measure the consistency relationship between the text content and the visual evidence and to generate multimodal fusion features based on the consistency relationship. The principle of this first processing module is the same as that of step S110, and will not be repeated here.

[0106] The second processing module is used to input the multimodal fusion features into the first verification layer of the pre-trained adaptive routing model for processing to output initial prediction values. The pre-trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer. The first verification layer is configured to lightweight filter the multimodal claims to be detected, the second verification layer is configured to enhance the verification of the multimodal claims to be detected based on external evidence, and the third verification layer is configured to use multiple agents to infer the multimodal claims to be detected. The principle of this first processing module is the same as that of step S120, and will not be repeated here.

[0107] The first determining module is used to determine the selection probabilities of the first verification layer, the second verification layer, and the third verification layer based on the initial predicted values ​​and the multimodal fusion features. The principle of this first processing module is the same as that of step S130, and will not be repeated here.

[0108] The second determination module is used to determine the target verification layer from the first verification layer, the second verification layer, and the third verification layer based on the selection probability and hazard score corresponding to each of the three verification layers. The hazard score represents the social risk level after the multimodal claim to be detected is propagated. The principle of this first processing module is the same as that of step S140, and will not be repeated here.

[0109] The first generation module is used to input the multimodal fusion features into the target verification layer for processing to output the target prediction value. The principle of this first processing module is the same as that of step S150, and will not be repeated here.

[0110] The second generation module is used to generate detection text based on the target prediction value. The principle of this first processing module is the same as that of step S160, and will not be repeated here.

[0111] According to embodiments of this disclosure, the first processing module is further configured to: input the multimodal screening model to be detected into the trained multimodal screening model for processing to output multimodal fusion features, including: processing the text content in the multimodal declaration to be detected through the text encoder to output a text semantic representation; processing the visual evidence in the multimodal declaration to be detected through the visual encoder to output a visual semantic representation; measuring the consistency relationship between the text semantic representation and the visual semantic representation through the gating unit, and outputting multimodal fusion features based on the consistency relationship, the text semantic representation, and the visual semantic representation.

[0112] The first determining module is further configured to determine the selection probabilities of the first verification layer, the second verification layer, and the third verification layer based on the initial predicted values ​​and multimodal fusion features. This includes: determining complexity features based on the initial predicted values ​​and consistency relationships, whereby the complexity features characterize the verification and recognition difficulty of the multimodal declaration to be detected; and determining the selection probabilities of the first verification layer, the second verification layer, and the third verification layer based on the complexity features and the multimodal fusion features. It is also configured to determine routing features based on the complexity features and the multimodal fusion features; and determine the selection probabilities of the first verification layer, the second verification layer, and the third verification layer based on the routing features. It is also used for initial predicted values, including true class predicted values ​​and false class predicted values; and to determine complexity features based on the initial predicted values ​​and consistency relationships, including: determining the maximum class confidence, prediction information entropy, and class prediction interval based on the true class predicted values ​​and false class predicted values; the maximum class confidence represents the maximum predicted value between the true class predicted value and the false class predicted value, the prediction information entropy represents the degree of ambiguity in the determination of the true or false nature of the multimodal declaration to be detected, and the class prediction interval represents the difference between the true class predicted value and the false class predicted value; and to determine complexity features based on the maximum class confidence, prediction information entropy, class prediction interval, and consistency relationships.

[0113] The second determining module is further configured to determine a target verification layer from the first verification layer, the second verification layer, and the third verification layer based on the selection probabilities and hazard scores corresponding to the first verification layer, the second verification layer, and the third verification layer, including: sorting the selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer by size; taking the verification layer with the highest selection probability as a candidate verification layer; taking the third verification layer as the target verification layer when the hazard score is greater than or equal to a first preset threshold and the candidate verification layer is the first verification layer or the second verification layer; or taking the candidate verification layer as the target verification layer when the hazard score is less than the first preset threshold.

[0114] The second processing module, also used in the second verification layer, includes: a query generation unit, an evidence retrieval unit, an evidence reordering unit, and an evidence enhancement verification unit; the multimodal statement to be detected also includes metadata; the query generation unit generates a retrieval query from the text content, visual evidence, and metadata; based on the retrieval query, the evidence retrieval unit determines multiple candidate external evidence from a pre-stored database; the evidence reordering unit determines the target external evidence from the multiple candidate external evidence; and based on the target external evidence and multimodal fusion features, the evidence enhancement verification unit outputs the predicted value of the second verification layer.

[0115] The second processing module is also used in the third verification layer, which includes: a first agent, a second agent, a third agent, and a fourth agent. The first agent processes visual evidence to generate analysis results, which characterize the risk of video image modification. The second agent determines the credibility of external evidence of the target. The third agent processes the multimodal claim to be detected, external evidence of the target, and metadata to generate inference results, which characterize the factual consistency of the multimodal claim to be detected. The fourth agent processes the analysis results, credibility, inference results, and multimodal fusion features to output the predicted value of the third verification layer.

[0116] The second generation module is further configured to generate detection text based on the target predicted value and the verification information of the target verification layer, including: when the target verification layer is the first verification layer, the detection text includes a hazard score, complexity features, and the predicted value of the first verification layer; when the target verification layer is the second verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, and the predicted value of the second verification layer; or, when the target verification layer is the third verification layer, the detection text includes a hazard score, complexity features, external evidence of the target, the analysis result output by the first agent, the credibility output by the second agent, the reasoning result output by the third agent, and the predicted value of the third verification layer output by the fourth agent.

[0117] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for detecting false information based on adaptive routing, characterized in that, include: The multimodal declaration to be detected is input into the pre-trained multimodal screening model for processing and output of multimodal fusion features; The multimodal claim to be detected includes text content and visual evidence. The trained multimodal screening model is used to measure the consistency relationship between the text content and the visual evidence and to generate the multimodal fusion feature based on the consistency relationship between the text content and the visual evidence. The multimodal fusion features are input into the first verification layer of the pre-trained adaptive routing model for processing to output the initial prediction value; The trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer. The first verification layer is configured to lightweight filter the multimodal statements to be detected. The second verification layer is configured to enhance the verification of the multimodal statements to be detected based on external evidence. The third verification layer is configured to use multiple agents to reason about the multimodal statements to be detected. The selection probabilities of the first verification layer, the second verification layer, and the third verification layer are determined based on the initial predicted value and the multimodal fusion features. The target verification layer is determined from the first verification layer, the second verification layer, and the third verification layer based on the selection probability and hazard score corresponding to each of the three verification layers; the hazard score characterizes the social risk level after the multimodal statement to be detected is propagated. The multimodal fusion features are input into the target verification layer for processing to output the target prediction value; The detection text is generated based on the target prediction value and the verification information of the target verification layer.

2. The method according to claim 1, characterized in that, The trained multimodal screening model includes a text encoder, a visual encoder, and a gating unit; The step of inputting the multimodal declaration to be detected into the pre-trained multimodal screening model for processing to output multimodal fusion features includes: The text encoder processes the text content in the multimodal declaration to be detected and outputs a semantic representation of the text. The visual encoder processes the visual evidence in the multimodal claim to be detected and outputs a visual semantic representation. The gating unit measures the consistency relationship between the text semantic representation and the visual semantic representation, and outputs multimodal fusion features based on the consistency relationship, the text semantic representation, and the visual semantic representation.

3. The method according to claim 1, characterized in that, The step of determining the selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer based on the initial predicted value and the multimodal fusion features includes: The complexity feature is determined based on the initial predicted value and the consistency relationship, and the complexity feature characterizes the verification and recognition difficulty of the multimodal statement to be detected; The selection probabilities of the first verification layer, the second verification layer, and the third verification layer are determined based on the complexity features and the multimodal fusion features.

4. The method according to claim 3, characterized in that, The initial predicted values ​​include true class predicted values ​​and false class predicted values; The step of determining the complexity features based on the initial predicted values ​​and the consistency relationship includes: The maximum class confidence, prediction information entropy, and class prediction interval are determined based on the true class prediction value and the false class prediction value; the maximum class confidence value represents the maximum prediction value between the true class prediction value and the false class prediction value; the prediction information entropy represents the ambiguity of the determination of the true or false nature of the multimodal claim to be detected; and the class prediction interval represents the difference between the true class prediction value and the false class prediction value. Complexity features are determined based on the maximum category confidence, the prediction information entropy, the category prediction interval, and the consistency relationship.

5. The method according to claim 3, characterized in that, The step of determining the target verification layer from the first verification layer, the second verification layer, and the third verification layer based on the selection probability and hazard score corresponding to each of the first verification layer, the second verification layer, and the third verification layer includes: The selection probabilities corresponding to the first verification layer, the second verification layer, and the third verification layer are sorted by size. The verification layer corresponding to the highest selection probability is used as the candidate verification layer; If the hazard score is greater than or equal to the first preset threshold, and the candidate verification layer is the first verification layer or the second verification layer, the third verification layer will be used as the target verification layer. Alternatively, if the hazard score is less than the first preset threshold, the candidate verification layer can be used as the target verification layer.

6. The method according to claim 5, characterized in that, The multimodal declaration to be detected also includes metadata; The hazard score is determined based on the metadata, the text content, and the visual evidence.

7. The method according to claim 6, characterized in that, The second verification layer includes: a query generation unit, an evidence retrieval unit, an evidence reordering unit, and an evidence enhancement verification unit; The query generation unit generates a retrieval query from the text content, the visual evidence, and the metadata. Based on the retrieval query, multiple candidate external evidences are determined from the pre-stored database through the evidence retrieval unit; The target external evidence is determined from the plurality of candidate external evidence by the evidence reordering unit. Based on the target external evidence and the multimodal fusion features, the evidence enhancement verification unit outputs the predicted value of the second verification layer.

8. The method according to claim 7, characterized in that, The third verification layer includes: a first intelligent agent, a second intelligent agent, a third intelligent agent, and a fourth intelligent agent; The first intelligent agent processes the visual evidence to generate analysis results, which characterize the risk of video image modification. The credibility of the external evidence against the target is determined by the second intelligent agent; The third intelligent agent processes the multimodal claim to be detected, the target external evidence, and the metadata to generate a reasoning result, which characterizes the factual consistency of the multimodal claim to be detected. The fourth agent processes the analysis results, the credibility, the reasoning results, and the multimodal fusion features to output the predicted value of the third verification layer.

9. The method according to claim 8, characterized in that, The step of generating detection text based on the target predicted value and the verification information of the target verification layer includes: When the target verification layer is the first verification layer, the detected text includes the hazard score, the complexity feature, and the predicted value of the first verification layer; When the target verification layer is a second verification layer, the detection text includes the hazard score, the complexity feature, the target external evidence, and the predicted value of the second verification layer; Alternatively, if the target verification layer is a third verification layer, the detection text includes the hazard score, the complexity feature, the target external evidence, the analysis result output by the first agent, the credibility output by the second agent, the reasoning result output by the third agent, and the prediction value of the third verification layer output by the fourth agent.

10. An information detection device based on adaptive routing, characterized in that, include: The first processing module is used to input the multimodal declaration to be detected into the pre-trained multimodal screening model for processing and output multimodal fusion features; The multimodal claim to be detected includes text content and visual evidence. The trained multimodal screening model is used to measure the consistency relationship between the text content and the visual evidence and to generate the multimodal fusion feature based on the consistency relationship between the text content and the visual evidence. The second processing module is used to input the multimodal fusion features into the first verification layer of the trained adaptive routing model for processing to output the initial prediction value; The trained adaptive routing model includes a first verification layer, a second verification layer, and a third verification layer. The first verification layer is configured to lightweight filter the multimodal statements to be detected. The second verification layer is configured to enhance the verification of the multimodal statements to be detected based on external evidence. The third verification layer is configured to use multiple agents to reason about the multimodal statements to be detected. The first determining module is used to determine the selection probability of the first verification layer, the second verification layer and the third verification layer respectively based on the initial predicted value and the multimodal fusion feature; The second determining module is used to determine a target verification layer from the first verification layer, the second verification layer, and the third verification layer based on the selection probability and hazard score corresponding to each of the first verification layer, the second verification layer, and the third verification layer; the hazard score characterizes the social risk level after the multimodal claim to be detected is propagated; The first generation module is used to input the multimodal fusion features into the target verification layer for processing to output the target prediction value; The second generation module is used to generate detection text based on the target prediction value and the verification information of the target verification layer.

Images