Multi-modal fidelity hallucination detection method and system based on closed-loop multi-dimensional verification

Abstract

The application provides a multimodal fidelity hallucination detection method and system based on closed-loop multi-dimensional verification, belonging to the technical field of large visual language models, comprising: S1: decomposing the user instruction input into the LLM and the answer of the LLM into structured units; S2: through the collaborative mode of the LLM and the visual tool, the consistency score of the assertion is calculated from three dimensions of text-instruction consistency, text-image consistency and logical consistency, all verification results are bound to a unique evidence ID, and the assertion without evidence support is determined as unknown; S3: performing type-specific calibration on the consistency score, and then generating a unified hallucination score H through weighted fusion; S4: providing traceable evidence support for the detection result, and triggering the abstention mechanism when the evidence is insufficient, so as to ensure the credibility and practical value of the detection result. The method realizes accurate detection of three types of hallucinations, i.e., instruction inconsistency, text-image separation and logical inconsistency.

Description

Technical Field

[0001] This invention belongs to the field of large-scale visual language model technology, and specifically relates to a multimodal fidelity illusion detection method and system based on closed-loop multidimensional verification. Background Technology

[0002] Multimodal large language models, by fusing visual encoders with large language models, have demonstrated outstanding performance in tasks such as visual question answering, visual reasoning, and image understanding. However, the "fidelity illusion" severely limits their practicality and reliability—the model's output appears reasonable, but in reality, it makes "factual" assertions that cannot be supported by user instructions or provided images. This illusion is not a simple information error, but rather a "fabrication" or "over-speculation" by the model in the absence of sufficient evidence. This poses a serious safety hazard when deployed in high-risk fields such as medical diagnosis, financial analysis, legal evidence review, and autonomous driving.

[0003] To address the problem of hallucinations, academia and industry have proposed various detection methods, which can be mainly divided into four technical approaches. The core ideas and implementation methods of each approach are as follows:

[0004] (1) Text-based hallucination detection method

[0005] This type of method determines hallucinations solely through textual semantic analysis, without introducing any visual evidence. The core idea is to utilize Natural Language Inference (NLI) and semantic similarity calculation techniques to verify the logical consistency between the answer and the query. In practice, one approach is to use a pre-trained NLI model to determine whether the answer falls within the semantic scope defined by the user's instructions; another approach is to use a text encoding model to convert the answer and user instructions into vector representations, calculate their cosine similarity, and determine the presence of a hallucination if the similarity is below a preset threshold. The typical characteristics of this type of method are its simple implementation process, lack of reliance on visual tools, and rapid deployment.

[0006] (2) Visual-based hallucination detection methods

[0007] This type of method uses visual tools as the core processing unit, focusing on the extraction and verification of coarse-grained visual features, primarily targeting the detection of basic illusion types such as object illusion. Its technical logic involves using low-level visual tools such as object detection, instance segmentation, and OCR (Optical Character Recognition) to extract coarse-grained features from images, including the existence, quantity, location bounding boxes, and surface text of objects, thereby verifying the truthfulness of core visual-related assertions in the response. For example, open-domain object detection tools like GroundingDINO are used to locate objects in an image to verify whether "the object mentioned in the response exists in the image"; OCR tools are used to recognize text information in an image to verify whether "the numerical values, names, and other textual content in the response appear in the image." The core advantage of this method is its ability to directly connect with visual evidence in the image, establishing a verification channel linking the response to visual information.

[0008] (3) Fine-grained hallucination detection method based on manual annotation

[0009] To overcome the limitations of visual-based hallucination detection methods in terms of granularity, this type of method constructs a fine-grained supervised dataset through manual annotation, and then trains a dedicated hallucination detection model. The annotation process requires defining fine-grained visual feature dimensions for specific scenarios, including object attributes, entity relationships, and action states. For example, in a medical scenario, attributes such as "color, material, and sealing status of medicine bottles" are manually labeled. During model training, the labeled features serve as supervisory signals to train a classifier or regression model. Hallucination detection is achieved by comparing the matching degree between attribute assertions in the responses and the labeled results. The core characteristic of this type of method is its focus on fine-grained visual feature verification, enabling accurate detection of specific attribute descriptions.

[0010] (4) Hallucination detection methods without calibration and interpretability

[0011] Most existing detection methods employ a single output mode, generating only a binary classification result of "hallucination / no hallucination," or outputting a "hallucination probability value" without probability calibration. For example, some methods directly output a binary label, simply stating "hallucination present" or "no hallucination"; others output a quantitative score such as "hallucination probability 0.8," but without any explanation of the score's reliability or the basis for the judgment. The focus of these methods is on rapidly outputting detection conclusions, without designing for verification of the result's credibility or traceability of the reasoning process.

[0012] Existing hallucination detection methods have the following drawbacks:

[0013] (1) Incomplete detection dimensions: Text-based hallucination detection methods suffer from a fundamental lack of modal information, relying solely on semantic associations within the text to determine hallucinations, completely ignoring the core basis of visual evidence from the image. In the Visual Question Answering (VQA) task, hallucinations where "the text is logically coherent but contradicts the image" account for over 40%. For example, when the user's instruction is "the color of the medicine bottle in the image," the answer "blue" may be semantically consistent with the user's instruction, but the medicine bottle in the image is actually red. Such hallucinations are completely outside the detection blind spot of this type of method, resulting in extremely low overall detection coverage. The root cause of this problem lies in the fact that the method design does not consider the core attribute of "text-image cross-modal fusion" in the VQA task, relying only on single-modal information as the basis for judgment, and thus failing to effectively identify cross-modal contradictory hallucinations.

[0014] (2) Insufficient Granularity Adaptability: Although vision-based hallucination detection methods introduce visual evidence, they suffer from incomplete feature granularity coverage due to limitations in the capabilities of underlying visual tools. Existing tools such as object detection and OCR can only extract coarse-grained features such as object existence, quantity, and surface text, and cannot support the verification of fine-grained attributes, such as the object attribute "the medicine bottle is made of plastic", the entity relationship "the traffic light is directly above the vehicle", and the action state "the child is about to cry". In key scenarios such as medical and financial fields, fine-grained hallucinations account for more than 50%, which makes it impossible for such methods to achieve comprehensive hallucination coverage detection, resulting in application limitations.

[0015] (3) Difficulty in controlling annotation costs and quality: Fine-grained illusion detection methods based on manual annotation face the dual challenges of high annotation costs and unstable annotation quality, which seriously restrict their large-scale application and detection reliability. From a cost perspective, each image needs to define and annotate dozens of fine-grained visual features, and the annotation cycle for large-scale datasets can last for several months. Moreover, manual annotation requires professional background support, resulting in extremely high time and economic costs. From a quality perspective, the annotation results are significantly affected by the annotator's subjective cognition, professional level, and experience differences, which can easily lead to annotation deviations. For example, in judging "sealed state," different annotators may have disagreements due to different interpretations of the standard, and the annotation of ambiguous features such as "component content clarity" is even more prone to inconsistencies. This strong reliance on manual annotation makes it difficult for the method to achieve large-scale expansion and to guarantee the uniformity and accuracy of the detection benchmark.

[0016] (4) Lack of score calibration and insufficient interpretability: Uncalibrated and uninterpretable hallucination detection methods suffer from inaccurate result quantification and lack of evidence traceability, severely reducing their practical value in real-world scenarios. On the one hand, the detection results lack effective uncertainty quantification, and the output "hallucination probability value" is not calibrated, failing to reflect the true credibility of the conclusion. For example, in samples where the model judges "hallucination probability as 0.8," the actual proportion of those without hallucinations may be as high as 40%, making it difficult for users to make trust decisions based on this result. On the other hand, the detection process lacks interpretability design, neither clearly labeling the specific type of hallucination nor providing evidence to support "why it was judged as a hallucination," failing to provide effective guidance for subsequent error correction, model optimization, or manual review. In reliability-sensitive scenarios such as medical diagnosis and financial analysis, such uncalibrated and uninterpretable detection results are difficult to meet the trust requirements of decision-making, greatly weakening their practical value.

[0017] These shortcomings severely restrict the deployment and application of multimodal large language models in reliability-sensitive scenarios. First, there is a modal fragmentation problem in the detection dimensions. Existing text-based hallucination detection methods rely solely on the semantic association between user commands and responses, completely ignoring the core basis of visual question-answering tasks—image visual evidence. This results in hallucinations where "textual logic is consistent but contradicts the image" remaining in the detection blind spot, failing to meet the needs of cross-modal collaborative verification. Second, there are significant limitations in feature granularity coverage. Although vision-based detection methods incorporate visual tools, they are limited by the capabilities of tools such as object detection and OCR, and can only handle coarse-grained features such as object existence and quantity. They cannot verify fine-grained assertions such as object attributes, entity relationships, and action states. In scenarios such as healthcare and finance, fine-grained hallucinations account for more than 50%, leading to insufficient detection completeness. Furthermore, existing fine-grained detection methods rely on manual annotation, requiring the definition and annotation of dozens of fine-grained visual features for each image. Annotation of large-scale datasets can take months, and the results are significantly affected by the annotator's subjective perception and skill level, leading to annotation bias. This makes it difficult to scale up and ensure the uniformity of the detection benchmark. Finally, the reliability and interpretability of the detection results are insufficient. Most methods output uncalibrated "hallucination probability values" or only provide binary classification judgments, failing to quantify the uncertainty of the results and providing no attribution or evidence supporting the hallucination type. This fails to provide effective guidance for subsequent error correction, model optimization, or manual review. Summary of the Invention

[0018] To address the aforementioned technical problems, this invention provides a multimodal fidelity illusion detection method based on closed-loop multidimensional verification, comprising the following steps:

[0019] Step S1: Decompose the user instructions input into the LLM and the LLM's responses into structured units;

[0020] Step S2: Using a collaborative model of LLM and vision tools, calculate the assertion consistency score from three dimensions: text-instruction consistency, text-image consistency, and logical consistency. , , All verification results are bound to a unique evidence ID, and assertions without supporting evidence are judged as unknown.

[0021] Step S3: Calculate the consistency score , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion;

[0022] Step S4: Provide traceable evidence to support the test results and trigger the abstention mechanism when the evidence is insufficient to ensure the credibility and practical value of the test results.

[0023] Beneficial effects:

[0024] 1. This invention provides a multimodal fidelity illusion detection method based on closed-loop multidimensional verification, which solves the modal fragmentation problem. By constructing a multidimensional collaborative verification mechanism of "text-visual-logic", it covers three types of illusions: instruction inconsistency, text-image separation, and logical inconsistency, which is different from traditional single-modal detection.

[0025] 2. This invention breaks through the limitations of coarse-grained illusion detection and establishes a fine-grained feature automatic parsing and verification system. Relying on LLM semantic understanding capabilities and the collaboration of multiple vision tools, it can verify fine-grained assertions such as object attributes, entity relationships, and action states without the need for manual pre-definition of features, thereby improving the completeness and accuracy of detection.

[0026] 3. This invention reduces reliance on manual annotation and designs a detection paradigm without manual annotation. Through LLM-driven structured decomposition of instructions and responses and automatic extraction of visual evidence, it replaces manual annotation to build detection benchmarks, reduces scaling costs, and ensures the uniformity of detection benchmarks.

[0027] 4. This invention can enhance the credibility and interpretability of the results, construct a calibration and interpretability output mechanism, correct score deviations through temperature scaling, and output hallucination type attribution, contradictory assertions, and visual and textual evidence to support decision-making in high-risk scenarios. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the process of a multimodal fidelity illusion detection method based on closed-loop multidimensional verification according to the present invention;

[0029] Figure 2 This is a structural block diagram of a multimodal fidelity illusion detection system based on closed-loop multidimensional verification according to the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0031] Example 1

[0032] like Figure 1 As shown in the figure, the multimodal fidelity illusion detection method based on closed-loop multidimensional verification provided by the present invention includes the following steps:

[0033] Step S1: Decompose the user instructions input into the LLM and the LLM's responses into structured units;

[0034] Step S2: Using a collaborative model of LLM and vision tools, calculate the assertion consistency score from three dimensions: text-instruction consistency, text-image consistency, and logical consistency. , , All verification results are bound to a unique evidence ID, and assertions without supporting evidence are judged as unknown.

[0035] Step S3: Calculate the consistency score , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion;

[0036] Step S4: Provide traceable evidence to support the test results and trigger the abstention mechanism when the evidence is insufficient to ensure the credibility and practical value of the test results.

[0037] In one embodiment, step S1 above, decomposing the user instructions input into the LLM and the LLM's responses into structured units, specifically includes:

[0038] Step S11: Decompose the user instructions and construct an instruction graph. It includes: task slots, constraints, and a set of subproblems. The task slots define the task type, mandatory items, and reasoning requirement identifiers; the constraints define the format requirements, scope, unit specifications, and language type; and the set of subproblems is the set of minimum subproblems that are broken down from user instructions.

[0039] An LLM (such as GPT-5) receives the user command Q and outputs a command graph in JSON format. , It includes three core fields: task slots, constraints, and a set of subproblems.

[0040] Task slot definition:

[0041] Task type (task_type), such as "information summary" or "image description";

[0042] Required answers, such as the "color" of a man's clothes or the "dosage" of medicine;

[0043] Reasoning requirement flag (reasoning_require), indicating whether reasoning is required, such as "What is the weather like in the picture";

[0044] The constraints include:

[0045] Formatting requirements, such as "output value + unit";

[0046] Range, such as "weight < 500mg";

[0047] Unit specifications, such as "kilogram";

[0048] Language type (lang), such as "English";

[0049] The set of subproblems breaks down Q into minimizing subproblems. The sub-problem types should be labeled, including: object existence, attribute judgment, spatial relationship, and common sense knowledge (object / attribute / relation / knowledge). For example, the instruction "summarize the information of the medicines in the picture" needs to be broken down into... Does the medicine exist? Medicine color, The relative positions of medicines and tables are considered to ensure that each sub-question corresponds to a single visual or textual verification target.

[0050] Step S12: According to the instruction diagram Extracting atomic assertions, association mappings, off-topic markers, and inference chains from LLM responses:

[0051] In the assertion extraction step of the LLM response, based on the instruction graph LLM extracts atomic assertions from model response A. Each assertion contains:

[0052] Assert text, such as "The medicine bottle is blue" or "The stock index closed at 3260.34";

[0053] Association mapping (maps_to) — marks the subproblems corresponding to this assertion. For example, answering with the color of the medicine This ensures a strong correlation with the requirements of the instructions;

[0054] Off-topic flag (off_topic) — Determines whether the assertion deviates from the instruction range; 0 = not off-topic, 1 = off-topic;

[0055] The reasoning chain (π) involves extracting intermediate reasoning steps if the answer contains multiple steps, such as "motorcycle falls over → two vehicles collide → driver leaves safely". This provides a basis for subsequent logical verification.

[0056] For example, the user instruction is "Summarize the medicine bottle label (brand, type, dosage)". The `must_answer` option is set to "brand, type, dosage", and the sub-question is... (attribute: brand) (attribute: type) (attribute: dosage); The model answers "Vitamin C complex (ProHealth, 60 tablets)", extracting the assertion. (Text: The brand is ProHealth) ), (text: type is vitamin C complex,) ), (Text: Dosage is 60 tablets) ).

[0057] This invention categorizes "fidelity illusion" into three verifiable types, which are verified through the following step S2:

[0058] (1) Inconsistent instructions: This refers to a response that deviates from the explicit or implicit constraints in the user's instructions, specifically manifested as deviations in task type, format, numerical range, unit, or missing mandatory items. A typical case exists in the medical scenario: The user's instructions explicitly require summarizing the brand, type, and dosage on the drug label. The response generated by the multimodal large language model states, "This product is a vitamin C complex, branded ProHealth, containing 60 tablets." However, the drug label in the image actually shows the product type as probiotics and contains 100 tablets. The response deviates from the product type description and incorrectly alters the dosage information, which is an illusion of inconsistent instructions.

[0059] (2) Text-image separation: This refers to a mismatch between the assertions made in the response regarding visual elements of the image and the actual visual evidence. It encompasses contradictions in the existence of objects, attributes, quantities, spatial relationships, and textual information. A typical example exists in the financial context: the index chart in the image shows that the index fell to 3168.52 on January 15, but the response generated by the multimodal large language model states that "the index rose sharply, closing at 3260.34." The assertions in the response regarding the trend and specific value of the index are completely contrary to the visual evidence in the image, which is a text-image separation illusion.

[0060] (3) Logical inconsistency: This refers to contradictions between the reasoning steps in the answer, or incompatibility between the reasoning steps and the final conclusion. Specifically, it includes contradictions between steps, contradictions between steps and conclusions, and contradictions between logical operators. A typical case exists in public safety scenarios: The answer generated by the multimodal large language model states that "the motorcycle collided with the silver car, but neither vehicle was damaged and the driver left safely." However, the image actually shows that the motorcycle was lying on the ground with obvious damage to its body, and the motorcycle was located next to the front wheel of the car. The description of "collision" in the answer conflicts with the conclusion that "neither vehicle was damaged," and also contradicts the visual evidence in the image. This is a logical inconsistency illusion.

[0061] In one embodiment, step S2 above involves calculating the assertion consistency score from three dimensions—text-instruction consistency, text-image consistency, and logical consistency—through a collaborative model of LLM and vision tools. , , All verification results are bound to a unique evidence ID. Assertions without supporting evidence are judged as unknown, specifically including:

[0062] Step S21: Text-Instruction Consistency Verification Based on Instruction Graph Under the constraints, LLM calculates the instruction consistency score using a preset template. The formula is as follows:

[0063] ;

[0064] in, For format compliance, For task type matching degree, The coverage of required questions is represented by "Optopic" and the degree of irrelevance to the topic. These are the weighting coefficients.

[0065] The judgment is based on the format requirement field in the constraint conditions of the instruction diagram. A lightweight text classifier, fastText, is used to evaluate whether the answer conforms to the preset format. A complete conformance is assigned a value of 1, a non-conforming value is assigned a value of 0, and a partial conformance is assigned a value according to the matching ratio and normalized to [0,1]. Semantic matching is performed between the task type (task_type) in the instruction graph and the answer content. Cosine similarity is calculated using the pre-trained sentence encoder Sentence-BERT and normalized to [0,1]. Based on the list of mandatory answers in the instruction diagram, the percentage of mandatory answers covered in the response is used as the coverage score. For example, if the mandatory answer list has three items and the response covers two, then... . This analysis is based on the task slots and constraints in the instruction graph, summarizing all assertions for off-topic determination. First, the off-topic severity of each individual assertion is calculated by associating it with sub-problems in the instruction graph. If an assertion cannot be mapped to any sub-problem, it is directly determined as off-topic and recorded as 1. If it can be mapped, the semantic relevance score between the assertion and its corresponding sub-problem is further calculated. A pre-trained sentence encoder is used to vectorize both the assertion and the sub-problem, and cosine similarity is calculated. If the similarity is below a set threshold (0.5), it is determined as off-topic and recorded as 1; otherwise, it is determined as not off-topic and recorded as 0. Subsequently, the normalized overall off-topic severity score is obtained by calculating the proportion of off-topic assertions to the total number of assertions. For example, for the instruction "Describe the color of the medicine bottle in the image," if the assertion extracted by the model is "The production date of the medicine bottle is 2023," this assertion cannot be mapped to the "color" related sub-problem, so it is determined as off-topic and recorded as 1. If 10 assertions are extracted from the answers, and 2 of them are judged to be off-topic, then the overall degree of off-topicness is... .

[0066] , , , The values ​​are all standardized scores ranging from 0 to 1; in the embodiments of this invention The default value is 0.25. For example, when the answer does not include the required "dosage" information, the Coverage score decreases, thus lowering the overall score. .

[0067] Step S22: Text-image consistency verification uses multiple visual tools to extract image evidence, performing coarse-grained and fine-grained verification at two levels, and outputting an image consistency score. Specifically, it includes:

[0068] Step S221: The formula for calculating the coarse grain size fraction is as follows:

[0069]

[0070] Where e represents the assertion entity; d represents the detection result, including the category cls, bounding box bbox, and detection confidence. D is the set of detection results; For category matching degree, For location matching degree; , and Let be the weighting coefficient, satisfying ;

[0071] Among them, detection confidence Derived from GroundingDINO, this is the raw confidence score for the existence of objects in an image; category matching score. It is obtained by calculating the semantic similarity between the asserted entity and the predicted category label of the detection result; position matching degree. The intersection-union ratio (IUU) of the detection bounding box (bbox) and the positional constraint region described in the assertion is obtained by calculating the IUU between the bounding box and the positional constraint region. The positional constraint region refers to the image coordinate range transformed according to the orientation description in the assertion, such as "left side" or "center". , , All scores are standardized from 0 to 1; in this embodiment of the invention, α=0.4, β=0.3, and γ=0.3 are set.

[0072] The coarse-grained score is mainly used for object illusion calculation. It performs open-domain object detection through GroundingDINO and outputs detection results containing object category, bounding box, and detection confidence. It also verifies the matching score of assertions such as object existence, quantity, and category.

[0073] Step S222: Formula for calculating fine particle size fraction as follows:

[0074] ;

[0075] Where 'a' represents a fine-grained assertion; in the verification of binary spatial relations... For the detection results of entity 1, The detection result for entity 2; For attribute matching degree; The relationship score is calculated using TextMatch(t,a); TextMatch(t,a) represents the text matching score, where T is the OCR text set. The degree of matching between visual features and abstract state / action assertions;

[0076] Coarse-grained scores are primarily used to verify object attributes, such as color and material. They extract object color features using the HSV color space model and calculate attribute matching scores using a lightweight attribute detection head. For spatial relationships, a relationship score is calculated based on the orientation, distance, and intersection-over-union (IoU) ratio of the detection boxes. For the text set, text recognition is performed on the images using an OCR tool, and the matching degree TextMatch(t,a) between the assertion and the text is calculated. For the action state, the compatibility between the assertion and the image features is verified using a Visual Natural Language Inference (VisionNLI) model.

[0077] Step S223: To integrate the multi-dimensional verification results and avoid interference from single-dimensional bias on the overall authenticity assessment of the entity, calculate the mean of the fine-grained score of the assertion entity e. :

[0078] ;

[0079] in, correspond The score in the i-th fine-grained dimension, where n is the actual number of dimensions involved;

[0080] Calculate the consistency score of a single entity :

[0081] ;

[0082] Wherein, λ is the weighting coefficient, which is set to 0.5 by default in this embodiment of the invention;

[0083] Step S224: Aggregate all entity scores and introduce a quantity inconsistency penalty term. ,get :

[0084] ;

[0085] Where Agg is the aggregation function, specifically defined as the arithmetic mean function, i.e.:

[0086]

[0087] here, For the set of entities involved in the assertion, Indicates the number of entities in the set; This is a penalty term for entity number discrepancies, used to penalize entities mentioned in the assertion but lacking supporting visual evidence. Its value is defined as the ratio of the number of unmatched entities to the total number of entities. The ratio, i.e. =(Number of unmatched entities) / Specifically, the Hungarian matching algorithm is first used to identify the set of entities involved in the assertion. The optimal match is performed with the entity set in the image detection results. Then, the number of entities that failed to match is counted, and a penalty term is calculated according to the definition. This ratio is naturally normalized to [0,1].

[0088] Step S23: Logical consistency verification targets the reasoning chain. LLM combines structured visual evidence (such as the number of objects, spatial relationships, and OCR text) to verify logical consistency through natural language reasoning, outputting a logical score. :

[0089] ;

[0090] in, These are the reasoning steps, with A being the final conclusion. , , These are the weighting coefficients;

[0091] The mean entailment of adjacent inference steps is calculated using a pre-trained Natural Language Inference (NLI) model, RoBERTa-large-mnli, for each pair of adjacent steps in the inference chain. Calculation premise Implied conclusions The probability score is then calculated, and the arithmetic mean of the scores for all adjacent pairs is taken to obtain the normalized mean of the inter-step implication; NLI( To assess the support of the reasoning steps for the conclusion, the same model is used, concatenating all reasoning steps as premises and taking the final assertion A as the conclusion. The probability score of the premise implying the conclusion is calculated as the logical support of the reasoning chain for the conclusion, with a value between [0,1]. OpCheck detects logical operator contradictions by using rule matching and lightweight symbolic reasoning to detect whether there are explicit, absolute logical contradictions in the answers, such as "both greater than 5 and less than 3". Rule matching scans and identifies text based on a predefined pattern library of logical contradictions; lightweight symbolic reasoning converts natural language assertions into formal logical expressions and calls the reasoning engine for consistency verification and contradiction derivation. The specific calculation formula is OpCheck = 1 - (number of detected contradictory assertions) / (total number of assertions related to logical judgments). This score is normalized to between [0,1]. When no contradiction is detected, OpCheck = 1; the more contradictions, the lower the score.

[0092] MeanNLI, NLI, and OpCheck are all standardized scores of 0-1. , , The default values ​​are 0.4, 0.4, and 0.2.

[0093] In one embodiment, step S3 above: [The sentence is incomplete and requires more context to translate accurately.] , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion, specifically including:

[0094] Step S31: Use the temperature scaling method to calibrate the consistency scores of each dimension to eliminate bias caused by differences in the distribution of scores across different dimensions.

[0095] , ;

[0096] in, For the Sigmoid function; The corresponding dimension calibration temperature parameters are obtained through cross-validation on the validation set. For the original fractions, The scores are the calibrated scores, and their values ​​range from [0,1].

[0097] Step S32: Calculate the final hallucination score H based on the calibrated scores through weighted summation:

[0098] ;

[0099] in, These are the weighting coefficients. The value of H ranges from [0,1]. The closer H is to 1, the higher the degree of hallucination in the answer.

[0100] By comparing calibration scores , , The minimum value can determine the dominant hallucination type:

[0101] like The smallest error is attributed to inconsistent instructions;

[0102] like The smallest difference is attributed to text-image separation;

[0103] like The smallest error is attributed to logical inconsistency.

[0104] In one embodiment, step S4 above—providing traceable evidence to support the test results and triggering an abstention mechanism when evidence is insufficient—ensures the credibility and practical value of the test results, specifically includes:

[0105] Step S41: Output the set of evidence linked to the detection results, including: visual evidence, textual evidence, and score details;

[0106] Step S42: When the following three situations occur, it is judged as uncertain, and no clear hallucination / non-hallucination label is output. Only a prompt is made that manual review is required:

[0107] a. Assert that there is no corresponding visual or textual evidence;

[0108] b. The confidence level of key evidence is below the threshold; for example, OCR recognition confidence level < 0.5 or object detection score < 0.3.

[0109] c. If the score after calibration falls within the middle range, it is impossible to definitively determine whether hallucinations exist. For example: At this point, it is impossible to definitively determine whether it is a hallucination or not. , Determined through ROC curve optimization.

[0110] This step provides traceable evidence to support the detection results and triggers an abstention mechanism when evidence is insufficient, ensuring the credibility and practical value of the detection results. In the multi-type evidence output stage, a set of evidence linked to the detection results is output, including visual evidence such as target detection bounding boxes, instance segmentation masks, OCR text fragments and location coordinates, and spatial relationship calculation metrics such as distance and direction; textual evidence, i.e., the natural language interpretation generated by LLM; and score details, outputting the original scores, calibration scores, and weights for each dimension, explaining the calculation process of the illusion score H.

[0111] This invention addresses the core requirement of multimodal large language model fidelity illusion detection in VQA tasks by proposing a multimodal fidelity illusion detection method based on closed-loop multidimensional verification. Through LLM-driven structured parsing, multi-tool collaborative evidence verification, type-adaptive score calibration fusion, and traceable evidence output, it achieves accurate detection of three types of illusions: instruction inconsistency, text-image separation, and logical inconsistency.

[0112] This invention's method comprises four steps: decomposition, verification, fusion, and interpretation, sequentially linked to form a closed-loop detection process. Taking "image I, user command Q, and model response A" as input, it undergoes decomposition to obtain structured parsing, performs multi-dimensional consistency verification, fuses to generate a unified hallucination score and type attribution, and finally interprets the output hallucination label, attribution results, and traceable evidence. Each step interacts with structured data such as JSON-formatted command graphs and feature score matrices to ensure accurate and efficient information transmission. The entire process requires no manual intervention and can be directly adapted to various VQA scenarios, including medical, financial, and public safety fields.

[0113] Example 2

[0114] like Figure 2 As shown, this embodiment of the invention provides a multimodal fidelity illusion detection system based on closed-loop multidimensional verification, comprising the following modules:

[0115] Decomposition module 51 is used to decompose the user instructions input to the LLM and the LLM's response into structured units;

[0116] Verification module 52 is used to calculate the consistency score of assertions from three dimensions: text-instruction consistency, text-image consistency, and logical consistency, through a collaborative model of LLM and vision tools. , , All verification results are bound to a unique evidence ID, and assertions without supporting evidence are judged as unknown.

[0117] Fusion module 53 is used for consistency scores , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion;

[0118] The interpretation module 54 is used to provide traceable evidence to support the test results and to trigger an abstention mechanism when the evidence is insufficient, so as to ensure the credibility and practical value of the test results.

[0119] In terms of technical complexity, the time complexity of this invention is linearly related to the number of atomic assertions C, O(|C|). The LLM parsing of the decomposition module and the calibration and weighted fusion of the fusion module are both constant time. Although the verification module increases linearly with C, each assertion only calls a single adaptation tool, with no redundant computation. Compared with traditional detection methods with quadratic time complexity, it is more suitable for real-time detection of large-scale VQA datasets. In terms of scenario adaptability, through modular design and parameter adjustment, it can adapt to different VQA scenarios: medical scenarios enhance the verification of mandatory item coverage and fine-grained attribute detection; financial scenarios optimize OCR text matching accuracy and graph spatial relationship verification; public safety scenarios enhance inference chain logic verification and action state detection. Each scenario does not require framework reconstruction, only adjustment of instruction graph field definitions and tool parameters, demonstrating strong scalability. Furthermore, the technology employs a three-dimensional verification process—text-visual-logic—to cover cross-modal contradictions, fine-grained attributes, and logical reasoning-related hallucinations missed by existing methods, improving detection coverage by over 40%. It eliminates the need for manual annotation of fine-grained visual features, automatically extracting evidence through LLM decomposition and visual tools, reducing annotation costs by 90% and avoiding subjective annotation bias. After type-specific calibration, the actual hallucination percentage of samples with a "hallucination probability of 0.8" deviates from the predicted value by ≤5%, resolving the inaccuracy issues of traditional methods. The output visual evidence and natural language explanations can directly support manual review and model optimization decisions in reliability-sensitive scenarios such as healthcare and finance, demonstrating significant practical value.

[0120] A multimodal fidelity illusion detection device based on closed-loop multidimensional verification includes one or more electronic devices, wherein the one or more electronic devices are used to implement the multimodal fidelity illusion detection method based on closed-loop multidimensional verification.

[0121] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a multimodal fidelity illusion detection method based on closed-loop multidimensional verification.

[0122] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement a multimodal fidelity illusion detection method based on closed-loop multidimensional verification.

[0123] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a multimodal fidelity illusion detection method based on closed-loop multidimensional verification.

[0124] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A multimodal fidelity illusion detection method based on closed-loop multidimensional verification, characterized in that, include: Step S1: Decompose the user instructions input into the LLM and the LLM's responses into structured units; Step S2: Using a collaborative model of LLM and vision tools, calculate the assertion consistency score from three dimensions: text-instruction consistency, text-image consistency, and logical consistency. , , All verification results are bound to a unique evidence ID. Assertions without supporting evidence are judged as unknown, specifically including: Step S21: Text-Instruction Consistency Verification Based on Instruction Graph Under the constraints, LLM calculates the instruction consistency score using a preset template. The formula is as follows: = OffTopic; in, For format compliance, For task type matching degree, The coverage of required questions is represented by "Optopic" and the degree of irrelevance to the topic. , These are the weighting coefficients; Step S22: Text-image consistency verification uses multiple visual tools to extract image evidence, performing coarse-grained and fine-grained verification at two levels, and outputting an image consistency score. Specifically, it includes: Step S221: The formula for calculating the coarse grain size fraction is as follows: ； Where e represents the assertion entity; d represents the detection result, including the category cls, bounding box bbox, and detection confidence C. D is the set of detection results; For category matching degree, For location matching degree; , and Let be the weighting coefficient, satisfying ; Step S222: Formula for calculating fine particle size fraction as follows: ； Where 'a' represents a fine-grained assertion; in the verification of binary spatial relations... For the detection results of entity 1, The detection result for entity 2; For attribute matching degree; The relationship score is calculated using TextMatch(t,a); TextMatch(t,a) represents the text matching score, where T is the OCR text set. The degree of matching between visual features and abstract state / action assertions; Step S223: To integrate the multi-dimensional verification results and avoid interference from single-dimensional bias on the overall authenticity assessment of the entity, calculate the mean of the fine-grained score of the assertion entity e. : ； in, correspond The score in the i-th fine-grained dimension, where n is the actual number of dimensions involved; Calculate the consistency score of a single entity : ； Where λ is the weighting coefficient; Step S224: Aggregate all entity scores and introduce a quantity inconsistency penalty term. ,get : ； Where Agg is an aggregate function. For the set of entities involved in the assertion; Calculate the entity number conflict penalty term using the Hungarian matching algorithm; Step S23: Logical consistency verification targets the reasoning chain. LLM, combined with structured visual evidence, verifies logical consistency through natural language reasoning and outputs a logical score. : MeanNLI( NLI( OpCheck; in, It is a reasoning step; MeanNLI ( The mean of the implication values ​​of adjacent reasoning steps; NLI ( OpCheck represents the degree to which the reasoning steps support the conclusion; it is used to detect contradictions in logical operators. , , These are the weighting coefficients; Step S3: Calculate the consistency score , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion, specifically including: Step S31: Use the temperature scaling method to calibrate the consistency scores of each dimension to eliminate bias caused by differences in the distribution of scores across different dimensions. , ； in, For the Sigmoid function; The corresponding dimension calibration temperature parameters are obtained through cross-validation on the validation set. For the original fractions, The scores are the calibrated scores, and their values ​​range from [0,1]. Step S32: Calculate the final hallucination score H based on the calibrated scores through weighted summation: ； in, These are the weighting coefficients. The value of H ranges from [0,1]. The closer H is to 1, the higher the degree of hallucination in the answer. By comparing calibration scores , , The minimum value can determine the dominant hallucination type: like The smallest error is attributed to inconsistent instructions; like The smallest difference is attributed to text-image separation; like The smallest inconsistency is attributed to logical inconsistency; Step S4: Provide traceable evidence to support the test results and trigger the abstention mechanism when the evidence is insufficient to ensure the credibility and practical value of the test results.

2. The multimodal fidelity illusion detection method based on closed-loop multidimensional verification according to claim 1, characterized in that, Step S1: Decompose the user instructions input into the LLM and the LLM's response into structured units, specifically including: Step S11: Decompose the user instructions and construct an instruction graph. It includes: task slots, constraints, and a set of subproblems. The task slots define the task type, mandatory items, and reasoning requirement identifiers; the constraints define the format requirements, scope, unit specifications, and language type; and the set of subproblems is the set of minimum subproblems that are broken down from user instructions. Step S12: According to the instruction diagram Extract atomic assertions, association mappings, off-topic markers, and inference chains from LLM responses.

3. The multimodal fidelity illusion detection method based on closed-loop multidimensional verification according to claim 2, characterized in that, Step S4: Provides traceable evidence to support the test results and triggers an abstention mechanism when evidence is insufficient, ensuring the credibility and practical value of the test results. Specifically, this includes: Step S41: Output the set of evidence linked to the detection results, including: visual evidence, textual evidence, and score details; Step S42: When the following three situations occur, it is judged as uncertain, and no clear hallucination / non-hallucination label is output. Only a prompt is made that manual review is required: a. Assert that there is no corresponding visual or textual evidence; b. The confidence level of key evidence is below the threshold; c. The score after calibration is in the middle range, making it impossible to definitively determine whether hallucinations exist.

4. A multimodal fidelity illusion detection system based on closed-loop multidimensional verification, characterized in that, Includes the following modules: The decomposition module is used to decompose the user instructions input to the LLM and the LLM's responses into structured units; The verification module is used to calculate the consistency score of assertions from three dimensions: text-instruction consistency, text-image consistency, and logical consistency, through a collaborative model of LLM and vision tools. , , All verification results are bound to a unique evidence ID. Assertions without supporting evidence are judged as unknown, specifically including: Step S21: Text-Instruction Consistency Verification Based on Instruction Graph Under the constraints, LLM calculates the instruction consistency score using a preset template. The formula is as follows: = OffTopic; in, For format compliance, For task type matching degree, The coverage of required questions is represented by "Optopic" and the degree of irrelevance to the topic. , These are the weighting coefficients; Step S22: Text-image consistency verification uses multiple visual tools to extract image evidence, performing coarse-grained and fine-grained verification at two levels, and outputting an image consistency score. Specifically, it includes: Step S221: The formula for calculating the coarse grain size fraction is as follows: ； Where e represents the assertion entity; d represents the detection result, including the category cls, bounding box bbox, and detection confidence C. D is the set of detection results; For category matching degree, For location matching degree; , and Let be the weighting coefficient, satisfying ; Step S222: Formula for calculating fine particle size fraction as follows: ； Where 'a' represents a fine-grained assertion; in the verification of binary spatial relations... For the detection results of entity 1, The detection result for entity 2; For attribute matching degree; The relationship score is calculated using TextMatch(t,a); TextMatch(t,a) represents the text matching score, where T is the OCR text set. The degree of matching between visual features and abstract state / action assertions; Step S223: To integrate the multi-dimensional verification results and avoid interference from single-dimensional bias on the overall authenticity assessment of the entity, calculate the mean of the fine-grained score of the assertion entity e. : ； in, correspond The score in the i-th fine-grained dimension, where n is the actual number of dimensions involved; Calculate the consistency score of a single entity : ； Where λ is the weighting coefficient; Step S224: Aggregate all entity scores and introduce a quantity inconsistency penalty term. ,get : ； Where Agg is an aggregate function. For the set of entities involved in the assertion; Calculate the entity number conflict penalty term using the Hungarian matching algorithm; Step S23: Logical consistency verification targets the reasoning chain. LLM, combined with structured visual evidence, verifies logical consistency through natural language reasoning and outputs a logical score. : MeanNLI( NLI( OpCheck; in, It is a reasoning step; MeanNLI ( The mean of the implication values ​​of adjacent reasoning steps; NLI ( OpCheck represents the degree to which the reasoning steps support the conclusion; it is used to detect contradictions in logical operators. , , These are the weighting coefficients; The fusion module is used to process consistency scores. , , Perform type-specific calibration, and then generate a unified illusion score H through weighted fusion, specifically including: Step S31: Use the temperature scaling method to calibrate the consistency scores of each dimension to eliminate bias caused by differences in the distribution of scores across different dimensions. , ； in, For the Sigmoid function; The corresponding dimension calibration temperature parameters are obtained through cross-validation on the validation set. For the original fractions, The scores are the calibrated scores, and their values ​​range from [0,1]. Step S32: Calculate the final hallucination score H based on the calibrated scores through weighted summation: ； in, These are the weighting coefficients. The value of H ranges from [0,1]. The closer H is to 1, the higher the degree of hallucination in the answer. By comparing calibration scores , , The minimum value can determine the dominant hallucination type: like The smallest error is attributed to inconsistent instructions; like The smallest difference is attributed to text-image separation; like The smallest inconsistency is attributed to logical inconsistency; The interpretation module is used to provide traceable evidence to support the test results and to trigger an abstention mechanism when the evidence is insufficient, so as to ensure the credibility and practical value of the test results.

5. A multimodal fidelity illusion detection device based on closed-loop multidimensional verification, characterized in that, It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 3.

6. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to implement the method described in any one of claims 1 to 3.

8. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 3.

Images