Multi-modal model visual illusion detection and inhibition method based on causal intervention

Abstract

The invention discloses a multi-modal model visual illusion detection and inhibition method based on causal intervention, and the method comprises the steps: constructing a causal graph containing three variables, namely an image, a question and an answer, introducing background / language prior as a confusion factor, and forming a multi-modal structure causal model; inputting the original image and the user question into a multi-modal large model, and outputting the original score of the candidate answer; constructing anti-fact input based on an original image, inputting the multi-modal large model again in combination with a user question, and outputting a prejudice score after anti-fact causal intervention; calculating a debiased score through the original score and the prejudice score, quantifying the difference distance between the original score and the debiased score, and dynamically detecting and judging illusion; and finally, outputting the candidate answers after illusion suppression to realize effective control of illusion in multi-modal questions and answers. According to the method, the problem of generalization failure of a multi-modal model on data outside distribution can be solved, visual illusion is effectively inhibited, plug and play in a zero sample reasoning stage is realized, and the deployment cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multimodal model method for detecting and suppressing visual hallucinations based on causal intervention. Background Technology

[0002] In recent years, with the development of multimodal large language models (such as LLaVA, mPLUG-Owl, etc.), machines are able to answer questions based on image content. These models are typically trained on massive image-text pair datasets and improve task performance by capturing statistical correlations between modalities.

[0003] However, existing training paradigms have significant drawbacks: the statistical co-occurrence patterns implicit in massive training data are overlearned by the model, causing its inference process to rely excessively on high-frequency associations in the data, rather than on genuine visual evidence from the images. For example, in the training set, "toothbrush" often co-occurs with "bathroom background," leading the model to establish a strong association of "bathroom background → toothbrush presence," even ignoring the visual features of whether a toothbrush actually exists in the image. This reliance on co-occurrence patterns exposes serious problems in the following scenarios: 1. The problem of model illusion is prominent: When there is a deviation between the input image and the typical co-occurrence pattern in the training data (such as no toothbrush in the bathroom background), the model is prone to ignoring the actual content of the image and incorrectly inferring the existence of objects that have not appeared based solely on the background information, resulting in the illusion of "something out of nothing".

[0004] 2. High cost of debiasing: Traditional solutions rely on building large-scale balanced datasets for model retraining or fine-tuning, which consumes a lot of GPU computing power and time resources, making it difficult to promote and apply in resource-constrained scenarios.

[0005] 3. Rigid detection mechanism: Existing debiasing methods use a uniform threshold to process all samples, which lacks dynamic evaluation of the degree of sample bias. This can easily lead to misjudging normal samples as biased samples, resulting in a decrease in the accuracy of the model on routine problems.

[0006] Therefore, there is an urgent need for a dynamic debiasing method with low resource consumption and high generalization ability, which can effectively suppress model illusion caused by statistical co-occurrence while preserving the normal reasoning ability of the model. Summary of the Invention

[0007] This invention provides a multimodal model-based method for detecting and suppressing visual hallucinations based on causal intervention, in order to overcome the aforementioned technical problems.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows: A multimodal model-based method for visual hallucination detection and suppression based on causal intervention includes: S1: Construct a causal graph containing three nodes and introduce a confounding factor to form a multimodal structural causal model. The nodes are image variables, question variables, and answer variables, respectively. The confounding factor is the background context of the image or language prior. That is, the multimodal structural causal model is used to define a model that introduces a confounding factor representing spurious relevance and can characterize the causal relationship between image variables, question variables, and answer variables. S2: Introduce a multimodal large model, input the original image and user question into the multimodal large model, and output the original score of each candidate answer based on conditional probability; S3: Construct counterfactual input images based on the original images, input the counterfactual input images and user questions into a multimodal large model for counterfactual causal intervention, and output the bias scores of each candidate answer; S4: Calculate the debiased score for each candidate answer based on the original score and the bias score; S5: Calculate the difference distance between the original score and the debiased score to perform dynamic hallucination detection and judgment, and output the candidate answer determined after dynamic hallucination detection and judgment as the final answer to the input user question, thereby realizing the detection and suppression of hallucination.

[0009] Furthermore, the original image and user question are input into a multimodal large model, based on Bayesian probability. Perform reasoning and output the raw scores for each candidate answer.

[0010] Furthermore, a counterfactual input image is constructed based on the original image. This counterfactual input image and the user question are then fed into a multimodal large model for counterfactual causal intervention, outputting bias scores for each candidate answer, including: S31. Perform a masking or nulling operation on the objects in the original image to mask the target object information and generate a counterfactual image that retains only the background information. ; S32. Input the counterfactual image and the user question into the multimodal large model, calculate the conditional probability, and obtain the bias score of each candidate answer, as shown in formula (1). (1) in, Indicates bias score, This represents the answer variable, i.e., the candidate answer predicted by the multimodal large model; This indicates a counterfactual intervention performed on the image. This refers to a counterfactual image that has been masked or emptyed, preserving the background but masking the target object. This represents the original user question, i.e., the question variable.

[0011] Furthermore, based on the original score and the bias score, the debiased score of each candidate answer is calculated, including: By subtracting the bias score from the original score using the causal effect, the debiased score is obtained to eliminate the influence of Z, as shown in formula (2). (2) in, This represents the raw score. The preset adjustment coefficient, This represents the estimated causal effect of "background bias".

[0012] Furthermore, the difference distance between the original score and the debiased score is calculated to perform dynamic hallucination detection and judgment, and the true result is output to achieve hallucination detection and suppression, including: S51. Calculate the distance between the largest raw score and the biased score of the candidate answer corresponding to the largest raw score, as shown in formula (3). (3) S52, Set threshold ,Will and The comparison is as follows; like If the bias is found to be false, the model illusion is determined. The bias removal scores of all candidate answers are compared, and the candidate answer with the highest bias removal score is output as the final answer to the input user question. like If the model is deemed reliable, the candidate answer corresponding to the highest original score is output as the final answer to the user's input question.

[0013] Beneficial effects: This invention provides a multimodal model-based method for detecting and suppressing visual hallucinations based on causal intervention, which has the following advantages: 1. It solves the generalization failure problem of multimodal models on out-of-distribution (OOD) data and effectively suppresses visual illusions: Existing technologies rely on statistical co-occurrence patterns in training data (such as the high-frequency association between "bathroom" and "toothbrush"), leading to systemic hallucinations when models encounter counterintuitive scenarios (such as a bathroom without a toothbrush). This invention orthogonally decouples image background (confusion factor Z) from target object features using a multimodal structural causal model. It utilizes counterfactual intervention to quantify and eliminate the causal effects of background bias, enabling model inference to regress to true visual evidence in the image. This method significantly reduces the model's over-reliance on background priors, corrects "spurious correlation" predictions in long-tailed distribution samples, and drastically reduces the incidence of visual hallucinations.

[0014] 2. Enables plug-and-play functionality in the zero-shot inference stage, significantly reducing deployment costs: Traditional debiasing methods (such as resampling and reweighting) often require modifications to the model structure or full parameter fine-tuning on specific balanced datasets, facing high computational costs and the risk of "catastrophic forgetting." This invention proposes a non-training post-processing intervention mechanism, which eliminates the need to modify or update the training parameters of large multimodal models. It only adds a small amount of computational overhead during the inference stage (such as counterfactual image generation and secondary inference), achieving debiasing through post-processing operations. This method can be directly deployed on pre-trained large-scale pre-trained models (such as LLaVA and MiniGPT-4), possessing significant engineering practical value and cost advantages.

[0015] 3. Establish a dynamic confidence calibration mechanism that balances bias correction effectiveness with fundamental performance: Existing forceful debiasing methods tend to overcorrect, leading to a decrease in model accuracy when processing simple, routine samples. This invention designs a dynamic gating mechanism based on confidence difference (score difference): intervention is triggered only when strong background interference is detected in the model's prediction (the difference between the original score and the debiased score exceeds a threshold); otherwise, the original inference result is retained. This mechanism corrects illusory samples while preserving the model's recognition accuracy on in-distribution samples without loss, achieving a balance between debiasing effectiveness and basic performance. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a multimodal model-based visual hallucination detection and suppression method based on causal intervention provided by the present invention; Figure 2 The original model for the existence of hallucinations; Figure 3 The multimodal structural causal model constructed for this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This embodiment provides a multimodal model-based method for detecting and suppressing visual hallucinations based on causal intervention, such as... Figure 1 As shown, it includes: S1: Construct a causal graph containing three nodes and introduce a confounding factor to form a multimodal structural causal model. The nodes are image variables, question variables, and answer variables, respectively. The confounding factor is the background context of the image or language prior. That is, the multimodal structural causal model is used to define a model that introduces a confounding factor representing spurious relevance and can characterize the causal relationship between image variables, question variables, and answer variables. S2: Introduce a multimodal large model, input the original image and user question into the multimodal large model, and output the original score of each candidate answer based on conditional probability; S3: Construct counterfactual input images based on the original images, input the counterfactual input images and user questions into a multimodal large model for counterfactual causal intervention, and output the bias scores of each candidate answer; S4: Calculate the debiased score for each candidate answer based on the original score and the bias score; S5: Calculate the difference distance between the original score and the debiased score to perform dynamic hallucination detection and judgment, and output the candidate answer determined after dynamic hallucination detection and judgment as the final answer to the input user question, thereby realizing the detection and suppression of hallucination.

[0020] Specifically, such as Figure 2 As shown, a causal graph with three nodes is first constructed, namely the image variable, the question variable, and the answer variable. At the same time, an unobserved confusion factor Z (representing the background context of the image or the language prior) is introduced. Secondly, the "original confidence score" is calculated by inputting the original image and user question into the multimodal large model, performing inference based on Bayesian probability, and outputting the original scores of each candidate option (such as "Yes" or "No"); the "bias confidence score" is calculated by counterfactual intervention, subtracting the "bias" from the "original score" to calculate the "final debias confidence score", i.e., the debias score; Finally, the difference between the original score and the biased score is calculated. If the difference is greater than a set threshold, it indicates a severe hallucination, and the final biased confidence score is used as the output. If the difference is less than or equal to the threshold, it indicates a highly reliable model, and the original confidence score is used directly as the output.

[0021] In a specific embodiment, a causal graph containing three nodes is constructed and a confusion factor is introduced to form a multimodal causal model. The nodes are image variables, question variables, and answer variables, respectively. The confusion factor is either the background context of the image or a language prior. like Figure 3 As shown, Figure 3 An abstract representation of illusion generation: During training, the multimodal large model determines the image content due to background bias, resulting in the final answer generated by the image containing illusions; The multimodal structural causal model constructed in this scheme is as follows: Figure 4 As shown, a causal graph containing three nodes is constructed, and a confounding factor is introduced. The causal graph constructed by this scheme contains the true inference path from the image variable to the answer variable and the hallucinatory inference path from the confounding factor to the answer variable, forming a multimodal structural causal model. By severing the association between the background and the image, the hallucination propagation path is blocked. The multimodal structural causal model is used to define a model that introduces a confounding factor representing false relevance and can characterize the causal relationship between the image variable, the question variable, and the answer variable. The confusion factor is the background context or language prior of the image. The background context refers to the scene information in the image other than the main object. The model will be biased due to the statistical regularity that "specific objects often appear in specific scenes". Linguistic priors refer to patterns established by the model based on the question text (e.g., the "fruit" question easily activates "apple, banana"), leading to an over-reliance on linguistic statistics rather than image content; both interfere with the model's reasoning about the real image content and are the main causes of visual illusions.

[0022] The modal structure causal model constructed in this scheme transforms the "fuzzy model illusion" into a "quantifiable and interventionizable causal mechanism," providing a mathematical framework for subsequent counterfactual interventions. This upgrades the debiasing process from "empirical correction" to "controllable operation based on causal relationships," significantly improving the model's generalization ability and deployment efficiency.

[0023] In a specific embodiment, a multimodal large model is introduced. The original image and user question are input into the multimodal large model, and the original score of each candidate answer is output based on conditional probability. The original image and user question are input into a multimodal large model, based on Bayesian probability. Perform reasoning, output conditional probabilities, perform softmax normalization on the output conditional probabilities, and obtain the original scores of each candidate answer, which are "Yes" and "No". Multimodal Large Language Models (MLLMs) are intelligent systems that integrate visual, audio, and video information on top of Large Language Models (LLMs). They use modal encoders to transform different types of data into a unified semantic space representation, which is then used by the LLM for reasoning and generation, thereby achieving cross-modal understanding and content generation.

[0024] Using multimodal large models for data processing or generation is a common technique used by those skilled in the art. In this embodiment, models such as LLaVA, mPLUG-Owl, and MiniGPT-4 can be used for inference to output conditional probabilities. Therefore, the inference process of multimodal large models will not be elaborated in detail. For example, given an input image of a bathroom without a toothbrush, and the question "Is there a toothbrush?", the multimodal large model outputs a score of -1.26 for "Yes" and -0.86 for "No". The model incorrectly predicts "Yes" (due to background interference).

[0025] In a specific embodiment, a counterfactual input image is constructed based on the original image. This counterfactual input image and the user's question are then input into a multimodal large model for counterfactual causal intervention, outputting bias scores for each candidate answer, including: S31. Perform a masking or nulling operation on the original image to mask the target object information and generate a counterfactual image that retains only the background information. ; In this scheme, the pixel value of the area where the target object is located is set to 0 (black) to preserve the background and generate a counterfactual input image; S32. Input the counterfactual image and the user question into the multimodal large model, calculate the conditional probability, and obtain the bias score of each candidate answer, as shown in formula (4). (4) in, Indicates bias score, This represents the answer variable, i.e., the candidate answer predicted by the multimodal large model; This indicates a counterfactual intervention performed on the image. This refers to a counterfactual image that has been masked or emptyed, preserving the background but masking the target object. This represents the original user question, i.e., the question variable; The purpose of constructing counterfactual images is to artificially create an image input that "has only a background and no objects": Preserve background: This means keeping the background pixels in the image unchanged except for the target object (e.g., bathroom walls, sink, etc.). Masking objects: By using a mask or blanking operation, the target object information in the original image is completely obscured or removed. At this time, the counterfactual image only contains background context information (confusion factor Z), severing the connection between object features and the answer. The counterfactual image (an image without objects) and the original question, such as "Do you have a toothbrush?", are input into a multimodal large model for secondary inference (computational bias) to calculate the conditional probability, that is, to let the model answer the original question when looking at "the image with the object blocked"; Obtaining the bias score: The score output by the model is the bias score. This score represents the degree to which the model guesses based solely on "background experience".

[0026] For example: Suppose the original image is "There is a toothbrush in the bathroom", and the question is "Is there a toothbrush?": Instructions: Black out or erase the "toothbrush" in the picture to create a "bathroom picture without a toothbrush".

[0027] Input: Input this processed image into the model and input the question "Do you have a toothbrush?".

[0028] Result: If the model still answers "Yes" and scores very high (e.g., -1.50), it means the model didn't actually look at the object, but rather reflexively assumed "there's a toothbrush" upon seeing "bathroom." This -1.50 score is the bias score we're trying to capture.

[0029] In a specific embodiment, the debiased score of each candidate answer is calculated based on the original score and the bias score, including: By subtracting the bias score from the original score using the causal effect, the debiased score is obtained to eliminate the influence of Z, as shown in formula (5). (5) in, This represents the raw score. The preset adjustment coefficient, This represents the estimated causal effect of "background bias"; For example, the corrected "Yes" score became 1.73 (the value is only for example logic; the actual comparison is done through Softmax normalization), and the "No" score increased significantly, ultimately leading to the correct prediction of "No".

[0030] In a specific embodiment, the difference distance between the original score and the debiased score is calculated to perform dynamic detection and judgment of hallucinations, and the true result is output to achieve the detection and suppression of hallucinations, including: S51. Calculate the distance between the largest raw score and the biased score of the candidate answer corresponding to the largest raw score, as shown in formula (6). (6) S52, Set threshold ,Will and The comparison is as follows; like If the bias is found to be false, the model illusion is determined. The bias removal scores of all candidate answers are compared, and the candidate answer with the highest bias removal score is output as the final answer to the input user question. like If the model is deemed reliable, the candidate answer corresponding to the highest original score is output as the final answer to the user's input question.

[0031] In this plan, The setup process is as follows: Step 1: Constructing the validation set and labeling illusion samples 1. Construct a validation set containing two types of samples: Hallucination samples: There is a statistical co-occurrence bias between the image and the question, but the image does not actually contain the target object (e.g., "bathroom without a toothbrush", question "Do you have a toothbrush?"), and they are manually labeled as "hallucinations"; Normal samples: Images and questions that conform to a common co-occurrence pattern (e.g., "a bathroom with a toothbrush", question "do you have a toothbrush?"), or where the co-occurrence pattern is weak and the model inference is correct, are manually labeled as "normal".

[0032] Step 2: Calculate the difference distance D among all samples in the validation set. 1. For each sample in the validation set, calculate the raw score, bias score, debiased score, and D-value; 2. Record the D value and "hallucination / normal" label for each sample; Step 3: Determine the threshold τ based on the D-value distribution, usually set between [e.g., 0.1 to 0.5], to balance the hallucination elimination rate and the accuracy of the regular response.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal model-based method for detecting and suppressing visual hallucinations based on causal intervention, characterized in that, include: S1: Construct a causal graph containing three nodes and introduce a confounding factor to form a multimodal structural causal model. The nodes are image variables, question variables, and answer variables, respectively. The confounding factor is the background context of the image or language prior. That is, the multimodal structural causal model is used to define a model that introduces a confounding factor representing spurious relevance and can characterize the causal relationship between image variables, question variables, and answer variables. S2: Introduce a multimodal large model, input the original image and user question into the multimodal large model, and output the original score of each candidate answer based on conditional probability; S3: Construct counterfactual input images based on the original images, input the counterfactual input images and user questions into a multimodal large model for counterfactual causal intervention, and output the bias scores of each candidate answer; S4: Calculate the debiased score for each candidate answer based on the original score and the bias score; S5: Calculate the difference distance between the original score and the debiased score to perform dynamic hallucination detection and judgment, and output the candidate answer determined after dynamic hallucination detection and judgment as the final answer to the input user question, thereby realizing the detection and suppression of hallucination.

2. The method for detecting and suppressing visual hallucinations based on a multimodal model with causal intervention according to claim 1, characterized in that, The original image and user question are input into a multimodal large model, based on Bayesian probability. Perform reasoning and output the raw scores for each candidate answer.

3. The method for detecting and suppressing visual hallucinations based on a multimodal model with causal intervention according to claim 1, characterized in that, A counterfactual input image is constructed based on the original image. This counterfactual input image and the user's question are then fed into a multimodal large model for counterfactual causal intervention. The output is a bias score for each candidate answer, including: S31. Perform a masking or nulling operation on the objects in the original image to mask the target object information and generate a counterfactual image that retains only the background information. ; S32. Input the counterfactual image and the user question into the multimodal large model, calculate the conditional probability, and obtain the bias score of each candidate answer, as shown in formula (1). （1） in, Indicates bias score, This represents the answer variable, i.e., the candidate answer predicted by the multimodal large model; This indicates a counterfactual intervention performed on the image. This refers to a counterfactual image that has been masked or emptyed, preserving the background but masking the target object. This represents the original user question, i.e., the question variable.

4. The method for detecting and suppressing visual hallucinations based on a multimodal model with causal intervention according to claim 3, characterized in that, Based on the original score and the bias score, calculate the debiased score for each candidate answer, including: By subtracting the bias score from the original score using the causal effect, the debiased score is obtained to eliminate the influence of Z, as shown in formula (2). （2） in, This represents the raw score. The preset adjustment coefficient, This represents the estimated causal effect of "background bias".

5. The method for detecting and suppressing visual hallucinations based on a multimodal model with causal intervention according to claim 4, characterized in that, The difference distance between the original score and the debiased score is calculated to perform dynamic hallucination detection and judgment, and the true result is output to achieve hallucination detection and suppression, including: S51. Calculate the distance between the largest raw score and the biased score of the candidate answer corresponding to the largest raw score, as shown in formula (3). （3） S52, Set threshold ,Will and The comparison is as follows; like If the bias is found to be false, the model illusion is determined. The bias removal scores of all candidate answers are compared, and the candidate answer with the highest bias removal score is output as the final answer to the input user question. like If the model is deemed reliable, the candidate answer corresponding to the highest original score is output as the final answer to the user's input question.

Images