Method and system for hallucination suppression of multi-modal large model based on attention feature anomaly detection

Abstract

The present application relates to a kind of hallucination suppression method and system based on attention feature anomaly detection of multimodal large model, method includes: offline training stage: collection training image and query text input target large visual language model and generate vector, extract attention feature vector in each token decoding step, as token segment sample and assign whether hallucination label, to input token segment level hallucination detection model and train;Online inference stage: given input image and query text, by token extraction from target large visual language model, extract attention feature vector in each token decoding step respectively, after normalization and aggregation, input token segment level hallucination detection model, obtain hallucination risk score, to carry out hallucination suppression.Compared with prior art, the present application triggers the selective precise intervention on demand only when detecting abnormal token segment, effectively alleviate object hallucination while reducing the negative impact on normal generation content.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multimodal large-model hallucination suppression method and system based on attention feature anomaly detection. Background Technology

[0002] In recent years, large-scale visual language models (LVLMs) have demonstrated powerful capabilities in multimodal tasks such as image captioning, visual question answering, and visual reasoning. These models typically consist of three parts: a visual encoder, a modal alignment module, and a language model. The visual encoder maps the input image to a set of visual tokens, the modal alignment module projects these tokens into the shared embedding space of the language model, and then the language model generates text responses token-by-token in an autoregressive manner. Representative models include Qwen3VL and InternVL3.5.

[0003] However, the widespread application of large-scale visual language models has also exposed serious problems in the reliability of their output. Research shows that LVLMs are prone to object hallucination, where the generated text descriptions contain objects inconsistent with the given visual input. For example, a model might incorrectly mention non-existent objects like "dog" or "table" when describing a picture containing only cats. This tendency not only impairs the credibility of the model's output but also poses potential security risks in high-risk applications such as autonomous driving assistance, medical image analysis, and security monitoring. Therefore, reliably detecting and effectively mitigating hallucination is a critical requirement for building trustworthy large-scale visual language models.

[0004] To alleviate object illusions in large visual language models, researchers have proposed various methods, among which contrastive decoding, as an intervention paradigm that does not require additional training during inference, has received widespread attention. The core idea of ​​contrastive decoding is to construct a biased contrastive branch based on the original inference process, and to suppress the illusory token by comparing the output distributions of the two branches.

[0005] Based on the different ways of constructing the comparison signals, existing comparison decoding methods can be broadly divided into two categories: The first category: Input perturbation-based methods. These methods amplify the illusion bias by perturbing the visual or text input, and then subtract the amplified bias from the original output distribution. Representative methods include VCD, which adds noise to the visual input to construct a contrast branch; M3ID, which constructs a pure language prior branch by removing the visual input; AVISC and SID, which enhance the utilization of visual information by adjusting attention weights; and Self-Aug, which constructs an enhanced branch through an adaptive strategy.

[0006] The second category: methods based on inter-layer comparison. These methods identify and correct illusory content by comparing the output distributions of different layers of the model and utilizing the differences in information between layers. A representative method is DoLa, which improves factual accuracy by comparing the logits of earlier and later layers.

[0007] Although the aforementioned methods and strategies differ, they share a common paradigm: intervening in the token-level output probability distribution at every decoding step, i.e., full-stage indiscriminate intervention. However, the full-stage indiscriminate intervention strategy introduces significant side effects, specifically in the following aspects: Defect 1: Full-stage intervention disrupts normal generation. Contrast decoding modifies the output distribution at each decoding step, rather than facing the risk of hallucinations at all steps. For token positions that could be correctly generated, unnecessary intervention may actually alter the originally correct output trajectory, causing previously hallucination-free samples to exhibit hallucinations after intervention. Experimental observations show that full-stage contrast decoding causes significant sample-level performance degradation across different models, datasets, and methods; some methods even exacerbate hallucinations under certain settings.

[0008] Defect 2: Limitations of existing token-level hallucination detection methods. To achieve selective intervention, hallucinations need to be detected in real-time during the decoding process. Existing token-level hallucination detection methods primarily focus on identifying the hallucinated noun tokens themselves, determining whether a particular noun token refers to a real object or a hallucinated object. However, recent research provides strong evidence that hallucinations do not necessarily originate from the hallucinated noun tokens themselves, but rather from triggering patterns associated with hallucinations already appearing in a continuous token sequence before the noun tokens are generated. This means that isolated token-level judgments are insufficient to capture the progressively anomalous patterns that lead to hallucinations. To achieve selective trigger-based contrastive decoding, the detector should determine whether the current decoding trajectory has entered a hallucination-prone segment, rather than simply whether a single token position is abnormal.

[0009] Deficiency 3: Lack of systematic modeling of the hallucination generation process. Recent work has attempted targeted intervention strategies, such as detecting anomalous patterns in attention distribution or adaptively selecting intervention programs. However, the triggering mechanisms of these methods still rely on heuristic rules or single-step feature judgments, lacking a systematic modeling of the hallucination generation process. They fail to systematically analyze and utilize hallucination precursor signals at the token segment level, resulting in limited accuracy and robustness of interventions.

[0010] Defect 4: Existing detection methods are limited to post-hoc analysis. Current span-level hallucination detection methods are limited to post-hoc annotation, that is, retrospectively marking the hallucination span after generation, and cannot trigger real-time intervention during the reasoning process. This post-hoc analysis approach cannot meet the actual need to trigger comparative decoding on demand during the generation process. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the existing technology, which has the defects of disrupting normal generation through full-stage intervention and failing to capture the abnormal patterns that emerge progressively in the continuous token sequence before the generation of hallucinatory terms, and to provide a multimodal large-model hallucination suppression method and system based on attention feature anomaly detection.

[0012] The objective of this invention can be achieved through the following technical solutions: A multimodal large-model hallucination suppression method based on attention feature anomaly detection includes: Offline training phase: Collect training images and query text and input them into the target large visual language model to generate vectors. Extract attention feature vectors at each token decoding step of the target large visual language model as token segment samples and assign labels to determine whether it is a hallucination. Input the token segment samples and labels into the pre-built token segment-level hallucination detection model, output hallucination risk scores, and train according to the corresponding labels. Online inference phase: The trained token segment-level hallucination detection model is attached as an external module to the original decoding process of the target large visual language model; given an input image and query text, the target large visual language model extracts tokens, extracts attention feature vectors at each token decoding step, selects multiple attention feature vectors in chronological order based on a preset sliding window, and inputs them into the trained token segment-level hallucination detection model after normalization and aggregation to obtain a hallucination risk score. Based on the hallucination risk score, it is determined whether to enable the contrastive decoding branch used to suppress hallucinations.

[0013] Furthermore, the attention feature vector extraction process includes: Obtain the attention distribution of the current decoding position to all visible tokens in each token decoding step; Based on the attention distribution, extract the visual attention ratio, cue attention, generated text attention, text-visual ratio, and outgoing attention entropy, and concatenate them to form the attention feature vector; The visual attention ratio is used to quantify the attention allocated to the visual token; The cue attention is used to measure the degree of attention given to the cue token at the current decoding position, and the corresponding calculation expression is: In the formula, For decoding steps A reminder to pay attention, The number of layers in a large-scale visual language model is the target. The target is the number of Heads in each layer of a large visual language model. For the set of cue tokens, For the first l The first layer Attention is focused on the token at the current decoding position. i Attention distribution; The generated text attention is used to define the average attention to previously generated tokens, and the corresponding calculation expression is: In the formula, For decoding steps The generated text attention, For the already generated set of text tokens; The text-visual ratio is used to quantify the balance between text and visual dependencies, and the corresponding calculation expression is: In the formula, For decoding steps Text-to-visual ratio, For decoding steps A compact measure of overall visual dependence; The outward attention entropy is used to quantify the dispersion of the current token's attention across all visible tokens, and the corresponding calculation expression is: In the formula, For decoding steps The outgoing attention entropy of For decoding steps The number of visible tokens, Entropy at the head level.

[0014] Furthermore, the token-segment-level hallucination detection model is a multilayer perceptron detector, and the processing procedure of the multilayer perceptron detector includes: The input data is mapped to the hidden space through a linear layer, and then processed sequentially through a normalization layer, an activation function layer, and a regularization layer. A residual multilayer perceptron is introduced to enhance the nonlinear modeling capability. Finally, a hallucination risk score is output through a linear classification layer and a sigmoid function.

[0015] Furthermore, the aggregation process of multiple attention feature vectors input to the token-segment-level illusion detection model includes: Each attention feature vector is aggregated along the token dimension using different statistics to form a segment identifier of a fixed dimension. This segment identifier is used as input to the token segment-level illusion detection model. The statistics include the mean of the entire segment, the tail mean of the last 5 tokens, the standard deviation, the temporal slope, and the maximum value.

[0016] Furthermore, the normalization process of the multiple attention feature vectors input to the token-segment-level illusion detection model includes: Based on the mean and standard deviation of the feature vectors that have been initialized or adaptively calibrated using prior data, each attention feature vector is normalized.

[0017] Furthermore, the method compares the obtained hallucination risk score with preset activation and deactivation thresholds to determine whether to activate the contrast decoding branch for suppressing hallucinations, wherein the activation threshold is greater than the deactivation threshold.

[0018] Furthermore, the method also includes a dual-threshold hysteresis mechanism to adjust the opening and closing times of the comparison decoding branch. The dual-threshold hysteresis mechanism includes: If the current comparison decoding branch is closed and the hallucination risk score is greater than the activation threshold, then the preset preheating counter is increased and the preset cooling counter is reset. When the preheating counter reaches the preset preheating count limit, the comparison decoding branch is opened. If the current comparison decoding branch is open and the hallucination risk score is less than the deactivation threshold, then the cooling counter is increased and the warm-up counter is reset. When the cooling counter reaches the preset cooling count limit, the comparison decoding branch is closed. In other cases, the preset inactive counter is reset to maintain the state of the comparison decoding branch.

[0019] Furthermore, when the comparison decoding branch is closed, a preset original decoding branch is opened to generate a token.

[0020] Furthermore, the target large-scale visual language model is used for image description, visual question answering, or visual reasoning based on the input image and query text.

[0021] The present invention also provides a multimodal large-model hallucination suppression system based on attention feature anomaly detection, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0022] Compared with the prior art, the present invention has the following advantages: (1) This invention proposes a token-segment-level hallucination detection paradigm. Through attention feature extraction, hallucination precursor pattern analysis, token-segment-level anomaly detection model training, and the processing of sliding window deployment and dual-threshold hysteresis triggering during inference, multi-dimensional implicit features are extracted from the attention distribution within a large visual language model to construct a token-segment-level hallucination precursor detection model. During inference, the model selectively triggers comparative decoding in a plug-and-play manner. This extends hallucination detection from single-token post-event judgment to token-segment-level real-time anomaly recognition. It can capture the precursor anomaly patterns that emerge progressively before the generation of hallucination terms, providing real-time early warning for selective intervention. The detection AUROC achieves higher early warning accuracy in both model architectures.

[0023] (2) The design covers a multi-dimensional attention feature system that includes visual attention ratio, cue attention, generated text attention, text-visual ratio and overall attention entropy. The system characterizes the hallucination precursor signal in cross-modal fusion and has good multi-architecture applicability.

[0024] (3) The whole-stage indiscriminate intervention was transformed into selective intervention on demand. While maintaining the hallucination relief effect, the damage to normal generation was significantly reduced. Sample-level analysis showed that about 60% of the samples that were degraded due to the whole-stage intervention were saved, and the maximum hallucination rate on AMBER was reduced by 27.6%.

[0025] (4) It adopts a lightweight plug-and-play 2-layer MLP design with minimal inference overhead. It can seamlessly integrate various mainstream comparison decoding methods such as VCD, M3ID, AVISC, SID, and Self-Aug without modifying its internal logic.

[0026] (5) Extensive experiments on two architectures, Qwen3VL-8B and InternVL3.5-8B, and four benchmarks, MSCOCO, AMBER, MMHal-Bench and Bingo, have verified the effectiveness and robustness of this method in different evaluation scenarios. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a multimodal large-model hallucination suppression method based on attention feature anomaly detection provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the main process of a multimodal large-model hallucination suppression method based on attention feature anomaly detection provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a model architecture for an illusion detector provided in an embodiment of the present invention; Figure 4 This is a comparison chart of hallucination anomaly patterns provided in an embodiment of the present invention; Figure 5This diagram illustrates the side effect ratios of various state-of-the-art contrast decoding methods provided in this embodiment of the invention, as well as the side effect ratios after incorporating the hallucination detector of this invention, showing an average reduction of approximately 60% or more. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0029] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0030] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0031] Example 1 like Figure 1 As shown, this embodiment provides a multimodal large-model hallucination suppression method based on attention feature anomaly detection, including: Offline training phase S1: Collect training images and query text input into the target large visual language model to generate vectors, and extract attention feature vectors at each token decoding step of the target large visual language model as token segment samples and assign labels to whether it is a hallucination; input the token segment samples and labels into the pre-built token segment-level hallucination detection model, output hallucination risk scores, and train according to the corresponding labels; Online Inference Phase S2: The online inference phase involves attaching the trained token-segment-level hallucination detection model as an external module to the original decoding process of the target large-scale visual language model. Given an input image and query text, the target large-scale visual language model extracts tokens and extracts attention feature vectors at each token decoding step. Based on a preset sliding window, multiple attention feature vectors are selected in chronological order, normalized, and aggregated before being input into the trained token-segment-level hallucination detection model to obtain a hallucination risk score. Based on the hallucination risk score, it is determined whether to enable the contrastive decoding branch used to suppress hallucinations.

[0032] Specifically, such as Figure 2As shown, given an input image and a text query, a large-scale visual language model extracts visual tokens and cue tokens through a visual encoder and a text encoder, respectively, and generates a text response token by token in an autoregressive manner. At each decoding step, this method extracts multi-dimensional implicit features in real time from the model's internal attention distribution, including the visual attention ratio, cue attention, generated text attention, text-visual ratio, and overall attention entropy, and appends them to the feature buffer. Subsequently, a sliding window is used to extract the feature sequences of the K most recent steps from the buffer, which are then normalized and statistically aggregated before being input into a lightweight MLP gating model, outputting the illusion risk score of the current token segment. This score is used to determine the intervention state through a trend-aware dual-threshold hysteresis mechanism: when the risk score continuously exceeds the activation threshold, intervention is initiated, and a contrastive decoding branch is used to generate tokens; when the risk score continuously falls below the deactivation threshold, intervention is deactivated, and the original decoding branch is restored.

[0033] Through the above mechanism, this method transforms the traditional full-stage indiscriminate comparative decoding intervention into a selective and precise intervention that is triggered only when an abnormal token segment is detected, effectively alleviating object illusions while reducing the negative impact on normally generated content.

[0034] The core steps of this method are described in detail below: 1. Extraction of attention feature vectors This section constructs a set of implicit features based on cross-modal attention distribution to characterize the internal state of the current decoding position from four dimensions: visual dependence, text dependence, relative balance between text and visual information, and complexity of the attention distribution. Let... For decoding steps The generated token, for the first l Layer Pay attention to this point and remember: Let be the attention distribution of the current decoding position with respect to all visible tokens, where For steps The number of visible tokens. Further note... These are the sets of visual tokens, cue word tokens, and generated text tokens, respectively. Under this notation, the following five types of attention features are defined: Feature 1: Visual Attention Ratio (VAR), which quantifies the attention allocated to the visual token. in Decoding steps are provided A compact measure of overall visual dependency. In specific implementations, Represented as a The matrix and flattened into 3D vector, preserving the entire layer The representation is used to capture illusion-related discrimination information distributed in different layers.

[0035] Feature 2: Prompt Attention. Measures the degree of dependence on input cues: This feature reflects the level of attention the current decoding position pays to the hint token.

[0036] Feature 3: Generated-text Attention. Defines the average attention paid to previously generated tokens: This value is set to zero when no text has been generated.

[0037] Feature 4: Text-Visual Ratio. Quantifies the balance between textual and visual dependencies. Larger values ​​indicate higher attention to the generated text compared to visual attention.

[0038] Feature 5: Outgoing Attention Entropy. This quantifies the dispersion of attention to the current token across all visible tokens. The head-level entropy is defined as: The final normalized attention entropy is: in, Measure the level of attention at the current decoding position.

[0039] Features two, three, and four are used to quantify the current model's dependence on generated tokens. When hallucinations occur, the model may be influenced by generated tokens, leading to an overemphasis on linguistic priors and reduced attention to visual tokens, potentially resulting in hallucinations. By defining features two, three, and four, the distribution of linguistic and visual tokens at this time can be effectively quantified, thereby determining whether the model has experienced visual attentional drift.

[0040] Feature 2 measures the cue text, processing cases where the cue contains obvious information. Feature 3 measures the generated text, calculating whether the model's subsequent answers strongly rely on the text while ignoring visual facts. Feature 4 measures the balance of attention distribution between text and vision. Feature 5 primarily quantifies the degree of attention dispersion of the current token. A relatively dispersed attention level indicates a lower certainty in the model's answer, which can help pinpoint the likelihood of an illusion.

[0041] Combining the above five types of features, each decoding step eigenvectors ,in +4 ( of (4 dimensions including the other four scalar features).

[0042] 2. Construction of a token-segment level dataset based on token-segment samples and labels, and analysis of hallucination precursor anomaly patterns. To construct samples for anomalous pattern analysis, this invention randomly samples 5000 images from the MSCOCO train2014 dataset. For each image, a target large-scale visual language model is driven to generate a response, and the aforementioned attention features are recorded at each token decoding step. The CHAIR protocol is used to locate phantom object names in the generated text. Based on the located name positions, each response is decomposed into multiple token segment samples: for each name token, attention features of all tokens from the start of the response to the position preceding that name token are collected, and the segment is labeled "phantom" or "normal" depending on whether the corresponding name is a phantom name. To avoid ambiguous supervision, all non-phantom name segments after the first phantom name in the same response are discarded, as subsequent generation may have deviated from the normal decoding trajectory. Therefore, a single response may generate multiple labeled token segment samples.

[0043] Through systematic statistical analysis, significant differences in the distribution of hallucinatory token segments and normal token segments were found across all five attentional characteristics: (1) VAR analysis: Visual attention is mainly concentrated in the middle layer and a few high-response heads. Normal samples showed stronger visual attention in the early layers, indicating earlier visual anchoring; while hallucination samples showed higher visual attention in the middle and deep layers, and the difference gradually weakened in the deep layers, indicating that the increase in visual attention at a later time does not necessarily correct the deviated generation trajectory.

[0044] (2) Analysis of the remaining four types of features: As decoding approaches the noun token position, cue attention, attention to generated text, and text-visual ratio consistently show negative differences (hallucination samples are lower than normal samples), indicating that hazy samples rely less on cues and generated history before noun generation. Overall attention entropy shows positive differences and gradually increases, indicating that hazy samples are more scattered and less focused. These differences become more significant as they approach the noun token.

[0045] The above findings support the core hypothesis of this invention: hallucination precursor signals emerge progressively in the token segment preceding the noun token, rather than appearing only at the noun position. This pattern was validated on two large visual language models with different architectures, Qwen3VL-8B and InternVL3.5-8B, demonstrating the architectural adaptability of the designed features.

[0046] 3. Token-level hallucination detection model 3.1 Model Architecture like Figure 3 As shown, the hallucination detector is implemented as a two-layer MLP (Multilayer Perceptron) structure, specifically including: a first linear layer that maps the input segment representation to a 256-dimensional hidden space, followed by LayerNorm normalization, GELU activation function, and Dropout regularization; and the introduction of a residual MLP block to enhance nonlinear modeling capabilities. Finally, the linear classification head outputs a scalar logit, which is converted into a hallucination risk score by a sigmoid function.

[0047] 3.2 Input Feature Processing For a generated prefix (the sequence of tokens before the noun tokens) of length n-1, the input sequence is defined as: in To preserve the tail ratio, To capture the local token-level feature segments of the evolution of the attention pattern before noun tokens, where each element z represents the attention feature of each token, and the dimension is D.

[0048] The so-called hallucination phenomenon essentially concerns the authenticity of object noun tokens, specifically whether the noun "dog" in the model's response truly exists. Therefore, the last token in the token sequence is the token preceding the object noun, which determines the generation of the next object noun token. According to some related work, the hallucination does not occur at the instant the object noun tokens are generated, but rather an anomaly occurs some time before token generation. Therefore, for generating an n-1 prefix, it is not necessary to use all tokens; instead, W can be used to select a token segment of suitable length.

[0049] 3.3 Feature Aggregation Because the token segment lengths vary across different samples, five statistics are used along the token dimension to... Aggregates into segment representations of a fixed dimension: overall segment mean, tail mean of the last 5 tokens, standard deviation, time slope, and maximum value. These statistics are concatenated into a segment with dimension [missing information]. Fixed-dimensional segment representation .

[0050] Here X It is the above. It simply means that the token segment length is determined to be L.

[0051] The mean of the entire segment, the mean of the tails, the standard deviation, the time slope (the token position is the x-axis of the independent variable, and the D-dimensional feature is the y-axis of the dependent variable, solving the trend of the dependent variable changing with the independent variable), and the maximum value are all obtained by performing arithmetic operations on each feature, thereby compressing the features of multiple tokens into a single feature representation.

[0052] This process ensures that all attention feature vector samples have the same length, facilitating subsequent real-time detection.

[0053] 3.4 Risk Score Output Segmentation Input Gating Model Output hallucination risk score: 3.5 Training Strategies The dataset is hierarchically divided into training and validation sets based on positive and negative labels, and class reweighting is used to mitigate the imbalance between hallucination and non-hallucination samples. The AdamW optimizer is used, along with linear warm-up and cosine learning rate decay. Mixed-precision training and gradient pruning are employed to improve stability. AUROC is used as the primary metric for model selection, and checkpoints showing the best performance on the validation set are preserved.

[0054] 4. Deployment strategy during the online inference phase 4.1 Feature Buffering and Normalization During inference, the illusion detector is attached as an external lightweight module to the original decoding process. Attention feature vectors are extracted at each decoding step. The values ​​are appended to the feature buffer. To mitigate feature distribution shifts between different responses, the mean and variance of the training set are used as priors, and a Welford update is performed on the first sample of the current response. The updated statistics are fixed and used to normalize all subsequent window features in the same response.

[0055] 4.2 Sliding Window Detection At each step t, extract the K most recent feature vectors from the buffer to form a local window, and calculate: in and The mean and standard deviation of the attention feature vectors after prior initialization and one-step adaptive calibration are given. To adjust the sliding window size, The aggregation method during training. Hallucination risk score. Window size. The window size is set to be slightly smaller than the average effective length of the training samples. This maintains consistency with the training segment-level paradigm while also providing early warning for anomalous segments. The window size is the length of the continuous token segments input to the hallucination detector. The average token segment length of the training data is statistically analyzed using the training samples; therefore, the window size should be slightly smaller than the average length.

[0056] 4.3 Trend-aware dual-threshold hysteresis triggering mechanism A double-threshold hysteresis mechanism is applied to the risk score sequence to reduce false triggers caused by local noise. The specific rules are as follows: (1) The current intervention status is closed. and Time: Add a preheating counter Reset the cooling counter ;when When the preset upper limit is reached, set (Activate intervention), otherwise keep .

[0057] (2) The current intervention status is enabled. )and Time: Add cooling counter Reset the preheating counter ;when When the preset upper limit is reached, set (Turn off intervention), otherwise keep .

[0058] (3) In other cases, reset the inactive counter and maintain the current intervention status.

[0059] in As the activation threshold, To deactivate the threshold, and This creates a hysteresis interval, reducing frequent state switching caused by short-term score fluctuations.

[0060] 4.4 Threshold Calibration Key inference hyperparameters include local window size. Activation threshold and deactivation threshold .fixed Then, the threshold pair is calibrated on the validation set based on a tradeoff between recall and false positive rate (FPR). Specifically, in the FPR budget... Under constraints, select the threshold combination that maximizes recall: Prioritizing high recall to reduce false negatives, while limiting false triggers through FPR constraints. A conservative FPR budget was used in the experiment. The default settings are 0.10 and 0.10 hysteresis interval.

[0061] 4.5 Selective Intervention Implementation According to the intervention status ,when When using the comparison decoding branch to generate a token, The original decoding branch is used to generate tokens. Thus, this invention transforms indiscriminate intervention across all stages into precise intervention triggered only when an abnormal segment is detected.

[0062] 4.6 Plug and Play Integration As an external module, the hallucination detector proposed in this invention can be seamlessly deployed on any full-stage contrast decoding method (including VCD, M3ID, AVISC, SID, Self-Aug, etc.), only controlling whether to enable the contrast decoding branch at the current time step, without changing the contrast formula, logit adjustment and decoding rules of the contrast decoding method itself.

[0063] The aforementioned large-scale visual language model is used for image description, visual question answering, or visual reasoning based on the input image and query text.

[0064] The following is an example of a specific usage process of the above solution: like Figure 2As shown, this method consists of an offline training phase and an online inference phase. In the offline training phase, 5000 images are randomly sampled from the MSCOCO train2014 dataset. A large-scale visual language model (such as Qwen3VL-8B or InternVL3.5-8B) is driven on a single NVIDIA A800-SXM4-80GB GPU to generate image descriptions. Attention distribution across all L layers × H heads is recorded at each decoding step, and visual attention ratio, cue attention, generated text attention, text-visual ratio, and outgoing attention entropy are extracted and concatenated into a D-dimensional feature vector. The CHAIR protocol is used to locate phantom words. Each response is segmented into token segments based on word position and labeled with positive and negative tags. Non-phantom segments after the first phantom word are discarded to avoid ambiguous supervision. For each sample, five statistics are calculated along the token dimension: mean of the entire segment, mean of the last 5 tokens, standard deviation, temporal slope, and maximum value. These are concatenated into a 5D fixed-segment representation and input into a 2-layer MLP detector (256-dimensional hidden layers, including LayerNorm, GELU activation, Dropout, and residual connections). Training employs the AdamW optimizer, combined with linear warm-up, cosine learning rate decay, mixed precision, and gradient clipping. Class reweighting is used to handle positive and negative sample imbalance, and the optimal checkpoint is selected on the validation set according to the AUROC metric.

[0065] During the online inference phase, the trained MLP detector is attached as an external module to the original decoding process. Attention features are extracted and appended to a buffer at each decoding step. Normalization statistics are initialized using the training set mean and variance as priors, combined with the Welford update of the first sample in the current response step; these are then fixed for subsequent feature normalization. At each step, the K most recent feature vectors are truncated from the buffer to form a sliding window, which, after normalization and aggregation, is input to the detector to obtain the hallucination risk score. A dual-threshold hysteresis mechanism (activation threshold) is applied to the score sequence. With deactivation threshold The hysteresis interval is set to 0.05. The intervention state is determined in conjunction with warm-up and cool-down counters: the contrast decoding branch is activated when the risk score consistently exceeds θ_high, and the original decoding branch is restored when it consistently falls below θ_low. Threshold calibration on the validation set is constrained by a conservative FPR budget α=0.05, selecting the threshold combination with the highest recall rate. This module can be directly integrated into any full-stage contrast decoding method such as VCD, M3ID, AVISC, SID, and Self-Aug, controlling only the branch selection at each time step without changing the logic of the contrast decoding itself.

[0066] from Figure 4 It can be seen that there are significant differences in the attentional characteristics presented between hallucination and non-hallucination cases.

[0067] from Figure 5It can be seen that adding the hallucination detector in this invention significantly reduces the proportion of decoding side effects, further improving the hallucination relief ability. This embodiment also provides a multimodal large model hallucination suppression system based on attention feature anomaly detection, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the multimodal large model hallucination suppression method based on attention feature anomaly detection as described above.

[0068] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A multimodal large-model hallucination suppression method based on attention feature anomaly detection, characterized in that, include: Offline training phase: Collect training images and query text and input them into the target large visual language model to generate vectors. Extract attention feature vectors at each token decoding step of the target large visual language model as token segment samples and assign labels to determine whether it is a hallucination. Input the token segment samples and labels into the pre-built token segment-level hallucination detection model, output hallucination risk scores, and train according to the corresponding labels. Online inference phase: The original decoding process of attaching the trained token-segment-level illusion detection model as an external module to the target large visual language model; Given an input image and query text, a target large-scale visual language model extracts tokens. Attention feature vectors are extracted at each token decoding step. Based on a preset sliding window, multiple attention feature vectors are selected in chronological order. After normalization and aggregation, they are input into a trained token segment-level hallucination detection model to obtain a hallucination risk score. Based on the hallucination risk score, it is determined whether to enable the contrast decoding branch used to suppress hallucinations.

2. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The attention feature vector extraction process includes: Obtain the attention distribution of the current decoding position to all visible tokens in each token decoding step; Based on the attention distribution, extract the visual attention ratio, cue attention, generated text attention, text-visual ratio, and outgoing attention entropy, and concatenate them to form the attention feature vector; The visual attention ratio is used to quantify the attention allocated to the visual token; The cue attention is used to measure the degree of attention given to the cue token at the current decoding position, and the corresponding calculation expression is: In the formula, For decoding steps A reminder to pay attention, The number of layers in a large-scale visual language model is the target. The target is the number of Heads in each layer of a large visual language model. For the set of cue tokens, For the first l The first layer Attention is focused on the token at the current decoding position. i Attention distribution; The generated text attention is used to define the average attention to previously generated tokens, and the corresponding calculation expression is: In the formula, For decoding steps The generated text attention, For the already generated set of text tokens; The text-visual ratio is used to quantify the balance between text and visual dependencies, and the corresponding calculation expression is: In the formula, For decoding steps Text-to-visual ratio, For decoding steps A compact measure of overall visual dependence; The outward attention entropy is used to quantify the dispersion of the current token's attention across all visible tokens, and the corresponding calculation expression is: In the formula, For decoding steps The outgoing attention entropy of For decoding steps The number of visible tokens, Entropy at the head level.

3. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The token-segment-level hallucination detection model is a multilayer perceptron detector, and the processing procedure of the multilayer perceptron detector includes: The input data is mapped to the hidden space through a linear layer, and then processed sequentially through a normalization layer, an activation function layer, and a regularization layer. A residual multilayer perceptron is introduced to enhance the nonlinear modeling capability. Finally, a hallucination risk score is output through a linear classification layer and a sigmoid function.

4. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The method's aggregation process for multiple attention feature vectors input to the token-segment-level illusion detection model includes: Each attention feature vector is aggregated along the token dimension using different statistics to form a segment identifier of a fixed dimension. This segment identifier is used as input to the token segment-level illusion detection model. The statistics include the mean of the entire segment, the tail mean of the last 5 tokens, the standard deviation, the temporal slope, and the maximum value.

5. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The normalization process of the method for multiple attention feature vectors input to the token-segment-level illusion detection model includes: Based on the mean and standard deviation of the feature vectors that have been initialized or adaptively calibrated using prior data, each attention feature vector is normalized.

6. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The method compares the obtained hallucination risk score with preset activation and deactivation thresholds to determine whether to activate the contrast decoding branch for suppressing hallucinations, wherein the activation threshold is greater than the deactivation threshold.

7. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 6, characterized in that, The method also includes a dual-threshold hysteresis mechanism to adjust the opening and closing times of the comparison decoding branch, the dual-threshold hysteresis mechanism comprising: If the current comparison decoding branch is closed and the hallucination risk score is greater than the activation threshold, then the preset preheating counter is increased and the preset cooling counter is reset. When the preheating counter reaches the preset preheating count limit, the comparison decoding branch is opened. If the current comparison decoding branch is open and the hallucination risk score is less than the deactivation threshold, then the cooling counter is increased and the preheating counter is reset. When the cooling counter reaches the preset cooling count limit, the comparison decoding branch is closed. In other cases, the preset inactive counter is reset to maintain the state of the comparison decoding branch.

8. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 7, characterized in that, When the comparison decoding branch is closed, the preset original decoding branch is opened to generate a token.

9. The multimodal large-model hallucination suppression method based on attention feature anomaly detection according to claim 1, characterized in that, The target large-scale visual language model is used to perform image description, visual question answering, or visual reasoning based on the input image and query text.

10. A multimodal large-model hallucination suppression system based on attention feature anomaly detection, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor calling the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images