Visual language model hallucination mitigation methods, systems, and electronic devices
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-29
- Publication Date
- 2026-08-11
AI Technical Summary
[0005]本发明提供一种视觉语言模型幻觉缓解方法、系统和电子设备，用以解决现有的视觉语言模型幻觉缓解方法忽视模型内部语言先验偏差的影响，幻觉缓解效果差，且在生成阶段依赖双路对比解码导致计算开销大、推理延迟高的缺陷
Smart Images
Figure CN122547902A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, and electronic device for alleviating hallucinations using a visual language model. Background Technology
[0002] In recent years, large-scale visual language models, by integrating visual and linguistic modalities, have demonstrated enormous application potential in tasks such as image recognition, image description, and visual question answering. However, these models are generally plagued by the illusion problem, where the generated text content is inconsistent with the visual input, or even incorrectly asserts objects that do not exist in the image. This problem severely undermines the reliability of visual language models in high-risk real-world industrial scenarios such as medical image diagnosis and autonomous driving.
[0003] Existing training-free hallucination mitigation methods primarily rely on contrastive decoding. These methods typically disrupt the visual basis by injecting noise or blurring the image input, causing the model to indirectly expose its linguistic priors. Subsequently, intervention is achieved during the generation phase by calculating the difference in probability distributions through two forward passes.
[0004] However, existing technologies have significant drawbacks: they ignore the fact that the essence of model illusion stems from the excessive reliance of the backbone network of a large language model on its own prior language knowledge, and existing methods only indirectly alleviate this by manipulating the visual modality, which is ineffective; the inference latency is high, and existing contrastive decoding methods need to perform two forward inference passes at each step of autoregressive generation, which leads to a significant increase in inference latency and makes it difficult to meet the real-time requirements of industrial deployment. Summary of the Invention
[0005] This invention provides a visual language model hallucination mitigation method, system, and electronic device to address the shortcomings of existing visual language model hallucination mitigation methods, which ignore the influence of prior language bias within the model, have poor hallucination mitigation effects, and rely on dual-path contrast decoding in the generation stage, resulting in high computational overhead and high inference latency.
[0006] This invention provides a visual language model-based hallucination relief method, comprising: During the process of the visual language model performing the target reasoning task and outputting response text, an attention intervention operation is performed on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to the visual region of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pair samples and perturbed image-text pair samples. The original image-text pair samples include image samples and the original descriptive text of the image samples. The perturbed image-text pair samples include the image samples and the perturbed descriptive text of the image samples. The original image-text pair sample and the perturbation image-text pair sample are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually marked region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually marked region of the perturbation image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visual marker region is calculated, and based on the attention weight offset, the text sensitive layer is determined from all the attention layers.
[0007] In some embodiments, obtaining the original image-text pair samples and the perturbed image-text pair samples includes: Obtain an image sample and its original descriptive text; combine the image sample and the original descriptive text to obtain the original image-text pair sample. Descriptive text for reference image samples is randomly selected from a pre-defined sample database; The original description text is concatenated with the description text to generate the perturbation description text with conflict semantics; The image sample and the perturbation description text are combined to obtain the perturbation image-text pair sample.
[0008] In some embodiments, calculating the attention weight offset of each attention layer for the visually marked region based on the first attention weight and the second attention weight includes: The difference between the first attention weight and the second attention weight is calculated to obtain the attention weight offset of each attention layer for the visual marker region.
[0009] In some embodiments, determining a text-sensitive layer from all attention layers based on the attention weight offset includes: The attention weight offset is standardized to obtain a standard score for each attention layer. The standard score is compared with a preset threshold. If the standard score is greater than the preset threshold, the corresponding attention layer is determined to be a text-sensitive layer.
[0010] In some embodiments, enhancing the attention weights of the text-sensitive layer to visual regions of the input image includes: Determine the original attention weights of each text-sensitive layer for the visual region of the input image; Determine the enhancement coefficient, and based on the enhancement coefficient and the original attention weight, calculate the enhanced attention weight of each text-sensitive layer for the visual region.
[0011] In some embodiments, determining the enhancement coefficient includes: Obtain the preset basic enhancement parameters; When the preset threshold is greater than or equal to 1, the enhancement coefficient is obtained based on the basic enhancement parameters and the standard score of each text-sensitive layer; When the preset threshold is less than 1, the maximum and minimum values of the standard scores of all text-sensitive layers are extracted. Based on the basic enhancement parameters and the standard scores of each text-sensitive layer, as well as the maximum and minimum values, the enhancement coefficient is calculated.
[0012] The present invention also provides a visual language model hallucination relief system, comprising: The enhancement unit is used to perform attention intervention operations on the text-sensitive layer of the visual language model during the process of the visual language model performing target reasoning tasks and outputting response text, thereby enhancing the attention weight of the text-sensitive layer to the visual regions of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pair samples and perturbed image-text pair samples. The original image-text pair samples include image samples and the original descriptive text of the image samples. The perturbed image-text pair samples include the image samples and the perturbed descriptive text of the image samples. The original image-text pair sample and the perturbation image-text pair sample are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually marked region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually marked region of the perturbation image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visual marker region is calculated, and based on the attention weight offset, the text sensitive layer is determined from all the attention layers.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the visual language model hallucination relief method as described above.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the visual language model hallucination relief method as described above.
[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the visual language model hallucination relief method as described above.
[0016] This invention provides a method, system, and electronic device for alleviating hallucinations in a visual language model. By inputting original and perturbed image-text pairs into a visual language model, the method extracts the first attention weight of each attention layer for the visually labeled regions of the original image-text pairs and the second attention weight of each attention layer for the visually labeled regions of the perturbed image-text pairs. It calculates the attention weight offset of each attention layer for the visually labeled regions and, based on this offset, determines the text-sensitive layer from all attention layers. Attention intervention is performed on the text-sensitive layer to enhance its attention weight to the visual regions of the input image. By introducing perturbed descriptive text as a probe, the method directly quantifies the model's dependence on its own language prior bias, thereby achieving precise localization of the text-sensitive layer causing hallucinations. Only a single forward pass combined with weight modifications of a specific layer is needed to guide the model to focus on the true visual basis. This effectively alleviates model hallucinations while avoiding the exponential computational overhead of dual-path computation, essentially maintaining the model's original inference efficiency and real-time performance. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1 This is one of the flowcharts illustrating the visual language model hallucination relief method provided by the present invention.
[0019] Figure 2 This is the second flowchart of the visual language model hallucination relief method provided by the present invention.
[0020] Figure 3 This is a schematic diagram of the visual language model hallucination relief system provided by the present invention.
[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0023] Figure 1 This is one of the flowcharts illustrating the visual language model hallucination relief method provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 110: Obtain the original image-text pair samples and the perturbed image-text pair samples. The original image-text pair samples include the image sample and the original descriptive text of the image sample. The perturbed image-text pair samples include the image sample and the perturbed descriptive text of the image sample.
[0024] In this context, original image-text pairs refer to standard image-text data combinations containing both visual and linguistic modalities, used for benchmarking or normal business processing scenarios. Image samples can be visual data such as digital images or video frames containing pixel feature matrices. Original descriptive text is the initial prompt or default descriptive text corresponding to the content of the image sample. Perturbed image-text pairs refer to image-text data combinations containing interfering and conflicting semantics, specifically constructed to detect internal defects in the visual-language model. Perturbed descriptive text refers to descriptive text containing content that is inconsistent with the objective content of the image sample or has semantic conflicts.
[0025] In this step, keeping the image samples unchanged while only changing the text data is to use the visual modality as a control variable and the active text perturbation as a diagnostic probe, thereby accurately detecting the degree of dependence of each attention layer of the visual language model on its own language prior knowledge.
[0026] Step 120: Input the original image-text pair samples and the perturbed image-text pair samples into the visual language model. During the forward inference process of the visual language model, extract the first attention weight of the visual label region of the original image-text pair sample for each attention layer of the visual language model, and extract the second attention weight of the visual label region of the perturbed image-text pair sample for each attention layer.
[0027] The visual language model refers to an artificial intelligence model architecture that integrates a visual encoding network and a large language model backbone network, and possesses cross-modal feature understanding capabilities. Forward inference refers to the data flow process from the bottom-level network nodes of the visual language model, through the calculation of network parameters at each layer, until it reaches the output.
[0028] The attention layer is a self-attention network module in the visual language model network structure used to calculate the correlation between different input sequence segments. Visually labeled regions refer to the set of high-dimensional visual feature vectors mapped within the visual language model after image samples have been sliced and feature-encoded.
[0029] The first attention weight refers to the basic weight values assigned by each attention layer to the visual region of the image when the visual language model processes the baseline input. The second attention weight refers to the weight values assigned by the visual language model to the visually labeled region when it is subjected to conflicting semantic text interference. By simultaneously obtaining the attention weights in both the baseline and perturbed states, basic data support can be provided for subsequent quantification of the internal structural biases of the visual language model.
[0030] Step 130: Based on the first attention weight and the second attention weight, calculate the attention weight offset of each attention layer to the visual marker region, and determine the text-sensitive layer from all attention layers based on the attention weight offset.
[0031] Here, attention weight shift refers to the change in the degree of attention directed towards basic visual features within the same attention layer before and after being perturbed by text. The text-sensitive layer refers to the network layer where visual attention drops sharply and shifts to the text region after the text is perturbed; that is, a specific layer within the visual language model that is prone to illusions and vulnerable to language biases.
[0032] Optionally, text-sensitive layers can be selected based on preset comparison rules or numerical judgment conditions. By comparing and analyzing the changes in attention weights before and after the perturbation, the deep-seated vulnerable locations that cause the visual language model to produce illusions can be accurately quantified and located, achieving precise diagnosis and localization of network layers.
[0033] Step 140: During the process of the visual language model performing the target reasoning task and outputting response text, an attention intervention operation is performed on the text-sensitive layer to enhance the attention weight of the text-sensitive layer to the visual regions of the input image.
[0034] In this context, the target reasoning task refers to the visual language model receiving and executing real-world instructions in practical industrial applications or user interaction scenarios, including tasks such as image recognition, image detection, image classification, image description analysis, or visual question answering. The input image is the visual scene to be analyzed provided by the user in the target reasoning task. The response text is the final text output generated by the visual language model through autoregression during the decoding phase.
[0035] Optionally, without altering the original fixed model parameters of the visual language model and without requiring retraining, attention intervention can be applied to the text-sensitive layer during the network's forward propagation computation, dynamically modifying the internal elements of the attention matrix of that layer. Through specific numerical scaling or mapping strategies, the attention weights of the text-sensitive layer pointing to the visual region can be amplified.
[0036] In the standard autoregressive generation stage, this embodiment abandons the dual-path contrast decoding calculation mode and only performs a single-path forward pass. When the calculation reaches the recognized text sensitive layer, dynamic weight amplification is implemented to force the visual language model to generate text based on real visual features, thereby overcoming the dominance and interference of internal language prior knowledge.
[0037] In this embodiment of the invention, by introducing perturbation-descriptive text as a diagnostic probe to directly quantify the internal language bias of the visual language model, the accurate localization of the text-sensitive layer that produces hallucinations is achieved. In the actual generation of response text, only a single forward pass is required and attention weight enhancement is applied to the text-sensitive layer to force the visual language model to focus on the real input image. This effectively overcomes language prior interference and alleviates the hallucinations in the model output, while avoiding the exponential computational overhead caused by traditional dual-path comparison calculations, thus improving the inference efficiency and real-time performance of the visual language model during the deployment phase.
[0038] In some embodiments, a visual language model hallucination relief method is provided, the method comprising: During the process of the visual language model performing target reasoning tasks and outputting response text, attention intervention operations are performed on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to the visual regions of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pairs and perturbed image-text pairs. The original image-text pairs include image samples and the original descriptive text of the image samples. The perturbed image-text pairs include image samples and perturbed descriptive text of the image samples. The original image-text pair samples and the perturbed image-text pair samples are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually labeled region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually labeled region of the perturbed image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visually marked region is calculated. Based on the attention weight offset, the text-sensitive layer is determined from all attention layers.
[0039] In the above embodiments, the diagnostic process of the text-sensitive layer of the visual language model is decoupled from the actual reasoning and generation process. If the text-sensitive layer of the visual language model has been predetermined, then in the practical application stage, it is not necessary to perform the step of determining the text-sensitive layer.
[0040] In some embodiments, obtaining original image-text pair samples and perturbed image-text pair samples includes: Obtain image samples and their original descriptive text, and combine them to obtain the original image-text pair samples; Descriptive text for reference image samples is randomly selected from a pre-defined sample database; The original description text is concatenated with the description text to generate a perturbed description text with conflicting semantics; Combine the image samples and the perturbation description text to obtain perturbation image-text pairs.
[0041] The original descriptive text consists of natural language prompts that correspond to the actual visual content of the image samples.
[0042] The preset sample database refers to a pre-constructed collection of image and text data containing a large number of different image samples and their corresponding descriptive texts. Reference image samples refer to other image samples extracted from the database whose visual content is completely different from the currently processed image samples. The perturbation descriptive text contains misleading information such as objects not present in the currently processed image samples, incorrect color attributes, or incorrect descriptions of relationships between objects.
[0043] Optionally, a pseudo-random algorithm or a specific sampling strategy can be used to extract descriptive text from other image samples in the sample database to ensure that the extracted descriptive text is diverse, unpredictable, and irrelevant to the current image sample.
[0044] Optionally, the image sample and the original descriptive text can be spliced or fused to obtain the original image-text pair sample; or the image sample and the perturbed descriptive text can be spliced or fused to obtain the perturbed image-text pair sample.
[0045] Specifically, for a given visual language model Define the original image and text pair sample set ,in The default suggestion word, i.e., the original description text. Image samples. Obtain the raw output. Then, by concatenating the descriptive text of other image samples... In the middle, construct the perturbation description text , .
[0046] In this embodiment of the invention, by keeping the image samples unchanged, the perturbation description text is constructed by concatenating the description text of randomly sampled reference image samples with the original description text. This enables the batch generation of highly perturbation probe data in a low-cost and automated manner. This not only avoids tedious manual data annotation, but also provides a reliable data foundation for the subsequent effective quantification of the dependence of visual language models on language prior bias by introducing semantic conflicts.
[0047] In some embodiments, based on a first attention weight and a second attention weight, the attention weight offset of each attention layer to the visually marked region is calculated, including: Calculate the difference between the first attention weight and the second attention weight to obtain the attention weight offset of each attention layer for the visually labeled region.
[0048] Among them, the attention weight offset intuitively reflects the specific severity of the shift or loss of the internal computation focus of each attention layer from visual features when facing semantic conflicts.
[0049] Optionally, when obtaining the original image-text pair samples and the perturbed image-text pair samples respectively when generating the labels at the same position, the first... The average attention weight of each attention layer for all visually labeled regions is calculated, namely the first attention weight and the second attention weight; the difference between the two is then calculated to obtain the first attention weight. Attention weight offset of each attention layer ,in .
[0050] In this embodiment of the invention, by directly calculating the difference in attention weights pointing to the visual marker region before and after the perturbation, the physical degradation degree of each attention layer due to conflicting text interference can be quantified intuitively and efficiently. This concretizes the abstract model language prior bias into an objective numerical indicator that can be recognized by the underlying computer network, providing a rigorous logical basis and calculation standard for the subsequent accurate screening of text-sensitive layers that are prone to hallucination.
[0051] In some embodiments, determining the text-sensitive layer from all attention layers based on attention weight offsets includes: The attention weight offset is standardized to obtain the standard score for each attention layer. The standard score is compared with a preset threshold. If the standard score is greater than the preset threshold, the corresponding attention layer is determined to be a text-sensitive layer.
[0052] Standardization refers to the data scaling process that uses specific statistical and mathematical calculation methods to eliminate the influence of dimensionality between various attention weight offsets. For example, the mean of the attention weight offsets of all attention layers in a visual language model can be extracted, and then the global mean and global standard deviation can be calculated. The difference between a specific attention offset and the global mean can then be divided by the global standard deviation.
[0053] The standard score is the dimensionless evaluation value obtained after the above data scaling process. This value can objectively reflect the relative severity of text prior interference to a certain attention layer.
[0054] Alternatively, the formula for calculating standard scores is as follows: ; in, Standard score, The mean of the attention weight offsets across all attention layers. This represents the standard deviation of the attention weight offsets across all attention layers. The higher the value, the more susceptible the layer is to textual prior interference.
[0055] The preset threshold refers to the numerical boundary conditions set in advance by the researchers to classify the attributes of network layers. In practical applications, the higher the standard score of an attention layer, the more easily it is influenced by prior textual information and ignores visual features. Identifying the corresponding attention layer as a text-sensitive layer essentially involves marking the attention layers that meet the above numerical conditions in computer memory so that they can be used as target layers requiring computational intervention during subsequent actual generation tasks.
[0056] In this embodiment of the invention, by standardizing the attention weight offset and combining it with a preset threshold for screening, the differences in parameter scale between different model architectures or different network layers are eliminated, a unified and objective quantitative evaluation index is established, and the accurate positioning of the text-sensitive layer inside the model is achieved.
[0057] In some embodiments, enhancing the attention weight of the text-sensitive layer to visual regions of the input image includes: Determine the original attention weights of each text-sensitive layer for the visual regions of the input image; Determine the enhancement coefficient, and based on the enhancement coefficient and the original attention weight, calculate the enhanced attention weight of each text-sensitive layer for the visual region.
[0058] Specifically, during a single forward pass computation of the visual language model to perform the target reasoning task, when the computation graph reaches the position that has been marked as a text-sensitive layer, the basic tensor parameter matrix of the visual region currently pointed to by that layer is captured in real time through program instructions to obtain the original attention weights.
[0059] The enhancement coefficient is a calculation factor used to dynamically scale the original attention weights.
[0060] Optionally, the processor can call the computation unit to perform numerical mapping operations, such as multiplication, on the acquired original attention weights and enhancement coefficients. This step directly amplifies the attention to visual features at the underlying data structure level, allowing the updated high-weight tensor parameters to continue flowing into subsequent neural networks. The enhanced attention weights are then input into the multi-head attention projection layer to calculate the vocabulary probability distribution of the currently generated tags, completing the decoding. This process involves only one forward inference.
[0061] In this embodiment of the invention, by extracting the original attention weights of the text-sensitive layer during a single forward inference process and combining them with the enhancement coefficient for dynamic calculation and amplification, the model can be forced to pay close attention to the visual image of the objective input when generating text, effectively suppressing the model's own language illusion divergence, while limiting the intervention action to a specific text-sensitive layer, ensuring the efficiency of the underlying inference calculation.
[0062] In some embodiments, determining the enhancement factor includes: Obtain the preset basic enhancement parameters; When the preset threshold is greater than or equal to 1, the enhancement coefficient is obtained based on the basic enhancement parameters and the standard score of each text sensitivity layer; If the preset threshold is less than 1, extract the maximum and minimum values of the standard scores of all text-sensitive layers, and calculate the enhancement coefficient based on the basic enhancement parameters, the standard scores of each text-sensitive layer, and the maximum and minimum values.
[0063] Among them, the basic enhancement parameter is a global adjustment constant used to control the overall intervention intensity benchmark. The maximum and minimum values represent the extreme ranges of the degree of interference experienced by all located text-sensitive layers in the current model structure.
[0064] The enhancement coefficients can be calculated by dividing the difference between the standard score and the minimum score of the attention layer by the difference between the maximum and minimum scores to achieve linear normalization, and then combining this result with the basic enhancement parameters. This piecewise mapping mechanism ensures that the calculated enhancement coefficients always meet the requirements for positive enhancement, regardless of the original data distribution.
[0065] Optionally, the enhancement coefficient is calculated as follows: when When ≥1, ; when <1 hour, ; in, The preset basic enhancement hyperparameters, and Represent all that satisfy Standard score in attention layer The minimum and maximum values.
[0066] In this embodiment of the invention, by obtaining the basic enhancement parameters and using a multi-condition piecewise function strategy that includes the maximum and minimum values to calculate the enhancement coefficients, the intervention operation is ensured to always maintain strict positive monotonicity, preventing over-scaling or negative suppression of the underlying attention weights, and maintaining the numerical stability of the overall forward propagation of the visual language model network in the process of enhancing attention weights.
[0067] Figure 2 This is the second flowchart of the visual language model hallucination relief method provided by the present invention, as shown below. Figure 2 As shown, the method includes the following: In the first stage, the original image-text pairs and the perturbed image-text pairs are input into the visual language model, respectively.
[0068] The internal architecture of the visual language model includes a visual encoder, a projection layer, and multiple attention layers based on the multi-head attention mechanism. Image features are processed sequentially by the visual encoder and projection layer, and then fused with text sequence features in the language model for computation.
[0069] The system extracts the feature matrices generated by each attention layer when processing the two types of image-text pairs mentioned above. Figure 2 The grid diagram on the right represents the distribution of the extracted attention weights. By comparing the changes in matrix element values before and after the perturbation for the same attention layer, the layer-level attention weight offset is calculated. Specific network layers with a significant increase in offset can be diagnosed and marked as text-sensitive layers.
[0070] In the second stage, if the calculated hierarchical attention weight offset exceeds the preset threshold, an intervention operation is performed when the actual inference task is executed and the forward computation is passed to the text-sensitive layer.
[0071] The attention weight matrix is logically divided into a system instruction region, a visual input region, a user prompt region, and a model output region. The intervention here precisely targets the underlying tensor element representing the visual input region. By dynamically amplifying the numerical weights of this specific matrix region, the visual language model is forced to prioritize and rely on the underlying real image features when generating autoregressive response text.
[0072] In this embodiment of the invention, the decoupling logic between the diagnostic and inference generation stages is intuitively demonstrated by combining the network hierarchical architecture. Based on the precise quantification of hierarchical attention shifts, fine-grained feature value enhancement down to specific matrix regions is achieved, improving the reliability of visual language model inference. This embodiment's method requires no additional data fine-tuning or reinforcement learning and can be directly applied to existing visual language model architectures, exhibiting high plug-and-play characteristics. Compared to traditional contrastive decoding methods, this embodiment's method adds almost no additional inference latency, greatly improving the feasibility of model deployment.
[0073] The visual language model hallucination relief system provided by the present invention is described below. The visual language model hallucination relief system described below can be referred to in correspondence with the visual language model hallucination relief method described above.
[0074] Figure 3 This is a schematic diagram of the visual language model hallucination relief system provided by the present invention. Figure 3 As shown, the visual language model hallucination relief system 300 includes: The acquisition unit 310 is used to acquire original image-text pair samples and perturbed image-text pair samples. The original image-text pair samples include image samples and original descriptive text of the image samples. The perturbed image-text pair samples include image samples and perturbed descriptive text of the image samples. Extraction unit 320 is used to input the original image-text pair samples and the perturbed image-text pair samples into the visual language model. During the forward inference process of the visual language model, it extracts the first attention weight of each attention layer of the visual language model for the visually labeled region of the original image-text pair sample, and extracts the second attention weight of each attention layer for the visually labeled region of the perturbed image-text pair sample. The determining unit 330 is used to calculate the attention weight offset of each attention layer to the visual marker region based on the first attention weight and the second attention weight, and to determine the text sensitive layer from all attention layers based on the attention weight offset. The enhancement unit 340 is used to perform attention intervention operations on the text-sensitive layer during the process of the visual language model performing target reasoning tasks and outputting response text, thereby enhancing the attention weight of the text-sensitive layer to the visual regions of the input image.
[0075] In some embodiments, a visual language model hallucination relief system is provided, the system comprising: The enhancement unit is used to perform attention intervention operations on the text-sensitive layer of the visual language model during the process of the visual language model performing target reasoning tasks and outputting response text, thereby enhancing the attention weight of the text-sensitive layer to the visual regions of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pairs and perturbed image-text pairs. The original image-text pairs include image samples and the original descriptive text of the image samples. The perturbed image-text pairs include image samples and perturbed descriptive text of the image samples. The original image-text pair samples and the perturbed image-text pair samples are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually labeled region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually labeled region of the perturbed image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visually marked region is calculated. Based on the attention weight offset, the text-sensitive layer is determined from all attention layers.
[0076] Optionally, obtaining the original image-text pair samples and the perturbed image-text pair samples includes: Obtain image samples and their original descriptive text, and combine them to obtain the original image-text pair samples; Descriptive text for reference image samples is randomly selected from a pre-defined sample database; The original description text is concatenated with the description text to generate a perturbed description text with conflicting semantics; Combine the image samples and the perturbation description text to obtain perturbation image-text pairs.
[0077] Optionally, based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visually labeled region is calculated, including: Calculate the difference between the first attention weight and the second attention weight to obtain the attention weight offset of each attention layer for the visually labeled region.
[0078] Optionally, based on attention weight offsets, text-sensitive layers are determined from all attention layers, including: The attention weight offset is standardized to obtain the standard score for each attention layer. The standard score is compared with a preset threshold. If the standard score is greater than the preset threshold, the corresponding attention layer is determined to be a text-sensitive layer.
[0079] Optionally, the attention weights of the text-sensitive layer on visual regions of the input image are enhanced, including: Determine the original attention weights of each text-sensitive layer for the visual regions of the input image; Determine the enhancement coefficient, and based on the enhancement coefficient and the original attention weight, calculate the enhanced attention weight of each text-sensitive layer for the visual region.
[0080] Optionally, determining the enhancement factor includes: Obtain the preset basic enhancement parameters; When the preset threshold is greater than or equal to 1, the enhancement coefficient is obtained based on the basic enhancement parameters and the standard score of each text sensitivity layer; If the preset threshold is less than 1, extract the maximum and minimum values of the standard scores of all text-sensitive layers, and calculate the enhancement coefficient based on the basic enhancement parameters, the standard scores of each text-sensitive layer, and the maximum and minimum values.
[0081] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute a visual language model hallucination mitigation method. This method includes: during the process of the visual language model performing a target inference task and outputting response text, performing attention intervention operations on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to visual regions of the input image; the text-sensitive layer is determined based on the following steps: acquiring original image-text pairs and perturbed image-text pairs, the original image-text pairs including image samples and original descriptive text of the image samples, and the perturbed image-text pairs including image samples and perturbed descriptive text of the image samples; inputting the original image-text pairs and perturbed image-text pairs into the visual language model; during the forward inference process of the visual language model, extracting a first attention weight of each attention layer of the visual language model for the visually marked regions of the original image-text pairs, and extracting a second attention weight of each attention layer for the visually marked regions of the perturbed image-text pairs; calculating the attention weight offset of each attention layer for the visually marked regions based on the first and second attention weights; and determining the text-sensitive layer from all attention layers based on the attention weight offset.
[0082] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0083] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the visual language model hallucination mitigation method provided by the above methods. The method includes: during the process of the visual language model performing a target reasoning task and outputting response text, performing an attention intervention operation on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to the visual region of the input image; the text-sensitive layer is determined based on the following steps: obtaining original image-text pair samples and perturbed image-text pair samples, the original image-text pair samples including... The process includes image samples and their original descriptive text, and perturbed image-text pairs, including image samples and their perturbed descriptive text. The original and perturbed image-text pairs are input into a visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually labeled region of the original image-text pair is extracted, and the second attention weight of each attention layer for the visually labeled region of the perturbed image-text pair is extracted. Based on the first and second attention weights, the attention weight offset of each attention layer for the visually labeled region is calculated. Based on the attention weight offset, the text-sensitive layer is determined from all attention layers.
[0084] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the visual language model hallucination mitigation method provided by the methods described above. This method includes: during the process of the visual language model performing a target inference task and outputting response text, performing an attention intervention operation on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to the visual region of the input image; the text-sensitive layer is determined based on the following steps: acquiring original image-text pair samples and perturbed image-text pair samples, wherein the original image-text pair samples include image samples and original descriptions of the image samples. The text and perturbed image-text pairs include image samples and perturbed descriptive text for the image samples. The original image-text pairs and perturbed image-text pairs are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually labeled region of the original image-text pairs is extracted, and the second attention weight of each attention layer for the visually labeled region of the perturbed image-text pairs is extracted. Based on the first and second attention weights, the attention weight offset of each attention layer for the visually labeled region is calculated. Based on the attention weight offset, the text-sensitive layer is determined from all attention layers.
[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A visual language model-based hallucination relief method, characterized in that, include: During the process of the visual language model performing the target reasoning task and outputting response text, an attention intervention operation is performed on the text-sensitive layer of the visual language model to enhance the attention weight of the text-sensitive layer to the visual region of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pair samples and perturbed image-text pair samples. The original image-text pair samples include image samples and the original descriptive text of the image samples. The perturbed image-text pair samples include the image samples and the perturbed descriptive text of the image samples. The original image-text pair sample and the perturbation image-text pair sample are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually marked region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually marked region of the perturbation image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visual marker region is calculated, and based on the attention weight offset, the text sensitive layer is determined from all the attention layers.
2. The visual-language model hallucination relief method according to claim 1, characterized in that, The process of obtaining the original image-text pair samples and the perturbed image-text pair samples includes: Obtain an image sample and its original descriptive text; combine the image sample and the original descriptive text to obtain the original image-text pair sample. Descriptive text for reference image samples is randomly selected from a pre-defined sample database; The original description text is concatenated with the description text to generate the perturbation description text with conflict semantics; The image sample and the perturbation description text are combined to obtain the perturbation image-text pair sample.
3. The visual-language model hallucination relief method according to claim 1, characterized in that, The step of calculating the attention weight offset of each attention layer for the visually marked region based on the first attention weight and the second attention weight includes: The difference between the first attention weight and the second attention weight is calculated to obtain the attention weight offset of each attention layer for the visual marker region.
4. The visual-language model hallucination relief method according to claim 1, characterized in that, Based on the attention weight offset, a text-sensitive layer is determined from all the attention layers, including: The attention weight offset is standardized to obtain a standard score for each attention layer. The standard score is compared with a preset threshold. If the standard score is greater than the preset threshold, the corresponding attention layer is determined to be a text-sensitive layer.
5. The visual language model hallucination relief method according to claim 4, characterized in that, The enhancement of the attention weights of the text-sensitive layer to the visual regions of the input image includes: Determine the original attention weights of each text-sensitive layer for the visual region of the input image; Determine the enhancement coefficient, and based on the enhancement coefficient and the original attention weight, calculate the enhanced attention weight of each text-sensitive layer for the visual region.
6. The visual language model hallucination relief method according to claim 5, characterized in that, The determination of the enhancement coefficient includes: Obtain the preset basic enhancement parameters; When the preset threshold is greater than or equal to 1, the enhancement coefficient is obtained based on the basic enhancement parameters and the standard score of each text-sensitive layer; When the preset threshold is less than 1, the maximum and minimum values of the standard scores of all text-sensitive layers are extracted. Based on the basic enhancement parameters and the standard scores of each text-sensitive layer, as well as the maximum and minimum values, the enhancement coefficient is calculated.
7. A visual language model-based hallucination relief system, characterized in that, include: The enhancement unit is used to perform attention intervention operations on the text-sensitive layer of the visual language model during the process of the visual language model performing target reasoning tasks and outputting response text, thereby enhancing the attention weight of the text-sensitive layer to the visual regions of the input image. The text sensitivity layer is determined based on the following steps: Obtain original image-text pair samples and perturbed image-text pair samples. The original image-text pair samples include image samples and the original descriptive text of the image samples. The perturbed image-text pair samples include the image samples and the perturbed descriptive text of the image samples. The original image-text pair sample and the perturbation image-text pair sample are input into the visual language model. During the forward inference process of the visual language model, the first attention weight of each attention layer of the visual language model for the visually marked region of the original image-text pair sample is extracted, and the second attention weight of each attention layer for the visually marked region of the perturbation image-text pair sample is extracted. Based on the first attention weight and the second attention weight, the attention weight offset of each attention layer to the visual marker region is calculated, and based on the attention weight offset, the text sensitive layer is determined from all the attention layers.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the visual language model hallucination relief method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the visual language model hallucination relief method as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the visual language model hallucination relief method as described in any one of claims 1 to 6.