Visual language model illusion suppression method based on adaptive dynamic attention intervention

Abstract

The invention discloses a visual language model illusion suppression method based on self-adaptive dynamic attention intervention. The method is used for reducing the problem that a visual language model generates wrong associated information. The method comprises the steps that text input, visual input and historical response are acquired, and an attention accumulation vector is initialized; in the layer-by-layer calculation process of the language model, dynamically adjusting a non-normalized attention matrix, enhancing the weight of a visual sensitive attention head, and performing visual Token pruning in a deep network to optimize cross-modal information interaction; and finally, generating an output Token based on the adjusted attention mechanism, and carrying out loop iteration until a complete response is generated. According to the method, a method of combining text deviation correction through self-adaptive attention head modification and visual attention convergence-based Token pruning is adopted, so that the performance of the model in a multi-modal task is remarkably improved, and the illusion phenomenon caused by a language modal dominant reasoning process is effectively relieved.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention. Background Art

[0002] In large-scale vision-language models (LVLMs), hallucination occurs when the model ignores or misinterprets visual information, or even fabricates non-existent objects or attributes, resulting in generated content that does not match the actual image. This phenomenon impacts tasks such as visual question answering and image description, reducing the accuracy and credibility of generated content and limiting the model's application value. The primary cause of hallucination is the model's overreliance on the language modality, which weakens or incorrectly integrates visual information within multi-layer attention modules. The presence of a large number of irrelevant visual tokens dilutes critical visual information, exacerbating the hallucination problem. Existing methods for mitigating hallucinations face challenges such as high computational cost, potential visual information loss from pruning, and the difficulty of balancing computational efficiency and accuracy in high-resolution scenarios. Effectively mitigating hallucinations is crucial for improving the model's real-world perception capabilities and enhancing its reliability in practical applications.

[0003] With the rapid development of artificial intelligence and deep learning, large models have demonstrated tremendous potential in areas such as multimodal understanding and intelligent interaction. Indeed, many scholars have conducted in-depth research on hallucinations in large models, finding that attention sinks and visual attention redundancy are the primary factors contributing to these hallucinations. During data processing, large models often over-rely on the language modality, resulting in a weakening of visual information in attention allocation. Furthermore, processing a large number of redundant visual tokens dilutes key visual content, causing the generated results to deviate from the true image information, exacerbating the hallucination problem. To alleviate this problem, attention intervention and visual token pruning have been proposed as effective solutions. Attention intervention adjusts the model's internal attention allocation, prioritizing visual content during data processing, reducing the dominance of the language modality and ensuring a more balanced fusion of multimodal information. Visual token pruning filters and removes redundant visual tokens, reducing low-value or irrelevant visual information and focusing on key visual features, ensuring that visual information receives appropriate weight in data processing.

[0004] However, the performance of existing methods often needs to be optimized. Therefore, a hallucination suppression method based on dynamic attention intervention is proposed, which reduces computational overhead while enhancing the utilization of visual information. It has important research value and broad application prospects. Summary of the Invention

[0005] To solve the problems in the prior art, the present invention proposes a hallucination suppression method for a visual language model based on adaptive dynamic attention.

[0006] The technical solutions of the present invention are as follows:

[0007] A method for suppressing hallucinations in a visual language model based on adaptive dynamic attention, wherein the visual language model comprises an image encoder, a visual-language connector, and a language model. The language model has a multi-layer architecture, and each layer comprises a multi-head self-attention module. The method comprises:

[0008] S1: Gets text input and visual input processed by the image encoder and vision-language connector;

[0009] S2: Initialize the self-attention accumulation vector and the mixed attention accumulation vector;

[0010] S3: Iterate substeps S3.1-S3.4 layer by layer in the language model:

[0011] S3.1: If the current layer is the first layer, the input of the current layer must be dynamically constructed based on whether there is a historical response. If there is no historical response, the text input and visual input are concatenated to obtain a multimodal input. If there is a historical response, the text input, visual input, and historical response are concatenated to obtain a multimodal input. The resulting multimodal input serves as the input of the current layer.

[0012] If the current layer is not the first layer, the output of the previous layer is directly used as the input of the current layer;

[0013] S3.2: Calculate the unnormalized attention matrix based on the multimodal input, update the self-attention accumulation vector and the mixed attention accumulation vector according to the unnormalized attention matrix, and calculate the output of each attention head of the multi-head self-attention module in the current layer;

[0014] S3.3: If the current layer is the first or last layer, skip this step and execute S3.4 directly; if the current layer is the second to fifth layer, identify the visually sensitive attention head based on the output of each attention head, and adjust the unnormalized attention weight of the identified attention head; if the current layer is after the fifth layer, perform a pruning operation on the token corresponding to the visual input, and adjust the unnormalized attention matrix based on the pruning result; the unnormalized attention weight is obtained according to the unnormalized attention matrix;

[0015] S3.4: Normalize the adjusted unnormalized attention matrix and calculate the output of the current layer;

[0016] S4: Generate a token based on the output of the last layer of the language model and add the generated token to the historical response;

[0017] S5: Repeat steps S2-S4 based on the text input and the pruned visual input until the generated token is an EOS token (end token) or the preset maximum number of tokens generated is reached. The final token output sequence is obtained in the token generation order as the output of the visual language model.

[0018] The present invention also proposes a visual language model hallucination suppression system based on adaptive dynamic attention intervention, which is used to implement the above-mentioned hallucination suppression method.

[0019] Compared with the existing technology, the invention has the following outstanding features:

[0020] This paper proposes a method that combines adaptive attention head modification to correct for textual bias with token pruning based on visual attention convergence. By strengthening the weight of visual information in the generation process, it reduces the model's over-reliance on the language modality. This strategy significantly improves the model's performance on multimodal tasks, effectively mitigates hallucinations caused by the language modality dominating the reasoning process, and enhances the model's understanding and utilization of visual information. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a method for suppressing hallucinations in one embodiment of the present invention;

[0022] Figure 2 2 is a comparison chart of the experimental results of LLaVA and the improved method of the present invention. DETAILED DESCRIPTION

[0023] The following detailed description of selected implementation cases in conjunction with the accompanying drawings is intended to more fully reveal the core ideas of the present invention, but is not intended to constitute any form of limitation thereto. We encourage diverse innovations and adjustments without departing from the basic principles and spirit of the present invention, including but not limited to structural fine-tuning, functional optimization, and expansion of application scenarios. All these variations and innovations should be deemed to fall within the scope of protection of the present invention, fully reflecting the broad tolerance and respect for the boundaries of technological innovation.

[0024] After experiments and research, the applicant has summarized a method that combines adaptive attention head modification and token pruning based on visual attention convergence. It can effectively enhance the information integration ability in multimodal tasks and reduce excessive reliance on language modalities. However, the model often falls into the problem of visual information loss and aggravated text-dominant bias in the later stages of token reasoning. In order to solve these problems, the present invention proposes an adaptive dynamic attention intervention strategy for hallucination suppression, dynamically balancing the contributions of visual and language modalities. Ultimately, the present invention obtains a method for hallucination suppression of multimodal large models with better performance.

[0025] Example 1

[0026] Relying on the architecture of the LLaVA-v1.5 model, this paper proposes a hallucination suppression method based on adaptive dynamic attention intervention and visual guidance optimization. By dynamically adjusting the attention distribution of visual and language modalities, the model's excessive dependence on the language modality is reduced, thereby enhancing the model's real perception ability.

[0027] The LLaVA-v1.5 model includes a CLIP model (image encoder), a visual language connector and an LLM model (large language model); the CLIP model is used to encode the visual input, and the encoded visual features are projected into the embedding space of the LLM model through the visual language connector. The projected visual features are input into the LLM model together with the text input and the historically generated LLM model output for processing. The LLM consists of multiple stacked Transformer layers, each Transformer layer contains a multi-head self-attention module and a feedforward neural network, and each sublayer has a residual connection and layer normalization. The LLM model of the LLaVA-v1.5 model described in this embodiment includes 32 layers of Transformer layers.

[0028] The main steps of the visual-language model hallucination suppression method based on adaptive dynamic attention proposed in this embodiment are as follows: (1) Process the input of the visual-language task, and combine the text input, visual input and historically generated responses into a multimodal input; (2) Calculate the adaptive attention distribution layer by layer in the large language model of the visual language model, including calculating the attention distribution, adjusting the weight of the visually sensitive attention head, and pruning the low-contribution visual tokens to make them pay more attention to visual information; (3) Generate the current token based on the attention distribution; (4) Dynamically update the input and return to step (2) to enter the next round of calculation; (5) Construct the final token generation result. It has been found through experiments that the visual language model will have better performance through the scheme of the present invention. The specific implementation method is as follows and a flow chart is made as follows Figure 1 .

[0029] (1) Processing the input of the vision-language task, combining text input, visual input, and historically generated responses into multimodal input;

[0030] (1.1): Get input information;

[0031] In the present invention, the input information consists of three parts: text input X I , visual input X V and historical response X HIn this embodiment, the visual input is the image features, and the historical response refers to all the outputs generated by the vision-language model. After appropriate processing, these inputs are combined into a multimodal input X for subsequent generation tasks. The specific input structure is as follows:

[0032] Text input is represented by X I , is a I A sequence of text tokens:

[0033] X I =[x1,x2,…,x Ltext ]#(1)

[0034] The visual input is represented as X V , which is composed of visual Tokens processed by the image encoder. image A sequence of visual tokens:

[0035]

[0036] The historically generated responses are denoted as X H , which is a history Sequence of historical response tokens:

[0037]

[0038] (1.2): Initialize algorithm configuration;

[0039] Before starting to generate a new token, it is necessary to initialize key algorithm parameters to ensure the correct execution of the pruning strategy and the stability of the model. A threshold parameter τ is defined to control the proportion of retained visual tokens.

[0040] Attention accumulation vector Indicates the importance of the accumulation of visual tokens in the attention module. The attention accumulation vector C can be further divided into two categories: self-attention accumulation vector C self and the mixed attention accumulation vector C hybrid Specifically, the attention accumulation vector C is defined as follows:

[0041]

[0042] Among them, L image Indicates the number of visual tokens.

[0043] Each time a new Token is generated, the threshold parameter τ is reinitialized to 1. At the same time, all elements C in the attention accumulation vector C are set to 0 during initialization.

[0044] (1.3): Data splicing;

[0045] Enter text into X I , visual input X V and historical response X H The concatenated multimodal input X is expressed as follows:

[0046] X=concat(X I [1:L system ],X V ,X I [L system +1:L text ],X H ]#(5)

[0047] where X I [1:L system ] is the system token, L system is the number of system tokens, X I [L system +1:L text ] is represented by a question token. The system token is an instruction preset by the developer that defines the model behavior pattern or task, and the question token is an instruction proposed by the user that represents a specific task.

[0048] In particular, in the Transformer layer after the first layer of the LLM model, inter-modal feature concatenation is no longer performed, but the output of the previous layer is directly used as the multimodal input X.

[0049] Enter text into X I and history generation X H The combined input X text The concatenated text-only input X text It is expressed as follows:

[0050] X text =concat(X I ,X H )#(6)

[0051] (2) Calculate adaptive attention distribution layer by layer in the large language model of the vision-language model, including calculating attention distribution, adjusting the weight of visually sensitive attention heads, and pruning low-contribution visual tokens to make them focus more on visual information;

[0052] (2.1): Calculate the query matrix Q of the attention head h h , key matrix K h Sum matrix V h ;

[0053] The multimodal input X is linearly transformed to generate the query matrix Q for each attention head h. h , key matrix K h Sum matrix V h , the specific calculation formula is as follows:

[0054]

[0055] in, The weight matrices representing the query, key, and value of the h-th attention head respectively.

[0056] (2.2): Calculate the unnormalized attention matrix

[0057] In the multi-head self-attention module, the unnormalized attention matrix Through the query matrix Q h and bond matrix K h The dot product of is calculated and scaled. Its mathematical expression is as follows:

[0058]

[0059] Among them, d k Represents the query matrix Q h and bond matrix K h The dimension of represents the feature space dimension of the query and key, which is used to scale the result of the dot product.

[0060] (2.3): Calculate the normalized attention weight matrix A h ;

[0061] To obtain the normalized attention weight matrix A h , the unnormalized attention matrix needs to be Perform softmax operation. The calculation formula is as follows:

[0062]

[0063] Among them, the mathematical expression of the softmax operation is as follows:

[0064]

[0065] (2.4): Perform visual token importance calculation;

[0066] Calculate the average attention distribution matrix M of all attention heads in the current layer. The specific calculation formula is as follows:

[0067]

[0068] Where H represents the total number of attention heads in the current layer.

[0069] In order to accurately identify the visual tokens that serve as attention sinks during the reasoning process, the average attention distribution matrix M is decomposed into two sub-region self-attention matrices M self and the mixed attention matrix M hybrid .

[0070] Self-attention matrix M self represents the attention interaction between visual tokens, which is defined as:

[0071] M self =M[L system :L system +L image ,L system :L system +L image ]#(12)

[0072] Mixed attention matrix M hybrid It represents the attention contribution of all tokens to the visual token, which is defined as:

[0073] M hybrid =M[∶,L system :L system +L image ]#(13)

[0074] Among them, L system Indicates the number of system tokens.

[0075] For the jth visual Token, its corresponding self-attention accumulation vector C self and the mixed attention accumulation vector C hybrid The j-th element in is recorded as the cumulative score of self-attention and the mixed attention cumulative score They are updated iteratively in the following way:

[0076]

[0077] (2.5): Calculate the output O of the attention head h h ;

[0078] Normalize the attention weight matrix A h Applied to the value matrix V h , to aggregate the feature information of different key-value pairs and obtain the output O of the attention head h h The calculation formula is as follows:

[0079] O h =A h Vh #(16)

[0080] (2.6): Execute different strategies according to the number of the Transformer layer currently being executed;

[0081] If the current layer layer∈[2,5], then perform recognition and enhancement of adaptive visually sensitive attention heads;

[0082] Based on formulas (7), (8), (9), and (16), replace the multimodal input X in the formula with the input X containing only text modality text Calculate and get the attention output containing only text modality

[0083] Calculate the visual sensitivity VAS of the attention head h h , visual acuity VAS h It measures the sensitivity of the attention head h to visual information. The formula is as follows:

[0084]

[0085] Here, ‖·‖2 is the Euclidean norm, which means calculating the Euclidean distance between two vectors.

[0086] Calculate the language sensitivity LAS of the attention head h h , language sensitivity LAS h It measures the sensitivity of the attention head h to language information. The formula is as follows:

[0087]

[0088] By VAS h and LAS h By calculating , we can identify the visually sensitive attention head set VDS. It is defined as follows:

[0089] VDS={h|VAS h <μ(VAS)+σ(VAS),LAS h <μ(LAS)+σ(LAS)}#(19)

[0090] Among them, μ(VAS) and μ(LAS) are the means of the visual sensitivity and language sensitivity of all attention heads in the current layer, respectively, and σ(VAS) and σ(LAS) are the standard deviations of the visual sensitivity and language sensitivity of all attention heads in the current layer, respectively.

[0091] For the attention head h selected by VDS, that is, in the unnormalized attention matrix The last line of the visual Token set T visionThe elements in The corresponding columns in the are adjusted. The specific calculation method is as follows:

[0092]

[0093] Among them, α is a hyperparameter used to control the strength of the enhancement; n represents the last row index of the matrix.

[0094] If the current layer layer∈[6,21], perform adaptive visual Token pruning;

[0095] In order to measure the relative importance of each visual token in the attention distribution, each visual token is sorted in descending order according to the element C in the attention accumulation vector C, and the visual tokens are rearranged in the sorted order:

[0096]

[0097] in represents the jth visual Token after rearrangement according to the accumulated self-attention vector, Represents the j-th visual Token after rearrangement according to the mixed attention accumulation vector.

[0098] In order to adaptively determine the number of visual tokens p that need to be retained, the calculation method is as follows:

[0099]

[0100] Hybrid Attention Pruning Set D hybrid and the self-attention pruning set D self The definition is as follows:

[0101]

[0102] Based on formulas (21), (22), and (23), the hybrid attention pruning set D is calculated and determined. hybrid and the self-attention pruning set D self , which is used to evaluate the relative importance of visual tokens in attention distribution. By calculating the mixed attention pruning set D hybrid and the self-attention pruning set D self The intersection of the visual token set D that needs to be pruned in the current layer is determined purning , which is calculated as follows:

[0103] D purning =D hybrid ∩D self #(twenty four)

[0104] For the pruning set D purningThe visual token in the unnormalized attention matrix The last line in sets the element value of the corresponding column to negative infinity:

[0105]

[0106] In order to control the extent of pruning at each layer, the threshold τ is adjusted after pruning each Transformer layer so that:

[0107] τ=τ-δ#(26)

[0108] Among them, δ is a hyperparameter, which serves as an attenuation factor to control the amplitude of pruning at each layer and gradually adjust the proportion of visual token retention during the inference process.

[0109] If the current layer layer∈(21,31], the threshold τ is not updated, and the unnormalized attention matrix is not adjusted. The pruning strategy used in the 21st layer is continued. When the current layer layer=32, the pruning operation is stopped.

[0110] (2.7): Calculate the attention output O Attention ;

[0111] For the unnormalized attention matrix According to step (2.3), the new attention weight matrix A is calculated h , and then calculate the output O of the h-th attention head according to step (2.5) h .

[0112] The output O of all attention heads h Splicing is performed and linear transformation W is performed o Generate multi-head attention output O:

[0113] O=concat(O1,O2,…,O H )W o #(27)

[0114] Among them, W o is the projection weight matrix, which is used to transform the concatenated multi-head output back to the feature space of the model.

[0115] For the multi-head attention output O, a residual connection is further used after generation to add it to the multimodal input X obtained by splicing and calculating formula (5) to enhance information retention and gradient stability. Attention The mathematical expression is as follows:

[0116] O Attention =O+X#(28)

[0117] (2.8): Calculate the MLP transformation output O MLP ;

[0118] Attention output O Attention Perform RMS normalization to obtain the normalized attention output o norm , which is calculated as follows:

[0119]

[0120] Among them, γ is a trainable parameter used to adjust the output scale after normalization; i Represents the attention output O Attention The element in the i-th dimension; n represents O Attention The characteristic dimension of , that is, the dimension of its hidden layer, ∈ is a minimum constant.

[0121] After obtaining the normalized attention output O norm Afterwards, this value will be input into the Multi-Layer Perceptron (MLP) for feature transformation. The specific calculation process is as follows:

[0122]

[0123] Among them, W d is the weight matrix of the dimension reduction projection layer, which is used to map the features back to the original dimension size; W g is the gated projection layer weight matrix; W u is the weight matrix of the extended projection layer; σ SwiGLU is the activation function.

[0124] σ SwiGLU It is an activation function that combines the Swish activation function and the gating mechanism. It enhances the adaptability of the model to different features by introducing the gating mechanism. The specific calculation formula is:

[0125]

[0126] in, is the Swish activation function, W k is the weight matrix of the gating mechanism, b k is the bias term of the gating mechanism.

[0127] (2.9): Calculate the final output of the current layer;

[0128] For the MLP output O MLP , after generation, a residual connection is further used to connect it with the attention output O Attention Add. The final output of the current layer is X out The mathematical expression is as follows:

[0129] X out=O MLP +O Attention #(32)

[0130] (2.10): If the current layer layer = 32, then go to step (3); otherwise, the final output X of the current layer out Return to step (1.3) as the multimodal input X of the next layer and enter the calculation of the next layer.

[0131] (3) Generate the current token based on the attention distribution;

[0132] (3.1): Calculate the probability distribution of generated tokens;

[0133] The final output X of the 32nd layer out Perform RMS normalization to obtain the normalized result X norm , which is calculated as follows:

[0134]

[0135] Among them, γ is a trainable parameter used to adjust the output scale after normalization; X i Represents the attention output X out The element in the i-th dimension; n represents the feature dimension of X, that is, the dimension size of its hidden layer.

[0136] Tensor X after RMS normalization norm It needs to pass through a linear mapping layer to map it to the size of the model vocabulary. The specific calculation formula is as follows:

[0137] logit = X norm ×W proj +b proj #(34)

[0138] Among them, logit represents the prediction score of each token in the vocabulary, W proj is the weight matrix of the linear mapping layer, which is responsible for mapping the input tensor to the vocabulary size; b proj is a bias term used to adjust the mapped result.

[0139] Through the softmax function, logit is converted into a normalized probability distribution to generate the probability P(Token i ) is represented as follows:

[0140]

[0141] Where V is the size of the vocabulary, logit i and logit jRepresents the i-th and j-th elements of logit.

[0142] (3.2): Determine the generated Token;

[0143] Greedy Search algorithm is based on the generation probability P(Token i ), select the Token with the highest probability as the output.

[0144] (4) If the currently generated token is an EOS token or the preset maximum number of tokens generated has been reached, the loop ends and proceeds to step (5); otherwise, return to step (1.2) to continue generating the next token;

[0145] (5) Output the complete Token sequence in the generation order to construct the final generation result.

[0146] Use the above method to test, such as Figure 2 As shown, Figure 2 The left side is the visual input, and the LlaVA model is required to describe the visual input. Figure 2 The right side shows the output generated by the original LLaVA-v1.5 model and the output generated by the LLaVA-v1.5 model combined with the method of the present invention. The results show that the method of the present invention can effectively improve the output accuracy of the LLaVA model and successfully achieve hallucination suppression.

[0147] This example conducts an in-depth analysis based on the POPE and CHAIR tasks to evaluate the performance of different algorithms in hallucination suppression. The POPE task primarily measures the accuracy of a model in text generation tasks, while the CHAIR task focuses on assessing the degree of hallucination and content consistency during the description generation process.

[0148] Different algorithms were used to evaluate the hallucination suppression ability of POPE and CHAIR tasks. The results are shown in Table 1. The F1 score in the table is used to measure the accuracy of classification or information retrieval tasks. S It is used to measure the proportion of sentences containing hallucinated objects in all generated sentences. I The recall rate measures the proportion of hallucinated objects among all mentioned objects. The recall rate measures whether the target word or information mentioned in the generated description actually exists in the image. The output length represents the average length of the output generated by the model, with ↑ indicating higher values are better, and ↓ indicating lower values are better. Test results show that the method of the present invention performs best among all tested algorithms. Compared with existing methods, this method has significant advantages in reducing error generation and improving text quality, demonstrating excellent hallucination suppression capabilities, further demonstrating its reliability and practical value in multimodal tasks.

[0149] Table 1: Test results of each model on POPE and CHAIR tasks

[0150]

[0151]

[0152] Example 2

[0153] This embodiment also provides a visual language model hallucination suppression system based on adaptive dynamic attention intervention, which is used to implement the above-mentioned embodiment. The terms "module", "unit", etc. used below can be a combination of software and / or hardware that implements the predetermined function. Although the system described in the following embodiments is preferably implemented in software, it is also possible to implement it in hardware, or a combination of software and hardware.

[0154] The visual language model hallucination suppression system based on adaptive dynamic attention intervention includes:

[0155] The multimodal input preprocessing module receives image and text inputs, extracts visual feature tokens and text tokens respectively, and provides basic input for subsequent multimodal fusion;

[0156] The multimodal input construction module is used to construct a multimodal input sequence suitable for each layer of Transformer based on text input, visual tokens and historical responses;

[0157] A multi-layer Transformer inference module, which processes multimodal input layer by layer in the language model, enabling cross-modal information fusion and token-level generative reasoning.

[0158] The attention accumulation module is used to continuously count and update the self-attention and mixed attention scores of visual tokens during the multi-layer Transformer inference process;

[0159] The visual attention head control module is used to identify attention heads that are highly sensitive to visual information and enhance their attention weights;

[0160] Visual Token Pruning module, which dynamically identifies and prunes low-contribution visual tokens based on the accumulated attention score;

[0161] The token generation and response update module is used to predict and generate the next token based on the final layer output and append it to the historical response sequence for the next multimodal input;

[0162] The generation result output module is used to output the complete token sequence as the final result according to the token generation order after the generation process is completed.

[0163] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0164] Embodiments of the system of the present invention can be applied to any device with data processing capabilities, such as a computer or other device. System embodiments can be implemented through software, hardware, or a combination of software and hardware. For example, a software implementation, as a logical device, is implemented by a processor of any device with data processing capabilities, reading corresponding computer program instructions from non-volatile memory into internal memory and executing them.

[0165] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Persons skilled in the art will readily appreciate that variations and modifications may be made without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention, wherein the visual language model comprises an image encoder, a visual-language connector, and a language model. The language model is a pre-trained multi-layer Transfromer architecture, each layer of which comprises a multi-head self-attention module. The method comprises: S1: Gets text input and visual input processed by the image encoder and vision-language connector; S2: Initialize the self-attention accumulation vector and the mixed attention accumulation vector; S3: Iterate substeps S3.1-S3.4 layer by layer in the language model: S3.1: If the current layer is the first layer, the input of the current layer must be dynamically constructed based on whether there is a historical response. If there is no historical response, the text input and visual input are concatenated to obtain a multimodal input. If there is a historical response, the text input, visual input, and historical response are concatenated to obtain a multimodal input. The resulting multimodal input serves as the input of the current layer. If the current layer is not the first layer, the output of the previous layer is directly used as the input of the current layer; S3.2: Calculate the unnormalized attention matrix based on the multimodal input, update the self-attention accumulation vector and the mixed attention accumulation vector according to the unnormalized attention matrix, and calculate the output of each attention head of the multi-head self-attention module in the current layer; S3.3: If the current layer is the first or last layer, skip this step and go directly to S3.

4. If the current layer is the second to fifth layer, identify the visually sensitive attention heads based on the output of each attention head and adjust the unnormalized attention weights of the identified attention heads. If the current layer is after the fifth layer, perform pruning on the tokens corresponding to the visual input and adjust the unnormalized attention matrix based on the pruning results. S3.4: Normalize the adjusted unnormalized attention matrix and calculate the output of the current layer; S4: Generate a token based on the output of the last layer of the language model and add the generated token to the historical response; S5: Repeat steps S2-S4 based on the text input and the pruned visual input until the generated token is an EOS token or the preset maximum number of tokens generated is reached. The final token output sequence is obtained in the token generation order as the output of the visual language model.

2. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S2, the self-attention accumulation vector and the mixed attention accumulation vector are defined as follows: Among them, C self is the self-attention accumulation vector, C hybrid is the mixed attention accumulation vector, is the cumulative score of the i-th self-attention, is the cumulative score of the i-th mixed attention, L image Indicates the number of tokens in the visual input; Each time the attention accumulation vector is initialized, the attention accumulation score is 0.

3. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.1, the input of the current layer is determined based on whether a historical response has been generated: If no historical response has been generated, the text input and visual input are concatenated to form a multimodal input and used as the input of the current layer. The concatenation formula is: X=concat(X l [1:L system ],X V ,X I [L system +1:L text ]) If a historical response has been generated, the text input and the visual input are concatenated to form a multimodal input, which is used as the input of the current layer. The concatenation formula is: X=concat(X I [1:L system ],X V ,X I [L system +1:L text ],X H ) Among them, X is the multimodal input, concat(·) represents concatenation, X I [1:L system ] is the system token, L system is the number of system tokens, X I [L system +1: L text ] is the question token, L text The number of tokens for text input, X V is the visual input, X H A historical response.

4. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.2, the unnormalized attention matrix is calculated based on the multimodal input, specifically: Calculate the query matrix and key matrix of each attention head, and calculate the unnormalized attention matrix through the query matrix and key matrix. The calculation formula is: Where X is the multimodal input, are the query and key weight matrices of the h-th attention head respectively; is the unnormalized attention matrix of the h-th attention head, d k Represents the query matrix Q h and bond matrix K h dimension.

5. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.2, the self-attention accumulation vector, the mixed attention accumulation vector, and the output of each attention head of the current layer are calculated according to the unnormalized attention matrix, specifically: Normalize the unnormalized attention matrix, calculate the average attention distribution matrix through the normalized attention matrix, decompose the average attention distribution matrix into the self-attention matrix and the mixed attention matrix, and calculate the self-attention accumulation vector and the mixed attention accumulation vector based on the self-attention matrix and the mixed attention matrix. The calculation formula is: M self =M[L system :L system +L image ,L system :L system +L image ] M hybrid =M[:,L system :L system +L image ] Among them, A h is the normalized attention weight matrix of the hth attention head, softmax(·) is the s0ftmax operation, M is the average attention distribution matrix, H represents the total number of attention heads in the current layer, and M self and M hybrid are the self-attention matrix and the mixed attention matrix, L system is the number of system tokens in the text input, L image is the number of tokens input by the visual input, is the jth element in the self-attention accumulation vector, i.e., the self-attention accumulation score, is the jth element in the mixed attention accumulation vector, i.e., the mixed attention accumulation score; Apply the normalized attention weight matrix to the value matrix to get the output of the attention head, which is: About h =A h In h Among them O h is the output of the h-th attention head.

6. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.3, the visually sensitive attention head is identified based on the output of each attention head, and the unnormalized attention weight of the identified attention head is adjusted. The specific formula is: VDS={h|VAS h <μ(VAS)+σ(VAS),LAS h <μ(LAS)+σ(LAS)} Among them, VAS h is the visual sensitivity of the h-th attention head, O h is the output of the h-th attention head, is the output of the h-th attention head without visual information input, LAS h is the language sensitivity of the h-th attention head, ||·||2 is the Euclidean norm, VDS is the set of visually sensitive attention heads, μ(VAS) and μ(LAS) are the means of the visual sensitivity and language sensitivity of all attention heads in the current layer, σ(VAS) and σ(LAS) are the standard deviations of the visual sensitivity and language sensitivity of all attention heads in the current layer, respectively. is the unnormalized attention matrix, n is the last row index of the matrix, is the element in the jth column of the last row of the unnormalized attention matrix, α is a hyperparameter, T vision It is the set of all Tokens in the visual input.

7. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.3, the pruning operation on the tokens in the visual input is performed in two cases: When the current layer is the first half of the remaining layers after the 5th layer, the specific steps are: Each token in the visual input is sorted twice in descending order according to the corresponding element values in the self-attention accumulation vector and the mixed attention accumulation vector, respectively, to obtain two sets of rearranged visual tokens. The adaptive pruning strategy is used to calculate the visual tokens to be pruned based on the rearranged visual tokens, and the visual tokens to be pruned are removed from the visual input. The specific formula of the adaptive pruning strategy is: D purning =D hybrid ∩D self Among them, p self 、p hybrid is the number of visual tokens to be retained, is a set of positive integers, is the jth element in the self-attention accumulation vector, is the jth element in the mixed attention accumulation vector, L image is the number of visual input tokens, τ is the threshold, and the initial value of the threshold τ is 0; D hybrid and D self is the mixed attention pruning set and the self-attention pruning set, D purning is the visual Token set that needs to be pruned in the current layer. is the jth visual Token after rearrangement according to the self-attention accumulation vector, is the jth visual Token after rearrangement according to the mixed attention accumulation vector; After pruning, the last row of the unnormalized attention matrix corresponds to the set D purning The elements of each visual token in are set to negative infinity, and the threshold τ is reduced at the same time; When the current layer is the first half of the remaining layers after the 5th layer, the token order, unnormalized attention matrix and threshold τ in the visual input are not changed, and the adaptive pruning strategy is used to prune the visual input.

8. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S3.4, the adjusted unnormalized attention matrix is normalized using a softmax operation, and the output of the current layer is obtained based on the adjusted normalized attention matrix. The specific formula is: About h =A h In h O=concat(O1,O2,…,O H )W O The Attention =O+X O norm =RMS(O Attention ) O MLP =MLP(O norm ) X out =O MLP +O Attention Among them, O h is the output of the h-th attention head, A h is the normalized attention weight matrix, V h is the value matrix of the h-th attention head, O is the multi-head attention output, W O is the projection weight matrix, H is the number of attention heads, concat(·) is the concatenation operation, RMS(·) is the RMS normalization, O norm is the normalized attention output, O MLP is the MLP output, MLP(·) is the multi-layer perceptron, X out is the final output of the current layer.

9. The method for suppressing hallucinations in a visual language model based on adaptive dynamic attention intervention according to claim 1, characterized in that: In S4, a token is generated based on the output of the last layer of the language model, specifically: The final output X of the last layer out Perform RMS normalization to obtain the normalized tensor X norm ; Then normalize the tensor X norm The size of the vocabulary of the visual language model is mapped to obtain the prediction score of each token in the vocabulary. The prediction score is normalized by softmax to obtain the generation probability, and the token with the highest generation probability is selected as the token generated in this iteration.

10. A visual language model hallucination suppression system based on adaptive dynamic attention intervention, used to implement the hallucination suppression method according to claim 1, characterized in that: The system includes: The multimodal input preprocessing module receives image and text inputs, extracts visual feature tokens and text tokens respectively, and provides basic input for subsequent multimodal fusion; The multimodal input construction module is used to construct a multimodal input sequence suitable for each layer of Transformer based on text input, visual tokens and historical responses; A multi-layer Transformer inference module, which processes multimodal input layer by layer in the language model, enabling cross-modal information fusion and token-level generative reasoning. The attention accumulation module is used to continuously count and update the self-attention and mixed attention scores of visual tokens during the multi-layer Transformer inference process; The visual attention head control module is used to identify attention heads that are highly sensitive to visual information and enhance their attention weights; Visual Token Pruning module, which dynamically identifies and prunes low-contribution visual tokens based on the accumulated attention score; The token generation and response update module is used to predict and generate the next token based on the final layer output and append it to the historical response sequence for the next multimodal input; The generation result output module is used to output the complete token sequence as the final result according to the token generation order after the generation process is completed.

Images