A visual localization backtracking method for alleviating the illusion of multimodal large language models

Abstract

This application discloses a visual localization backtracking method for mitigating the illusion of multimodal large language models, relating to the fields of artificial intelligence and multimodal information processing. The method acquires visual input and text input as the current text cue, constructs original visual representations and text representations, and monitors the output uncertainty of the target decoding layer during autoregressive generation. When the output uncertainty meets a preset trigger condition, a cue-related editing signal is obtained based on the hidden representation of the current generation position. The cue-related editing signal and the original visual representation are input into a visual localization module to determine the editing intensity of each visual token and generate a target visual representation. The target visual representation is introduced into the decoder feedforward network, the visual backtracking output is calculated, and fused with the original feedforward output to obtain an updated feedforward output. Based on the updated feedforward output, the autoregressive generation of text results continues. This method can reduce the back-injection of irrelevant visual information and improve the consistency between the generated results and visual facts.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a visual localization backtracking method for alleviating the illusion of multimodal large language models. Background Technology

[0002] Multimodal large language models typically use a visual encoder, a visual-text alignment module, and a language decoder to jointly process visual and text inputs, enabling the model to generate natural language results in tasks such as image description, visual question answering, scene understanding, human-computer interaction, and decision support. These models generally first convert images, image frames, or other visual data into visual representations, then input the visual and text representations together into a decoder, which gradually generates the text result using an autoregressive approach. However, multimodal large language models are prone to the "illusion problem" during generation, where the generated content is inconsistent with the input visual reality. For example, the model might describe objects that do not exist in the image, incorrectly determine object attributes, or make statements about spatial relationships between objects that do not conform to the image's reality. This problem reduces the reliability of the model's output, particularly affecting its application in scenarios such as medical assistance, robot navigation, industrial inspection, intelligent question answering, and automated decision support.

[0003] Existing solutions for mitigating the illusion of multimodal large language models mainly include retrieval enhancement, additional fine-tuning, attention intervention, contrastive decoding, and visual backtracking. Among these, retrieval enhancement typically relies on external knowledge bases or additional retrieval processes, resulting in high system deployment complexity; additional fine-tuning requires building training data and updating model parameters, leading to high training and transfer costs; while attention intervention and contrastive decoding can enhance the role of visual information to some extent, they usually introduce additional inference overhead and may affect the model's original generative capabilities.

[0004] Visual backtracking schemes can reintroduce visual information into the model's internal reasoning process when model generation is uncertain or lacks sufficient visual dependence, thereby supplementing visual factual evidence and reducing the risk of hallucinations. However, existing visual backtracking schemes often focus more on whether to perform visual back-injection, while failing to adequately distinguish the relevance between the back-injected visual information and the current text prompt. If a complete visual representation or a large range of visual tokens is directly back-injected into the model, it is easy to introduce visual information irrelevant to the current question, resulting in irrelevant visual noise interfering with the subsequent generation process. In addition, in the model's autoregressive generation process, the visual regions focused on by different generation steps are not the same. For example, when the text prompt asks about object category, object color, or spatial relationship, the visual evidence that the model actually needs to backtrack may correspond to different visual tokens. If the visual backtracking process cannot dynamically determine the relevant visual tokens based on the current generation position and the current text prompt, it is difficult to balance the hallucination suppression effect with the model's original general capabilities. On the other hand, if visual localization is based solely on the original hidden state or attention distribution, the visual localization results may also be affected by linguistic prior bias, resulting in visual tokens that are truly relevant to the current visual facts not being accurately reinforced.

[0005] Therefore, it is necessary to propose a new visual localization backtracking scheme to target visual tokens based on the current text prompts and the current generation state during the multimodal large language model generation process, and to introduce the target visual representation that is more relevant to the current generation process into the decoder feedforward network, thereby reducing the interference of irrelevant visual noise on the generation results and improving the consistency between the model output and the input visual facts. Summary of the Invention

[0006] This application provides a visual localization backtracking method for alleviating hallucinations in multimodal large language models, in order to solve the problems in existing visual backtracking schemes that directly inject complete visual information, making it difficult to filter relevant visual evidence based on the current text prompts, easily introducing irrelevant visual noise, and affecting the balance between hallucination suppression effect and the original general capabilities of the model.

[0007] To address the aforementioned issues, this application provides a visual localization backtracking method for mitigating the illusion of multimodal large language models. The method includes: acquiring visual input and text input as the current text prompt, constructing original visual representations and text representations; inputting the original visual representations and text representations into the decoder of the multimodal large language model, and monitoring the output uncertainty of the target decoding layer during autoregressive generation; when the output uncertainty meets a preset trigger condition, obtaining a prompt-related editing signal based on the hidden representation corresponding to the current generation position in the target decoding layer; inputting the prompt-related editing signal and the original visual representation into a visual localization module, which determines the editing intensity of each visual token relative to the current text prompt; weighting the original visual representation based on the editing intensity to generate a target visual representation; introducing the target visual representation into the computation process of the decoder's feedforward network, obtaining a visual backtracking output based on the current hidden state of the decoder at the current generation position and the target visual representation, and fusing the visual backtracking output with the original feedforward output obtained by the decoder's feedforward network before the target visual representation was injected, to obtain an updated feedforward output; and continuing autoregressive generation based on the updated feedforward output to obtain the text result.

[0008] Furthermore, monitoring the output uncertainty of the target decoding layer includes: obtaining the probability distribution of the next word corresponding to the target decoding layer on the vocabulary, calculating the entropy value or normalized entropy value based on the probability distribution, and determining that the output uncertainty meets the preset triggering condition when the entropy value or normalized entropy value is greater than a preset threshold.

[0009] Furthermore, obtaining the cue-related editing signal based on the hidden representation corresponding to the current generation position in the target decoding layer includes: extracting the hidden state vector corresponding to the current generation position in the target decoding layer as the cue-related editing signal; or, performing a linear or nonlinear mapping on the hidden state vector corresponding to the current generation position in the target decoding layer to obtain the cue-related editing signal. Here, the current generation position is the last position of the current input sequence.

[0010] Furthermore, the visual localization module includes a first mapping sub-network, a second mapping sub-network, an interactive computing unit, an intensity constraint unit, and a visual weighting unit. The first mapping sub-network is used to perform feature mapping on the original visual representation, the second mapping sub-network is used to perform feature mapping on the cue-related editing signals, the interactive computing unit is used to calculate the editing intensity of each visual token based on the mapped original visual representation and the mapped cue-related editing signals, the intensity constraint unit is used to constrain the editing intensity to a preset numerical range, and the visual weighting unit is used to generate the target visual representation based on the editing intensity.

[0011] Furthermore, the interactive computing unit calculates the correlation between each visual token and the cue-related editing signal through at least one of dot product matching, bilinear matching, attention matching, or nonlinear matching based on multilayer perceptron, and obtains the editing intensity of each visual token based on the correlation.

[0012] Furthermore, the intensity constraint unit limits the editing intensity of each visual token to a preset numerical range by using at least one of the following methods: sigmoid function, softmax function, normalization function, or truncation constraint based on a preset threshold.

[0013] Further, the original visual representation is weighted based on the edit intensity to generate a target visual representation, including: multiplying the original visual representation of each visual token by the corresponding edit intensity to obtain a weighted visual representation, and using the weighted visual representation as the target visual representation.

[0014] Furthermore, the target visual representation is introduced into the computation process of the decoder feedforward network, including: calculating the visual backtracking output based on the matching response between the current hidden state of the decoder at the current generation position and the target visual representation; and weighting and fusing the visual backtracking output with the original feedforward output according to the preset injection ratio parameters to obtain the updated feedforward output.

[0015] Furthermore, the range of the preset injection ratio parameter is [0, 1]; when the output uncertainty does not meet the preset triggering condition, the original feedforward output remains unchanged.

[0016] Furthermore, this visual localization backtracking method also includes training the visual localization module. During training, the backbone parameters of the multimodal large language model are frozen, and the target visual representation generated during the training phase is introduced into the decoder feedforward network through an auxiliary injection path to obtain the training editing signal for training the visual localization module. The visual token representation corresponding to the target decoding layer is perturbed, and the contribution label of each visual token representation is determined based on the change in the attention output of the target decoding layer before and after the perturbation, so that the editing intensity distribution output by the visual localization module approximates the contribution label.

[0017] The above technical solution has at least the following beneficial effects: by performing visual backtracking when the output uncertainty meets the preset triggering conditions, the model introduces visual supplementary information only in the generation stage where there is a high risk of hallucination or insufficient visual dependence, avoiding indiscriminate intervention in the entire generation process; by obtaining the prompt-related editing signal based on the hidden representation corresponding to the current generation position of the target decoding layer, the visual localization process can combine the current text prompt and the current generation state; by determining the editing intensity of each visual token through the visual localization module and generating the target visual representation based on the editing intensity, the visual information injected back to the decoder feedforward network is more focused on the visual evidence required by the current generation process, thereby reducing the interference of irrelevant visual noise on the language generation path and improving the consistency between the model output and the input visual facts.

[0018] The above technical solutions also have the following advantages: triggering visual backtracking by using entropy or normalized entropy can improve the computability and interpretability of trigger judgment; co-calculating editing intensity by using a mapping subnetwork, interactive computing unit, strength constraint unit, and visual weighting unit can improve the stability of the visual token selection process; fusing visual backtracking output with original feedforward output by injecting proportional parameters or dynamic gating parameters can enhance visual factual evidence while preserving the model's original language generation capabilities; freezing backbone parameters and training the visual localization module can reduce model modification costs; and perturbing the visual token representation and using changes in attention output to construct contribution labels can improve the learning effect of the visual localization module on relevant visual evidence. Attached Figure Description

[0019] Figure 1 A schematic diagram of the overall process of the visual positioning backtracking method provided in this application; Figure 2 A schematic diagram of the fusion structure of the visual positioning module and the feedforward network provided in this application; Figure 3 This application provides a schematic diagram of the output uncertainty triggering and injection control process; Figure 4 A schematic diagram of the training process for the visual positioning module provided in this application. Detailed Implementation

[0020] The content of this application will be further described in detail below with reference to specific embodiments.

[0021] In the description of this application, the terms "first," "second," etc., are used only to distinguish identical or similar objects, components, steps, or data, and are not used to limit the order, quantity, or importance. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, module, or apparatus that comprises a list of elements may include not only the expressly listed elements but also other elements not expressly listed, or elements inherent to the process, method, module, or apparatus. The term "acquire" can be understood as receiving, reading, calling, collecting, extracting, or generating corresponding data from existing data, not limited to receiving directly from an external source. The term "based on" can be understood as processing at least partially based on the corresponding object, and does not exclude processing based on other objects. The term "preset" can be understood as determined before the corresponding step is performed, or dynamically determined during model operation based on configuration parameters, historical data, or the current state.

[0022] In the description of this application, the term "illusion" refers to the phenomenon where the text result generated by a multimodal large language model is inconsistent with the visual facts expressed by the visual input, such as generating objects that do not exist in the visual input, incorrectly describing object attributes, incorrectly describing the number of objects, or incorrectly describing spatial relationships. The term "current text prompt" refers to the text input, question, instruction, or dialogue context upon which the model relies in the current generation process. This can include user-inputted text content or contextual text content that has already been generated and participates in subsequent predictions. The term "visual token" refers to a visual representation unit formed after visual encoding or visual text alignment of the visual input. A visual token can correspond to an image patch, image region, local visual feature, or other visual feature unit. The term "original visual representation" refers to the visual representation of the visual input after visual encoding or visual text alignment, but before being weighted or filtered based on editing intensity. The term "target visual representation" refers to the visual representation obtained after weighting, filtering, or combining the original visual representation according to editing intensity. The target visual representation is more focused on visual evidence related to the current text prompt compared to the original visual representation.

[0023] The term "target decoding layer" refers to the decoding layer selected in the decoder for monitoring output uncertainty, extracting hidden representations, or performing visual backtracking. The target decoding layer can be a single decoding layer or any of multiple candidate decoding layers. The term "current generation position" refers to the position of the currently generated lexical unit during autoregressive generation. In a decoding structure using causal masks, the current generation position can be the last position of the current input sequence. The term "current hidden state" refers to the hidden state of the decoder at the current generation position. The term "output uncertainty" refers to an indicator characterizing the degree of uncertainty of the target decoding layer's prediction of the next lexical unit, such as the entropy value or normalized entropy value of the probability distribution of the next lexical unit on the vocabulary. The term "preset trigger condition" refers to the condition used to determine whether to perform visual localization backtracking, such as when the output uncertainty exceeds a preset threshold.

[0024] The term "cue-related editing signal" refers to a signal obtained based on the hidden representation corresponding to the current generation position in the target decoding layer, used to guide the visual localization module to determine relevant visual tokens. This cue-related editing signal can characterize the semantic requirements of the current text cue and the current generation state. The term "edit intensity" refers to a numerical value used to characterize the importance or relevance of a visual token relative to the current text cue. The term "edit intensity distribution" refers to the overall distribution of the edit intensities corresponding to multiple visual tokens. The term "visual backtracking output" refers to the supplementary visual information output calculated based on the current hidden state and the target visual representation. The term "original feedforward output" refers to the output obtained by the decoder feedforward network based on the current hidden state without introducing the target visual representation. The term "updated feedforward output" refers to the output obtained after fusing the visual backtracking output and the original feedforward output. The terms "injection ratio parameter" and "dynamic gating parameter" are both used to control the degree to which the visual backtracking output participates in the feedforward output fusion. The injection ratio parameter can be a pre-set fixed parameter, while the dynamic gating parameter can be dynamically determined based on output uncertainty, editing intensity distribution, the layer number of the target decoding layer, or the current autoregressive generation step sequence. The term "auxiliary injection path" refers to the auxiliary computational path used during the training phase to introduce the target visual representation into the decoder feedforward network. The term "contribution label" refers to the supervision label determined based on the change in the attention output of the target decoding layer before and after perturbation of the visual token representation, used to characterize the degree of contribution of the visual token to the current text prompt or the current generation process.

[0025] This application provides a visual localization backtracking method for mitigating the illusion of multimodal large language models. This method can be applied to multimodal generative models capable of receiving visual and text inputs and generating text results. Visual input can be images, video frames, or other data that can be visually encoded to obtain visual tokens; text input can be questions, instructions, descriptive prompts, or dialogue context. The model may include a visual encoder, a visual-text alignment module, and a decoder. The visual encoder is used to extract features from the visual input, the visual-text alignment module is used to map the visual features to an embedding space compatible with the language model, and the decoder is used to generate text results based on the visual and text representations in an autoregressive manner.

[0026] When executing the method of the above embodiments, visual input and text input as the current text prompt are first acquired. The visual input is processed by a visual encoder to obtain visual features, and then processed by a visual-text alignment module to obtain the original visual representation; the text input is processed by word segmentation and embedding to obtain the text representation. The original visual representation and text representation can be concatenated in a preset order, or they can be fused according to the input format supported by the multimodal large language model, and then jointly input into the decoder. The original visual representation is a visual representation that has not yet been weighted or filtered by prompt-related editing signals, and it includes visual token representations corresponding to multiple visual tokens.

[0027] Figure 1 This is a schematic diagram illustrating the overall process of the visual positioning backtracking method provided in this application. Figure 1 As shown, in S101, visual input and text input as the current text prompt are acquired to construct the original visual representation and text representation. In S102, the original visual representation and text representation are input into the decoder of the multimodal large language model. In S103, the output uncertainty of the target decoding layer is monitored during the autoregressive generation process. In S104, it is determined whether the output uncertainty meets the preset triggering condition. If the output uncertainty meets the preset triggering condition, S105 to S109 are executed; if the output uncertainty does not meet the preset triggering condition, S110 is executed, keeping the original feedforward output unchanged and continuing the autoregressive generation.

[0028] In some embodiments, the target decoding layer can be a preset layer in the decoder or any layer among multiple candidate decoding layers. The target decoding layer can be determined based on the model structure, inference stage, generation sequence, or experimental calibration results. By monitoring output uncertainty at the target decoding layer, visual localization backtracking can be triggered when the model's prediction of the next token is unstable or visual dependence is insufficient, instead of continuously introducing visual backtracking intervention throughout the generation process, thereby reducing unnecessary computational overhead and the impact on the original generation path.

[0029] In one specific implementation, the output uncertainty can be calculated using the probability distribution of the next word in the vocabulary corresponding to the target decoding layer. Let the probability distribution of the next word in the vocabulary corresponding to the target decoding layer be... The vocabulary size is The output uncertainty can then be expressed using normalized entropy as:

[0030] in, This indicates output uncertainty. Indicates the first The predicted probability corresponding to each word element This indicates the vocabulary size. A larger normalized entropy indicates greater uncertainty in the target decoding layer's prediction of the next word. It can be... Compared with a preset threshold, when When the output uncertainty exceeds a preset threshold, the output uncertainty is determined to meet a preset trigger condition; when If the output uncertainty is not greater than a preset threshold, it is determined that the preset triggering condition is not met. In other implementations, the entropy value, the maximum value among multiple candidate layer uncertainties, the change in uncertainty of multiple consecutive generation steps, or a combination thereof can be used directly to determine whether the preset triggering condition is met.

[0031] Furthermore, in a more specific parameter configuration example, the output uncertainty can be calculated based on the top-10 logits of the target decoding layer output. Specifically, the top 10 logits with the largest values ​​can be selected from the next word prediction results corresponding to the target decoding layer, and softmax normalization can be performed on these top 10 logits to obtain the probability distribution of the top-10 candidate words. The normalized entropy can then be calculated based on this probability distribution. Let the first of the top 10 logits be... One logit is Then the normalized probability It can be represented as:

[0032] Furthermore, the normalized entropy calculated based on the probability distribution of the top-10 candidate lexical units can be expressed as:

[0033] in, This represents the normalized entropy calculated based on the top-10 logits. This indicates the th logits after softmax normalization. The probability of each candidate word. The larger the value, the more uncertain the target decoding layer's prediction of the next word. By using the normalized entropy corresponding to the top-10 logits, the output uncertainty of the target decoding layer can be quantified while reducing the computational overhead of the entire vocabulary. It should be understood that the aforementioned method of calculating normalized entropy based on top-10 logits is a specific implementation of the aforementioned method of calculating output uncertainty based on vocabulary probability distribution, and does not exclude the calculation of output uncertainty based on the probability distribution of the entire vocabulary or other candidate word sets.

[0034] In one specific implementation, the preset triggering condition may include multiple sub-conditions, and visual localization backtracking processing is triggered when multiple sub-conditions are simultaneously met. Specifically, if the normalized entropy of the top-10 logits of the current target decoding layer is greater than a preset entropy threshold, visual backtracking injection has not been triggered during this forward propagation, the layer number of the current target decoding layer is strictly greater than a preset starting layer number, and the layer number of the current target decoding layer is strictly less than a preset ending layer number, then the output uncertainty is determined to meet the preset triggering condition. The above preset triggering condition can be expressed as:

[0035]

[0036]

[0037]

[0038] in, This represents the normalized entropy of the top-10 logits of the current target decoding layer. This represents the preset entropy threshold. This indicates whether visual backtracking injection has been triggered during this forward propagation process. This indicates that visual backtracking injection has not been triggered yet. Indicates the layer number of the current target decoding layer. Indicates the preset starting layer number. Indicates the preset end layer number. This can be set via... This can limit visual backtracking injection to only one time during a single forward propagation, avoiding excessive intervention caused by repeated injections; by setting... and This allows visual backtracking injection to be limited to a preset layer range, avoiding feature disturbances caused by injection at too early or too late layers.

[0039] In a set of example parameter configurations, the preset starting layer number can be configured. Set to 5, and set the preset end layer number. Set it to 18, and set the preset entropy threshold. The value is set to 0.75. That is, visual localization backtracking is triggered when the current target decoding layer's layer number is greater than 5 and less than 18, the normalized entropy of the current target decoding layer's top-10 logits is greater than 0.75, and visual backtracking injection has not been triggered during this forward propagation. The above values ​​are merely an example; those skilled in the art can adjust the preset entropy threshold, preset starting layer number, and preset ending layer number based on the model's layer count, vocabulary size, number of visual tokens, task type, or illusion suppression effect on the validation set.

[0040] Furthermore, at the start of the forward propagation corresponding to each autoregressive generation step, a visual backtracking trigger flag can be set. Initialize to When any target decoding layer triggers visual backtracking injection, it will Updated to This prevents subsequent decoding layers in the same forward propagation from repeatedly triggering visual backtracking injection. Therefore, visual backtracking processing can be limited to triggering at most once per forward propagation, avoiding excessive intervention caused by repeatedly injecting visual information into multiple decoding layers.

[0041] In one specific implementation, the decoder can perform forward computation layer by layer, and for layers with layer numbers greater than [a certain value], [the following applies]. and less than The candidate decoding layers are sequentially evaluated for output uncertainty. When any candidate decoding layer meets a preset trigger condition, it can be used as the target decoding layer, and the relevant editing signal is obtained based on the hidden representation corresponding to the current generation position in the target decoding layer. If there is no candidate decoding layer that meets the preset trigger condition within the preset layer range, the visual positioning backtracking process is not performed, and the original feedforward output remains unchanged.

[0042] In step S105, when the output uncertainty meets a preset trigger condition, a cue-related editing signal is obtained based on the hidden representation corresponding to the current generation position. The current generation position can be the last position of the current input sequence, and the hidden state output by the target decoding layer at this position can aggregate the semantic information of the current visible context, the current text cue, and the generated text. Therefore, the hidden state vector corresponding to the current generation position in the target decoding layer can be extracted as the cue-related editing signal. The cue-related editing signal is used to guide the subsequent visual localization module to identify visual tokens related to the current text cue and the current generation state.

[0043] In some embodiments, using the hidden state vector corresponding to the current generation position directly as the cue-related editing signal can reduce additional computation. If it is necessary to enhance its matching ability with the visual representation, the hidden state vector can be linearly mapped, nonlinearly mapped, or normalized to obtain the cue-related editing signal. For example, the hidden state vector corresponding to the current generation position can be denoted as... , It can be used directly as a prompt-related editing signal, or it can be used as a prompt-related editing signal after being mapped by a multilayer perceptron. In this way, the visual localization process no longer relies solely on static visual information, but can combine the semantic requirements of the current text prompt and the current generation step.

[0044] In step S106, the relevant editing signals and the original visual representation are input into the visual localization module to determine the editing intensity of each visual token. Editing intensity characterizes the importance or relevance of each visual token relative to the current text prompt. If the current text prompt asks for the color of an object in an image, the visual token corresponding to the area where that object is located can have a higher editing intensity, while visual tokens corresponding to background areas or irrelevant objects can have a lower editing intensity. This method allows for prompt-related filtering and weighting of the original visual representation before visual backtracking, reducing the interference of irrelevant visual tokens on the subsequent generation process.

[0045] Figure 2 This is a schematic diagram of the fusion structure of the visual positioning module and the feedforward network provided in this application. Figure 2 As shown, the visual localization module may include a first mapping sub-network, a second mapping sub-network, an interactive computing unit, an intensity constraint unit, and a visual weighting unit. The original visual representation is input to the first mapping sub-network, and the relevant editing signals are input to the second mapping sub-network. The first mapping sub-network maps the original visual representation to a preset feature space, and the second mapping sub-network maps the relevant editing signals to the same feature space or a feature space that matches that feature space. The interactive computing unit calculates the correlation of each visual token based on the mapped original visual representation and the mapped relevant editing signals, and obtains the editing intensity based on this correlation. The intensity constraint unit applies numerical constraints to the editing intensity, and the visual weighting unit weights the original visual representation according to the constrained editing intensity to obtain the target visual representation.

[0046] In one specific implementation, let the original visual representation be: The relevant editing signal is indicated. The first mapping subnetwork is The second mapping subnetwork is The strength constraint function is Then the edit strength corresponding to each visual token can be expressed as:

[0047] in, This indicates the edit strength corresponding to multiple visual tokens. This represents the features of the original visual representation after being mapped by the first mapping sub-network. This indicates the characteristics of the relevant editing signals after being mapped by the second mapping sub-network. The function can be a sigmoid function, a softmax function, a normalization function, or a truncation constraint function based on a preset threshold. The interactive computing unit is not limited to dot product matching; it can also use bilinear matching, attention matching, or nonlinear matching based on a multilayer perceptron. Through the first and second mapping sub-networks, visual token representations and related editing signals can be matched within the same semantic space, improving the stability of editing intensity calculation.

[0048] In one specific implementation, the visual localization module can be implemented using a VGMMapper structure. The input dimension of the VGMMapper can be 4096, consistent with the hidden layer dimension of a multimodal large language model; the intermediate dimension can be 1024; and the number of multi-head cross-attention heads can be 8. The VGMMapper can sequentially include LayerNorm, BaseResMapper, L2 normalization, multi-head cosine matching, and temperature scaling. Specifically, BaseResMapper can include two multilayer perceptron layers and residual connections for nonlinear mapping of input features; L2 normalization normalizes the mapped visual features and cue-related editing signals; multi-head cosine matching calculates the matching relationship between visual tokens and cue-related editing signals across multiple attention heads; and temperature scaling adjusts the distribution range of the matching scores.

[0049] In a set of example configurations, the temperature scaling parameter The initial value can be It is approximately 2.303; The value range can be limited to [0.1, 100.0]; the scaling factor in multi-head matching can be... When the hidden layer dimension is 4096 and the number of attention heads is 8, =4096 / 8=512, the scaling factor is approximately =22.63. The above parameters are only an example configuration. Those skilled in the art can adjust the input dimension, intermediate dimension, number of attention heads, and temperature scaling parameters of VGMMapper according to the hidden layer dimension, number of visual tokens, and computational resource constraints of the multimodal large language model.

[0050] In S107, a target visual representation is generated based on the edit intensity-weighted original visual representation. In one specific implementation, let the first... The original visual representation corresponding to each visual token is: , No. The edit strength corresponding to each visual token is: Then the first The target visual representation corresponding to each visual token can be represented as:

[0051] in, Indicates the first A weighted visual representation of a set of visual tokens. Multiple weighted visual token representations can be combined to form the target visual representation. The target visual representation is more focused on visual evidence related to the current text prompt compared to the original visual representation. In other implementations, visual tokens can be filtered based on editing intensity first, and then the visual token representations that meet the preset intensity threshold can be used to form the target visual representation; alternatively, the target visual representation can be formed by selecting the top preset number of visual token representations according to the sorting results from high to low editing intensity. The weighting method, threshold filtering method, and sorting filtering method described above can be used individually or in combination.

[0052] In S108, the target visual representation is introduced into the computation process of the decoder feedforward network. Based on the current hidden state and the target visual representation, a visual backtracking output is obtained, and this output is fused with the original feedforward output to obtain an updated feedforward output. For example... Figure 2 As shown, the target visual representation and the current hidden state are jointly input into the visual backtracking calculation unit. The visual backtracking calculation unit calculates the visual backtracking output based on the matching response between the current hidden state and the target visual representation. The visual backtracking output is input into the fusion unit, and the original feedforward output is also input into the fusion unit. The fusion unit fuses the visual backtracking output and the original feedforward output according to the injection ratio parameter or dynamic gating parameter, and outputs the updated feedforward output.

[0053] In one specific implementation, let the current hidden state of the decoder at the current generation position be: The target visual representation is Then the visual backtracking output can be represented as:

[0054] in, This indicates visual backtracking output. This refers to the activation function, matched response function, or normalized weight function. Indicates the current hidden state and the first The matching relationship between the visual representations of the target. Through this calculation method, the current hidden state can obtain supplementary visual information related to the current generation process from the visual representations of the target.

[0055] In one specific implementation, the matching response function can employ a softmax normalization function. Specifically, the inner product similarity between the current hidden state and each target visual representation can be calculated first, and then softmax normalization can be applied to multiple inner product similarities to obtain the matching weights corresponding to each target visual representation. Subsequently, based on these matching weights, multiple target visual representations are weighted and summed to obtain the visual backtracking output. Thus, the visual backtracking output can extract visual supplementary information more relevant to the current generation position from the target visual representations according to the current hidden state.

[0056] Furthermore, the dimension of the visual backtracking output can be consistent with the dimension of the original feedforward output of the decoder feedforward network. If the dimensions of the visual backtracking output and the original feedforward output are different, a linear mapping layer can be used to map the visual backtracking output to the same hidden layer dimension as the original feedforward output, and then the mapped visual backtracking output and the original feedforward output can be weighted and fused. Through the above dimension alignment process, it can be ensured that the visual backtracking output and the original feedforward output can be fused in the same feature space.

[0057] Furthermore, let the first Layered feedforward networks rely on the current hidden state when no visual representation of the target is introduced. The resulting raw feedforward output is The injection ratio parameter is Then the fused feedforward output can be expressed as:

[0058] in, This represents the updated feedforward output after introducing the target visual representation, denoted by "". "The visual representation of the target is introduced into the feedforward network computation process corresponding to the current hidden state, and it is not limited to whether there is a mathematical proportional relationship between the two." Used to control the proportion of visual backtracking output in the updated feedforward output. The larger the value, the stronger the impact of the visual retrospective output on the subsequent generation process; The smaller the value, the stronger the influence of the original feedforward output on the subsequent generation process. By fusing the visual backtracking output with the original feedforward output, rather than completely replacing the original feedforward output with the visual backtracking output, we can enhance the visual factual basis while preserving the model's original language generation capabilities.

[0059] In a set of example parameter configurations, the preset injection ratio parameter is... This can be set to 0.12, allowing the visual backtracking output to participate in feedforward output fusion at a rate of 12%, while the original feedforward output participates at a rate of 88%. In this case, the updated feedforward output can be represented as:

[0060] Here, 0.12 is an example value for the preset injection ratio parameter. The preset injection ratio parameter can be in the range of 0 to 1. The larger the value, the higher the proportion of the visual backtracking output in the updated feedforward output; the smaller the value, the higher the proportion of the original feedforward output in the updated feedforward output. Those skilled in the art can adjust the preset injection ratio parameter according to the model size, the target decoding layer position, the type of generation task, or the illusion suppression effect on the validation set.

[0061] In S109, autoregressive generation continues based on the updated feedforward output until a termination marker is generated or a preset generation length is reached, yielding the text result. Since the updated feedforward output has already incorporated the target visual representation related to the current text prompt, the subsequent generation process can obtain more concentrated visual evidence, thereby reducing the risk of inconsistency between the generated content and the visual input facts.

[0062] In S110, if the output uncertainty does not meet the preset triggering condition, the original feedforward output remains unchanged and autoregressive generation continues. Through this branch, the model can use the original generation path in the relatively stable generation stage, avoiding indiscriminate visual backtracking intervention on the entire generation process.

[0063] Figure 3 This is a schematic diagram illustrating the output uncertainty triggering and injection control process provided in this application. (For example...) Figure 3 As shown, in S201, the probability distribution of the next word corresponding to the target decoding layer on the vocabulary is obtained. In S202, the entropy value or normalized entropy value is calculated based on the probability distribution. In S203, it is determined whether the entropy value or normalized entropy value is greater than a preset threshold. If the determination result is no, the original feedforward output remains unchanged in S204; if the determination result is yes, visual localization backtracking is triggered in S205. In S206, at least one control criterion among output uncertainty, editing intensity distribution, the layer number of the target decoding layer, and the current autoregressive generation step sequence is obtained. In S207, a preset injection ratio parameter or dynamic gating parameter is determined. In S208, the visual backtracking output and the original feedforward output are fused according to the preset injection ratio parameter or dynamic gating parameter. In S209, the updated feedforward output is output and autoregressive generation continues.

[0064] In some embodiments, the injection ratio parameter can be a pre-set fixed parameter with a range of [0, 1], or it can be a parameter dynamically determined based on the current inference state. For example, when the output uncertainty is high, the injection ratio of the visual backtracking output can be increased; when the output uncertainty is low, the injection ratio of the visual backtracking output can be decreased. As another example, when the editing intensity distribution is relatively concentrated, it indicates that the relevant visual tokens located by the visual positioning module are relatively clear, and the injection ratio can be appropriately increased; when the editing intensity distribution is relatively dispersed, it indicates that the visual positioning results are not concentrated enough, and the injection ratio can be appropriately decreased to avoid introducing too much uncertain visual information into the subsequent generation process. Furthermore, different injection ratios can be configured for different target decoding layers, and different dynamic gating parameters can be configured when the current autoregressive generation sequence is at different stages.

[0065] In one specific implementation, the dynamic gating parameters can be adjusted based on the output uncertainty. Concentration of editing intensity distribution This is confirmed. For example, dynamic gating parameters can be obtained using the sigmoid function. :

[0066] in, Indicates dynamic gating parameters. This indicates output uncertainty. This indicates the concentration of the editing intensity distribution. , and These are preset or trainable parameters. This formula is only one example; in other implementations, the dynamic gating parameters can also be determined based on the layer number of the target decoding layer, the current autoregressive generation step order, or a combination of the above factors. When using dynamic gating parameters, the updated feedforward output can be expressed as:

[0067] in, This indicates the updated feedforward output after dynamic gating parameter fusion. Through the above gating control, the intensity of visual backtracking can be adjusted according to the current uncertainty of the model and the visual localization result, thereby improving the adaptability of backtracking intervention.

[0068] Figure 4 This is a schematic diagram of the training process for the visual positioning module provided in this application. Figure 4As shown, in S301, the backbone parameters of the multimodal large language model are frozen. The backbone parameters may include at least a portion of the visual encoder parameters, visual text alignment module parameters, and decoder main parameters. In S302, the target visual representation generated during the training phase is introduced into the decoder feedforward network via an auxiliary injection path. In S303, the editing signal used to train the visual localization module is obtained. In S304, the visual token representation corresponding to the target decoding layer is perturbed. In S305, the attention output of the target decoding layer before perturbation is obtained. In S306, the attention output of the target decoding layer after perturbation is obtained. In S307, the contribution label of each visual token representation is determined based on the change in attention output before and after perturbation. In S308, the cue-related editing signal and the original visual representation are input into the visual localization module to obtain the editing intensity distribution. In S309, the editing intensity distribution is made to approximate the contribution label. In S310, the parameters of the visual localization module are updated.

[0069] During training, freezing the backbone parameters and updating only the visual localization module parameters reduces model modification costs and minimizes disruption to the original multimodal generation capabilities. An auxiliary injection path can be used during training to generate an editing signal more suitable for training the visual localization module, ensuring that this signal simultaneously incorporates the semantic requirements of the current text prompt and visual features related to the visual input. Therefore, the visual localization module does not need to rely entirely on the original hidden state or the original attention distribution for learning, mitigating the impact of linguistic prior biases on the visual token localization results.

[0070] In one specific implementation, the auxiliary injection path can be connected to the input of the decoder feedforward network or the fusion unit. During the training phase, it converts the target visual representation into auxiliary visual features with the same dimension as the current hidden state, and inputs these auxiliary visual features and the current hidden state into the decoder feedforward network or the fusion unit. The training edit signal can be obtained from the hidden state corresponding to the currently generated position in the target decoding layer, or it can be obtained from the current hidden state after introducing the target visual representation through a linear or nonlinear mapping. The training edit signal is used to characterize the correlation between the current text prompt and the target visual representation during the training phase, and to guide the visual localization module to learn the edit intensity corresponding to each visual token.

[0071] During the training phase, the training edit signal can serve as a training stage form for the cue-related edit signal. After receiving the training edit signal and the original visual representation, the visual localization module outputs the edit intensity distribution corresponding to each visual token. By approximating this edit intensity distribution to a contribution label obtained based on the change in attention output before and after perturbation, the visual localization module can learn the degree of contribution of each visual token to the current text cue or the current generation process. Thus, during the inference phase, the visual localization module can determine the target visual representation related to the current text cue based on the cue-related edit signal and the original visual representation.

[0072] In one specific implementation, the target decoding layer can be... The visual token represents the perturbation. Let the target decoding layer have the [number]th [type of token]. Each visual token is represented as Then its perturbation form can be expressed as:

[0073] in, Indicates the first The first layer before the first layer A visual token represents This indicates a random perturbation term or a preset perturbation term. This represents the visual token representation after perturbation. The attention output of the target decoding layer before and after perturbation can be obtained separately, and the value of the first attention token can be determined based on the change between the two. The contribution of each visual token is represented. If the attention output changes significantly after perturbing a visual token, it indicates that the visual token contributes highly to the current text prompt or the current generation process; if the attention output changes little after perturbing, it indicates that the visual token contributes less.

[0074] In some embodiments, the attention output of the target decoding layer before perturbation is assumed to be Disturbance Each visual token represents the attention output of the post-target decoding layer. Then the first Each visual token, representing the corresponding contribution tag, can be represented as follows:

[0075] in, Indicates the first A visual token representing a contribution label. The formula represents the change in attention output before and after the perturbation, and the denominator is used to normalize the changes corresponding to multiple visual token representations. The above formula is only one example of how to construct contribution labels. In other implementations, contribution labels can also be constructed using L2 norm, cosine distance, KL divergence, or other indicators that can characterize the difference in attention output before and after the perturbation.

[0076] After obtaining the contribution labels, the editing intensity distribution output by the visual positioning module can be made to approximate the contribution labels. For example, the editing intensity distribution can be denoted as... Let the distribution of contribution labels be denoted as C, then the attribution loss can be expressed as:

[0077] in, Indicates attribution of loss, Denotes KL divergence, This represents the distribution formed by contribution labels. This represents the edit intensity distribution output by the visual localization module. By minimizing the attribution loss, the visual localization module can learn a visual token importance distribution consistent with the perturbation contribution.

[0078] In a more specific implementation, the training loss of the visual localization module may further include enhancement loss and suppression loss. Enhancement loss is used to increase the intensity of visual token editing relevant to the current text prompt, while suppression loss is used to decrease the intensity of visual token editing unrelated to the current text prompt. The total training loss can be expressed as:

[0079] in, Indicates total training loss. This indicates an increased loss. Indicates the suppression of loss. Indicates attribution of loss, , and This represents the weight parameters of the corresponding loss term. Using the training method described above, the visual localization module can be optimized without updating the backbone parameters, enabling it to more accurately determine the visual tokens related to the current text prompt during the inference phase.

[0080] To facilitate understanding, this embodiment is illustrated below using a visual question-answering scenario. Assume the visual input is an image containing multiple objects, and the text input is a question asking about the color of one of those objects. During the autoregressive generation process, if the target decoding layer's prediction of the next token is uncertain, visual localization backtracking is triggered. The model extracts the hidden representation corresponding to the current generation position as a cue-related editing signal. The visual localization module calculates the editing intensity of each visual token based on this cue-related editing signal and the original visual representation. Visual tokens corresponding to the area where the questioned object is located receive higher editing intensity, while visual tokens corresponding to the background area or other irrelevant objects receive lower editing intensity. Subsequently, a target visual representation is generated based on the editing intensity and introduced into the decoder feedforward network. This ensures that subsequent generation processes have visual factual basis related to the color of the questioned object, thereby reducing the risk of generating non-existent objects, incorrect colors, or incorrect attribute descriptions.

[0081] For example, when the text input asks about the spatial relationship between two objects in an image, the visual localization module can increase the editing intensity of visual tokens corresponding to the two objects and their relative positions based on the prompt-related editing signals, and decrease the editing intensity of visual tokens unrelated to spatial relationship judgment. After the target visual representation is introduced into the feedforward network, it can supplement the spatial relationship visual evidence required for the current generation process, making the model output more consistent with the factual relationships in the visual input. Therefore, this embodiment can dynamically determine relevant visual tokens for different text prompts and different generation positions, rather than indiscriminately injecting the complete visual representation back into the model.

[0082] Through the above implementation methods, this embodiment only performs visual localization backtracking when the output uncertainty meets the preset triggering conditions, which can avoid continuous intervention in the entire generation process; by prompting relevant editing signals to guide the visual localization module to calculate the editing intensity, the visual supplementary information can be matched with the current text prompt and the current generation state; by generating the target visual representation based on the editing intensity, irrelevant visual tokens can be reduced from entering the feedforward network during the visual backtracking process; by fusing the visual backtracking output and the original feedforward output, the original language generation path can be preserved while enhancing the visual factual basis; by training the visual localization module with auxiliary injection path, perturbation analysis and contribution label, the consistency between the visual token localization result and the actual visual contribution can be improved, thereby enhancing the consistency between the model generation result and the visual input fact.

[0083] The above embodiments are merely illustrative of the technical concept and features of this application, intended to enable those skilled in the art to understand the content of this application and implement it accordingly, and should not be construed as limiting the scope of protection of this application. It is obvious to those skilled in the art that this application is not limited to the details of the above exemplary embodiments, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of this application is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within this application.

Claims

1. A visual localization backtracking method for alleviating the illusion of multimodal large language models, characterized in that, include: Obtain visual input and text input as the current text prompt, and construct the original visual representation and text representation; The original visual representation and the text representation are input into the decoder of the multimodal large language model, and the output uncertainty of the target decoding layer is monitored during the autoregressive generation process; When the output uncertainty meets the preset triggering condition, a prompt-related editing signal is obtained based on the hidden representation corresponding to the current generation position in the target decoding layer; The prompt-related editing signals and the original visual representation are input into the visual positioning module, and the visual positioning module determines the editing intensity of each visual token relative to the current text prompt. The original visual representation is weighted based on the editing intensity to generate the target visual representation; The target visual representation is introduced into the computation process of the decoder feedforward network. Based on the current hidden state of the decoder at the current generation position and the target visual representation, a visual backtracking output is obtained. The visual backtracking output is then fused with the original feedforward output obtained by the decoder feedforward network when the target visual representation is not injected to obtain an updated feedforward output. The autoregressive generation continues based on the updated feedforward output to obtain the text result.

2. The visual positioning backtracking method according to claim 1, characterized in that, The output uncertainty of the monitoring target decoding layer includes: Obtain the probability distribution of the next word corresponding to the target decoding layer on the vocabulary, calculate the entropy value or normalized entropy value based on the probability distribution, and determine that the output uncertainty satisfies the preset triggering condition when the entropy value or normalized entropy value is greater than a preset threshold.

3. The visual positioning backtracking method according to claim 1, characterized in that, The step of obtaining the cue-related editing signal based on the hidden representation corresponding to the currently generated position in the target decoding layer includes: Extract the hidden state vector corresponding to the current generation position in the target decoding layer as the prompt-related editing signal; or, perform a linear or nonlinear mapping on the hidden state vector corresponding to the current generation position in the target decoding layer to obtain the prompt-related editing signal; wherein, the current generation position is the last position of the current input sequence.

4. The visual positioning backtracking method according to claim 1, characterized in that, The visual positioning module includes a first mapping subnetwork, a second mapping subnetwork, an interactive computing unit, an intensity constraint unit, and a visual weighting unit. The first mapping subnetwork is used to perform feature mapping on the original visual representation, the second mapping subnetwork is used to perform feature mapping on the cue-related editing signal, the interaction computing unit is used to calculate the editing intensity of each visual token based on the mapped original visual representation and the mapped cue-related editing signal, the intensity constraint unit is used to constrain the editing intensity to a preset value range, and the visual weighting unit is used to generate the target visual representation according to the editing intensity.

5. The visual positioning backtracking method according to claim 4, characterized in that, The interactive computing unit calculates the correlation between each visual token and the cue-related editing signal using at least one of dot product matching, bilinear matching, attention matching, or nonlinear matching based on a multilayer perceptron, and obtains the editing intensity of each visual token based on the correlation.

6. The visual positioning backtracking method according to claim 4, characterized in that, The intensity constraint unit limits the editing intensity of each visual token to the preset numerical range by using at least one of the following methods: sigmoid function, softmax function, normalization function, or truncation constraint based on a preset threshold.

7. The visual positioning backtracking method according to claim 1, characterized in that, The step of weighting the original visual representation based on the editing intensity to generate the target visual representation includes: The original visual representation of each visual token is multiplied by its corresponding edit intensity to obtain a weighted visual representation, and the weighted visual representation is used as the target visual representation.

8. The visual positioning backtracking method according to claim 1, characterized in that, The process of incorporating the target visual representation into the computation of the decoder feedforward network includes: Based on the matching response between the current hidden state corresponding to the current generation position of the decoder and the target visual representation, the visual backtracking output is calculated; according to the preset injection ratio parameter, the visual backtracking output and the original feedforward output are weighted and fused to obtain the updated feedforward output.

9. The visual positioning backtracking method according to claim 8, characterized in that, The range of the preset injection ratio parameter is [0, 1]; When the output uncertainty does not meet the preset triggering condition, the original feedforward output remains unchanged.

10. The visual positioning backtracking method according to claim 1, characterized in that, It also includes training the visual positioning module; During training, the backbone parameters of the multimodal large language model are frozen, and the target visual representation generated during the training phase is introduced into the decoder feedforward network through an auxiliary injection path to obtain training editing signals for training the visual localization module. The visual token representation corresponding to the target decoding layer is perturbed, and the contribution label of each visual token representation is determined based on the change in the attention output of the target decoding layer before and after the perturbation, so that the editing intensity distribution output by the visual positioning module approximates the contribution label.

Images