Inference intervention method and device for relieving illusion of large visual language model, medium and equipment

Abstract

The invention discloses an inference intervention method and device for relieving large visual language model illusion, a medium and equipment, and relates to the technical field of artificial intelligence, intelligent medical treatment and intelligent finance. The method comprises the following steps: aiming at a current to-be-generated output token, constructing a multi-modal input stream containing an image token and a text token and a plain text input stream only containing the text token; in the forward reasoning process of the language decoder, attention enhancement processing is carried out on an image token in the multi-modal input stream, forward reasoning calculation is carried out on the multi-modal input stream and the plain text input stream after attention enhancement, and multi-modal confidence distribution and plain text confidence distribution are obtained; carrying out suppression processing on the plain text confidence distribution on the basis of the multi-modal confidence distribution to obtain probability distribution of a current to-be-generated output token; and decoding to generate a current output token so as to update the autoregressive text of the language decoder. The method can reduce the probability of occurrence of model illusion.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence, smart healthcare, and smart finance, and in particular to a reasoning intervention method, apparatus, medium, and device for alleviating hallucinations caused by large visual language models. Background Technology

[0002] In recent years, large vision-language models (LVLMs) have become an important development direction in the field of multimodal artificial intelligence. These models are usually composed of a visual encoder, a projector, and a large language model (LLM) connected in series. They can achieve semantic understanding of image content and generate text descriptions, question-and-answer or dialogue content corresponding to the image.

[0003] Currently, LVLMs commonly suffer from the illusion problem, where the text descriptions generated by the model do not match the image content. To address this illusion problem, existing technologies primarily focus on improvements in three directions: first, training-based optimization methods, such as using higher-quality data or performing multimodal alignment training; second, post-processing-based correction methods, such as calling external visual models to verify and correct the generated results; and third, methods based on specific decoding strategies, such as adjusting the attention mechanism or introducing visual uncertainty factors during the decoding stage.

[0004] However, while the aforementioned improvements can alleviate the illusion problem to some extent, they cannot fundamentally solve the text inertia problem caused by the model structure. Text inertia refers to the tendency in LVLMs, where the language model has far more parameters than the visual encoder, leading to an over-reliance on textual context and language priors during inference, while neglecting continuous attention to image tokens. This results in the model potentially continuing the same error generation pattern even without image input. Summary of the Invention

[0005] In view of this, embodiments of this application provide a reasoning intervention method, apparatus, medium and device to alleviate the illusion of large visual language models. The main purpose is to solve the technical problem that existing large visual language models rely too much on text content during the reasoning process, resulting in text inertia and thus model illusion.

[0006] According to one aspect of this application, a reasoning intervention method for alleviating hallucinations in large visual language models is provided, the method being applied to a large visual language model comprising an image encoder, a projector, and a language decoder, wherein the method is performed during autoregressive text generation by the language decoder, the method comprising: For the output tokens that the language decoder is currently generating, a multimodal input stream containing image tokens and text tokens, and a plain text input stream containing only text tokens are constructed. During the forward inference process of the language decoder, attention enhancement processing is performed on the image tokens in the multimodal input stream, and forward inference calculation is performed once on the attention-enhanced multimodal input stream and the plain text input stream to obtain the multimodal confidence distribution and the plain text confidence distribution. Based on the multimodal confidence distribution, the plain text confidence distribution is suppressed to obtain the probability distribution of the output token to be generated. Based on the probability distribution of the output token to be generated, the current output token is decoded and generated, and the autoregressive text of the language decoder is updated based on the output token.

[0007] According to another aspect of this application, a reasoning intervention device for alleviating the illusion of a large visual language model is provided. The device is disposed within a large visual language model comprising an image encoder, a projector, and a language decoder, wherein the device is executed when the language decoder performs autoregressive text generation. The device includes: The input stream construction module is used to construct a multimodal input stream containing image tokens and text tokens, as well as a plain text input stream containing only text tokens, for the output tokens that the language decoder is currently to generate. The attention enhancement module is used to perform attention enhancement processing on the image tokens in the multimodal input stream during the forward inference process of the language decoder, and to perform forward inference calculation on the attention-enhanced multimodal input stream and the plain text input stream respectively to obtain the multimodal confidence distribution and the plain text confidence distribution. The confidence adjustment module is used to suppress the plain text confidence distribution based on the multimodal confidence distribution to obtain the probability distribution of the output token to be generated. The text update module is used to decode and generate the current output token according to the probability distribution of the current output token to be generated, and update the autoregressive text of the language decoder according to the output token.

[0008] According to another aspect of this application, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described reasoning intervention method for alleviating large visual-language model hallucinations.

[0009] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described reasoning intervention method for alleviating large visual-language model illusions.

[0010] By employing the above technical solutions, this application provides a reasoning intervention method, apparatus, medium, and device for alleviating the illusion of large visual language models. By executing the aforementioned reasoning intervention method during autoregressive text generation in the language decoder, this method can be directly deployed on existing pre-trained models, effectively lowering the technical application threshold. The method does not rely on specific model architectures or decoding algorithms and can be integrated into various large visual language models as a general-purpose plugin, showing broad application prospects while avoiding expensive model retraining costs. Furthermore, compared to standard reasoning methods, the above reasoning method only adds forward propagation computation of the plain text input stream and computational intervention of the probability distribution of the output tokens. The overall reasoning time is almost the same as the original model, far faster than methods requiring complex post-processing or bundle search decoding, with lower computational overhead. In addition, by forcing the model to pay more attention to image tokens, the method enables the model's attention to more accurately focus on the corresponding objects in the image, thereby making the generated text description more relevant to the image content, more accurate, and significantly reducing the probability of model illusion.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a reasoning intervention method for alleviating large visual-language model hallucinations provided in an embodiment of this application is shown. Figure 2 A schematic diagram of the structure of a reasoning intervention device for alleviating large visual-language model illusions provided in an embodiment of this application is shown. Detailed Implementation

[0013] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0014] Existing large-scale visual language models (LVLMs) generally suffer from a serious hallucination problem. Specifically, the text descriptions generated by the model do not match the input visual content. For example, the model may fabricate objects that do not exist in the image, or incorrectly describe the attributes, quantity, and relationships of objects.

[0015] Currently, existing technological solutions to the hallucination problem in LVLM mainly fall into the following categories: 1. Training-based optimization methods: These methods alleviate the illusion by improving model training. For example, using higher-quality labeled data, improving data filtering methods, or optimizing the fusion of visual and linguistic modalities through alignment training. However, these methods require significant computational resources and time for model retraining or fine-tuning, making them costly and time-consuming to implement.

[0016] 2. Post-processing-based correction methods: These methods typically validate and correct the output after the model generates a response by introducing external tools or additional modules. For example, the LURE (Likelihood Unfaithfulness Reasoning and Editing) method trains a state detector and calls the repair model to regenerate the content when an illusion is detected; the Woodpecker method uses an external visual model to check the entities mentioned in the response and then lets the model regenerate the answer. However, these methods rely on external tools or models, which increases the complexity and latency of the inference process and also increases the deployment cost of the model.

[0017] 3. Methods based on specific decoding strategies: These methods typically optimize during the decoding stage. For example, OPERA (Object-Hallucination Reduction via Attention-Map Analysis and Refinement) alleviates hallucinations by identifying anomalous attention patterns, while VCD (Visual Contrastive Decoding) guides contrastive decoding by introducing visual uncertainty. However, these methods are usually tied to specific decoding algorithms, such as Beam Search or Nucleus Sampling, lacking versatility and limited applicability to all decoding scenarios.

[0018] Furthermore, existing technologies have failed to effectively address a deep-seated problem prevalent in large visual language models: the "text inertia" problem. This problem refers to the dominance of LLMs in multimodal understanding due to the size difference between the visual encoder and the language model. This leads the model to over-rely on contextual information and prior linguistic knowledge when generating text, neglecting the actual visual input. A typical example is that even after removing the image input and providing only the text history, the model still generates the exact same illusory content as before. The root cause is that image tokens do not receive sufficient attention during generation, causing the inference process to resemble text-based autocomplete rather than continuously focusing on image generation.

[0019] To address the issues of high training and deployment costs, poor versatility, and ineffective suppression of text inertia in the aforementioned model-based hallucination mitigation schemes, in one embodiment, such as... Figure 1 As shown, an inference intervention method for alleviating hallucinations in large visual language models is provided. This method can be applied to large visual language models containing image encoders, projectors, and language decoders, and is executed when the language decoder performs autoregressive text generation. The method may include the following steps: Step 101: For the output tokens to be generated by the language decoder, construct a multimodal input stream containing image tokens and text tokens, as well as a plain text input stream containing only text tokens.

[0020] In this context, image tokens refer to vector sequences representing visual information, obtained by feature extraction from the input image by the image encoder and mapping by the projector; text tokens include instruction text tokens and history text tokens, corresponding to the task description text and the segmented sequence of the generated text, respectively; multimodal input streams refer to input sequences containing both image tokens and text tokens, while pure text input streams refer to input sequences containing only text tokens. Output tokens refer to the units predicted and generated by the language decoder in each iteration during the generation of autoregressive text; these units can be words, subwords, or symbols.

[0021] Specifically, after receiving image and text commands from the user, the large-scale visual language model first converts the input image into a fixed-length image token sequence using a visual encoder and projector. Simultaneously, it segments the text commands to obtain a command text token sequence. Furthermore, if this is not the first generation, all previously generated output tokens are used as historical text token sequences. Subsequently, two input sequences can be constructed in memory: the first is a multimodal input stream, which is obtained by sequentially concatenating the image token sequence, command text token sequence, and historical text token sequence in a preset order; the second is a plain text input stream, which is obtained by concatenating only the command text token sequence and historical text token sequence in the same order, while setting the image token sequence to null or marking it as missing. In this way, parallel and content-distinguishable input data can be provided for subsequent interventional comparative analysis.

[0022] Step 102: During the forward inference process of the language decoder, attention enhancement processing is performed on the image tokens in the multimodal input stream, and forward inference calculation is performed once on the attention-enhanced multimodal input stream and the plain text input stream to obtain the multimodal confidence distribution and the plain text confidence distribution.

[0023] Among them, attention enhancement processing refers to selectively amplifying the attention score corresponding to the image token during the computation of the self-attention mechanism of the language decoder; multimodal confidence distribution and plain text confidence distribution refer to the normalized log probability distribution of the entire vocabulary generated at the output layer after the language decoder performs forward computation based on two different input streams.

[0024] Specifically, when the language decoder performs forward computation on the multimodal input stream, it can intervene in the self-attention module of each layer or a specified layer. Specifically, when calculating the attention of the currently generated output token to all input tokens, the original attention score matrix before the Softmax function is applied can be located, and the scores corresponding to the positions of all image tokens in the current row can be extracted. Then, these scores are calculated using a preset enhancement algorithm, and the enhanced scores are written back to the attention score matrix. After normalization using the Softmax function, new attention weights are obtained, completing one attention enhancement. This process can be repeated at each layer or a specified layer of the language decoder. Subsequently, a forward propagation can be performed on the enhanced multimodal input stream to obtain the multimodal confidence distribution. Simultaneously, a forward computation without enhancement can be performed on the plain text input stream in parallel to obtain the plain text confidence distribution. By dynamically amplifying the weight of image tokens in the attention mechanism, the model can focus more on and integrate visual information during the generation process, thereby guiding the model's generation to be closer to the content of the image.

[0025] Step 103: Based on the multimodal confidence distribution, suppress the plain text confidence distribution to obtain the probability distribution of the output token to be generated.

[0026] Suppression processing refers to the process of reducing the dependence on the plain text confidence distribution in the generation decision through specific linear operations and highlighting the dominant role of the multimodal confidence distribution; the probability distribution of the output token refers to the normalized probability distribution obtained after suppression processing and used for final decoding.

[0027] Specifically, after obtaining the multimodal confidence distribution and the plain text confidence distribution, the current confidence distribution can be adjusted using a pre-defined refinement equation. For example, the multimodal confidence distribution can be multiplied by a penalty coefficient to obtain an amplified distribution, and then the plain text confidence distribution can be multiplied by a weight smaller than the penalty coefficient to obtain a distribution representing text inertia. Finally, the latter is subtracted from the former to obtain the refined distribution. These steps, by combining visual enhancement and suppression of text inertia—two complementary intervention strategies—can synergistically address the problem of text inertia from two dimensions, thereby ensuring that the final text content is largely derived from comprehensive information including visual information.

[0028] Step 104: Based on the probability distribution of the current output token to be generated, decode and generate the current output token, and update the autoregressive text of the language decoder based on the output token.

[0029] Decoding and generation refers to selecting and determining a specific output token based on the probability distribution of the output token; updating the autoregressive text refers to adding the newly generated token to the historical text sequence as context information for generating the next token.

[0030] Specifically, after obtaining the probability distribution of the output token to be generated, a preset decoding strategy, such as a greedy strategy, bundle search, or sampling strategy, can be used to determine the unique output token to be generated at the current moment. After generating the current output token, the token ID of the output token can be added to the end of the historical text token sequence. Then, it can be determined whether the token is a predefined text end token. If not, the entire process (i.e., steps 101 to 104) can use the updated historical text token sequence as the new input for the next iteration and generate the next token; if it is a text end token, the generation process terminates. By iteratively applying this intervention to the entire text sequence generation process, it can be ensured that the generated response text is constrained by the correction mechanism of the preceding steps from beginning to end. Furthermore, the above inference intervention method does not require additional model training or fine-tuning and can be directly intervened in the model's inference stage. The entire process is completed within the LVLM model and does not rely on any external tools, models, or data, thus maintaining the simplicity and efficiency of the system. Furthermore, by intervening in the intermediate stages of inference, the above method allows the inference intervention method to be independent of the final decoding algorithm. Whether it is greedy decoding, bundle search, or core sampling strategy, this method can be combined with and play a role, thus having strong versatility and flexibility.

[0031] The aforementioned reasoning intervention method can be applied to large-scale visual language models across various fields and can effectively suppress the illusion problem caused by textual inertia in large-scale visual language models. For example, in the financial field, when a user inputs a bar chart showing the revenue share of each business line in a company's quarterly financial report and enters the command "analyze the contribution of each business line," the uninterrupted model might generate the conclusion "the financial services business line makes a significant contribution" based on common financial report text patterns, even though the actual height of the bar for that business line in the chart is very low. However, after applying the method of this embodiment, before generating the term "financial services," the attention enhancement processing forces the model to pay more attention to the visual features representing each business line in the image, and the suppression processing reduces the confidence generated solely by the linguistic prior of "financial report text frequently mentions financial services," making the model more likely to ultimately generate a description faithful to the chart, such as "the consumer goods business line is the main contributor," based on the facts in the image.

[0032] For another example, in the medical field, when a user inputs a patient's chest X-ray and the command "describe what is seen in the image," an uninterrupted model might, due to the frequent co-occurrence of words like "cough" and "increased lung texture" in the training data, generate a phantom description of "visible increased lung texture" even when the X-ray actually shows no clear abnormalities. However, after applying the method described in this embodiment, when generating the word "increased," the attention enhancement processing guides the model to strengthen its attention to the X-ray image tokens and focus on the actual pixel areas, while the suppression processing can suppress the inherent tendency to associate "increased texture" solely with the complaint of "cough," thus making the generated probability distribution more inclined to output descriptions that faithfully reflect the objective appearance of the image, such as "clear lung fields, no substantial lesions seen."

[0033] As can be seen from the above examples, in fields such as finance and healthcare where the accuracy of facts is highly critical, applying the reasoning intervention method described in this embodiment to a large visual language model can effectively constrain the generation process of the large visual language model, allowing its output to be derived to the greatest extent possible from the input charts or video evidence. This reduces fictitious or erroneous descriptions caused by the model's over-reliance on prior language, thereby improving the reliability and credibility of analysis reports and medical impact assessments.

[0034] By applying the technical solution of this embodiment, and executing the aforementioned inference intervention method during autoregressive text generation in the language decoder, this method can be directly deployed on existing pre-trained models, thereby effectively lowering the technical application threshold. This method does not rely on specific model architectures or decoding algorithms and can be integrated as a general-purpose plugin into various large-scale visual language models, showing broad application prospects while avoiding expensive model retraining costs. Furthermore, compared to standard inference methods, this inference method only adds forward propagation computation to the plain text input stream and computational intervention for the probability distribution of the output tokens. The overall inference time is almost the same as the original model and is much faster than methods requiring complex post-processing or bundle search decoding, with lower computational overhead. In addition, by forcing the model to pay more attention to image tokens, this method enables the model's attention to more accurately focus on the corresponding objects in the image, resulting in a stronger correlation between the generated text description and the image content, higher accuracy, and a significant reduction in the probability of model illusions.

[0035] In one embodiment, step 101 can be implemented as follows: For the output token to be generated by the language decoder, construct a multimodal input stream and a plain text input stream, wherein the multimodal input stream includes an image token, an instruction text token, and a history text token, and the plain text input stream includes an instruction text token and a history text token. Further, after decoding and generating the current output token, the history text token can be updated based on the current output token, and based on the updated history text token, construct the multimodal input stream and the plain text input stream for the next output token to be generated.

[0036] Specifically, when generating the first output token, the input image is processed by a visual encoder and projector to obtain an image token sequence. Meanwhile, by segmenting the user-input text commands, a command text token sequence can be obtained. At this point, the historical text token sequence can be used. Initialize to empty. Then, construct two input sequences logically in parallel, i.e., by sequentially concatenating them. , and This constitutes a multimodal input stream, and simultaneously, through sequential concatenation... and This constitutes a plain text input stream. Furthermore, after generating an output token based on the probability distribution of the currently generated output token, the token's lexical ID can be added to the historical text token sequence. At the end, and when generating the next output token, re-update the... With the original and By concatenating the data, a multimodal input stream can be dynamically constructed for the next iteration. and plain text input stream This process is repeated, with the interface providing dual-path input data containing the latest context for each step in the autoregressive generation.

[0037] This embodiment dynamically constructs and maintains a pair of content-comparable input streams for each output token to be generated, which can provide an accurate data foundation for subsequent attention enhancement and confidence refinement, and ensure that the intervention mechanism can play a continuous and iterative role as the text is generated, thereby ensuring the coherence and accuracy of the generation sequence from beginning to end.

[0038] In one embodiment, step 102 can be implemented by the following method: locating the attention score matrix in the attention module of at least one layer of the language decoder, locating the attention score row corresponding to the output token to be generated in the attention score matrix, extracting the attention score corresponding to the image token from the attention score row, enhancing the attention score corresponding to the image token based on a preset amplification factor, updating the enhanced attention score in the attention score matrix, and normalizing the updated attention score matrix to obtain the attention-enhanced multimodal input stream.

[0039] Specifically, when the language decoder performs forward inference computation, when the process reaches at least one layer (e.g., each layer or a specified layer) of its self-attention module, it can obtain the original attention score matrix generated by that module before applying the Softmax normalization function. Within this matrix, it locates the row of score vectors corresponding to the currently generated nth output token. This row vector represents the original attention level of the current generation position to all input tokens. Then, based on the known index range of the image tokens in the input sequence, for example, from the nth... To the At each position, a subset of attention scores corresponding to all image tokens is extracted from the row vector. Then, a pre-defined enhancement algorithm is invoked to process these scores. For example, each extracted original attention score is multiplied by a positive amplification factor pre-defined according to the model type, and the product is added to the original score to obtain the enhanced attention score. After calculation, these enhanced attention scores can be written back to their corresponding positions in the original attention score matrix to replace the old values. Finally, the entire matrix of updated image token scores can be normalized using the Softmax function to obtain an updated attention weight distribution. This weight distribution constitutes the attention-enhanced multimodal information flow and can be used to continue the calculation of this layer and subsequent layers.

[0040] This embodiment can effectively enhance the weight of visual features in the model's internal representation by dynamically identifying and amplifying the original attention score of the current generation position to the image token in at least one layer of the attention module of the language decoder. This can effectively combat the text inertia problem caused by the language model dominating and ignoring visual input, thereby improving the correlation between the model output content and the image content.

[0041] In one embodiment, in step 102, the attention score corresponding to the image token can be enhanced in the following way: First, the amplification factor corresponding to the image token is determined according to the type of the current large visual language model. Then, the amplification factor is multiplied by the absolute value of the attention score corresponding to the image token to obtain the gain of the attention score. Finally, the gain of the attention score is added to the attention score corresponding to the image token to obtain the enhanced attention score.

[0042] Specifically, during attention enhancement processing, the amplification factor α can be retrieved from a pre-configured mapping table based on the specific architecture and name of the currently loaded large visual language model, such as LLaVA (Large Language and Vision Assistant), Shikra (a falcon model symbolizing visual localization and pointing capabilities), or MiniGPT-4 (MiniatureGenerative Pre-trained Transformer 4 for Vision-Language). This amplification factor is empirically set based on the typical degree of image token neglect by the model without intervention and is a positive value. Then, the extracted attention score can be enhanced using Formula 1: (Formula 1) in, These are preset hyperparameters, i.e., amplification factors, used to control the magnitude of enhancement. For example, they can be set to 0.5 for the LLaVA model, 0.6 for Shikra, and 0.2 for MiniGPT-4. To improve the attention score after processing, This represents the original attention score corresponding to the image token.

[0043] In this embodiment, for each extracted image token, the corresponding original attention score... Its absolute value can be calculated. and the absolute value With magnification factor Perform scalar multiplication to obtain the gain corresponding to the fraction. Then, a vector addition operation is performed, which calculates the gain. Compared with the original attention score The elements are added together to output the final, enhanced attention score. This is used to enhance the single attention score.

[0044] This embodiment dynamically and adaptively amplifies the attention score corresponding to the image token during the self-attention calculation process of the decoder, and amplifies it based on the original attention value, rather than simply assigning a fixed gain to the image token. This can further strengthen the visual signal in the direction that the model considers relevant, thereby guiding the model to generate text content that is more faithful to the image content.

[0045] In one embodiment, step 103 can be implemented by the following method: First, the multimodal confidence distribution is scaled up based on a preset penalty coefficient to obtain an image enhancement distribution. Then, the penalty weight is obtained based on the difference between the penalty coefficient and the preset penalty intensity. The plain text confidence distribution is scaled up based on the penalty weight to obtain the text inertia distribution to be suppressed. Finally, the text inertia distribution is subtracted from the image enhancement distribution to obtain the probability distribution of the output token to be generated.

[0046] Specifically, after obtaining the multimodal confidence distribution and the plain text confidence distribution, the probability distribution of the output token to be generated can be calculated using Formula 2: (Formula 2) in, This represents the probability distribution of the output tokens to be generated. is a hyperparameter used to control the severity of the penalty, i.e., the penalty coefficient, which is usually set to a value slightly greater than 1, such as 1.1 or 1.2, etc. For multimodal confidence distribution, For plain text confidence distribution, For image tokens, For instruction text tokens, For historical text tokens.

[0047] In this embodiment, when suppressing the confidence distribution of plain text, a preset penalty coefficient γ, slightly greater than 1, can be used. Then, the multimodal confidence distribution... Each value in the algorithm (i.e., the original confidence score for each candidate word) is multiplied by γ to obtain an image enhancement distribution that is numerically amplified overall. Then, the difference between the penalty coefficient γ and the pre-set penalty strength, γ-1, is calculated and used as the penalty weight. This penalty weight is then used to proportionally amplify the plain text confidence distribution, resulting in a numerically amplified text inertia distribution. This distribution can quantify the model. The generation tendency when relying solely on textual information. Finally, a vector subtraction operation is performed, subtracting the text inertia distribution element-wise from the image enhancement distribution. The result is the refined probability distribution of the output token to be generated. .

[0048] This embodiment calculates the confidence distribution of plain text without image input and subtracts it from the confidence distribution of multimodal text with image input. This can accurately punish the tendency to generate content driven solely by language logic, thereby forcing the model to rely on visual information to make decisions and effectively suppressing the tendency to generate hallucinatory content.

[0049] In one embodiment, step 104 can be implemented by the following method: First, based on the probability distribution of the current output token to be generated, the current output token is generated by a preset decoding strategy. Then, the generated output token is added to the historical text token of the next output token to be generated in order to update the autoregressive text of the language decoder.

[0050] Specifically, the probability distribution of obtaining the refined, currently to-be-generated output token. Then, the final output token can be determined from the distribution according to a pre-configured decoding strategy. For example, if a greedy decoding strategy is used, the token can be directly selected. The candidate token ID with the highest probability value is selected; if a beam search or sampling strategy is used, the selection can be made according to the rules of the corresponding algorithm. Furthermore, after generating the output token, its corresponding lexical ID can be added to the historical text token sequence. This completes the update of the autoregressive text context at the end. Afterwards, it can be based on the updated historical text token sequence. The multimodal input stream and plain text input stream containing the new history are reconstructed, and all intervention steps starting from attention enhancement are repeated to generate the next output token.

[0051] This embodiment transforms the refined probability distribution into output tokens and dynamically updates the generated historical text tokens based on the output tokens. This enables a closed-loop iteration of the autoregressive generation and intervention mechanism, thereby ensuring that the correction effects of attention enhancement and confidence refinement can continuously affect the generation decision of each token in the text sequence, thus guaranteeing the fidelity and accuracy of the final generated response text.

[0052] In one embodiment, the above-described inference intervention method for mitigating the illusion of large visual language models may further include the following steps: when the current output token is a text end token, stop generating the next output token, and generate the response text of the large visual language model based on the currently updated autoregressive text.

[0053] Specifically, in each iteration of the autoregressive generation, after decoding and generating the current output token based on the probability distribution of the current output token, this output token can be combined with a text end token representing the end of generation, such as... <eos>The two tokens are compared, and if they match, the entire loop of constructing input, enhancing attention, refining confidence, and decoding updates is immediately terminated, and no new round of computation is started for the next token. Subsequently, the updated historical text token sequence, i.e., the ordered set of all generated tokens from the first token to the last token, can be used as the final output. By calling the decoding function of the token segmenter, the token ID sequence is converted back into a natural language string, which is the complete and final response text generated by the large visual language model for this user request.

[0054] This embodiment ensures that the text generation process can end in a timely and accurate manner when the model considers the expression to be complete by real-time detection of the text end token, thereby outputting complete and non-redundant response text, thus guaranteeing the effectiveness and usability of the generated content.

[0055] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the labels corresponding to each step in the above embodiments are only for identification purposes and are not intended to limit the execution order of the steps. The execution order of the steps in each embodiment can be set according to the actual situation.

[0056] Furthermore, as Figure 1 In a specific implementation of the method shown, this application provides a reasoning intervention device to alleviate the illusion of large visual language models. The device is installed within a large visual language model comprising an image encoder, a projector, and a language decoder. The device executes when the language decoder performs autoregressive text generation, such as... Figure 2 As shown, the device includes: The input stream construction module 21 can be used to construct a multimodal input stream containing image tokens and text tokens, as well as a plain text input stream containing only text tokens, for the output tokens that the language decoder is currently to generate. The attention enhancement module 22 can be used to perform attention enhancement processing on the image tokens in the multimodal input stream during the forward inference process of the language decoder, and perform forward inference calculation on the attention-enhanced multimodal input stream and the plain text input stream respectively to obtain the multimodal confidence distribution and the plain text confidence distribution. The confidence adjustment module 23 can be used to suppress the plain text confidence distribution based on the multimodal confidence distribution to obtain the probability distribution of the output token to be generated. The text update module 24 can be used to decode and generate the current output token according to the probability distribution of the current output token to be generated, and update the autoregressive text of the language decoder according to the output token.

[0057] In a specific application scenario, the attention enhancement module 22 can be used to locate the attention score matrix in at least one layer of the attention module of the language decoder, and locate the attention score row corresponding to the output token to be generated in the attention score matrix; extract the attention score corresponding to the image token in the attention score row; enhance the attention score corresponding to the image token based on a preset amplification factor; update the enhanced attention score to the attention score matrix, and normalize the updated attention score matrix to obtain the attention-enhanced multimodal input stream.

[0058] In specific application scenarios, the attention enhancement module 22 can also be used to determine the amplification coefficient corresponding to the image token according to the type of the current large-scale visual language model; multiply the amplification coefficient by the absolute value of the attention score corresponding to the image token to obtain the gain of the attention score; and add the gain of the attention score to the attention score corresponding to the image token to obtain the enhanced attention score.

[0059] In specific application scenarios, the confidence adjustment module 23 can be used to scale up the multimodal confidence distribution based on a preset penalty coefficient to obtain an image enhancement distribution; obtain a penalty weight based on the difference between the penalty coefficient and the preset penalty intensity, and scale up the plain text confidence distribution based on the penalty weight to obtain a text inertia distribution to be suppressed; subtract the text inertia distribution from the image enhancement distribution to obtain the probability distribution of the output token to be generated.

[0060] In specific application scenarios, the text update module 24 can be used to generate the current output token based on the probability distribution of the current output token to be generated, through a preset decoding strategy; and add the generated output token to the historical text token of the next output token to be generated, so as to update the autoregressive text of the language decoder.

[0061] In specific application scenarios, the input stream construction module 21 can be used to construct a multimodal input stream and a plain text input stream for the output token to be generated by the language decoder. The multimodal input stream includes an image token, an instruction text token, and a history text token, and the plain text input stream includes an instruction text token and a history text token. The input stream construction module 21 can also be used to construct a multimodal input stream and a plain text input stream for the next output token to be generated based on the updated history text token.

[0062] In specific application scenarios, the text update module 24 can also be used to stop generating the next output token when the current output token is the text end token, and generate the response text of the large visual language model based on the currently updated autoregressive text.

[0063] It should be noted that other corresponding descriptions of the functional units involved in the reasoning intervention device for alleviating large visual-language model hallucinations provided in the embodiments of this application can be found in the following references. Figure 1 The corresponding descriptions in the methods shown will not be repeated here.

[0064] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0065] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0066] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0067] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / eos>

Claims

1. A reasoning intervention method for alleviating large visual-language model hallucinations, characterized in that, The method is applied to a large visual language model comprising an image encoder, a projector, and a language decoder, wherein the method is executed during autoregressive text generation by the language decoder, and the method includes: For the output tokens that the language decoder is currently generating, a multimodal input stream containing image tokens and text tokens, and a plain text input stream containing only text tokens are constructed. During the forward inference process of the language decoder, attention enhancement processing is performed on the image tokens in the multimodal input stream, and forward inference calculation is performed once on the attention-enhanced multimodal input stream and the plain text input stream to obtain the multimodal confidence distribution and the plain text confidence distribution. Based on the multimodal confidence distribution, the plain text confidence distribution is suppressed to obtain the probability distribution of the output token to be generated. Based on the probability distribution of the output token to be generated, the current output token is decoded and generated, and the autoregressive text of the language decoder is updated based on the output token.

2. The method according to claim 1, characterized in that, The attention enhancement processing performed on the image tokens in the multimodal input stream during the forward inference process of the language decoder includes: In at least one layer of the attention module of the language decoder, the attention score matrix is ​​located, and the attention score row corresponding to the output token to be generated is located in the attention score matrix; Extract the attention score corresponding to the image token from the attention score row; Based on a preset magnification factor, the attention score corresponding to the image token is enhanced: The enhanced attention score is updated into the attention score matrix, and the updated attention score matrix is ​​normalized to obtain the attention-enhanced multimodal input stream.

3. The method according to claim 2, characterized in that, The enhancement processing of the attention score corresponding to the image token based on a preset magnification factor includes: The magnification factor corresponding to the image token is determined based on the type of the current large-scale visual language model; The gain of the attention score is obtained by multiplying the magnification factor by the absolute value of the attention score corresponding to the image token. The attention score is obtained by adding the gain of the attention score to the attention score corresponding to the image token.

4. The method according to claim 1, characterized in that, The step of suppressing the plain text confidence distribution based on the multimodal confidence distribution to obtain the probability distribution of the output token to be generated includes: The multimodal confidence distribution is scaled up based on a preset penalty coefficient to obtain an image enhancement distribution; The penalty weight is obtained based on the difference between the penalty coefficient and the preset penalty intensity, and the confidence distribution of the plain text is proportionally amplified based on the penalty weight to obtain the text inertia distribution to be suppressed. Subtracting the text inertia distribution from the image enhancement distribution yields the probability distribution of the output token to be generated.

5. The method according to claim 1, characterized in that, The step of decoding and generating the current output token based on the probability distribution of the current output token to be generated, and updating the autoregressive text of the language decoder based on the output token, includes: Based on the probability distribution of the output token to be generated, the current output token is generated using a preset decoding strategy; The generated output token is added to the history text token of the next output token to be generated, in order to update the autoregressive text of the language decoder.

6. The method according to claim 5, characterized in that, The construction of a multimodal input stream containing image tokens and text tokens, and a plain text input stream containing only text tokens, for the output tokens currently to be generated by the language decoder, includes: For the output token to be generated by the language decoder, a multimodal input stream and a plain text input stream are constructed, wherein the multimodal input stream includes an image token, an instruction text token, and a history text token, and the plain text input stream includes an instruction text token and a history text token; After adding the generated output token to the historical text token of the next output token to be generated, the method further includes: constructing a multimodal input stream and a plain text input stream of the next output token to be generated based on the updated historical text token.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: When the current output token is the end-of-text token, stop generating the next output token and generate the response text of the large visual language model based on the currently updated autoregressive text.

8. A reasoning intervention device for alleviating large-scale visual-linguistic model hallucinations, characterized in that, The apparatus is disposed within a large visual language model comprising an image encoder, a projector, and a language decoder, wherein the apparatus is executed during autoregressive text generation by the language decoder, and the apparatus includes: The input stream construction module is used to construct a multimodal input stream containing image tokens and text tokens, as well as a plain text input stream containing only text tokens, for the output tokens that the language decoder is currently to generate. The attention enhancement module is used to perform attention enhancement processing on the image tokens in the multimodal input stream during the forward inference process of the language decoder, and to perform forward inference calculation on the attention-enhanced multimodal input stream and the plain text input stream respectively to obtain the multimodal confidence distribution and the plain text confidence distribution. The confidence adjustment module is used to suppress the plain text confidence distribution based on the multimodal confidence distribution to obtain the probability distribution of the output token to be generated. The text update module is used to decode and generate the current output token according to the probability distribution of the current output token to be generated, and update the autoregressive text of the language decoder according to the output token.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images