Multi-modal large model object illusion relieving method for removing training prejudice based on efficient machine forgetting

Abstract

The invention discloses a multi-modal large model object illusion relieving method for removing training prejudice based on efficient machine forgetting. The method comprises the following steps: S1, collecting a deviation reasoning result: firstly, selecting an instruction related to an image description task from LVLM training data as a source data set; thirdly, the image and text instructions in the source data set are subjected to reasoning again through LVLM, and a reasoning result set is generated; by comparing the object existence conditions in the real output and the reasoning result, screening out the reasoning result containing the illusion phenomenon; and S2, efficient depolarization training: based on the collected deviation reasoning result, carrying out depolarization training on the LM head by adopting a machine forgetting method based on gradient rise. The invention provides an efficient depolarization method by specifically eliminating the illusion phenomenon in a large visual language model (LVLM). According to the method, a deviation reasoning result of a model on training data is collected, and a language model head (LM head) is finely adjusted, so that efficient utilization of data and parameters is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal large language models (LVLMs), and particularly to a method for alleviating object hallucinations in multimodal large models based on efficient machine forgetting to remove training biases. Background Art

[0002] In recent years, despite significant progress in the field of multimodal large language models (LVLMs), the hallucination problem remains a major challenge hindering their accurate understanding of visual inputs. Existing technologies mainly alleviate the hallucination problem in LVLMs through two approaches: preference learning-based methods and training-free decoding enhancement methods. Preference learning-based methods, such as technologies like LLaVA-RLHF and RLHF-V, integrate preference learning algorithms (such as PPO and DPO) into the training process of LVLMs and use human-annotated preference data to align the model output with human preferences, thereby enabling it to generate descriptions that are more in line with real visual content. Training-free decoding enhancement methods include technologies like VCD and VDD, which introduce perturbations to visual inputs during the decoding stage, compare the output differences of the model under the original input and the perturbed input, and thus reduce the over-reliance of the model output on language priors. These methods directly optimize the model's generation process through post-processing or improvement of decoding strategies without additional training.

[0003] However, existing technologies still have some limitations: Although training-based methods can optimize model behavior through human feedback, they rely on a large amount of human-annotated data and computing resources. Although training-free decoding enhancement methods have advantages in computational efficiency, they rely on perturbation strategies for visual inputs, which may lead to insufficient stability of the generated results and may also impair the model's own generation ability. Therefore, although existing technologies have made some progress in alleviating the hallucination problem in LVLMs, there is still room for improvement in balancing the accuracy and quality of the generated results and reducing the dependence on annotated data. This patent aims to further solve or improve the above problems through innovative technical solutions.

[0004] The information disclosed in this background art section is only intended to deepen the understanding of the overall background art of the present invention and should not be regarded as an admission or any form of implication that this information constitutes prior art already known to those skilled in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for alleviating object hallucinations in multimodal large models based on efficient machine forgetting to remove training biases, so as to solve the technical problems existing in the prior art.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] The present invention provides a method for alleviating object hallucination in a multimodal large model based on efficient machine forgetting to remove training biases, including the following steps:

[0008] S1. Collection of deviation inference results: First, select instructions related to image description tasks from the training data of the LVLM as the source dataset; where each piece of data includes an image input, a text instruction, and a true output; then, use the LVLM to re-infer the images and text instructions in the source dataset to generate an inference result set; by comparing the object presence in the true output and the inference results, filter out the inference results containing hallucination phenomena;

[0009] S2. Efficient debiasing training: Based on the collected deviation inference results, use the machine forgetting method based on gradient ascent to perform debiasing training on the LM head to eliminate specific biases learned by the model while retaining its general performance.

[0010] Furthermore, in step S1, debiasing is only performed on the clauses containing hallucinated objects; specifically, use common delimiters to split the sentence into multiple clauses, and further use the correctly generated objects as delimiters to avoid accidentally deleting correct information; for each hallucinated object, only collect the clause where it first appears in the description to reduce the over-punishment for repeated objects and minimize the number of tokens for debiasing training, improving efficiency; finally, obtain a debiasing training dataset, where each piece of data includes an image, a text instruction, a true output, an inference result, and a set of hallucination clauses.

[0011] Furthermore, in step S2, a loss function is designed to forget the biased clauses, specifically:

[0012]

[0013] where I i and x i are the input image and the question respectively, δ i,j is the token in the biased clause in the model inference result, is the token in the model inference result before δ i,j . By optimizing this loss function, the probability of the model generating biased clauses can be minimized, thus achieving the debiasing goal; however, training only with the debiasing goal may weaken the original generation ability of the model; therefore, introduce the classic autoregressive loss as a regularization term in the loss function, that is:

[0014]

[0015] where y i,jis a token in the true label; by optimizing this loss function, the probability of the model generating true tokens can be maximized to guide the model to learn correct knowledge from the true output; finally, the two loss functions are combined to optimize the parameters of the LM head:

[0016]

[0017] where θ LM are the parameters of the LM Head, and α is a hyperparameter that controls the degree of machine forgetting. By adjusting the hyperparameter, the degree of bias removal can be controlled, so as to ensure that the model can fully learn the knowledge in the original training data while eliminating bias.

[0018] Adopting the above technical solution, the present invention has the following beneficial effects:

[0019] This patent screens out the inference results containing hallucination phenomena from the training data, and only performs debiasing on the hallucination clauses, avoiding misdeleting correct information, reducing duplicate penalties at the same time, minimizing the number of training tokens, and improving efficiency. In the training stage, this patent adopts a machine forgetting method based on gradient ascent, designs a loss function that combines a debiasing objective and autoregressive regularization, and retains the original generation ability of the model while eliminating bias. This method significantly reduces the training memory overhead and can complete the debiasing of a 7B-parameter model on a GPU with 24GB video memory. Through innovative technical solutions, this patent solves the deficiencies of existing methods in balancing generation quality and data dependence, realizes more efficient and stable debiasing training, and significantly improves the accuracy and quality of the model generation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 is the system architecture diagram of the multi-modal large model object hallucination mitigation method based on efficient machine forgetting to remove training bias provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0024] This patent aims to propose an efficient debiasing method by specifically eliminating the hallucination phenomenon in large vision - language models (LVLMs). This method collects the biased inference results of the model on the training data and fine - tunes the language model head (LM head) to achieve efficient utilization of data and parameters. The specific content of this method is as follows:

[0025] I. Collection of Biased Inference Results:

[0026] To construct a debiased training dataset, this patent first selects the instructions related to the image captioning task from the training data of the LVLM as the source dataset, where each piece of data contains an image input, a text instruction, and a ground - truth output. Then, this patent uses the LVLM to re - infer the images and text instructions in the source dataset to generate an inference result set. By comparing the object presence in the ground - truth output and the inference results, the inference results containing hallucination phenomena are filtered out.

[0027] Since hallucinated inference results usually contain both real objects and hallucinated objects, it is not appropriate to directly perform debiased training on the entire sentence. Therefore, this patent only performs debiasing on the clauses containing hallucinated objects. Specifically, we use common delimiters (such as commas and periods) to split the sentence into multiple clauses, and further use the correctly generated objects as delimiters to avoid accidentally deleting correct information. For each hallucinated object, this patent only collects the clause in which it first appears in the description to reduce the over - penalty for repeated objects and minimize the number of tokens for debiased training, improving efficiency. Finally, we obtain a debiased training dataset, where each piece of data contains an image, a text instruction, a ground - truth output, an inference result, and a set of hallucinated clauses. The overall process is as Figure 1 shown on the left.

[0028] II. Efficient Debiased Training:

[0029] Based on the collected deviation inference results, this patent adopts a machine forgetting method based on gradient ascent to perform debiasing training on the LM head, aiming to eliminate the specific biases learned by the model while retaining its general performance. Specifically, this patent designs a loss function for forgetting the biased clauses. This loss function achieves the debiasing goal by minimizing the probability of the model generating the biased clauses. However, training only with the debiasing goal may weaken the original generation ability of the model. Therefore, this patent introduces the classic autoregressive loss as a regularization term in the loss function to guide the model to learn correct knowledge from the true outputs. Finally, this patent combines the two loss functions to optimize the parameters of the LM head. By adjusting the hyperparameters to control the degree of debiasing, it ensures that the model can fully learn the knowledge in the original training data while eliminating biases. This method can not only efficiently eliminate the biases in the model but also reduce the memory overhead during the training process. This patent can complete the debiasing of a 7B-parameter model on a GPU with 24GB of video memory. The specific process is as Figure 1 shown on the right.

[0030] Multimodal large models have broad application prospects in real-world scenarios, such as in intelligent customer service, content generation, educational assistance, and other fields. However, the hallucination problem of multimodal large models (i.e., the model generates content that does not match the input or does not conform to the facts) seriously affects their reliability in real-world scenarios, especially in fields with extremely high accuracy requirements, such as auxiliary medical diagnosis and autonomous driving.

[0031] In auxiliary medical diagnosis, multimodal large models can combine patients' medical records, imaging data (such as X-rays, CT scans), and laboratory results to generate diagnostic suggestions or treatment plans. However, if the model has a hallucination problem, it may generate incorrect diagnostic or treatment suggestions, leading to serious medical accidents. For example, the model may misidentify a benign tumor as a malignant tumor or overlook key pathological features. The efficient debiasing method provided by this patent can effectively alleviate this problem, making the diagnostic suggestions generated by the model more accurate and reliable, thus improving the efficiency and safety of medical diagnosis.

[0032] In the field of autonomous driving, multimodal large models can integrate data from sensors such as cameras, radars, and lidars to generate perceptions and decision-making suggestions about the surrounding environment. However, the hallucination problem may cause the model to misidentify traffic signs, pedestrians, or other vehicles, thus triggering traffic accidents. For example, the model may misidentify a roadside billboard as a traffic signal or misidentify a stationary object as a moving obstacle. The debiasing method of this patent can significantly improve the model's perception accuracy of the real world, reduce misjudgments, and enhance the safety and reliability of the autonomous driving system.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for alleviating object hallucination in a multimodal large model based on efficient machine forgetting to remove training biases, characterized in that It includes the following steps: S1. Collection of deviation inference results: First, select the instructions related to the image description task from the training data of LVLM as the source dataset; each piece of data contains an image input, a text instruction, and a ground truth output; then, use LVLM to re-infer the images and text instructions in the source dataset to generate an inference result set; by comparing the object presence in the ground truth output and the inference results, filter out the inference results containing hallucination phenomena; S2. Efficient debiasing training: Based on the collected deviation inference results, use the machine forgetting method based on gradient ascent to debias the LMhead to eliminate the specific biases learned by the model while retaining its general performance.

2. The method for alleviating object hallucination of a multimodal large model based on efficiently removing training bias by machine forgetting according to claim 1, wherein In step S1, only the clauses containing hallucinated objects are debiased; specifically, use common delimiters to split the sentence into multiple clauses, and further use the correctly generated objects as delimiters to avoid accidentally deleting correct information; for each hallucinated object, only collect the clause where it first appears in the description to reduce the over-punishment for repeated objects and minimize the number of tokens for debiasing training to improve efficiency; Finally, obtain a debiased training dataset, where each piece of data contains an image, a text instruction, a ground truth output, an inference result, and a set of hallucination clauses.

3. The method for alleviating object hallucination of a multimodal large model based on efficient machine forgetting to remove training bias according to claim 1, characterized in that In step S2, a loss function is designed to forget the biased clauses, specifically: where i i and x i are the input image and the question respectively, and δ i,j is a token in the deviation clause in the model inference result, and i is a token before δ j in the model inference result; by optimizing this loss function, the probability of the model generating a deviation clause can be minimized, thus achieving the debiasing goal; however, training only with the debiasing goal may weaken the original generation ability of the model; therefore, a classical autoregressive loss is introduced as a regularization term in the loss function, that is: where y i,j is a token in the true label; by optimizing this loss function, the probability of the model generating true tokens is maximized to guide the model to learn correct knowledge from the true output; finally, the two loss functions are combined to optimize the parameters of the LM head: where θ LM is a parameter of the LM Head, and α is a hyperparameter that controls the degree of machine forgetting; by adjusting the hyperparameter, the degree of debiasing is controlled, so as to ensure that the model can fully learn the knowledge in the original training data while eliminating the bias.

Images