Large model illusion mitigation method and apparatus, storage medium

Abstract

The present application provides a large model illusion alleviation method and device, and a storage medium. The method comprises the following steps: receiving a given image and prompt text as input, generating a plurality of text samples through a large-scale visual language model; comparing the inconsistencies in the plurality of text samples, searching for a contradiction point, and determining an illusion factor; generating a question for the illusion factor; performing question and answer according to the question, analyzing the image content by using a visual question and answer model, outputting an answer to the question, and correcting the prompt text by using the answer; and generating an optimized text sample by using the corrected prompt text. The method can effectively alleviate the illusion generated by the large-scale visual language model and improve the effect of the large model generation.

Description

Technical Field

[0001] This invention belongs to the field of visual language large model, specifically relating to a method, device, and storage medium for alleviating large model illusions. Background Technology

[0002] In recent years, large-scale language models (LLMs) have had a tremendous impact on the field of artificial intelligence. With training data reaching hundreds of billions of parameters, they have unleashed powerful emergent capabilities and garnered widespread attention and application. Following this trend, large-scale visual language models (LVLMs) extend LLMs to understand visual input and handle multimodal tasks through pre-trained visual-language alignment. These works have driven the development of multimodal general artificial intelligence, particularly demonstrating surprising capabilities in zero-shot tasks.

[0003] Currently, LVLMs, like LLMs, face some new challenges, one of the most serious being the tendency for models to generate illusions. In language models, illusions are defined as inaccurate or misleading factual information generated by the model, mostly stemming from incorrect knowledge learned from training data. Existing work has utilized external knowledge bases to assist generation and mitigate illusions. However, for visual language models, illusions manifest as incorrect references or descriptions of the input image in the output response. Accurately representing visual information in abstract visual encoding is a greater challenge. In addressing these problems, multimodal illusions are often more difficult to detect automatically, requiring costly human supervision. Existing work typically explores solutions based on instruction tuning or decoding constraints. LRV-Instruction uses a relatively intuitive approach, limiting the text length of the data through instructions to reduce illusions, but this obviously prevents the model from generating very detailed descriptions. VIGC employs a multi-step generation scheme, mitigating illusions by sacrificing the cost and efficiency of fine-tuning. Opera penalizes specific words during decoding to alleviate illusions, but this only applies to certain special tokens and also significantly shortens the generated text. These instruction tuning models and decoding constraint methods typically require a large amount of data and computing resources from humans.

[0004] Therefore, how to alleviate the illusions generated by large-scale visual language models in a low-cost and efficient manner, so as to improve their accuracy and reliability in real-world applications, remains an urgent problem to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method, device, and storage medium for alleviating large-scale model hallucinations. Based on contradiction point localization and question-and-answer correction, it can effectively alleviate hallucinations generated by large-scale visual language models and improve the effect of large-scale model generation.

[0006] To achieve the above objectives, the present invention provides a method for alleviating large-scale hallucinations, comprising:

[0007] Given an image and prompt text as input, generate multiple text samples using a large-scale visual language model;

[0008] By comparing inconsistencies among multiple text samples, contradictions were identified, and hallucination factors were determined.

[0009] Generate questions targeting this hallucination factor;

[0010] Based on the question, a visual question-answering model is used to analyze the image content, output the answer to the question, and use the answer to correct the prompt text.

[0011] Using the revised prompt text, an optimized text sample is regenerated.

[0012] In one embodiment, for the plurality of text samples, inconsistencies in the content of the plurality of text samples are identified by comparison. These inconsistencies represent words with a flat probability distribution in the generated prediction, and are used as illusion factors.

[0013] In one embodiment, a distillation learning method is used to train a small-scale question generation model to generate questions for the hallucination factor.

[0014] In one embodiment, the small-scale problem generation model is trained through the following steps:

[0015] Generate a large amount of training data in a specified format using a generative pre-trained model;

[0016] Using this training data, the small-scale problem generation model was fine-tuned.

[0017] Using these multiple text samples as input, the small-scale question generation model learns to generate questions about hallucination factors from these multiple text samples.

[0018] In one embodiment, multiple text samples are obtained using various sampling methods.

[0019] In one embodiment, it further includes:

[0020] The optimized text samples were evaluated to verify the effectiveness of hallucination relief.

[0021] In another aspect, the present invention provides a large-scale hallucination relief device, which employs the above-described large-scale hallucination relief method, and the device comprises at least:

[0022] The sampling generation module receives a given image and prompt text as input and generates multiple text samples through a large-scale visual language model.

[0023] The hallucination localization module is used to compare inconsistencies among multiple text samples, search for contradictions, and determine hallucination factors.

[0024] The question generation module is used to generate questions related to this hallucination factor;

[0025] The correction module performs a question-and-answer process based on the question, analyzes the image content using a visual question-and-answer model, outputs the answer to the question, and uses the answer to correct the prompt text; and

[0026] Using the revised prompt text, an optimized text sample is regenerated.

[0027] In one embodiment, the hallucination localization module is further configured to:

[0028] For these multiple text samples, inconsistencies in the content of the multiple text samples were identified by comparison. These inconsistencies represent words with a flat probability distribution in the prediction process, which serve as illusion factors.

[0029] In one embodiment, the problem generation module is further configured to:

[0030] Using a distillation learning method, a small-scale question generation model is trained to generate questions related to this hallucination factor, including:

[0031] Generate a large amount of training data in a specified format using a generative pre-trained model;

[0032] Using this training data, the small-scale problem generation model was fine-tuned.

[0033] Using these multiple text samples as input, the small-scale question generation model learns to generate questions about hallucination factors from these multiple text samples.

[0034] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described large model illusion relief method.

[0035] As can be seen from the above solutions, the advantages of the present invention are:

[0036] The large model illusion mitigation method disclosed in this invention is based on contradiction point localization and question-and-answer correction. The method can perform comparative detection, proactive questioning and answer correction, etc., on uncertain information during the model generation process, thereby effectively optimizing the generation effect of large models without relying on additional manually labeled data and instruction fine-tuning training. Attached Figure Description

[0037] Figure 1A schematic diagram of the overall process of a large-scale hallucination relief method provided by an embodiment of the present invention is shown;

[0038] Figure 2 A demonstration diagram showing a specific example of applying the large-model hallucination relief method of the present invention is shown;

[0039] Figure 3 A framework diagram corresponding to the large-scale hallucination relief device is shown;

[0040] Figure 4 A schematic block diagram of an electronic device is shown.

[0041] in:

[0042] 200: Large-scale hallucination relief device;

[0043] 210: Generate sampling module;

[0044] 220: Illusion Location Module;

[0045] 230: Problem generation module;

[0046] 240: Correction module;

[0047] 300: Electronic devices;

[0048] 301: Computational unit;

[0049] 302: Read-only memory;

[0050] 303: Random Access Memory;

[0051] 304: Bus;

[0052] 305: I / O interface;

[0053] 306: Input unit;

[0054] 307: Output unit;

[0055] 308: Storage unit;

[0056] 309: Communication unit. Detailed Implementation

[0057] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0058] Please see Figure 1 , Figure 2 As shown, Figure 1 This diagram illustrates the overall flow of a large-scale hallucination relief method according to an embodiment of the present invention. Figure 2A demonstration diagram shows a specific example of applying the large-model hallucination relief method of the present invention. This large-model hallucination relief method includes the following steps:

[0059] Step S1: Receive the given image and prompt text as input, and generate multiple text samples through a large-scale visual language model.

[0060] In this embodiment, a given image and a prompt text describing the image are taken as input. Through a large-scale visual language model, the task objective is to output text describing the content of the image based on the requirements of the prompt text, thereby generating multiple text samples.

[0061] Given the randomness inherent in sampling from the generative model, the same prompt text may produce different output texts. In some embodiments, multiple sampling methods are employed to obtain these multiple text samples. Examples include top-k and top-p sampling methods. In top-k sampling, at each generation step, the model selects the top K tokens with the highest probabilities from all possible tokens (words or tokens) as a candidate set, and then randomly selects one token from this set as the output. This sampling strategy, by limiting the size of the candidate set, can control the diversity and quality of the generated text to some extent. Top-p sampling is a strategy that dynamically selects candidate words from the probability distribution predicted by the model. At each generation step, the model generates a set of candidate tokens (words or tokens), then selects the smallest set from these candidate words whose cumulative probability reaches or exceeds a set threshold P, and randomly selects one token from this set as the output. Top-p sampling, by dynamically adjusting the size and randomness of the candidate word set, can increase the diversity of the text while maintaining the quality of the generated text.

[0062] For example Figure 2 As shown, based on the input image and the prompt text "Briefly describe the image", multiple sample texts are generated, such as "A woman with a brown dog playing on the beach", "The woman is petting the dog on the beach.", "A woman is holding her cell phone and looking at the dog.", and "A girl and a white dog."

[0063] Step S2: Compare the inconsistencies among the multiple text samples, search for contradictions, and determine the hallucination factors.

[0064] In this embodiment, after acquiring multiple text samples, inconsistencies among these samples are further compared to identify contradictions. These contradictions serve as uncertainty information for the hallucination-inducing model, and hallucination factors are generated based on these contradictions. For example... Figure 2 As shown, by comparing the inconsistencies between the sampled texts “Awoman with a brown dog playing on the beach,” “The woman is petting the dog on the beach,” “A woman is holding her cellphone and looking at the dog,” and “A girl and a white dog,” it can be identified that “a brown dog” and “a white dog,” and “petting the dog” and “holding her cell phone” are contradictions in the sampled texts. Based on these contradictions, the hallucination elements “The Object in the woman's hand” and “The color of the dog” can be generated.

[0065] In practical applications, large-scale visual language models are generative models that receive input instructions and generate responses. The model is pre-trained on a large text corpus, learning to predict the next word during generation. This process endows the model with powerful understanding and reasoning abilities, but also introduces a degree of randomness into the output. Considering the next word the model will predict, if the relevant elements are of high confidence to the model, then that factual word should be assigned a high probability score, meaning it can be generated relatively stably. Conversely, when predicting uncertain information, the model is more likely to produce a flatter probability distribution, resulting in variable outcomes across multiple generation attempts. During reasoning, these varying predicted words across multiple generation attempts are more likely to be non-factual words, a phenomenon known as model hallucination. This phenomenon allows for a connection between the model's uncertainty and the hallucination problem.

[0066] In view of this, in one embodiment, for the plurality of text samples, inconsistencies in the content of the plurality of text samples are identified by comparison. These inconsistencies represent words with a flat probability distribution in the generated prediction, which serve as illusion factors.

[0067] Step S3: Generate questions targeting this hallucination factor.

[0068] To further identify factual vocabulary related to hallucinations, an automated question-and-answer method will be used to extract information. In this embodiment, corresponding questions will be generated specifically targeting hallucination factors, such as... Figure 2In the above-mentioned hallucination factors "The Object in the woman's hand" and "The color of the dog", the corresponding questions are generated as "Is there a cell phone in her hand?" and "What color is the dog?", respectively.

[0069] In addition, to further improve processing efficiency and reduce computational costs, the method also employs distillation learning to train a small-scale problem generation model to generate problems targeting the illusion factor, thereby obtaining the capabilities required for the task at a lower cost.

[0070] Specifically, the small-scale question generation model is trained through the following steps: First, a large amount of training data in a specified format is generated using a generative pre-trained model; then, the small-scale question generation model is fine-tuned using this training data. Next, using these multiple text samples as input, the fine-tuned small-scale question generation model learns to generate questions about hallucination factors from these text samples, and can then output questions about those hallucination factors.

[0071] In some embodiments, the generative pre-trained model can be ChatGPT, and the small-scale problem generation model can be T5-small, but the present invention is not limited thereto.

[0072] Step S4: Based on the question, perform a question-and-answer session, analyze the image content using a visual question-and-answer model, output the answer to the question, and use the answer to correct the prompt text.

[0073] Since the above question addresses uncertain information for the model, the answer can be directly obtained by analyzing the given image content using a visual question-answering model. This answer represents accurate information. This answer is then added to the prompt text of the larger model, allowing it to be revised and compelling the model to focus on and accept this correct information while discarding previously uncertain predictions. For example... Figure 2As shown, based on the questions "Is there a cell phone in her hand?" and "What color is the dog?", the image is analyzed, and the answers are "There is no cell phone in the image" and "The dog is brown." Further using these answers, the initial prompt text "Briefly describe the image" is revised to "Briefly describe the image. Note that the dog is brown and there is no cell phone in the image."

[0074] Step S5: Using the corrected prompt text, regenerate the optimized text sample.

[0075] In this embodiment, after obtaining the corrected prompt text, an optimized text sample is generated using a large-scale visual language model, utilizing the given image and the corrected prompt text. Furthermore, the optimized text sample is evaluated to verify the hallucination relief effect. For example... Figure 2 As shown, the corrected prompt text "Briefly describe the image. Note that the dog is brown and there is no cell phone in the image" yields the optimized text sample "A woman is sitting on the beach and playing with a brown dog."

[0076] This shows that by using the corrected prompt text to regenerate the optimized text sample and remove uncertain information, the image content can be better described. In other words, the large-scale visual language model will achieve better generation results than the original, that is, the hallucinatory information in the generated content is significantly reduced, and the hallucination problem of the large model is effectively alleviated.

[0077] In summary, the large-scale model hallucination mitigation method disclosed in this invention is based on contradiction point localization and question-and-answer correction. The method can perform comparative detection, proactive questioning, and answer correction on uncertain information generated during the model generation process. This allows for effective optimization of the large model's generation performance without relying on additional manually labeled data and instruction-based fine-tuning. Specifically, the method first identifies contradiction points from multiple text samples generated by a large-scale visual language model as uncertain information in the model that causes hallucinations, i.e., hallucination factors. The detection of these factors also relies on the understanding and reasoning abilities of the large model. Then, corresponding questions are generated for the hallucination factors. To further improve processing efficiency and reduce computational costs, a distillation learning approach is used to train a small-scale question generation model, achieving the required capabilities at a lower cost. Finally, answers are provided to the questions addressing the hallucination factors, and the prompt text is corrected. Based on the corrected prompt text, optimized text samples are regenerated, eliminating uncertain information and achieving better generation results. Each step in this process is clear and transparent, providing good interpretability. Furthermore, because the questioning mechanism is based on a lightweight, pluggable, small-scale model, the framework is flexible and efficient.

[0078] In addition, corresponding to the aforementioned embodiments of the large model hallucination relief method, the present invention also provides embodiments of the large model hallucination relief device.

[0079] See Figure 3 The present invention provides a large-scale hallucination relief device 200, which includes:

[0080] The sampling generation module 210 receives a given image and prompt text as input and generates multiple text samples through a large-scale visual language model.

[0081] The hallucination localization module 220 is used to compare inconsistencies among multiple text samples, search for contradictions, and determine hallucination factors.

[0082] Question generation module 230 is used to generate questions related to the hallucination factor.

[0083] Correction module 240 performs question-and-answer processing based on the question, analyzes the image content using a visual question-and-answer model, outputs an answer to the question, and corrects the prompt text using the answer; and

[0084] Using the revised prompt text, an optimized text sample is regenerated.

[0085] Furthermore, in some embodiments, the illusion localization module 220 is also used to: for the plurality of text samples, compare and identify inconsistencies in the content of the plurality of text samples, the inconsistencies representing words with a flat probability distribution in the generated prediction, as illusion factors.

[0086] Furthermore, in some embodiments, the question generation module 230 is also used to: train a small-scale question generation model using a distillation learning method to generate questions for the hallucination factor, including: generating a large amount of training data in a specified form using a generative pre-trained model; fine-tuning the small-scale question generation model using the training data; and using the multiple text samples as input, having the small-scale question generation model learn to generate questions for the hallucination factor from the multiple text samples.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of each module of the described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0088] According to embodiments of the present invention, an electronic device is also provided.

[0089] Figure 4 A schematic block diagram of an electronic device 300 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown in this invention, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0090] Electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0091] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0092] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the processing of each step of the large model illusion mitigation method described above, such as steps S1 to S5. For example, in some embodiments, steps S1 to S5 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by computing unit 301, one or more steps of steps S1 to S5 of the large model illusion mitigation method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform steps S1 to S5 by any other suitable means (e.g., by means of firmware).

[0093] Various embodiments of the systems and techniques described above in this invention can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this invention does not impose any limitations on them.

[0097] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for alleviating large-scale hallucinations, characterized in that, Include: The system receives a given image and prompt text as input and generates multiple text samples using a large-scale visual language model; the multiple text samples are obtained by using various different sampling methods. For these multiple text samples, inconsistencies in the content of the multiple text samples were identified by comparison. These inconsistencies represent words with a flat probability distribution in the prediction generation process, which serve as illusion factors. Using the distillation learning method, a small-scale question generation model is trained to generate questions targeting this hallucination factor; Based on the question, a visual question-answering model is used to analyze the image content, output the answer to the question, and use the answer to correct the prompt text. Using the revised prompt text, regenerate the optimized text sample; The small-scale problem generation model is trained through the following steps: Generate a large amount of training data in a specified format using a generative pre-trained model; Using this training data, the small-scale problem generation model was fine-tuned. Using these multiple text samples as input, the small-scale question generation model learns to generate questions about hallucination factors from these multiple text samples.

2. The method according to claim 1, characterized in that, Also includes: The optimized text samples were evaluated to verify the effectiveness of hallucination relief.

3. A large-scale hallucination relief device, characterized in that, Include: A sampling generation module is used to receive a given image and prompt text as input, and generate multiple text samples through a large-scale visual language model; in this process, multiple text samples are obtained by using various sampling methods. The hallucination localization module is used to compare inconsistencies among multiple text samples, search for contradictions, and determine hallucination factors. The question generation module is used to generate questions related to this hallucination factor; The correction module performs a question-and-answer process based on the question, analyzes the image content using a visual question-and-answer model, outputs the answer to the question, and uses the answer to correct the prompt text; and Using the revised prompt text, regenerate the optimized text sample; The hallucination positioning module is also used for: For these multiple text samples, inconsistencies in the content of the multiple text samples were identified by comparison. These inconsistencies represent words with a flat probability distribution in the prediction generation process, which serve as illusion factors. The question generation module is also used for: Using a distillation learning method, a small-scale question generation model is trained to generate questions related to this hallucination factor, including: Generate a large amount of training data in a specified format using a generative pre-trained model; Using this training data, the small-scale problem generation model was fine-tuned. Using these multiple text samples as input, the small-scale question generation model learns to generate questions about hallucination factors from these multiple text samples.

4. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the large-model hallucination relief method according to any one of claims 1-2.

Images