Large model illusion relieving method and device and storage medium

Abstract

The invention provides a large model illusion relieving method and device and a storage medium, and the method comprises the steps: receiving a given image and a prompt text as input, and generating a plurality of text samples through a large-scale visual language model; comparing inconsistencies in the plurality of text samples, searching out contradiction points, and determining illusion factors; generating a problem for the illusion factor; performing question and answer according to the question, analyzing the image content by using a visual question and answer model, outputting an answer for the question, and correcting the prompt text by using the answer; and regenerating an optimized text sample by using the corrected prompt text. The method can effectively relieve illusion generated by a large-scale visual language model and improve the effect of large model generation.

Description

Technical Field

[0001] The invention belongs to the field of large models of visual language, and in particular relates to a large model hallucination relief method, device and storage medium. Background Art

[0002] In recent years, large-scale language models (LLMs) have had a huge impact on the field of artificial intelligence. With hundreds of billions of parameters in training data, they have unleashed powerful emergent capabilities and have received widespread attention and application. Following this trend, large-scale visual language models (LVLMs) have extended LLMs to understand visual inputs and handle multimodal tasks by pre-training visual-language alignment. These works have promoted the development of multimodal general artificial intelligence, especially demonstrating surprising capabilities on zero-shot tasks.

[0003] At present, LVLMs also face some new challenges like LLMs. One of the serious problems is that the model tends to produce hallucinations. In language models, hallucinations are defined as inaccurate or misleading factual information generated by the model. Most of these hallucinations are caused by incorrect knowledge learned from training data. Some existing works have used external knowledge bases to assist in generation to alleviate hallucinations. For visual language models, hallucinations are manifested as the output response has incorrect references or descriptions for the input image. Accurately expressing visual information in abstract visual encoding is a greater challenge. When solving such problems, multimodal hallucinations are often more difficult to detect automatically and require expensive manual supervision costs. Existing work usually explores solutions based on instruction tuning or decoding constraints. LRV-Instruction adopts a more intuitive method to reduce hallucinations by limiting the text length of the data through instructions, but obviously this will prevent the model from generating very detailed descriptions. VIGC adopts a multi-step generation scheme to alleviate hallucinations by sacrificing the cost and efficiency of fine-tuning. Opera penalizes specific words during decoding to alleviate hallucinations, but it only applies to some special tags and will also significantly shorten the generated text. These instruction tuning model-based and decoding constraint-based methods usually require humans to provide a large amount of data and computing resources.

[0004] Therefore, how to mitigate the hallucinations produced by large-scale visual language models in a low-cost and efficient manner to improve their accuracy and reliability in real-world applications remains an urgent problem to be solved. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a large model hallucination alleviation method, device, and storage medium, which are based on contradiction point positioning and question-answer correction, can effectively alleviate the hallucinations caused by large-scale visual language models and improve the effect of large model generation.

[0006] In order to achieve the above object, the present invention provides a large model hallucination relief method, comprising:

[0007] Receive a given image and prompt text as input and generate multiple text samples through a large-scale visual language model;

[0008] Compare the inconsistencies in the multiple text samples, search for contradictions, and determine the hallucination factors;

[0009] Generate questions that target this illusion factor;

[0010] Performing question-answering according to the question, analyzing the image content using a visual question-answering model, outputting an answer to the question, and modifying the prompt text using the answer;

[0011] Use the corrected prompt text to regenerate the optimized text sample.

[0012] In one embodiment, for the multiple text samples, conflicting points of inconsistent content in the multiple text samples are compared and identified, and the conflicting points represent words with a flat probability distribution in generating predictions, which serve as hallucination factors.

[0013] In one embodiment, a distillation learning method is used to train a small-scale question generation model to generate questions targeting the hallucination factor.

[0014] In one embodiment, the small-scale question generation model is trained by the following steps:

[0015] Use a generative pre-trained model to generate a large amount of training data in a specified format;

[0016] Using the training data, fine-tune the small-scale problem generation model;

[0017] Taking the multiple text samples as input, the small-scale question generation model learns to generate questions targeting hallucination factors from the multiple text samples.

[0018] In one embodiment, a plurality of different sampling methods are adopted to obtain the plurality of text samples.

[0019] In one embodiment, it also includes:

[0020] The optimized text samples are evaluated to verify the effect of hallucination alleviation.

[0021] Another aspect of the present invention is to provide a large model hallucination mitigation device, which adopts the large model hallucination mitigation method described above, and at least comprises:

[0022] A generation sampling module is used to receive a given image and prompt text as input and generate multiple text samples through a large-scale visual language model;

[0023] A hallucination location module is used to compare the inconsistencies in the multiple text samples, search for contradictions, and determine hallucination factors;

[0024] A question generation module, used for generating questions targeting the hallucination factor;

[0025] a correction module, which performs question-answering according to the question, analyzes the image content using a visual question-answering model, outputs an answer to the question, and corrects the prompt text using the answer; and

[0026] Use the corrected prompt text to regenerate the optimized text sample.

[0027] In one embodiment, the illusion positioning module is further used to:

[0028] For the multiple text samples, the inconsistency points of the content in the multiple text samples are compared and identified, and the inconsistency points represent words with a flat probability distribution in the generation prediction process, which serve as hallucination factors.

[0029] In one embodiment, the question generation module is further used to:

[0030] Using the distillation learning method, a small-scale question generation model is trained to generate questions targeting the hallucination factor, including:

[0031] Use a generative pre-trained model to generate a large amount of training data in a specified format;

[0032] Using the training data, fine-tune the small-scale problem generation model;

[0033] Taking the multiple text samples as input, the small-scale question generation model learns to generate questions targeting hallucination factors from the multiple text samples.

[0034] In yet another aspect, the present invention provides a computer-readable storage medium storing a computer program, which implements the steps of the large model hallucination mitigation method when executed by a processor.

[0035] It can be seen from the above scheme that the advantages of the present invention are:

[0036] The large model hallucination mitigation method disclosed in the present invention is based on contradiction point location and question and answer correction. The method can perform comparative detection, active questioning and answer correction on uncertain information in the model generation process, thereby effectively optimizing the generation effect of the large model without relying on additional manually labeled data and instruction fine-tuning training. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1A schematic diagram of the overall process of a large model hallucination mitigation method provided by an embodiment of the present invention is shown;

[0038] Figure 2 A demonstration diagram showing a specific example of applying the large model hallucination mitigation method of the present invention;

[0039] Figure 3 A framework diagram corresponding to a large-scale model illusion relief device is shown;

[0040] Figure 4 A schematic block diagram of an electronic device is shown.

[0041] in:

[0042] 200: Large model hallucination relief device;

[0043] 210: Generate a sampling module;

[0044] 220: hallucination positioning module;

[0045] 230: question generation module;

[0046] 240: Correction module;

[0047] 300: electronic equipment;

[0048] 301: computing unit;

[0049] 302: read-only memory;

[0050] 303: Random access memory;

[0051] 304: bus;

[0052] 305: I / O interface;

[0053] 306: input unit;

[0054] 307: output unit;

[0055] 308: storage unit;

[0056] 309: Communication unit. DETAILED DESCRIPTION

[0057] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0058] See also Figure 1 , Figure 2 As shown, Figure 1 The overall flow chart of the large model hallucination mitigation method provided by one embodiment of the present invention is shown. Figure 2A demonstration diagram of a specific example of applying the large model hallucination mitigation method of the present invention is shown. The large model hallucination mitigation method comprises the following steps:

[0059] Step S1: receiving a given image and prompt text as input, and generating multiple text samples through a large-scale visual language model.

[0060] In this embodiment, a given image and a prompt text used to describe the image are taken as input. Through a large-scale visual language model, the task goal is to output text describing the image content according to the requirements of the prompt text and generate multiple text samples.

[0061] Given that sampling from a generative model involves randomness, the same prompt text may produce different output texts. In some embodiments, a variety of different sampling methods are used to obtain the multiple text samples. For example, top-k and top-p sampling methods. Top-k sampling is that in each step of the generation process, the model selects the top K tokens with the highest probability from all possible tokens (words or tags) as a candidate set, and then randomly selects a token from this set as the output. This sampling strategy can control the diversity and quality of the generated text to a certain extent by limiting the size of the candidate set. Top-p sampling is a strategy for dynamically selecting candidate words from the probability distribution predicted by the model. In each step of the generation process, the model generates a set of candidate tokens (words or tags), and then selects the minimum set with a cumulative probability that reaches or exceeds a set threshold P from these candidate words, and randomly selects a token from this set as the output. Top-p sampling can increase the diversity of text while maintaining the quality of the generated text by dynamically adjusting the size and randomness of the candidate word set.

[0062] For example Figure 2 As shown in , based on the input image and the prompt text “Briefly describe the image”, multiple sample texts are generated, such as “A woman with a brown dog playing on the beach”, “The woman is petting the dog on the beach.”, “A woman is holding her cell phone and looking at the dog.”, and “A girl and a white dog.”.

[0063] Step S2: Compare the inconsistencies in the multiple text samples, search for contradictions, and determine the hallucination factors.

[0064] In this embodiment, after obtaining multiple text samples, the inconsistencies in the multiple text samples are further compared to search for contradictions therein. These contradictions are used as uncertainty information of the model causing hallucinations, and hallucination factors are generated based on these contradictions. Figure 2 As shown in , by comparing the inconsistencies of the above sampled texts “A woman with a brown dog playing on the beach”, “The woman is petting the dog on the beach.”, “A woman is holding her cell phone and looking at the dog.”, and “A girl and a white dog.”, it can be identified that “a brown dog” and “a white dog”, “petting the dog” and “holding her cell phone” in the sampled texts are contradictions, and based on these contradictions, the hallucination factors “The Object in the woman's hand” and “The color of the dog” can be generated.

[0065] In practical applications, a large-scale visual language model is a generative model that receives instruction input and generates responses. The model is pre-trained on a large text corpus and learns to predict the next word when generating. This process gives the model powerful understanding and reasoning capabilities, and also makes the output have a certain degree of randomness. Considering the next word to be predicted by the model, if the relevant elements are of high confidence to the model, then the factual word should be assigned a high probability score, that is, it can be generated relatively stably. Conversely, when the model predicts uncertain information, it is more likely to produce a flatter probability distribution, so that the results of multiple generations will be variable. In the reasoning process, the predicted words that change in multiple generations are more likely to be non-factual words, that is, model hallucinations. This phenomenon makes it possible to link the uncertainty information of the model with the hallucination problem.

[0066] In view of this, in one embodiment, for the multiple text samples, the inconsistencies in the content of the multiple text samples are compared and identified. These inconsistencies represent words with a flat probability distribution in the generated prediction, which serve as hallucination factors.

[0067] Step S3: Generate questions targeting the hallucination factor.

[0068] In order to further learn the factual vocabulary of hallucination factors, an automatic question-answering method is used to extract information. In this embodiment, corresponding questions are further generated for hallucination factors, such as Figure 2In the example, for the above-mentioned illusion factors “The Object in the woman's hand” and “The color of the dog”, the corresponding questions generated are “Is there a cell phone in her hand?” and “What color is the dog?”.

[0069] In addition, in order to further improve processing efficiency and reduce computing costs, the method also uses distillation learning methods to train a small-scale question generation model to generate questions targeting the hallucination factor and obtain the capabilities required for the task at a lower cost.

[0070] Specifically, the small-scale question generation model is trained by the following steps: first, a large amount of training data in a specified form is generated using a generative pre-training model; and the small-scale question generation model is fine-tuned using the training data. Then, the fine-tuned small-scale question generation model learns to generate questions for hallucination factors from the multiple text samples using the multiple text samples, and can output questions for the hallucination factors.

[0071] In some embodiments, the generative pre-training model may select ChatGPT, and the small-scale problem generation model may select T5-small, but the present invention is not limited to this.

[0072] Step S4: conduct question-answering based on the question, use a visual question-answering model to analyze the image content, output an answer to the question, and use the answer to correct the prompt text.

[0073] Since the above questions are about uncertain information of the model, the visual question answering model can be used to directly analyze the given image content to obtain the answer to the question, which is accurate information. Then the answer is added to the prompt text of the large model, and the prompt text is corrected with the answer, forcing the large model to pay attention to and accept the correct information and abandon the original uncertain prediction. Figure 2As shown in FIG, the image is analyzed based on the questions “Is there a cell phone in her hand?” and “What color is the dog?” and the answers to the questions are “There is no cell phone in the image” and “The dog is brown”. The initial prompt text “Briefly describe the image” is further modified to “Briefly describe the image. Note that the dog is brown and there is no cell phone in the image” using the answers.

[0074] Step S5: Regenerate an optimized text sample using the corrected prompt text.

[0075] In this embodiment, after obtaining the corrected prompt text, the given image and the corrected prompt text are further used to generate an optimized text sample by a large-scale visual language model. In addition, the optimized text sample is further evaluated to verify the effect of hallucination relief. For example Figure 2 As shown in the figure, the optimized text sample "A woman is sitting on the beach and playing with a brown dog." is obtained from the corrected prompt text "Briefly describe the image. Note that the dog is brown and there is no cell phone in the image".

[0076] From this, it can be seen that by using the corrected prompt text, the uncertain information can be removed from the regenerated optimized text sample to better describe the image content. That is, the large-scale visual language model will obtain better generation effects than the original one, that is, the hallucination information in the generated content is significantly reduced, and the hallucination problem of the large model is effectively alleviated.

[0077] In summary, the large model hallucination mitigation method disclosed by the present invention is based on the location of contradiction points and question-answer correction. The method can perform operations such as comparative detection, active questioning and answer correction on uncertain information in the model generation process. Thus, it is possible to effectively optimize the generation effect of the large model without relying on additional manually labeled data and instruction fine-tuning training. Specifically, the method first finds contradictions from multiple text samples generated by a large-scale visual language model as uncertainty information of the model that causes hallucinations, that is, hallucination factors, and the detection of such factors also depends on the understanding and reasoning ability of the large model. Then, corresponding questions are generated for the hallucination factors. In this process, in order to further improve processing efficiency and reduce computing costs, the idea of ​​distillation learning is used to train a small-scale question generation model to obtain the capabilities required for the task at a lower cost. Finally, according to the questions about the hallucination factors, the prompt text is answered and corrected, and the optimized text sample is regenerated based on the corrected prompt text, so that the uncertain information is eliminated from the regenerated text sample, that is, a better generation effect is achieved. Each step in the process of the method is clear and transparent, which also provides good interpretability. And because the questioning mechanism is based on a lightweight, pluggable small-scale model, the framework is also flexible and efficient.

[0078] In addition, corresponding to the above-mentioned embodiment of the large model hallucination alleviation method, the present invention also provides an embodiment of the large model hallucination alleviation device.

[0079] See also Figure 3 , a large model hallucination mitigation device 200 provided by an embodiment of the present invention comprises:

[0080] The sampling generation module 210 is used to receive a given image and a prompt text as input, and generate multiple text samples through a large-scale visual language model.

[0081] The hallucination location module 220 is used to compare the inconsistencies in the multiple text samples, search for contradictions, and determine hallucination factors.

[0082] The question generating module 230 is used to generate questions targeting the hallucination factor.

[0083] A correction module 240 performs question-answering according to the question, analyzes the image content using a visual question-answering model, outputs an answer to the question, and corrects the prompt text using the answer; and

[0084] Use the corrected prompt text to regenerate the optimized text sample.

[0085] In addition, in some embodiments, the hallucination localization module 220 is further used to: compare and identify inconsistencies in the content of the multiple text samples, where the inconsistencies represent words with a flat probability distribution in the generated prediction, as hallucination factors.

[0086] In addition, in some embodiments, the question generation module 230 is also used to: use a distillation learning method to train a small-scale question generation model to generate questions for the hallucination factor, including: using a generative pre-training model to generate a large amount of training data in a specified format; using the training data to fine-tune the small-scale question generation model; using the multiple text samples as input, the small-scale question generation model learns to generate questions for the hallucination factor from the multiple text samples.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of each module of the described device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0088] According to an embodiment of the present invention, the present invention further provides an electronic device.

[0089] Figure 4 A schematic block diagram of an electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown in the present invention, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required in the present invention.

[0090] The electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0091] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0092] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs the processing of each step of the large model hallucination mitigation method described above, such as steps S1 to S5. For example, in some embodiments, steps S1 to S5 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of steps S1 to S5 of the large model hallucination mitigation method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform steps S1 to S5 in any other appropriate manner (eg, by means of firmware).

[0093] Various embodiments of the systems and techniques described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and the present invention is not limited here.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for alleviating large model hallucinations, It is characterized in that Include: Receive a given image and prompt text as input and generate multiple text samples through a large-scale visual language model; Compare the inconsistencies in the multiple text samples, search for contradictions, and determine the hallucination factors; Generate questions that target this illusion factor; Performing question-answering according to the question, analyzing the image content using a visual question-answering model, outputting an answer to the question, and modifying the prompt text using the answer; Use the corrected prompt text to regenerate the optimized text sample.

2. The method according to claim 1, It is characterized in that For the multiple text samples, the inconsistency points of the content in the multiple text samples are compared and identified, and the inconsistency points represent words with a flat probability distribution in the generation prediction process, which serve as hallucination factors.

3. The method according to claim 1, It is characterized in that Using the distillation learning method, a small-scale question generation model is trained to generate questions targeting the hallucination factor.

4. The method according to claim 3, It is characterized in that The small-scale question generation model is trained by the following steps: Use a generative pre-trained model to generate a large amount of training data in a specified format; Using the training data, fine-tune the small-scale problem generation model; Taking the multiple text samples as input, the small-scale question generation model learns to generate questions targeting hallucination factors from the multiple text samples.

5. The method according to claim 1, It is characterized in that A plurality of different sampling methods are adopted to obtain the plurality of text samples.

6. The method according to claim 1, It is characterized in that Also includes: The optimized text samples are evaluated to verify the effect of hallucination alleviation.

7. A large-scale hallucination relief device, It is characterized in that Include: A generation sampling module is used to receive a given image and prompt text as input and generate multiple text samples through a large-scale visual language model; A hallucination location module is used to compare the inconsistencies in the multiple text samples, search for contradictions, and determine hallucination factors; A question generation module, used for generating questions targeting the hallucination factor; A correction module performs question-answering according to the question, analyzes the image content using a visual question-answering model, outputs an answer to the question, and corrects the prompt text using the answer; as well as Use the corrected prompt text to regenerate the optimized text sample.

8. The device according to claim 7, It is characterized in that The hallucination positioning module is also used to: For the multiple text samples, the inconsistency points of the content in the multiple text samples are compared and identified, and the inconsistency points represent words with a flat probability distribution in the generation prediction process, which serve as hallucination factors.

9. The device according to claim 7, It is characterized in that The question generation module is also used to: Using the distillation learning method, a small-scale question generation model is trained to generate questions targeting the hallucination factor, including: Use a generative pre-trained model to generate a large amount of training data in a specified format; Using the training data, fine-tune the small-scale problem generation model; Taking the multiple text samples as input, the small-scale question generation model learns to generate questions targeting hallucination factors from the multiple text samples.

10. A computer-readable storage medium, It is characterized in that A computer program is stored, and when the computer program is executed by a processor, the steps of the large model hallucination mitigation method according to any one of claims 1 to 6 are implemented.

Images