Method and apparatus for evaluating and analyzing relationship hallucinations in large visual language models
By constructing a relational illusion benchmark (R-Bench), generating instance-level and image-level questions, and automatically generating questions using a large-scale language model (LLM), the relational illusion problem in understanding image content by large-scale visual language models is solved, enabling effective evaluation and analysis of the model and improving local visual understanding capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAMEN UNIV
- Filing Date
- 2024-03-25
- Publication Date
- 2026-07-31
AI Technical Summary
Existing large-scale visual language models are prone to relational illusions when understanding image content and lack comprehensive and rigorous evaluation benchmarks.
A relation illusion benchmark (R-Bench) is constructed by generating instance-level and image-level questions, automatically generating questions using a large language model LLM, evaluating the presence of relations in images and assessing local visual understanding, extracting bounding boxes using a scene graph parser and GroundingDINO, and setting color bounding boxes or masks to indicate targets.
An effective method and apparatus are provided to evaluate and analyze relational illusions in large-scale visual language models. It can identify relationships between specified targets, is applicable to existing models without retraining, and improves the model's local visual understanding ability.
Smart Images
Figure CN118312644B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of large visual language modeling technology, and in particular to a method and apparatus for evaluating and analyzing relational illusions in large visual language models. Background Technology
[0002] Recently, large-scale language models (LLMs), such as GPT4 and Llama, have emerged. The success of these models has prompted researchers to explore the combined use of LLMs with visual input, leading to the development of various large-scale visual language models (LVLMs). These efforts typically involve methods such as visual language pre-training or visual instruction tuning, aiming to integrate pre-trained visual encoders with LLMs to enhance their understanding of visual context.
[0003] However, despite their impressive performance, a major challenge these models face is the unavoidable hallucination problem. Existing large visual language models often tend to generate responses inconsistent with the image content. This problem is particularly critical for large visual language models, which are expected to accurately understand images and produce answers consistent with the visual input. Although previous research has thoroughly evaluated object hallucination and provided mitigation strategies through object detection or segmentation models, there is a significant gap in addressing hallucinations related to relationships between objects. Compared to object hallucination, relationship hallucination better reflects the ability of large visual language models to understand complex visual scenes, such as... Figure 1 As shown. Currently, there is a lack of comprehensive and rigorous benchmarks to assess these relationship illusions. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a method and apparatus for evaluating and analyzing relational illusions in large visual language models. By constructing a new relational illusion benchmark (R-Bench), and using automatically generated instance-level and image-level problems of large language models (LLM), the existence of relations in images and local visual understanding are evaluated, thereby realizing the evaluation of relational illusions in large visual language models.
[0005] In a first aspect, the present invention provides a method for evaluating and analyzing relational illusions in large-scale visual language models, comprising:
[0006] Step 10: Use the scene graph parser to extract relation triples from COCO subtitles to form a relation library;
[0007] Step 20: Use the scene graph parser to parse the nocaps subtitles into relation triples, and then match them with the relation library to form a relation seed from the matched relation set;
[0008] Step 30: Extract the target bounding box corresponding to the nocaps subtitle;
[0009] Step 40: Use the Large Language Model (LLM) to generate image-level questions with answers "yes" or "no" based on nocaps captions and the relation seed; use the Large Language Model (LLM) to generate instance-level questions with answers "yes" or "no" based on nocaps captions, the target bounding boxes obtained in step 30, and the bounding boxes in nocaps annotations. The image-level questions and instance-level questions constitute the relation illusion benchmark.
[0010] Step 60: Evaluate and analyze the relational illusion of the large visual language model using the relational illusion benchmark.
[0011] Furthermore, prior to step 60, the method includes: step 50, removing noisy image-level and instance-level problems according to human instructions to create a refined relational illusion benchmark.
[0012] Furthermore, in step 30, a color bounding box or mask is set for the extracted target bounding box, and in step 40, a color bounding box or mask suffix is added to the target corresponding to the problem to indicate the target in the figure, so that the large visual language model can pass the relational illusion benchmark evaluation without retraining.
[0013] Furthermore, setting the color bounding box or mask specifically involves randomly setting a red or green bounding box or mask.
[0014] Furthermore, in step 30, GroundingDINO is used to extract the target bounding box corresponding to the nocaps subtitle.
[0015] In a second aspect, the present invention provides an apparatus for evaluating and analyzing relational illusions in large-scale visual language models, comprising:
[0016] The relation library extraction module is used to extract relation triples from COCO subtitles using the scene graph parser to form a relation library;
[0017] The relation seed extraction module is used to parse nocaps subtitles into relation triples using a scene graph parser, and then match them with the relation library to form a relation seed from the set of matched relations.
[0018] The target bounding box extraction module is used to extract the target bounding boxes corresponding to the nocaps subtitles;
[0019] The question generation module is used to generate image-level questions with answers of "yes" or "no" based on nocaps captions and the relation seed using a large language model LLM; and to generate instance-level questions with answers of "yes" or "no" based on nocaps captions, target bounding boxes obtained by the target bounding box extraction module, and bounding boxes in nocaps annotations using a large language model LLM. The image-level questions and instance-level questions constitute the relation illusion benchmark.
[0020] The evaluation and analysis module is used to evaluate and analyze the relational illusions of large visual language models using relational illusion benchmarks.
[0021] Furthermore, it also includes a fine-tuning module for removing noisy image-level and instance-level problems based on human instructions to create a refined relational illusion benchmark.
[0022] Furthermore, in the target bounding box extraction module, a color bounding box or mask is set for the extracted target bounding box, and in the question generation module, a color bounding box or mask suffix is added to the target corresponding to the question to indicate the target in the image, so that the large visual language model can pass the relational illusion benchmark evaluation without retraining.
[0023] Furthermore, in the target bounding box extraction module, setting the color bounding box or mask specifically involves randomly setting a red or green bounding box or mask.
[0024] Furthermore, the target bounding box extraction module uses GroundingDINO to extract the target bounding box corresponding to the nocaps subtitle.
[0025] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:
[0026] A large visual language model (LLM) is employed to generate questions based on pre-extracted relation seeds derived from COCO captions. This approach is more conducive to analyzing the illusions of large visual language models (LVLMs) because most LVLMs share similar distributional biases with COCO caption data, and their visual instruction adjustment data is derived from COCO captions. Instance-level and image-level questions are constructed to evaluate the presence of relations in images and assess local visual understanding, with affirmative or negative answers, thus ensuring compatibility with existing target illusion benchmarks.
[0027] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0029] Figure 1 A schematic diagram of target illusion and relational illusion generated by large visual language models in existing technologies;
[0030] Figure 2 This is a flowchart illustrating the method in Embodiment 1 of the present invention;
[0031] Figure 3 This is a flowchart illustrating the generation of image-level and instance-level problems in Embodiment 1 of the present invention;
[0032] Figure 4(a) is a schematic diagram of the co-occurrence matrix between relations in Embodiment 1 of the present invention;
[0033] Figure 4(b) is a schematic diagram of the co-occurrence matrix between subjects and relationships in Embodiment 1 of the present invention;
[0034] Figure 4(c) is a schematic diagram of the co-occurrence matrix between relations and targets in Embodiment 1 of the present invention;
[0035] Figure 5 This is a schematic diagram illustrating examples of relational illusions generated by a large visual language model due to different reasons in Embodiment 1 of the present invention;
[0036] Figure 6 This is a schematic diagram illustrating examples of counterfactual relational illusions and illusionary relational illusions appearing in a large visual language model in Embodiment 1 of the present invention;
[0037] Figure 7 This is a schematic diagram showing the target pointing accuracy results of a large visual language model tested in Embodiment 1 of the present invention;
[0038] Figure 8 This is a schematic diagram of data statistics for R-Bench in Embodiment 1 of the present invention;
[0039] Figure 9 This is a schematic diagram of the results of the large visual language model in Embodiment 1 of the present invention on R-Bench;
[0040] Figure 10 This is a schematic diagram comparing relational illusion and target illusion in Embodiment 1 of the present invention;
[0041] Figure 11 This is a schematic diagram of the statistical results of relation words and target words in COCO image description in Embodiment 1 of the present invention;
[0042] Figure 12 This is a schematic diagram illustrating the performance of the large visual language model in other relational illusions according to Embodiment 1 of the present invention;
[0043] Figure 13 This is a schematic diagram of the device in Embodiment 2 of the present invention. Detailed Implementation
[0044] This invention provides a method and apparatus for evaluating and analyzing relational illusions in large visual language models. By constructing a new relational illusion benchmark (R-Bench), and using automatically generated instance-level and image-level problems of large language models (LLM), the invention evaluates the existence of relations in images and assesses local visual understanding, thereby achieving the evaluation of relational illusions in large visual language models.
[0045] The overall concept of the technical solutions in the embodiments of the present invention is as follows:
[0046] A novel relational illusion benchmark (R-Bench) is constructed specifically to evaluate relational illusions in large-scale visual language models. This benchmark comprises image-level and instance-level questions, labeled "yes" or "no." Image-level questions assess the presence of relations in images, while instance-level questions evaluate local visual understanding, focusing on specific target relations represented by color bounding boxes or masks. Instance-level questions demonstrate the local visual understanding capabilities of large-scale visual language models, are applicable to existing models, and require no retraining.
[0047] The problems constructed in this invention have affirmative or negative answers, making the benchmark of this invention compatible with existing target illusion benchmarks. This invention employs LLM to generate problems based on a pre-extracted set of relations. This set of relations comes from COCO captions, which will be more helpful for this invention in analyzing the illusions of large visual language models, because most large visual language models have the same distribution bias as COCO caption data, since their visual instruction adjustment data comes from COCO captions.
[0048] Image-level problems are too simplistic for large visual language models and cannot accommodate state-of-the-art models with enhanced local understanding capabilities. Furthermore, identifying relationships between specified targets is more relevant to practical needs. To address this issue, this invention introduces instance-level problems to evaluate relationships between specified targets, requiring large visual language models to possess local visual understanding capabilities. This is achieved by attaching colored bounding boxes or masks to the targets mentioned in the problem and visually representing these colored elements on the image.
[0049] The process of generating image-level and instance-level questions includes: parsing all COCO captions to create a comprehensive relation set; aligning parsed captions from nocaps captions with the relation set to establish a set of relation seeds; creating cues based on nocaps captions, relation seeds, and bounding boxes; and then automatically generating image-level and instance-level questions using a large language model (LLM). This invention generates image-level and instance-level questions using a large language model (LLM), thereby aiding in the evaluation and analysis of relational illusions in large visual language models.
[0050] Example 1
[0051] This embodiment provides a method for evaluating and analyzing relational illusions in large-scale visual language models, such as... Figure 2 and Figure 3 As shown, the steps can be as follows:
[0052] Step 10: Use the scene graph parser to extract relation triples from COCO captions to form a relation set.
[0053] Step 20: Use the scene graph parser to parse the nocaps captions into relation triples, and then match them with the relation library. The set of matched relations forms relation seeds. The relations come from COCO captions, which will be more helpful for this invention to analyze relation illusions in large visual language models (LVLMs), because most large visual language models have the same bias in the distribution of COCO caption data, since their visual instruction adjustment data comes from COCO captions.
[0054] Step 30: Use GroundingDINO to extract the target bounding box corresponding to the nocaps subtitle.
[0055] Step 40: Using the Large Language Model (LLM) based on nocaps captions and the relation seed obtained in Step 20, generate image-level questions with answers of "yes" or "no"; using the Large Language Model (LLM) based on nocaps captions, the target bounding box obtained in Step 30, and the bounding box in the nocaps annotation, generate instance-level questions with answers of "yes" or "no". The image-level questions and instance-level questions constitute the relation illusion benchmark.
[0056] Step 50: Remove noisy image-level and instance-level issues according to human instructions to create a refined relational illusion benchmark.
[0057] Step 60: Evaluate and analyze the relational illusion of large visual language models using relational illusion benchmarks (i.e., image-level problems and instance-level problems).
[0058] In one possible implementation, a color bounding box or mask is set for the extracted target bounding box in step 30, and a color bounding box or mask suffix is added to the target corresponding to the problem in step 40 to indicate the target in the image, so that the large visual language model can pass the relational illusion benchmark evaluation without retraining. Specifically, setting the color bounding box or mask can be a random setting of red or green bounding boxes or masks. Figure 3 As shown.
[0059] The relational illusion of a large visual language model was evaluated and analyzed using a relational illusion benchmark, identifying three types of relational co-occurrence that lead to the illusion: relation-relation, subject-relation, and relation-object. Figure 4(a) is a schematic diagram of the co-occurrence matrix between relation-relation (Rel-Rel), Figure 4(b) is a schematic diagram of the co-occurrence matrix between subject-relation (Sub-Rel), and Figure 4(c) is a schematic diagram of the co-occurrence matrix between relation-object (Rel-Obj). The matrix shows the conditional probability of an element on the y-axis occurring when another element on the x-axis occurs.
[0060] Figure 5 This is a schematic diagram illustrating examples of relational illusions generated by large visual language models obtained after evaluation and analysis due to different reasons. Figure 6 This is a schematic diagram illustrating examples of counterfactual and illusory relational illusions in a large visual language model obtained after evaluation and analysis.
[0061] The benchmark for this invention is based on evaluating the proposed model on a validation set of nocaps, consisting of 4500 images from OpenImage, each with 10 titles. These images can be categorized into 648 in-domain images (with only COCO classes), 2938 near-domain images (with both COCO and novel classes), and 914 out-of-domain images. To demonstrate the effectiveness of the proposed method, a series of quantitative experiments were conducted, with the results as follows:
[0062] The target pointing accuracy of large visual language models is as follows: Figure 7 As shown in the figure. The results indicate that large visual language models have the ability to provide targeted answers.
[0063] The statistical data of the relational illusion benchmark (R-Bench) proposed in this embodiment of the invention are as follows: Figure 8 Show.
[0064] Results of large visual language models on R-Bench in embodiments of the present invention Figure 9 As shown. The results indicate that large visual language models suffer from a severe relational illusion problem.
[0065] Examples of relational illusions and target illusions in large-scale visual language models in this invention include: Figure 10 As shown in the figure. The results indicate that the relational illusion problem is more severe than the target illusion problem in large visual language models.
[0066] In this embodiment of the invention, the statistics of relation words and target words in COCO image descriptions are as follows: Figure 11 As shown, the results indicate that the relational terms and target terms in the data are unbalanced.
[0067] The performance of the large visual language model in other relational illusions in the embodiments of the present invention is as follows: Figure 12 As shown, this includes counterfactual and illusionary relational illusions.
[0068] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.
[0069] Example 2
[0070] This embodiment provides an apparatus for evaluating and analyzing relational illusions in large-scale visual language models, such as... Figure 13 As shown, it includes:
[0071] The relation library extraction module is used to extract relation triples from COCO subtitles using the scene graph parser to form a relation library;
[0072] The relation seed extraction module is used to parse nocaps subtitles into relation triples using a scene graph parser, and then match them with the relation library to form a relation seed from the set of matched relations.
[0073] The target bounding box extraction module is used to extract the target bounding boxes corresponding to the nocaps subtitles;
[0074] The question generation module is used to generate image-level questions with answers of "yes" or "no" based on nocaps captions and the relation seed using a large language model LLM; and to generate instance-level questions with answers of "yes" or "no" based on nocaps captions, target bounding boxes obtained by the target bounding box extraction module, and bounding boxes in nocaps annotations using a large language model LLM. The image-level questions and instance-level questions constitute the relation illusion benchmark.
[0075] The evaluation and analysis module is used to evaluate and analyze the relational illusions of large visual language models using relational illusion benchmarks.
[0076] In a more optimized implementation, a fine-tuning module is also included to remove noisy image-level and instance-level problems based on human instructions, in order to create a refined relational illusion benchmark.
[0077] In one possible implementation, a color bounding box or mask is set for the extracted target bounding boxes in the target bounding box extraction module, and a color bounding box or mask suffix is added to the target corresponding to the question in the question generation module to indicate the target in the figure, so that the large visual language model can pass the relational illusion benchmark evaluation without retraining.
[0078] In one possible implementation, in the target bounding box extraction module, setting the color bounding box or mask specifically involves randomly setting a red or green bounding box or mask.
[0079] In one possible implementation, the target bounding box extraction module uses GroundingDINO to extract the target bounding box corresponding to the nocaps caption.
[0080] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.
[0081] This invention employs a Large Visual Language Model (LLM) to generate a problem based on pre-extracted relation seeds derived from COCO captions. This approach is more conducive to analyzing the illusions of Large Visual Language Models (LVLMs) because most LVLMs share similar biases with the distribution of COCO caption data, and their visual instruction adjustment data is derived from COCO captions. Instance-level and image-level questions are constructed to evaluate the presence of relations in images and to assess local visual understanding, with affirmative or negative answers, thus ensuring compatibility with existing target illusion benchmarks.
[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0086] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for evaluating and analyzing relational illusions in large-scale visual language models, characterized in that, include: Step 10: Use the scene graph parser to extract relation triples from COCO subtitles to form a relation library; Step 20: Use the scene graph parser to parse the nocaps subtitles into relation triples, and then match them with the relation library to form a relation seed from the matched relation set; Step 30: Extract the target bounding box corresponding to the nocaps subtitle; Step 40: Use a large language model (LLM) to generate image-level questions with answers "yes" or "no" based on nocaps captions and the relation seed; use a large language model (LLM) to generate instance-level questions with answers "yes" or "no" based on nocaps captions, the target bounding boxes obtained in step 30, and the bounding boxes in nocaps annotations. The image-level questions and instance-level questions constitute the relation illusion benchmark. Step 60: Evaluate and analyze the relational illusion of the large visual language model using the relational illusion benchmark.
2. The method according to claim 1, characterized in that, Prior to step 60, the process also includes: step 50, removing noisy image-level and instance-level problems according to human instructions to create a fine relational illusion benchmark.
3. The method according to claim 1, characterized in that: In step 30, a color bounding box or mask is set for the extracted target bounding box. In step 40, a color bounding box or mask suffix is added to the target corresponding to the problem to indicate the target in the figure, so that the large visual language model can pass the relational illusion benchmark evaluation without retraining.
4. The method according to claim 3, characterized in that: The setting of the color boundary box or mask specifically involves randomly setting a red or green boundary box or mask.
5. The method according to claim 1, characterized in that: In step 30, GroundingDINO is used to extract the target bounding box corresponding to the nocaps subtitle.
6. An apparatus for evaluating and analyzing relational illusions in large-scale visual language models, characterized in that, include: The relation library extraction module is used to extract relation triples from COCO subtitles using the scene graph parser to form a relation library; The relation seed extraction module is used to parse nocaps subtitles into relation triples using a scene graph parser, and then match them with the relation library to form a relation seed from the set of matched relations. The target bounding box extraction module is used to extract the target bounding boxes corresponding to the nocaps subtitles; The question generation module is used to generate image-level questions with answers of "yes" or "no" based on nocaps captions and the relation seed using a large language model LLM; and to generate instance-level questions with answers of "yes" or "no" based on nocaps captions, target bounding boxes obtained by the target bounding box extraction module, and bounding boxes in nocaps annotations using a large language model LLM. The image-level questions and instance-level questions constitute the relation illusion benchmark. The evaluation and analysis module is used to evaluate and analyze the relational illusions of large visual language models using relational illusion benchmarks.
7. The apparatus according to claim 6, characterized in that, Also includes: The fine-tuning module is used to remove noisy image-level and instance-level issues based on human instructions to create a fine-grained relational illusion benchmark.
8. The apparatus according to claim 6, characterized in that: In the target bounding box extraction module, a color bounding box or mask is set for the extracted target bounding box. In the question generation module, a color bounding box or mask suffix is added to the target corresponding to the question to indicate the target in the figure. This allows large visual language models to pass the relational illusion benchmark evaluation without retraining.
9. The apparatus according to claim 8, characterized in that: In the target bounding box extraction module, setting the color bounding box or mask specifically involves randomly setting a red or green bounding box or mask.
10. The apparatus according to claim 6, characterized in that: The target bounding box extraction module uses GroundingDINO to extract the target bounding box corresponding to the nocaps subtitle.