Visual vocabulary guided multi-modal large model illusion optimization method and storage medium

Abstract

The invention relates to the technical field of large model training, and discloses a visual vocabulary-guided multi-modal large model illusion optimization method and a storage medium, and the method comprises the steps: constructing a training sample comprising an image, a text prompt, an optimal answer and a rejection answer, carrying out the image detection of the image through a visual target detection model according to the optimal answer, and obtaining a visual target detection result; according to the method, related indexes consistent with visual words are extracted, the visual weight of each visual word is calculated according to the indexes, different types of words are subjected to differential weighting according to the visual weights, a dynamic adjustment coefficient is defined according to the proportion of a corrected fragment, and the dynamic adjustment coefficient, probability difference and differential weighting jointly construct a DPO loss function. And guiding the multi-mode large model to generate an answer with consistent image-text. According to the method, the multi-modal large model is dynamically endowed with differentiated optimization weights, so that the multi-modal large model pays attention to vocabularies highly related to image semantics, information description more conforming to visual facts is generated, and the consistency and authenticity of text generation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model training, in particular to a visual vocabulary guided multimodal large model hallucination optimization method and a storage medium. BACKGROUND

[0002] Since the advent of artificial intelligence technology and its rapid development, multimodal large language models (MLLMs) have made significant progress in image question answering, image-text generation, text understanding, content description, and other tasks, and are widely used in human-computer interaction, auxiliary annotation, scientific education, intelligent medical care, social security, and many other fields, and are deeply integrated into people's daily life. Such multimodal large language models usually combine visual encoders and language generators to achieve the ability to understand image semantic content and the fusion of natural language expression.

[0003] Although existing MLLMs perform well in image-text content understanding, coherent generation, and language fluency, they still have an unavoidable "large model hallucination" problem. Large model hallucination refers to the phenomenon that a multimodal large language model outputs information that does not match the actual content of an image when processing image question answering or description generation tasks, such as Figure 5 、 6 、7, which can be divided into three categories: (1) hallucination of fictitious entities, i.e., the model generates objects that do not exist in the image, such as "a cat lying on the sofa" in an image with no cat; (2) hallucination of attribute misjudgment, i.e., the model identifies real objects in the image but describes their attributes (such as color, number, position, etc.) incorrectly, for example, describing "a black sofa" as "a red sofa"; (3) hallucination of missing entities, i.e., some important elements in the image are not recognized or reflected by the model, such as ignoring objects on the edge of the image and generating statements like "there is no obvious content in the image." The above three types of hallucination phenomena are more likely to occur in multi-target images, complex backgrounds, or non-central regions, which not only reduces the consistency of the model's image-text, but also seriously affects its practical value in high-reliability application scenarios, especially in fields such as medical image analysis and autonomous driving scene perception, which can pose a significant risk.

[0004] To alleviate the hallucination problem of large models, existing researches mostly use reinforcement learning based on human feedback (RLHF) or direct preference optimization (DPO) methods, which introduce user preference signals to guide the model to generate results that are more consistent with human expectations. Although these methods perform well in pure language tasks, they still have obvious limitations in multi-modal scenarios involving image-text collaboration, mainly in three aspects: first, there is a lack of effective modeling of image-text semantic consistency, making it difficult to identify key words in the text that need to be accurately aligned with image content; second, it is unable to measure the importance of each word in the visual semantic, and all words are treated equally in training, making it difficult for the model to focus on key visual elements; third, for incorrect answers containing hallucinations, existing methods lack a dynamic punishment mechanism that can adjust with the severity of hallucinations, limiting the improvement of the model's correction ability. SUMMARY

[0005] In view of the deficiencies in the prior art, the purpose of the present application is to provide a visual vocabulary guided multi-modal large model hallucination optimization method.

[0006] To achieve the above purpose, the present application provides the following technical solutions: The visual vocabulary guided multi-modal large model hallucination optimization method is an optimization method with word-level visual recognition capability, fine-grained preference adjustment mechanism and dynamic hallucination content punishment, which enhances the multi-modal large model's understanding of real image content in the preference learning framework by introducing a visual guidance mechanism and a weighted preference optimization strategy, effectively reducing hallucination output and improving the accuracy and reliability of multi-modal large model generation results.

[0007] The multi-modal large model hallucination optimization method specifically includes the following steps: Training data construction: construct training samples including images, text prompts, preferred answers and rejected answers, wherein the preferred answers are highly corresponding to the images, and the rejected answers contain hallucination components as a reference for comparison; Visual information extraction and word weight calculation: use a visual object detection model to detect images based on preferred answers, extract relevant indicators consistent with visual words, and calculate the visual weight of each visual word accordingly; Visual guided multi-modal large model modeling: differentially weight different types of words according to the visual weight to improve the multi-modal large model's attention to image-text alignment words during modeling; Dynamic hallucination punishment adjustment: dynamically adjust according to the proportion of corrected segments in the rejected answers, so that answers containing more incorrect segments receive stronger constraint signals in training; The visual guidance preference loss function is constructed by fusing the probability difference of the preferred answer and the rejected answer, combining the differential weighting and the dynamic adjustment mechanism, and constructing the DPO loss function of the multi-modal large model, so as to guide the model to generate answers consistent with the image semantics.

[0008] In the present application, preferably, in each of the training samples, a quadruple (I, x, y+, y-) is constructed, , wherein I represents an input image, x is a text prompt, y+ is a preferred answer, and y- is a rejected answer.

[0009] In the present application, preferably, the open target detection model performs image detection on the image according to the noun in the preferred answer, and when the target region corresponding to the noun is found, the noun entity is a visual word, and the visual weight of the visual word is calculated.

[0010] In the present application, preferably, the correlation indicators include: a detection confidence : a degree of confidence of the target detection model on the correspondence between the visual word and the entity in the image; a target area ratio : representing the area ratio of the detection region in the whole image, the larger the area ratio, the more significant the target area is; a center deviation : measuring the position of the target corresponding to the visual word in the image, the closer to the center, the stronger the visual saliency is.

[0011] In the present application, preferably, the visual weight of each visual word is calculated as follows:

[0012] wherein is the detection confidence, is the target area ratio, is the center deviation. The final visual weight will be used to guide the optimization intensity of the word in subsequent language modeling, ensuring that the model focuses on the core entity words that actually exist in the image, thereby significantly reducing the generation probability of entity hallucination.

[0013] In the present application, preferably, a multi-modal large model modeling objective function corresponding to the differential weighting is constructed:

[0014] wherein is the visual weight of the visual word, is the overall weighting factor of the attribute word, and N is the total number of words in the sequence, which is used as a normalization factor, as GroundingDINO identified fragments, as unidentified nouns, adjectives or quantifiers, as the rest of the unidentified words; attribute words need to be weighted depending on the visual target words: no weighting for unbound visual words, independent weighting for each bound visual word. In the present application, a dynamic adjustment coefficient is introduced to scale the probability difference between the preferred answer and the rejected answer, wherein the is defined as:

[0015] wherein is the length of the corrected fragment, is the total length of the answer, is the base scaling coefficient. During training, will be used in the DPO loss function: can be directly used in the loss calculation of each sample instead of , or take the average of all in a batch as of this iteration. The length of the corrected fragment is calculated in token granularity, and the judgment adopts an automatic contrast combination method, which aligns the preferred and rejected answers through the model and differentiates them. If necessary, manual verification is supplemented to ensure the accuracy of the judgment. In the present application, the preferred DPO loss function of the multi-modal large model is constructed as: , wherein, represents the multi-modal model currently being trained, is a reference model whose parameters remain frozen and are not updated during the training process, and is used as a contrast benchmark to provide a stable reference distribution. In order to ensure the effectiveness of the loss function optimization, the reference model π_ref and the current model are constructed based on the same training data; is a regularization coefficient used to control the scaling amplitude of the probability ratio difference; is a Sigmoid function used to normalize the output and ensure numerical stability.

[0016] Compared with the prior art, the present application has the following beneficial effects: The method of the present application dynamically assigns differentiated optimization weights by introducing an image-guided word-level importance mechanism, so that the multi-modal large model pays more attention to the words highly related to image semantics, thereby generating information description more consistent with visual facts, improving the consistency and authenticity of text generation; improving the hallucination recognition ability, greatly reducing the hallucination rate in multiple standard evaluation tasks; the hallucination recognition ability of the multi-modal large model is improved, and the hallucination rate of the generated text description is greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of the visual vocabulary guided multi-modal large model hallucination optimization method of the present application.

[0018] Figure 2 An effect diagram of a specific embodiment in the present application.

[0019] Figure 3 An effect diagram of another specific embodiment in the present application.

[0020] Figure 4 An effect diagram of another specific embodiment in the present application.

[0021] Figure 5 A schematic diagram of a large model hallucination.

[0022] Figure 6 Another schematic diagram of a large model hallucination.

[0023] Figure 7 Another schematic diagram of a large model hallucination. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0026] Please refer to Figure 1The preferred embodiment of the present application provides a visual vocabulary guided multimodal large model hallucination optimization method. On the basis of the traditional DPO framework, image perception information is introduced, visual feature keywords are used as clues to dynamically adjust the sensitivity of the multimodal large model to hallucination content, and a differentiated loss function is constructed to guide the multimodal large model to more accurately generate text descriptions consistent with the image content.

[0027] The specific steps are as follows: S1, training data construction: constructing training samples including images, text prompts, preferred answers and rejected answers, wherein the preferred answers are highly corresponding to the images, and the rejected answers contain hallucination components as contrast references; S2, visual information extraction and word weight calculation: using a visual target detection model to perform image detection on the images according to the preferred answers, extracting related indicators consistent with visual words, and calculating the visual weights of each visual word accordingly; S3, visual guided multimodal large model modeling: differentiating and weighting different types of words according to the visual weights, and improving the attention ability of the multimodal large model to the image-text alignment vocabulary during modeling; S4, dynamic hallucination penalty adjustment: dynamically adjusting according to the proportion of the corrected segments in the rejected answers, so that the answers containing more error segments receive stronger constraint signals in the training; S5, visual guided preference loss function construction: fusing the probability difference of the preferred answers and the rejected answers, combining the differentiated weighting and the regularization coefficient, and constructing the DPO loss function of the multimodal large model to guide the model to generate answers consistent with the image semantics.

[0028] In step S1, in each of the training samples, a four-tuple (I, x, y+, y-) is constructed, wherein I represents an input image, x is a text prompt, , ), wherein I represents an input image, x is a text prompt, is a preferred answer, is a rejected answer. The preferred answer is a text description highly consistent with the image content and semantically clear and natural; and the rejected answer contains at least one piece of information (i.e., "hallucination word") contradictory to the image content, such as a target that does not exist in the image but is described, an incorrect color attribute, or a logical contradiction. The sample structure makes the training process not only learn "how to generate correct answers", but also clearly learn "which answers are not accepted", thereby establishing a clear image-text matching boundary.

[0029] In step S2, the conventional language model cannot combine the image to determine which keywords to focus on, but divides the attention evenly, so it is easy to generate words that are not related to the image, that is, "hallucinated words". To solve this problem, this step aims to identify core words that need image semantic support, namely "visual words", in the generated text, and evaluate the semantic importance of these words in the image, forming a word-level visual perception mechanism. Specifically, first, the advanced visual object detection model (such as GroundingDINO) is used to perform image detection on the noun entities in the preferred answer word by word. When the visual object detection model successfully finds the target area corresponding to the noun in the image, the noun entity is considered a "visual word", that is, the word has an irreplaceable core role in modeling image-text consistency. When no target area is found, the word is not given additional weight in the preferred answer, and the model weight of each word is calculated, which is set to 1.0 by default in specific implementations. The visual weight of the visual word is quantified in three dimensions. These three dimensions include: Detection confidence : The degree of confidence of the visual word in the entity correspondence in the image, which returns a confidence value after detection by the visual object detection model, which is the detection confidence ; Target area ratio : The area ratio of the detection area in the entire image, the larger the area ratio, the more significant it is, which is generated by the visual object detection model to frame the object, the area ratio of the rectangular frame to the entire image; Center deviation : Measures the position of the target corresponding to the visual word in the image. The visual object detection model generates a rectangular frame for the detected object, and calculates the offset value between the center position of the object rectangular frame and the center position of the entire image. The offset value is the center deviation , the closer to the center, the stronger the visual saliency. These three dimensions are the corresponding indicators.

[0030] The calculation of the visual weight of each visual word is as follows:

[0031] where is the detection confidence, is the target area ratio, is the center deviation. The final visual weight The optimization strength of the visual word in guiding subsequent multimodal large model modeling is ensured to focus on the core entity words that actually exist in the image, thereby significantly reducing the generation probability of entity class hallucination.

[0032] In step S3, after obtaining the visual weight of each word, the application further constructs a differentiated language training mechanism to enable the multimodal large model to focus on optimizing the words that actually exist in the image and maintain basic grammar modeling for the structural words without image correspondence, and give moderate strengthening to the attribute words (such as color, quantity) for modification, thereby proposing a visual-guided language modeling mechanism to introduce a differentiated optimization strategy from the type of vocabulary.

[0033] First, the preferred answer corresponding to the answer text y contains various nouns and adjectives, etc. The words in the answer text y are divided into three categories: Visual target words : including the noun entity fragments recognized by GroundingDINO, which are the key carriers of image-text consistency; Attribute words : descriptive words that modify target words, such as color, quantity, and orientation; Language structure words : such as prepositions and conjunctions, mainly serving grammatical structure and having no direct dependence on images.

[0034] According to this classification, a differentiated weighting corresponding multimodal large model modeling objective function is constructed:

[0035] wherein is the visual weight of the visual word, is the overall weighting factor of the attribute word, and , >1, N is the total number of words in the sequence, used as a normalization factor, is the fragment recognized by GroundingDINO, is the unrecognized noun, adjective, or quantifier, is the rest of the unrecognized words. The weighting of attribute words depends on visual target words: if the object modified by the attribute word is not recognized as a visual word, no weighting is applied; if an attribute word modifies multiple visual target words, it is bound to each visual target word and independently applies Weighting. The objective function is embedded into the DPO loss calculation as a weighted estimation of log π(y|x), which ensures that the multimodal large model can automatically strengthen the semantic core (such as entities and attributes) in the image during the generation process, while keeping the basic modeling of the language structure component, thereby maximizing the optimization ability of the model on the image-text consistency target.

[0036] In step S4, although the visual word weight and attribute weight have established a mechanism of "focusing on the real target word" in the training stage, effectively guiding the multimodal large model to focus on the semantic content visible in the image. However, in the multimodal task, simply optimizing the preferred answer is not enough to prevent the model from producing hallucinations. Especially in the reject sample , the error degree of hallucination content is dynamic, some only have slight deviation, and some completely describe the object that does not exist in the image.

[0037] If the uniform intensity constraint is still imposed on all reject answers, not only can it not effectively suppress answers containing a large number of error fragments, but it can also lead to problems of gradient instability or training insensitivity. Therefore, a dynamic adjustment mechanism based on the proportion of corrected fragments is proposed, which flexibly adjusts the contrast intensity according to the proportion of error fragments in each reject answer, thereby improving the sensitivity and suppression ability of the model to abnormal content.

[0038] In an embodiment of the present application, to realize the consistency of dynamic adjustment and gradient update of reject answers, a dynamic adjustment coefficient is introduced to scale the probability difference between the preferred answer and the reject answer. The dynamic adjustment coefficient is defined as:

[0039] wherein, is the basic scaling coefficient, represents the length of the corrected fragment in the answer, represents the total length of the answer. The length of the corrected fragment is calculated in token granularity, that is, based on the tokenization result of the multimodal large model, the difference area is mapped to the corresponding token number. In addition, the determination of the corrected fragment adopts an automatic contrast combination method, which aligns the text of the preferred answer and the reject answer through the model and performs differential analysis, and if necessary, manual verification is supplemented to ensure the accuracy of the determination. During training, will be used in the DPO loss function: can be replaced by in the loss calculation of each sample, or take the mean value of all in a batch as of this iteration By this definition, the higher the proportion of the corrected segment contained in the answer, the larger the corresponding dynamic adjustment coefficient , thereby imposing a stronger punishment on the rejected answer during training, achieving adaptive suppression of hallucinated content. This mechanism not only enhances the model's ability to distinguish difficult examples, but also ensures the stability and convergence of the training process.

[0040] In step S5, through the coordinated design of the first four steps, the present application has established a structured control mechanism at the levels of training data construction, visual semantic recognition, language generation optimization, and hallucination degree recognition. It has comprehensively enhanced the perception and response ability of the multi-modal large model for the consistency of text and image from four dimensions: input end, word-level modeling, loss construction, and punishment adjustment. In order to realize the unified deployment and gradient collaborative update of this series of mechanisms, a unified optimization framework that integrates the above information is constructed at the training target level, and a visual-guided DPO loss function is proposed as the convergence expression of the key technology. This optimization objective not only retains the efficiency of the DPO framework in preference comparison, but also combines the differentiated weight regulation of visual words and attribute words, achieving adaptive optimization under the condition of no manual annotation, making the model more inclined to generate preferred answers consistent with image semantics and reducing the occurrence of inconsistent content.

[0041] DPO loss function for constructing multi-modal large model: , wherein, represents the multi-modal model being trained, is a reference model whose parameters remain frozen and are not updated during training, serving as a comparison benchmark to provide a stable reference distribution. To ensure the effectiveness of the loss function optimization, the reference model π_ref and the current model are based on the same training data; is a regularization coefficient used to control the scaling amplitude of the probability ratio difference; is a Sigmoid function used to normalize the output and ensure numerical stability. This optimization objective uses a preference comparison mechanism to make the model more inclined to select preferred answers consistent with image semantics, reducing the risk of generating inconsistent or unreasonable content.

[0042] As Figure 2 , 3As shown in FIGS. 4, the visual guidance vocabulary preference optimization method provided by the present application effectively improves the consistency and authenticity of the multi-modal large language model in the text generation process. By introducing an image-guided word-level importance calculation mechanism, the correlation degree of different words and visual content can be distinguished during the training process, and differential optimization weights are dynamically assigned. In the training of the multi-modal large model, the generated data is not reprocessed before being input into the model, but is directly input into the training process in the form of an image-text pair. In the forward propagation stage, the visual encoder and the language encoder extract the features of the image and the words respectively, and calculate the correlation between the words and the visual semantics through the attention mechanism to obtain the word-level importance distribution. This distribution does not change the input data, but dynamically assigns different weights to different words in the loss function layer, so that the gradient of the words highly related to the image is amplified and the gradient of the weakly related words is weakened in the back propagation, thereby guiding the model to more accurately align the image and text semantics in the parameter update and reducing the inconsistency between the image and the text. The multi-modal large model pays more attention to the words highly related to the image semantics, thereby generating information description more consistent with the visual facts, and significantly improves the problems such as "image-text disconnection" and "semantic drift" in the traditional method.

[0043] At the same time, it has the advantages of strengthening the multi-modal large model hallucination recognition and inhibition capability. By constructing a word-level comparison mechanism, further combining the differences of high importance words in positive and negative samples, and dynamically adjusting the weight of negative samples in the training loss, the multi-modal large model is more sensitive to inconsistent expressions and can effectively distinguish them during the learning process, thereby reducing typical hallucination phenomena such as "out of nothing" and "attribute error". In multiple standard evaluation tasks, this method can significantly reduce the hallucination proportion in the generated content of the model without introducing additional supervision, thereby improving the overall text quality.

[0044] Under the premise of maintaining the language understanding and generation capability of the multi-modal large model, the stability and reliability of the overall training are improved. Through the joint regulation and control of the word weight and preference optimization process, the multi-modal large model can maintain more balanced gradient update when facing complex image-text combination data, and avoid performance fluctuations caused by unstable optimization strategies. At the same time, the introduction of the dynamic weight mechanism effectively adapts to the difficulty differences between different samples, thereby improving the training efficiency and consistency performance of the final model.

[0045] In some other preferred embodiments of the present application, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the method described in the above embodiments.

[0046] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0047] The above description is a detailed description of the preferred embodiments of the present application, but the embodiments are not intended to limit the scope of the patent application of the present application. Any equivalent changes or modifications made under the technical spirit of the present application should be included in the scope of the patent.

Claims

1. A method for visual vocabulary guided multi-modal large model hallucination optimization, characterized in that, The method comprises the steps of: constructing a training sample comprising an image, a text prompt, a preferred answer and a rejected answer, wherein the preferred answer corresponds to the image, and the rejected answer contains an illusion component as a contrast reference; using a visual target detection model to perform image detection on the image according to the preferred answer, extracting a related index consistent with a visual word, and calculating a visual weight of each visual word according to the related index; differentially weighting different types of words according to the visual weight, and improving the attention ability of a multi-modal large model to word alignment between an image and a text during modeling; introducing a dynamic adjustment coefficient based on a proportion of corrected segments, and constructing a DPO loss function of the multi-modal large model by combining the dynamic adjustment coefficient with a probability difference between the preferred answer and the rejected answer and the differential weighting, to guide the multi-modal large model to generate an image-text consistent answer.

2. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 1, wherein, In each of the training samples, a quadruple (I, x, y, y') is constructed, wherein I represents an input image, x is a text prompt, , ), wherein I represents an input image, x is a text prompt, is a preferred answer, is a rejected answer.

3. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 2, wherein, An open target detection model performs image detection on the image according to a noun in the preferred answer, and when a target region corresponding to the noun is found, the noun is a visual word, and a visual weight of the visual word is calculated.

4. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 3, wherein, The related index comprises: detection confidence : degree of confidence of the target detection model in the correspondence between the visual word and the entity in the image Target area ratio : indicates the area ratio of the detection region in the whole image; Centering deviation : measures the position of the target in the image to which the visual word corresponds.

5. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 4, wherein, The visual weight of each visual word is calculated in particular as follows: wherein is a detection confidence, is a target area area ratio, is a center deviation.

6. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 5, wherein, a modeling target function of a multi-modal large model corresponding to the differential weighting is constructed: wherein is a visual weight of the visual word, is an overall weighting factor for attribute words, N is the total number of words in the sequence, used as a normalization factor, is a segment recognized by GroundingDINO, is a noun, adjective, or quantifier that is not recognized, is the rest of the unrecognized words.

7. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 6, wherein, The multi-modal large model further introduces a dynamic adjustment coefficient when constructing a loss function for scaling the probability difference between the preferred answer and the rejected answer, wherein the dynamic adjustment coefficient is defined as: wherein is the length of the segment to correct, is the total length of the answer, is the base scaling factor.

8. The visual vocabulary guided multi-modal large model hallucination optimization method of claim 7, wherein, a DPO loss function of the multi-modal large model is constructed: ， wherein, denotes the multi-modal model currently being trained, is a reference model, is a regularization coefficient, is a Sigmoid function.

9. A storage medium, characterized by The computer program is stored, and when the computer program is executed by a processor, the processor executes the steps of the visual word guided multi-modal large model illusion optimization method according to any one of claims 1-8.

Images