Illusion relieving method and device for multi-modal large model, electronic equipment and medium

Abstract

The invention relates to an illusion relieving method and device for a multi-modal large model, electronic equipment and a medium. The method comprises the following steps: processing input image-text data to obtain a text token and an image token, aligning the text token and the image token to obtain an input sequence of the image-text data, and constructing an image token attention vector of each position in the input sequence; based on the image token attention vector, measuring the image understanding difference between different decoding layers through Jensen-Shannon divergence, so as to determine a target layer with the maximum distance from the image token attention vector of the tail layer according to a measurement result; and converting the hidden state of the target layer into probability distribution through the linear projection layer by using a preset advance exit mechanism so as to predict the probability distribution of the next token. Therefore, the problems that an existing technology for relieving the illusion phenomenon of the multi-modal large language model is too complex and high in cost, and the model performance is affected are solved, and the reliability and accuracy of the method in practical application are improved.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a method, device, electronic device and medium for alleviating hallucinations of a large multimodal model. Background Art

[0002] In recent years, multimodal large language models (MLLMs) have made significant progress in processing inputs from language and other modalities to generate open-ended responses. The latest MLLMs have performed well in a variety of visual tasks, such as object detection, image description, and visual question answering. However, despite their remarkable success, these models generally have a serious problem, namely, generating grammatically coherent but actually incorrect content, which is known as the "hallucination" phenomenon. The problems caused by this phenomenon are particularly serious in fields such as healthcare, autonomous driving systems, and robotics, because incorrect results can have catastrophic consequences. Therefore, how to effectively mitigate and manage the hallucination phenomenon in MLLMs and improve its reliability and accuracy in practical applications has become a technical problem that needs to be solved urgently.

[0003] In the related technologies, most methods solve the hallucination problem by increasing data or training costs, such as constructing new manually annotated datasets, introducing external knowledge bases, or performing additional training or fine-tuning; other methods try to reduce hallucinations during reasoning by leveraging the implicit knowledge of the model without increasing or only increasing almost negligible training or data costs.

[0004] However, on the one hand, the above method requires a large amount of high-quality labeled data and additional training costs, which significantly increases the complexity and cost of model development and deployment. On the other hand, it is easy to ignore the interaction between input and output, thus affecting the overall performance of the model, which needs to be solved urgently. Summary of the invention

[0005] The present application provides a method, device, electronic device and medium for alleviating hallucinations in a large multimodal model, so as to solve the problems that the existing technology for alleviating the hallucination phenomenon in a large multimodal language model is too complicated, costly and affects the performance of the model, and effectively reduce and manage the hallucination phenomenon in the large multimodal language model, and improve its reliability and accuracy in practical applications.

[0006] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a method for alleviating hallucinations of a multimodal large model, comprising the following steps:

[0007] Get the input image and text data;

[0008] Processing the graphic data to obtain text tokens and image tokens, aligning the text tokens and the image tokens to obtain an input sequence of the graphic data, and constructing an image token attention vector for each position in the input sequence;

[0009] Based on the image token attention vector of each position, measure the image understanding difference between different decoding layers by Jensen-Shannon divergence, and determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result;

[0010] Using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

[0011] According to an embodiment of the present application, the processing of the graphic data to obtain text tokens and image tokens includes:

[0012] Processing the text input of the graphic data using a preset large language model to obtain the text token;

[0013] The image input of the graphic data is converted into a visual embedding through a preset encoder, and the visual embedding is mapped into the image token through a visual-language alignment connector.

[0014] According to one embodiment of the present application, constructing the image token attention vector for each position in the input sequence includes:

[0015] Representing the attention weight distribution of the image token by the maximum weight in the multi-head attention;

[0016] The maximum weight in the multi-head attention is normalized using the softmax function to obtain the image token attention vector for each position in the input sequence.

[0017] According to one embodiment of the present application, the image token attention vector of each position is:

[0018]

[0019] in, is the maximum weight of each position, i s to e is the position range of the image token in the input sequence, n is the layer of the decoder, and t is each position.

[0020] According to one embodiment of the present application, before using a preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through a linear projection layer, it also includes:

[0021] Adjust the attention weight of the target layer.

[0022] According to the method for alleviating hallucinations of a large multimodal model proposed in the embodiment of the present application, the input text and image data is processed to obtain text tokens and image tokens, and the text tokens and image tokens are aligned to obtain the input sequence of text and image data, and the image token attention vector for each position in the input sequence is constructed; based on the image token attention vector, the image understanding differences between different decoding layers are measured by Jensen-Shannon divergence, so as to determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through the linear projection layer to predict the probability distribution of the next token. Thus, the existing technology for alleviating the hallucination phenomenon of large multimodal language models is solved, which is too complicated, costly, and affects the performance of the model, and the hallucination phenomenon in large multimodal language models is effectively reduced and managed, and its reliability and accuracy in practical applications are improved.

[0023] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a multi-modal large model hallucination relief device, comprising:

[0024] The acquisition module is used to obtain the input graphic data;

[0025] A construction module, used to process the graphic data to obtain text tokens and image tokens, align the text tokens and the image tokens to obtain an input sequence of the graphic data, and construct an image token attention vector for each position in the input sequence;

[0026] A determination module is used to measure the image understanding difference between different decoding layers by Jensen-Shannon divergence based on the image token attention vector of each position, and determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result;

[0027] The prediction module is used to utilize a preset early exit mechanism to transform the hidden state of the target layer into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

[0028] According to one embodiment of the present application, the building module is specifically used for:

[0029] Processing the text input of the graphic data using a preset large language model to obtain the text token;

[0030] The image input of the graphic data is converted into a visual embedding through a preset encoder, and the visual embedding is mapped into the image token through a visual-language alignment connector.

[0031] According to one embodiment of the present application, the building module is specifically used for:

[0032] Representing the attention weight distribution of the image token by the maximum weight in the multi-head attention;

[0033] The maximum weight in the multi-head attention is normalized using the softmax function to obtain the image token attention vector for each position in the input sequence.

[0034] According to one embodiment of the present application, the image token attention vector of each position is:

[0035]

[0036] in, is the maximum weight of each position, I s To I e is the position range of the image token in the input sequence, n is the layer of the decoder, and t is each position.

[0037] According to one embodiment of the present application, before using a preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through a linear projection layer, the prediction module is further used to:

[0038] Adjust the attention weight of the target layer.

[0039] According to the hallucination mitigation device of a multimodal large model proposed in the embodiment of the present application, the input text and image data is processed to obtain text tokens and image tokens, and the text tokens and image tokens are aligned to obtain the input sequence of text and image data, and the image token attention vector of each position in the input sequence is constructed; based on the image token attention vector, the image understanding difference between different decoding layers is measured by Jensen-Shannon divergence, so as to determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through the linear projection layer to predict the probability distribution of the next token. In this way, the problems of the existing technology for mitigating the hallucination phenomenon of multimodal large language models being too complex, costly, and affecting model performance are solved, and the hallucination phenomenon in the multimodal large language model is effectively reduced and managed, and its reliability and accuracy in practical applications are improved.

[0040] To achieve the above-mentioned objectives, the third aspect of the present application proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for alleviating hallucinations of a multimodal large model as described in the above-mentioned embodiments.

[0041] To achieve the above objectives, the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the method for alleviating hallucinations of a multimodal large model as described in the above embodiments.

[0042] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the method for alleviating hallucinations of a multimodal large model as described in the above embodiments.

[0043] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0045] Figure 1 A flowchart of a method for alleviating hallucinations of a multimodal large model provided according to an embodiment of the present application;

[0046] Figure 2 is a flowchart of image token attention enhanced decoding according to one embodiment of the present application;

[0047] Figure 3 A schematic block diagram of a multimodal large model hallucination alleviation device provided according to an embodiment of the present application;

[0048] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0050] The following describes the method, device, electronic device and medium for alleviating hallucinations of a multimodal large model proposed in accordance with an embodiment of the present application with reference to the accompanying drawings.

[0051] Figure 1 It is a flowchart of a method for alleviating hallucinations of a multimodal large model according to an embodiment of the present application.

[0052] Before introducing the method for alleviating hallucinations of a multimodal large model proposed in the embodiment of the present application, the relevant technical background is first introduced.

[0053] Understandably, the term hallucination was first defined in language models, which refers to the model outputting answers that appear to be grammatically or semantically correct, but are actually inconsistent with the facts or irrelevant to the context. Hallucinations are widely present in large language models, seriously affecting their practical applications in various real-world scenarios, such as medical consultations, autonomous driving, and robotics.

[0054] In this regard, in order to solve the hallucination problem in large language models, most methods increase data or training costs by means of, for example, building new manually annotated datasets, introducing external knowledge bases, or performing additional training or fine-tuning. Specific methods include training a posteriori hallucination correctors to detect and correct hallucination outputs; using external knowledge bases to perform retrieval enhancement to reduce hallucination problems; and using reinforcement learning from human feedback (RLHF) to align model outputs with human feedback. In addition, there are some methods that attempt to reduce hallucinations during reasoning by leveraging the implicit knowledge of the model. Such methods do not increase or only increase almost negligible training or data costs. For example, by comparing and decoding the output probability distributions of different layers, the hallucinations of large language models are reduced without increasing the inference time by no more than 10%.

[0055] For multimodal large language models, the main focus of the academic community on the hallucination problem is whether the output of the large language model is faithful to the image content. Compared with the solution to reduce the hallucination of the large language model, most methods introduce additional data and training costs, and only a small number of methods try to solve the hallucination problem in the inference stage without introducing additional data and training costs. For example, the VCD (Visual Contrastive Decoding) method borrows the idea of ​​contrastive decoding, inputs the original image and the noisy image into the multimodal large model respectively, to obtain the probability distribution under two different outputs, and performs contrastive decoding on these two probability distributions; the OPERA method (a new decoding method) proposes the over-trust penalty and retrospection allocation strategies based on the discovery that hallucination output is often accompanied by the phenomenon of attention aggregation. Although these methods do not introduce additional training and data costs, they bring exponential inference delays.

[0056] It can be seen that the relevant technologies have the following shortcomings: (1) Reliance on high-quality annotated data and additional training costs, that is, existing methods for reducing hallucinations often require a large amount of high-quality annotated data and additional training costs. These methods include building manually annotated datasets, integrating external knowledge bases, or performing additional training and fine-tuning, which increase the complexity and cost of model development and deployment; (2) Ignoring the interaction between input and output, that is, although some methods try to use the internal representation of large models to reduce hallucinations in the inference stage, these methods usually focus on the separate processing of input and output and ignore the interaction between them, which may lead to the inability to fully capture and utilize the complex relationship between input and output when processing multimodal inputs, thereby affecting the overall performance of the model; (3) The reasoning delay is significantly increased, that is, although some hallucination reduction methods implemented in the reasoning stage do not require additional training or data costs, they often significantly increase the reasoning delay, which is unacceptable for scenarios with high real-time requirements in practical applications, such as autonomous driving and robotics, because the increase in reasoning delay may cause the system response speed to decrease, affecting its actual application effect and safety.

[0057] Based on the above problems, the embodiment of the present application proposes a method for alleviating hallucinations of a multimodal large model. First, by processing the input image and text data, text tokens and image tokens can be obtained. The input sequence of image and text data can be obtained by aligning the text tokens and image tokens, and the image token attention vector for each position in the input sequence is constructed; secondly, based on the image token attention vector, the image understanding difference between different decoding layers can be measured by Jensen-Shannon divergence, so as to determine the target with the largest distance from the image token attention vector of the last layer according to the measurement result; finally, using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through the linear projection layer to predict the probability distribution of the next token. In this way, the existing technology for alleviating the hallucination phenomenon of multimodal large language models can be solved, which is too complicated, costly, and affects the performance of the model, and the hallucination phenomenon in the multimodal large language model can be effectively reduced and managed, and its reliability and accuracy in practical applications can be improved.

[0058] For example, Figure 1 As shown, the hallucination mitigation method of the multimodal large model includes the following steps:

[0059] In step S101, input graphic and text data is obtained.

[0060] Among them, graphic data refers to image data and text data. The multimodal large language model is designed to process and generate data from different modalities (such as images and text) to provide users with a rich and natural interactive experience. The specific graphic data input can be determined according to the actual needs of the user.

[0061] In step S102, the graphic data is processed to obtain text tokens and image tokens, and the text tokens and image tokens are aligned to obtain an input sequence of the graphic data, and an image token attention vector is constructed for each position in the input sequence.

[0062] That is to say, combined Figure 2 As shown in , after determining the image and text data to be input, the multimodal large language model can align the token space of the two modal data, that is, the image and text data can be processed separately to obtain the corresponding text tokens and image tokens. In order to enable images and texts to be processed in the same model, a mechanism (i.e., visual-language alignment connector or cross-modal converter) is required to align the representations of the two modalities into the same semantic space (i.e., represented in the same vector space), thereby obtaining the input sequence of the image and text data (which can be an alternating sequence of text and image tokens, or a parallel sequence containing text and image tokens), which can be recorded as Where L represents the length of the input token. Assume that the starting and ending positions of the image token in the input sequence are I and s and I e , then 1≤I s e ≤L. Based on the position of the image token in the input sequence, the image token attention vector corresponding to each position in the sequence can be constructed using the multi-head attention mechanism and the softmax function, thereby enhancing the model's understanding and utilization of image information, especially when processing text, it can focus on relevant image content.

[0063] The following describes in detail how to process graphic data to obtain text tokens and image tokens.

[0064] As a possible implementation method, in some embodiments, the graphic data is processed to obtain text tokens and image tokens, including: using a preset large language model to process the text input of the graphic data to obtain text tokens; converting the image input of the graphic data into a visual embedding through a preset encoder, and mapping the visual embedding into an image token through a visual-language alignment connector.

[0065] ​Among them, text tokens refer to the representation obtained by dividing the text into a series of discrete symbols or tags. These tokens can be words, characters, or other language elements, depending on the model used and the task requirements. For example, in natural language processing, text tokenization is to convert text into a format that the model can understand, such as dividing a sentence into words or phrases, and each word or phrase is converted into one or more tokens; image tokens refer to the representation obtained by decomposing an image into a series of discrete symbols or tags. These tags can represent specific areas, objects or features in the image. For example, in computer vision tasks, image tokenization divides the image into multiple small blocks, each of which is regarded as a token.

[0066] Specifically, for text input, the tokenization tools of preset large language models (such as LLM (Large Language Model)) (such as BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language model), GPT (Generative Pre-trained Transformer, a deep learning model trained on Internet data) and other models) can be used to process the text input in the graphic data and decompose the text input into a series of text tokens; for image input, the visual encoder (such as CNN (Convolutional Neural Network), Vision Transformer, etc.) can be used to process the image input in the graphic data and encode the image input into a series of visual embeddings (i.e., image feature vectors), which are mapped into image token sequences that match the dimensions of text tokens through visual-language alignment connectors (the length may be different from the text token sequence and requires further processing for alignment).

[0067] As a possible implementation method, in some embodiments, an image token attention vector for each position in the input sequence is constructed, including: representing the attention weight distribution of the image token by the maximum weight in the multi-head attention; using the softmax function to normalize the maximum weight in the multi-head attention to obtain the image token attention vector for each position in the input sequence.

[0068] Among them, the Multi-Head Attention (MHA) mechanism is one of the core components of the Transformer model, which allows the model to process information in parallel in different representation subspaces. When processing graphic data, the dependencies between image tokens can be calculated by applying the multi-head attention mechanism between image tokens. The image token attention vector refers to the vector representation of the important areas that the model pays attention to when processing an image. These vectors not only contain information about specific areas in the image, but also reflect the degree of "attention" of the model to these areas, that is, the model considers these areas to be important for understanding the entire image or completing specific tasks (such as object detection, image classification, etc.).

[0069] Specifically, the image token embedding is divided into multiple "heads", and each "head" will pass through a self-attention layer independently. In each self-attention layer, the image token will calculate the attention score with all other tokens in its sequence (including itself). For each "head", an attention weight matrix can be obtained, and each element in the matrix represents the attention weight of one image token to another image token.

[0070] For layer N, the attention weights are:

[0071]

[0072] Among them, l t Representation sequence The length of , that is, the total length of the input and output token sequences; Contains query For keys Each token in (from 1st to lth t positions), satisfying

[0073] Considering that the maximum weight in multi-head attention usually indicates a stronger confidence, the embodiment of the present application selects the maximum weight for each position:

[0074]

[0075] When the position range of the image token in the input sequence is I s to e At these positions, the maximum weight in the multi-head attention can be selected and the softmax function can be used to calculate After normalization, for the n-th layer decoder, the image token attention vector (i.e., iTaV) at position t (i.e., the image token attention vector at each position) is:

[0076]

[0077] in, is the maximum weight for each position, i s To I e is the position range of the image token in the input sequence, n is the number of layers in the decoder, and t is each position.

[0078] In step S103, based on the image token attention vector at each position, the image understanding differences between different decoding layers are measured by Jensen-Shannon divergence, and the target layer with the largest distance from the image token attention vector of the last layer is determined according to the measurement result.

[0079] Among them, Jensen-Shannon divergence (JSD for short) is an indicator to measure the similarity of two probability distributions, and is often used to compare the differences between two probability distributions.

[0080] Specifically, for each decoding layer, the image token attention vector of each position is extracted. For each pair of adjacent decoding layers (or selected layer pairs), the Jensen-Shannon divergence between their respective image token attention vector sets is calculated. Since each position has an image token attention vector, the Jensen-Shannon divergence of all position vector pairs can be calculated, and then the average or other aggregation methods are used to obtain a single divergence value to measure the difference in image understanding between different decoding layers (i.e., the distance between the corresponding iTaVs):

[0081]

[0082] Furthermore, all decoding layers (except the last layer) are traversed, and for each layer, the Jensen-Shannon divergence between it and the last layer is calculated, and the target layer with the largest distance from the image token attention vector of the last layer is determined based on the measurement result.

[0083] It can be understood that LLMs are usually composed of multiple stacked layers, each of which processes its input representation and passes it to the next layer. As the input data passes through the LLM hierarchy, the LLM's internal representation gradually improves as the input progresses through its layers, that is, the model gradually extracts and integrates more contextual information, semantic features, and patterns. Therefore, intuitively, the representation of the last decoding layer (i.e. ) than the middle layer (i.e. Where m ranges from 1 to N-1) is more rich and comprehensive. At the same time, the hallucination output of MLLMs usually occurs when the attention weight of the image token is reduced. Therefore, the embodiment of the present application proposes to extract and amplify the inter-layer improvement of image understanding in MLLMs to alleviate its hallucination.

[0084] Specifically, the input sequence h 0 can be directly input into the LLM to generate a response, that is, the LLM processes the input sequence h through multiple decoding layers consisting of a multi-head attention mechanism and a multilayer perceptron (MLP). 0 Decode. Assume that LLM consists of N layers, with h 0 As the hidden state input of the first layer, for each position t, we have:

[0085]

[0086] According to this model, we can get

[0087] In each decoding step t, the target layer M whose iTaV of the decoding layer is the largest from the iTaV of the last layer is selected:

[0088]

[0089] in, yes A subset of . Compared with other layers in ) can maximize the Improvements in image attention.

[0090] In step S104, the preset early exit mechanism is used to transform the hidden state of the target layer into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

[0091] LLM can be achieved through a linear projection layer Mapped to V-dimensional space, where V represents the vocabulary. Finally, the projection is transformed into a probability distribution for predicting the next token using the softmax function, as follows:

[0092]

[0093] In addition, the embodiment of the present application can also use a preset early exit mechanism (allowing the model to output results when reaching a certain intermediate layer without waiting until the last layer, which can speed up the reasoning process) to project the hidden state of the target layer M through the linear projection layer. Converted into a probability distribution:

[0094]

[0095] Therefore, for a given input, this application can obtain p N and p M, then, iTaD obtains the model’s final predicted token probability distribution p according to the parameter λ (λ=-∞optimal) iTaD , and so on, the decoding process is gradually implemented at each time t:

[0096] p iTaD =λp N +(1-λ)p M ;

[0097] Note that the resulting p iTaD Can satisfy the original output probability p N When the attention to the image token is less, the improvement of its attention relative to the hidden layer M on the image is amplified, thereby implicitly achieving enhanced attention to the image token and alleviating the hallucination problem.

[0098] Furthermore, in some embodiments, before using a preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through a linear projection layer, it also includes: adjusting the attention weight of the target layer.

[0099] It can be understood that before using the preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through the linear projection layer, the attention weight of the target layer can be adjusted according to preset strategies or conditions, such as renormalization, etc. The weight adjustment can optimize the target layer's attention distribution to important information in the input sequence so as to better capture key features.

[0100] In summary, the image token attention enhanced decoding proposed in the embodiment of the present application has the following advantages:

[0101] (1) Enhanced image understanding: By defining the image token attention vector (iTaV) and the image token attention amplifier (iTaMP), this application can extract and maximize the image understanding improvements between model levels, effectively enhancing the model's ability to process image information.

[0102] (2) Alleviating hallucination phenomena: This application introduces an image token attention amplifier to output token distribution, implicitly improving the model's attention to image tokens, thereby significantly alleviating hallucination phenomena in multimodal large language models.

[0103] (3) Versatility and ease of use: As a plug-and-play method, this application is applicable to different multimodal large language models, data sets, and benchmarks, and has wide applicability. At the same time, this application does not require major modifications to the original model, does not require any additional data sets and additional training, and maintains a low inference cost while alleviating the hallucination problem, meeting the requirements of scenarios where training resources are scarce and inference efficiency is required.

[0104] According to the method for alleviating hallucinations of a large multimodal model proposed in the embodiment of the present application, the input text and image data is processed to obtain text tokens and image tokens, and the text tokens and image tokens are aligned to obtain the input sequence of text and image data, and the image token attention vector for each position in the input sequence is constructed; based on the image token attention vector, the image understanding differences between different decoding layers are measured by Jensen-Shannon divergence, so as to determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through the linear projection layer to predict the probability distribution of the next token. Thus, the existing technology for alleviating the hallucination phenomenon of large multimodal language models is solved, which is too complicated, costly, and affects the performance of the model, and the hallucination phenomenon in large multimodal language models is effectively reduced and managed, and its reliability and accuracy in practical applications are improved.

[0105] Next, the multimodal large model hallucination alleviation device proposed in the embodiment of the present application is described with reference to the accompanying drawings.

[0106] Figure 3 It is a block diagram of a multimodal large model hallucination relief device according to an embodiment of the present application.

[0107] like Figure 3 As shown, the multimodal large model hallucination relief device 10 includes: an acquisition module 100, a construction module 200, a determination module 300 and a prediction module 400.

[0108] Wherein, the acquisition module 100 is used to acquire input graphic data;

[0109] A construction module 200 is used to process the image and text data to obtain text tokens and image tokens, align the text tokens and image tokens to obtain an input sequence of the image and text data, and construct an image token attention vector for each position in the input sequence;

[0110] A determination module 300 is used to measure the image understanding difference between different decoding layers by Jensen-Shannon divergence based on the image token attention vector of each position, and determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result;

[0111] The prediction module 400 is used to utilize a preset early exit mechanism to transform the hidden state of the target layer into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

[0112] Furthermore, in some embodiments, the construction module 200 is specifically used to:

[0113] Using a preset large language model to process the text input of the graphic data to obtain text tokens;

[0114] The image input of the graphic data is converted into visual embeddings through a preset encoder, and the visual embeddings are mapped to image tokens through a visual-language alignment connector.

[0115] Furthermore, in some embodiments, the construction module 200 is specifically used to:

[0116] The attention weight distribution of the image token is represented by the maximum weight in the multi-head attention;

[0117] The maximum weight in the multi-head attention is normalized using the softmax function to obtain the image token attention vector for each position in the input sequence.

[0118] Further, in some embodiments, the image token attention vector for each position is:

[0119]

[0120] in, is the maximum weight for each position, I s To I e is the position range of the image token in the input sequence, n is the number of layers in the decoder, and t is each position.

[0121] Furthermore, in some embodiments, before using a preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through a linear projection layer, the prediction module 400 is further used to:

[0122] Adjust the attention weights of the target layer.

[0123] It should be noted that the above explanation of the embodiment of the method for alleviating hallucinations of a large multimodal model is also applicable to the device for alleviating hallucinations of a large multimodal model of this embodiment, and will not be repeated here.

[0124] According to the hallucination mitigation device of a multimodal large model proposed in the embodiment of the present application, the input text and image data is processed to obtain text tokens and image tokens, and the text tokens and image tokens are aligned to obtain the input sequence of text and image data, and the image token attention vector of each position in the input sequence is constructed; based on the image token attention vector, the image understanding difference between different decoding layers is measured by Jensen-Shannon divergence, so as to determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through the linear projection layer to predict the probability distribution of the next token. In this way, the problems of the existing technology for mitigating the hallucination phenomenon of multimodal large language models being too complex, costly, and affecting model performance are solved, and the hallucination phenomenon in the multimodal large language model is effectively reduced and managed, and its reliability and accuracy in practical applications are improved.

[0125] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0126] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0127] When the processor 402 executes the program, the method for alleviating hallucinations of a multimodal large model provided in the above embodiment is implemented.

[0128] Furthermore, the electronic device further comprises:

[0129] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0130] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0131] The memory 401 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0132] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0133] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0134] The processor 402 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0135] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for alleviating hallucinations of a large multimodal model.

[0136] The embodiment of the present application also proposes a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for alleviating hallucinations of a large multimodal model.

[0137] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0138] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0139] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for alleviating hallucinations in a multimodal large model, characterized in that: The following steps are involved: Get the input image and text data; Processing the graphic data to obtain text tokens and image tokens, aligning the text tokens and the image tokens to obtain an input sequence of the graphic data, and constructing an image token attention vector for each position in the input sequence; Based on the image token attention vector of each position, measure the image understanding difference between different decoding layers by Jensen-Shannon divergence, and determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; Using the preset early exit mechanism, the hidden state of the target layer is converted into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

2. The method according to claim 1, characterized in that The processing of the graphic data to obtain text tokens and image tokens includes: Processing the text input of the graphic data using a preset large language model to obtain the text token; The image input of the graphic data is converted into a visual embedding through a preset encoder, and the visual embedding is mapped into the image token through a visual-language alignment connector.

3. The method according to claim 2, characterized in that The constructing of the image token attention vector for each position in the input sequence comprises: Representing the attention weight distribution of the image token by the maximum weight in the multi-head attention; The maximum weight in the multi-head attention is normalized using the softmax function to obtain the image token attention vector for each position in the input sequence.

4. The method according to claim 3, characterized in that The image token attention vector for each position is: in, is the maximum weight of each position, I s To I e is the position range of the image token in the input sequence, n is the layer of the decoder, and t is each position.

5. The method according to claim 1, characterized in that Before using the preset early exit mechanism to convert the hidden state of the target layer into a probability distribution through a linear projection layer, the method further includes: Adjust the attention weight of the target layer.

6. A multi-modal large-scale hallucination relief device, characterized in that: include: The acquisition module is used to obtain the input graphic data; A construction module, used to process the graphic data to obtain text tokens and image tokens, align the text tokens and the image tokens to obtain an input sequence of the graphic data, and construct an image token attention vector for each position in the input sequence; A determination module is used to measure the image understanding difference between different decoding layers by Jensen-Shannon divergence based on the image token attention vector of each position, and determine the target layer with the largest distance from the image token attention vector of the last layer according to the measurement result; The prediction module is used to utilize a preset early exit mechanism to transform the hidden state of the target layer into a probability distribution through a linear projection layer to predict the probability distribution of the next token.

7. The device according to claim 6, characterized in that The building blocks are specifically used for: Using a preset large language model to process the text input of the graphic data to obtain the text token; The image input of the graphic data is converted into a visual embedding through a preset encoder, and the visual embedding is mapped into the image token through a visual-language alignment connector.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for alleviating hallucinations of a multimodal large model as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for alleviating hallucinations of a multimodal large model as described in any one of claims 1 to 5.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, is used to implement the method for alleviating hallucinations of a multimodal large model as described in any one of claims 1 to 5.

Images