Replay generation method and device, electronic equipment and medium

Abstract

The embodiment of the invention is applicable to the technical field of artificial intelligence, and provides a reply generation method and device, electronic equipment and a medium, the method comprises the following steps: reasoning input data by using a first multi-modal large language model, the input data comprising a first text instruction and a picture, the first multi-modal large language model is used for generating an initial reply text of the input data; in the process of reasoning the input data by the first multi-modal large language model, performing illusion detection on the first multi-modal large language model; and if it is detected that the illusion feature appears in the first multi-modal large language model, processing the input data through a multi-modal large language model post-processing module to determine a target reply text of the input data. By means of the method, under the condition that an existing model structure or parameters are not modified, the illusion problem of the multi-modal large language model can be solved, the use difficulty of the multi-modal large language model is lowered, and reply accuracy can be guaranteed while reply generation efficiency is guaranteed.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to a response generation method, device, electronic device and medium. Background Art

[0002] The "Hallucination" phenomenon of large language models refers to the fact that when the model generates text, it may produce non-existent, wrong or unfounded information. This is because large language models cannot understand the real world and can only make predictions based on training data. If the data input to the model is biased or the model's prediction ability is insufficient, the large language model will experience "hallucination" phenomenon.

[0003] Multimodal Large Language Models (MLLMs) refer to models that take input data including text instructions and images and return text responses. When generating content, MLLMs may experience the phenomenon of "hallucination" where the text content generated by the model is inconsistent with the actual visual content of the image. This phenomenon of hallucination seriously hinders the practical application of MLLMs.

[0004] At present, the cost of solving the hallucination phenomenon of multimodal large language models is relatively high. At present, methods to solve the hallucination phenomenon of multimodal large language models may include: optimizing training data or optimizing model structure. However, optimizing training data requires a large amount of high-quality labeled data, which consumes human resources, and requires training or fine-tuning the model every time the multimodal large language model is improved. Optimizing model data may include modifying the visual encoder, connection module, or large language model module to adapt to different inputs and improve the accuracy of the output; this method requires adjustments to the model structure and requires retraining the model, which increases the technical difficulty and consumption of computing resources. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a response generation method, device, electronic device and medium, which can solve the hallucination problem of the multimodal large language model without modifying the existing model structure or parameters, reduce the difficulty of using the multimodal large language model, and ensure the accuracy of the response while ensuring the efficiency of response generation.

[0006] A first aspect of an embodiment of the present application provides a reply generation method, comprising:

[0007] Using a first multimodal large language model to infer input data, the input data includes a first text instruction and a picture, and the first multimodal large language model is used to generate an initial response text for the input data;

[0008] During the process of the first multimodal large language model reasoning about the input data, performing hallucination detection on the first multimodal large language model;

[0009] If it is detected that the first multimodal large language model has hallucination features, the input data is processed by a second processing module to determine a target reply text of the input data;

[0010] The target reply text is output.

[0011] In a possible implementation, a second multimodal large language model is deployed in the second processing module, and reasoning the input data by the second processing module to determine a target reply text of the input data includes:

[0012] The input data is input into the second multimodal large language model to obtain the target reply text output by the second multimodal large language model, wherein the parameter scale of the second multimodal large language model is greater than the parameter scale of the first multimodal large language model.

[0013] In a possible implementation, an open set visual basis model is deployed in the second processing module, and the processing of the input data by the second processing module to determine a target reply text of the input data includes:

[0014] Determining auxiliary information of the target contained in the picture using the open set visual basis model;

[0015] Determine a second text instruction according to the auxiliary information and the first text instruction;

[0016] The second text instruction and the picture are input into the first multimodal large language model to obtain the target reply text output by the first multimodal large language model.

[0017] In a possible implementation, a second multimodal large language model is deployed in the second processing module, and the processing of the input data by the second processing module to determine a target reply text of the input data includes:

[0018] Sending the input data to a server; the second processing module is deployed in the server;

[0019] Receive a target reply text of the input data fed back from the server.

[0020] In a possible implementation, the first multimodal large language model includes a self-attention layer, and performing hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning about the input data includes:

[0021] In the process of the first multimodal large language model inferring the input data, extracting the self-attention matrix of the last layer in the self-attention layer;

[0022] Based on the self-attention matrix, determining a hallucination evaluation feature value of the initial reply text, wherein the hallucination evaluation feature value is used to evaluate the influence of the summary word-element on the initial reply text;

[0023] It is determined whether the first multimodal large language model has a hallucination feature according to the hallucination evaluation feature value.

[0024] In a possible implementation, extracting the self-attention matrix of the last layer in the self-attention layer includes:

[0025] Extracting the multi-head self-attention matrix in the last layer of the self-attention layer;

[0026] Based on the multiple attention heads in the multi-head self-attention matrix, multiple matrices are obtained;

[0027] The weights of the corresponding positions of the multiple matrices are maximized, and the multiple matrices are normalized to obtain the self-attention matrix.

[0028] In a possible implementation, determining the hallucination evaluation feature value of the initial reply text based on the self-attention matrix includes:

[0029] Based on a sliding window, determining a plurality of sliding matrices from the self-attention matrix;

[0030] Performing column-by-column multiplication on the lower triangular matrix in each of the sliding matrices to obtain a plurality of cumulative multiplication values, wherein the cumulative multiplication values ​​are used to characterize the attention aggregation features in the sliding matrix;

[0031] The maximum accumulated value and the second largest accumulated value among the plurality of accumulated values ​​are used as the hallucination evaluation feature values.

[0032] In a possible implementation, determining whether the first multimodal large language model has a hallucination feature according to the hallucination evaluation feature value includes:

[0033] Calculate the product of the length of the sliding window and the second largest cumulative value;

[0034] If the maximum cumulative multiplication value is greater than the product, it is determined that the first multimodal large language model has hallucination features.

[0035] In a possible implementation, the first multimodal large language model includes a word segmenter, a visual feature extractor, and a large language model based on a self-attention mechanism, and the reasoning on the input data using the first multimodal large language model includes:

[0036] Using the word segmenter to obtain a digital sequence of the first text instruction;

[0037] Converting the digital sequence of the first text instructions into a first high-dimensional vector;

[0038] Extracting image features of the picture based on the visual feature extractor;

[0039] Converting the image feature into a second high-dimensional vector;

[0040] Performing feature alignment on the first high-dimensional vector and the second high-dimensional vector in the same feature space to obtain a fused feature, where the fused feature is used to represent a semantic relationship between the image and the first text instruction;

[0041] The fused features are input into the large language model to obtain the initial reply text.

[0042] In a possible implementation, the method further includes:

[0043] When it is determined that the first multimodal large language model does not exhibit hallucination features in the process of generating the initial reply text, the initial reply text is output.

[0044] In a possible implementation, the first multimodal large language model is deployed in a first processing module, and the first processing module and the second processing module are deployed in an electronic device; or,

[0045] The first processing module is deployed in the electronic device, and the second processing module is deployed in a server.

[0046] A second aspect of an embodiment of the present application provides a reply generation device, including:

[0047] A first processing module, configured to use a first multimodal large language model to infer input data, wherein the input data includes a first text instruction and a picture, and the first multimodal large language model is configured to generate an initial response text for the input data;

[0048] a hallucination assessment module, configured to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning the input data, and to provide the input data to the second processing module when hallucination features are detected in the first multimodal large language model;

[0049] The second processing module is used to process the input data to determine a target reply text of the input data;

[0050] The reply output module is used to output the target reply text.

[0051] A third aspect of an embodiment of the present application provides a reply generation system, including: an electronic device and a server, wherein the electronic device is used to execute the method described in the first aspect or any aspect of the first aspect, and the server is used to provide a target reply text of input data to the electronic device.

[0052] For example, the electronic device has a first processing module and a hallucination assessment module. The server is deployed with a second processing module. The first processing module uses a first multimodal large language model to infer input data, the input data includes a first text instruction and a picture, and the first multimodal large language model is used to generate an initial response text for the input data; and when it is determined that the first multimodal large language model does not produce hallucinations in the process of generating the initial response text, the initial response text is output;

[0053] The hallucination assessment module is used to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning on the input data;

[0054] The second processing module is used to process the input data to determine the target reply text of the input data and to provide the target reply text to the electronic device. The electronic device may also include a reply output module, which is used to output the target reply text.

[0055] A fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0056] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0057] A sixth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method described in the first aspect.

[0058] Compared with the prior art, the embodiments of the present application have at least the following beneficial effects:

[0059] By applying the method in the embodiment of the present application, a reply can be automatically generated based on a multimodal large language model. When generating a reply, the first multimodal large language model can be used to infer the input data to obtain an initial reply text. In the process of the first multimodal large language model inferring the input data, the first multimodal large language model can be hallucinated to evaluate whether the first multimodal large language model has hallucination features in the process of generating the initial reply text. If the first multimodal large language model has hallucination features in the process of generating the initial reply text, it indicates that the initial reply text generated by the first multimodal large language model is inaccurate. At this time, the input data can continue to be processed by the multimodal large language model post-processing module to generate a higher quality target reply text, and the target reply text is output. The method in the embodiment of the present application can use a multimodal large language model post-processing module to solve the hallucination problem of the multimodal large language model without modifying the existing model structure or parameters, thereby reducing the difficulty of using the multimodal large language model. The method in the embodiment of the present application can use the initial reply text of the first multimodal large language model as output when the first multimodal large language model does not have hallucination features, thereby ensuring the efficiency of reply generation. In the case where the first multimodal large language model has hallucination features, the multimodal large language model post-processing module can be called to output the target reply text, thereby improving the accuracy of the reply. The method in the embodiment of the present application can balance the efficiency of reply generation and the accuracy of reply in the process of using the multimodal large language model to generate replies. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0061] Figure 1 It is a schematic diagram of the steps of a reply generation method provided in an embodiment of the present application;

[0062] Figure 2 is a schematic diagram of using a first multimodal large language model to reason about input data provided by an embodiment of the present application;

[0063] Figure 3 is a schematic diagram of hallucination feature recognition provided by an embodiment of the present application;

[0064] Figure 4 This is an architecture diagram of a reply generation system provided in an embodiment of the present application;

[0065] Figure 5 is another architecture diagram of the reply generation system provided in an embodiment of the present application;

[0066] Figure 6 is a schematic diagram of a reply generation system provided in an embodiment of the present application;

[0067] Figure 7 It is a schematic diagram of the workflow of a reply generation system provided in an embodiment of the present application;

[0068] Figure 8 is a schematic diagram of a reply generating device provided in an embodiment of the present application;

[0069] Fig. 9 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.

[0071] The multimodal large language model combines technologies such as natural language processing, computer vision, and cross-modal learning, and can establish a deep understanding and connection between multiple modal data such as text, images, and audio, thereby generating more accurate results in application scenarios such as image description generation, visual question and answer, and multimodal dialogue. The multimodal large language model in the embodiment of the present application refers to a model whose input data includes text instructions and pictures, and whose output data is text replies. The use of a multimodal large language model in the embodiment of the present application can combine a large language model based on the Transformer architecture with a visual feature extractor, integrate visual information, and generate coherent text in an autoregressive form.

[0072] The self-attention mechanism is a core component of the multimodal large language model. It can capture the dependencies between the elements of the sequence by calculating the correlation between each element in the input sequence and other elements and dynamically assigning attention weights to each element. The self-attention mechanism allows the model to focus on all positions in the sequence at the same time when processing sequence data. The self-attention mechanism efficiently models long-distance dependencies through parallel computing, allowing the large language model to better understand the contextual information of the input data and take global information into account when generating responses.

[0073] Multimodal large language models can be used in a variety of application scenarios, but when they generate content, they may have the phenomenon of "hallucination" that the text content generated by the model is inconsistent with the actual visual content of the image. For example, in visual language tasks, the content automatically generated by the multimodal large language model may have problems such as describing non-existent objects, incorrect description of object attributes, or inaccurate judgment of object relationships. This hallucination phenomenon seriously hinders the practical application of multimodal large language models.

[0074] Existing processing methods can deal with the hallucination problem of multimodal models at the data level: when hallucination occurs, it can be reduced by optimizing the training data. However, this method requires a large amount of high-quality labeled data, consumes human resources, and each improvement requires training or fine-tuning the model. Existing processing methods can also include handling the hallucination problem of multimodal models at the model level: by modifying the visual encoder, connection module or large language model module to adapt to different inputs and improve the accuracy of the output. However, this method requires adjustments to the model structure and requires retraining the model, which increases the technical difficulty and consumption of computing resources.

[0075] In addition, the hallucination problem can also be dealt with at the level of the reasoning process. For example, a post-processing process can be added after the multimodal large language model reasoning is completed, and the hallucination problem of the generated text can be solved through the post-processing process. This method does not require modifying the existing model structure or parameters, and can be easily integrated into the existing model reasoning system, but it prolongs the model reasoning process, and the post-processing module needs to be executed for all answers, which may lead to duplication of computational work and waste of resources, bringing huge additional costs.

[0076] Based on this, an embodiment of the present application provides a response generation method based on a multimodal large language model, which is used to reduce the difficulty of solving the hallucination problem of the multimodal large language model and balance the generation efficiency and accuracy of the response.

[0077] The technical solution of the present application is described below through specific embodiments.

[0078] Reference Figure 1 , shows a schematic flow chart of the steps of a reply generation method provided in an embodiment of the present application, which may specifically include the following steps:

[0079] S101, using a first multimodal large language model to infer input data, the input data including a first text instruction and a picture, the first multimodal large language model being used to generate an initial response text for the input data.

[0080] The method in this embodiment can be executed on an electronic device. The electronic device can be a device that can deploy and run a multimodal large language model. This embodiment does not limit the specific type of the electronic device.

[0081] The first multimodal large language model can be deployed in the electronic device, and the first multimodal large language model is a multimodal large language model based on a self-attention mechanism with a relatively small parameter scale. For example, the first multimodal large language model can be a Qwen-VL model with a parameter scale of 7B.

[0082] The input data of the multimodal large language model may include text data and image data. The above-mentioned first text instruction is the text data in the input data, and the picture is the image data in the input data. Based on the input data, the multimodal large language model can generate a text reply. For example, the multimodal large language model can complete various types of visual question-answering tasks such as picture understanding, picture description generation, and picture question-answering according to the content of the first text instruction, and generate corresponding reply text.

[0083] In the process of generating an initial reply text based on input data, the first multimodal large language model can first extract text features from the first text instruction and image features from the picture; then perform feature fusion on the picture features and the text features to obtain fused features that can describe the semantic relationship between the picture and the first text instruction, and then automatically generate a reply text based on the fused features.

[0084] Figure 2 is a schematic diagram of using a first multimodal large language model to reason about input data provided by an embodiment of the present application. Figure 2 As shown, the first multimodal large language model can extract text features from text instructions and embed the features into a high-dimensional feature space. In one possible implementation, the first multimodal large language model may include a tokenizer, and based on the tokenizer, the first multimodal large language model can perform feature extraction of the first text instruction. Exemplarily, a digital sequence of the first text instruction can be obtained using a tokenizer; then the digital sequence can be converted into a first high-dimensional vector, and the first high-dimensional vector is the text feature embedded in the high-dimensional feature space. The first multimodal large language model can convert the digital sequence into a first high-dimensional vector by word embedding.

[0085] like Figure 2 As shown, the first multimodal large language model can perform visual feature extraction on the input image, thereby extracting image features. The first multimodal large language model can include a visual feature extractor, based on which the image features of the image can be extracted and converted into a second high-dimensional vector, which is the image feature embedded in the high-dimensional feature space.

[0086] In the high-dimensional feature space, image features and visual features can be aligned across modalities to obtain fused features. For example, the first multimodal large language model can align the first high-dimensional vector and the second high-dimensional vector in the same feature space through the input projection layer (Projector) to obtain fused features, which can describe the semantic relationship between the image and the first text instruction.

[0087] The first multimodal large language model may include a large language model based on a self-attention mechanism, and the large language model based on the self-attention mechanism is used to generate a reply text according to the input feature data. The above fusion feature is input into the large language model of the first multimodal large language model to obtain the above initial reply text.

[0088] S102: Perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model inferring the input data.

[0089] The first multimodal large language model includes a self-attention layer. When visualizing the last layer of the self-attention matrix during the inference process of the multimodal large language model, it is observed that when the model generates hallucinated content, the last layer of the self-attention matrix usually has an obvious attention aggregation pattern, that is, a token in the self-attention matrix has a high attention score for all other tokens.

[0090] Since the existing multimodal large language model is based on the large language model, the characteristics of its causal language model enable the self-attention layer of the multimodal large language model to aggregate the information of the previous word unit to a summary word unit at a shallow level. However, from the perspective of information flow, as the generated text sequence becomes longer and longer, the information provided by the word unit at the front of the sequence will be diluted in the process of information flow between summary words, that is, it is difficult for a summary word unit to fully record the information contained in all the previous words in the sequence. When the self-attention layer is too affected by the summary word unit during the processing process, it is easy to produce hallucination content. When the self-attention layer is too affected by the summary word unit during the processing process, the attention aggregation phenomenon occurs.

[0091] In the process of the first multimodal large language model reasoning about the input data, it is possible to determine whether the first multimodal large language model has hallucination characteristics by judging whether a specific attention aggregation pattern appears in the last layer of the self-attention matrix of the self-attention layer. If the self-attention layer has attention aggregation, it can indicate that a certain word in the self-attention matrix has a high attention score for all other words.

[0092] When performing hallucination detection on the first multimodal large language model, the self-attention matrix of the last layer in the self-attention layer can be extracted during the process of the first multimodal large language model reasoning about the input data. The electronic device can extract the multi-head self-attention matrix in the last layer of the self-attention layer of the first multimodal large language model. The calculation method of the multi-head self-attention matrix can be:

[0093]

[0094] Where Q, K, and D represent the query, key, and their corresponding feature dimensions in the large language model, respectively. In a multi-head attention matrix, the results of multiple attention heads can be calculated to obtain multiple matrices. The electronic device can take the maximum value of the weights of the corresponding positions of multiple matrices and renormalize them to obtain the processed self-attention matrix, thereby enhancing the information with high confidence in the self-attention matrix.

[0095] After extracting the self-attention matrix, the electronic device can determine the hallucination evaluation feature value of the initial reply text based on the self-attention matrix. The hallucination evaluation feature value can be used to evaluate the influence of the summary word on the initial reply text, so that the electronic device can determine whether the first multimodal large language model has hallucination phenomenon according to the hallucination feature value. Figure 3 It is a schematic diagram of hallucination feature recognition provided in an embodiment of the present application.

[0096] like Figure 3 As shown, the electronic device can set a local sliding window according to the self-attention matrix. The sliding window can be a sliding matrix formed by a sequence with the latest generated word as the end point and a length of window_size=k. Figure 3 As shown, based on the sliding window, multiple sliding matrices can be determined from the self-attention matrix. For the sliding matrix of the sliding window, the upper triangular matrix can be filled with zeros. Since the original value in the self-attention matrix is ​​too small, the value in the sliding window can be enlarged by σ times. After determining multiple sliding matrices, the lower triangular matrix in each sliding matrix can be multiplied column by column to obtain multiple cumulative values, which can be used to characterize the attention aggregation features in the sliding matrix. The maximum cumulative value and the second largest cumulative value among the multiple cumulative values ​​are used as hallucination evaluation feature values. The electronic device can calculate the product of the length of the sliding window and the second largest cumulative value; if the maximum cumulative value is greater than the product, it is determined that the first multimodal large language model has hallucination features.

[0097] For example, the maximum cumulative value can be represented by max(w), and its corresponding column is the summary word. The second largest cumulative value is represented by second_max(w). The hallucination phenomenon can be evaluated in the following way: if max(w)>k*sencond_max(w) (where k represents the length of the sliding window sequence), it means that the generated response is too greatly affected by the summary word, and there is a high probability of hallucination problem, that is, it is determined that the first multimodal large language model has hallucination characteristics.

[0098] S103: If it is detected that the first multimodal large language model has hallucination features, the input data is processed by a second processing module to determine a target reply text of the input data.

[0099] The method in the embodiment of the present application can be applied in a reply generation system, and the reply generation system may include a first processing module, an illusion evaluation module, and a second processing module. The first multimodal large language model can be deployed in the first processing module, the illusion evaluation module can be used to perform the above step S102, and the second multimodal large language model can be deployed in the second processing module. The second multimodal large language model can be a multimodal large language model with a larger parameter scale and higher reasoning accuracy. For example, the second multimodal large language model can be a GPT4-V model.

[0100] In a possible implementation, the reply generation system may include an electronic device, and the first processing module and the second processing module may both be deployed in the electronic device. Figure 4 This is an architecture diagram of the reply generation system provided in the embodiment of the present application. Figure 4 As shown, the second post-processing module can be deployed in an electronic device. When the hallucination feature is detected in the first multimodal large language model, the electronic device can input the input data into the second multimodal large language model, so as to obtain the target reply text output by the second multimodal large language model. Since the parameter scale of the second multimodal large language model is larger than the parameter scale of the first multimodal large language model, the reasoning accuracy of the second multimodal large language model is greater than the reasoning accuracy of the first multimodal large language model, and therefore, the reply quality of the target reply text is higher than the reply quality of the initial reply text.

[0101] In another possible implementation, the reply generation system may include an electronic device and a server. In order to save local storage space, the second multimodal large language model may be deployed in the server. Figure 5 is another architecture diagram of the reply generation system provided in the embodiment of the present application. Figure 5As shown, the first processing module and the hallucination assessment module can be deployed in the electronic device, and the second processing module can be deployed in the server. The server and the electronic device can belong to the same operator, and the server can deploy a second multimodal large language processing model developed by itself. In addition, the server can also be a server of a third-party developer, and the server can deploy the second multimodal large language processing model of the third-party developer.

[0102] When the second processing module is deployed in a cloud server, the electronic device can send input data to the server, and the server can infer the input data through the deployed second multimodal large language processing model to obtain the target reply text. For example, the electronic device may include a black box multimodal model service interface, and the electronic device can call the interface based on the input data. The interface can send the input data to the server, and the server can generate the target reply text based on the GPT4-V model and return the target reply text to the interface. The electronic device can obtain the target reply text based on the interface. Deploying the second multimodal large language model in the server can reduce the storage pressure of the electronic device while using a more powerful multimodal large language model to generate high-quality replies.

[0103] In addition, the second processing module can also solve the hallucination problem by improving the input data. In a possible implementation, an open set visual basis model can be deployed in the second processing module. The open set visual basis model can be used to obtain more information about the target in the image. For example, the open set visual basis model can be a Grouding DINO model.

[0104] By using the open set visual basic model, the electronic device can extract more information about the target in the picture, such as the number, color, shape, etc. of the target, so as to determine the auxiliary information of the target contained in the picture. The second text instruction can be determined based on the auxiliary information and the first text instruction. The auxiliary information can supplement the information in the first text instruction, so that the electronic device can obtain the second text instruction containing more feature information.

[0105] The electronic device can input the second text instruction and the picture as new input data into the first multimodal large language model, thereby obtaining the target reply text output by the first multimodal large language model. Since the second text instruction contains more feature data, after the new input data is input into the first multimodal large language model, the first multimodal large language model can extract more features, thereby improving the accuracy of the reply and solving the hallucination problem of the multimodal large language model.

[0106] S104: output the target reply text.

[0107] When hallucination features are detected in the first multimodal large language model, a target reply text with higher reasoning accuracy can be output as a final reply.

[0108] When it is determined that the first multimodal large language model does not have hallucinations in the process of generating the initial reply text, it can be determined that the accuracy of the initial reply text meets the requirements, so the initial reply text can be output. At this time, there is no need to generate the target reply text.

[0109] In the embodiment of the present application, when solving the hallucination problem, a second processing module can be added, without modifying the model structure or parameters of the existing first multimodal large language model, thereby reducing the difficulty of solving the hallucination problem of the multimodal large language model and facilitating the practical application of the multimodal large language model. In addition, the method in the embodiment of the present application can use the initial reply text of the first multimodal large language model as output when the first multimodal large language model does not have hallucination features, thereby ensuring the generation efficiency of the reply. In the case where the first multimodal large language model has hallucination features, the multimodal large language model post-processing module can be called to output the target reply text, thereby improving the accuracy of the reply. The method in the embodiment of the present application can balance the generation efficiency of the reply and the accuracy of the reply in the process of generating a reply using a multimodal large language model. The method in the embodiment of the present application can deploy a second multimodal large language model with a larger parameter scale on a server, thereby reducing the storage pressure of local electronic devices.

[0110] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0111] Figure 6 Schematic diagram of a reply generation system provided in an embodiment of the present application. Figure 6 As shown, the reply generation system includes a multimodal large language model reasoning module 61, a hallucination evaluation and selective post-processing module 62, and a multimodal large language model post-processing module 63, wherein:

[0112] The multimodal large language model inference module 61 may be used to use the first multimodal large language model to infer input data, the input data including the first text instruction and the image, the first multimodal large language model being used to generate an initial response text for the input data; and outputting the initial response text when it is determined that the first multimodal large language model does not experience hallucinations in the process of generating the initial response text;

[0113] The hallucination assessment and selective post-processing module 62 may be used to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning on the input data;

[0114] The multimodal large language model post-processing module 63 is used to process the input data to determine the target reply text of the input data when the hallucination feature is detected in the first multimodal large language model; and output the target reply text.

[0115] Figure 7 Schematic diagram of the workflow of a reply generation system provided in an embodiment of the present application. Figure 7 As shown, when the user inputs input data into the reply generation system, the first multimodal large language model deployed in the multimodal large language model reasoning module can receive text instructions and input images and perform reasoning process based on the images and text instructions.

[0116] The first multimodal large language model first converts the input text instructions into a sequence in digital form through a word segmenter, and converts the digital sequence into a high-dimensional vector representation through word embedding; for the input image data, the first multimodal large language model can use the ViT visual feature extractor based on the self-attention mechanism to extract image features and convert them into a high-dimensional vector representation that is convenient for subsequent model reasoning. Afterwards, the features of different modalities can be aligned in the same feature space through the input projection layer. When aligning features, feature mapping can be performed based on a multi-layer fully connected layer (MLP) or other complex structures, such as a cross-attention layer. The aligned features can contain better modality association information and can better represent the semantic relationship between images and texts. Afterwards, the fused features can be input into the large language model based on the self-attention mechanism. The large language model predicts the probability of generating the next sequence through forward propagation and generates reply text in the form of autoregression.

[0117] The hallucination evaluation and selective post-processing module and the multimodal large language model inference module can be deployed on the same electronic device, so that hallucination feature detection can be performed during the inference process of the multimodal large language model inference module. When the hallucination evaluation and selective post-processing module does not detect the hallucination feature, the reply text of the multimodal large language model inference module can be returned to the user.

[0118] When the hallucination assessment and selective post-processing module detects hallucination features, the multimodal large language model post-processing module can be called to generate a response to the user.

[0119] The multimodal large language model post-processing module can deploy a multimodal large language model with a larger parameter scale and higher reasoning accuracy than that in the multimodal large language model inference module, and use the higher quality replies generated by the multimodal large language model with a larger parameter scale and higher reasoning accuracy to return to the user, thereby alleviating the hallucination problem.

[0120] Alternatively, the multimodal large language model post-processing module can alleviate the hallucination problem by adding an additional visual auxiliary model. The multimodal large language model post-processing module uses the open set visual base model to first perform target detection and object attribute recognition on the input image, and adds the visual model reasoning result as auxiliary information to enhance the multimodal model reasoning after the text instruction. Then the image and the enhanced text instruction are input into the multimodal large language model of the multimodal large language model reasoning module to perform reasoning, and the generated content is returned to the user.

[0121] The multimodal large language model post-processing module can be deployed on local electronic devices or in a cloud service center. For example, the multimodal large language model post-processing module can call a black box multimodal model service interface with higher overall reasoning accuracy, and use a more powerful service interface to generate high-quality replies to return to users, thereby alleviating the hallucination problem. The black box multimodal model service interface can be used to call the GPT4-V model provided by a single developer.

[0122] Compared with the existing work of alleviating hallucinations through post-processing, the embodiment of the present application introduces a hallucination assessment and selective post-processing module in the response generation system, which can predict the occurrence of hallucination phenomena by identifying specific pattern features of the self-attention matrix in the multimodal large language model reasoning process before the post-processing process. The post-processing module is only executed when hallucination features are detected, solving the problem of low overall efficiency of the post-processing process.

[0123] In this embodiment, when the first multimodal large language model does not have hallucination problems, the first multimodal large language model, such as the Qwen-VL model, can be selected to perform conventional reply generation, thereby improving the reply generation efficiency. When the first multimodal large language model has hallucination problems, a second multimodal large language model with a larger parameter scale, such as the GPT4-V model, can be used to generate replies, thereby ensuring the accuracy of the replies. In this embodiment, hallucination evaluation and selective post-processing modules and multimodal large model post-processing modules can be added to the reasoning system, thereby alleviating the hallucination phenomenon of the multimodal large language model through selective post-processing, and balancing the accuracy and generation efficiency of the reply without modifying the existing model reasoning process.

[0124] The electronic device in the embodiment of the present application uses a first multimodal large language model with a smaller parameter scale to efficiently generate reply texts for most input data. During the process of the first multimodal large language model inferring the input data to generate an initial reply text, if the first multimodal large language model exhibits hallucination features, the electronic device can call a second processing module to post-process the input data to obtain a target reply text, thereby significantly improving the authenticity and credibility of the generated content while ensuring system performance.

[0125] Reference Figure 8 , shows a schematic diagram of a reply generation device provided in an embodiment of the present application, which may specifically include a first processing module 81, an illusion evaluation module 82, a second processing module 83 and a reply output module 84, wherein:

[0126] A first processing module 81 is used to use a first multimodal large language model to infer input data, the input data includes a first text instruction and a picture, and the first multimodal large language model is used to generate an initial response text of the input data;

[0127] a hallucination assessment module 82, configured to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model inferring the input data, and to provide the input data to the second processing module when hallucination features are detected in the first multimodal large language model;

[0128] The second processing module 83 is used to process the input data to determine the target reply text of the input data;

[0129] The reply output module 84 is used to output the above target reply text.

[0130] In a possible implementation, a second multimodal large language model is deployed in the second processing module, and the second processing module 83 includes:

[0131] The first target reply text generation submodule is used to input the above-mentioned input data into the above-mentioned second multimodal large language model to obtain the above-mentioned target reply text output by the above-mentioned second multimodal large language model, and the parameter scale of the above-mentioned second multimodal large language model is greater than the parameter scale of the above-mentioned first multimodal large language model.

[0132] In a possible implementation, an open set visual basis model is deployed in the second processing module, and the second processing module 83 includes:

[0133] An auxiliary information determination submodule, used to determine the auxiliary information of the target contained in the above picture using the above open set visual basis model;

[0134] A second text instruction generating submodule, used for determining a second text instruction according to the auxiliary information and the first text instruction;

[0135] The second target reply text generation submodule is used to input the above-mentioned second text instruction and the above-mentioned picture into the above-mentioned first multimodal large language model to obtain the above-mentioned target reply text output by the above-mentioned first multimodal large language model.

[0136] In a possible implementation, a second multimodal large language model is deployed in the second processing module, and the hallucination assessment module 82 includes:

[0137] A data sending submodule, used for sending the input data to a server; the second processing module is deployed in the server;

[0138] The target reply text receiving submodule is used to receive the target reply text of the input data fed back from the server.

[0139] In a possible implementation, the first multimodal large language model includes a self-attention layer, and the hallucination assessment module 82 includes:

[0140] A self-attention matrix extraction submodule, used for extracting the self-attention matrix of the last layer in the self-attention layer during the process of the first multimodal large language model inferring the input data;

[0141] A hallucination evaluation feature value determination submodule, used to determine the hallucination evaluation feature value of the initial reply text based on the self-attention matrix, wherein the hallucination evaluation feature value is used to evaluate the influence of the summary word on the initial reply text;

[0142] The judgment submodule is used to determine whether the first multimodal large language model has hallucination features according to the hallucination evaluation feature value.

[0143] In a possible implementation, the self-attention matrix extraction submodule includes:

[0144] A multi-head self-attention matrix extraction unit, used to extract the multi-head self-attention matrix in the last layer of the above self-attention layer;

[0145] A matrix determination unit, configured to obtain a plurality of matrices based on a plurality of attention heads in the multi-head self-attention matrix;

[0146] The self-attention matrix determination unit is used to take the maximum value of the weights of the corresponding positions of the multiple matrices above, and normalize the multiple matrices above to obtain the self-attention matrix above.

[0147] In a possible implementation, the above-mentioned hallucination assessment feature value determination submodule includes:

[0148] A sliding matrix determining unit, used to determine a plurality of sliding matrices from the self-attention matrix based on a sliding window;

[0149] A multiplication unit, used for performing column-by-column multiplication on the lower triangular matrix in each of the sliding matrices to obtain a plurality of accumulated multiplication values, wherein the accumulated multiplication values ​​are used to characterize the attention aggregation features in the sliding matrix;

[0150] The hallucination evaluation characteristic value determining unit is used to use the largest cumulative multiplication value and the second largest cumulative multiplication value among the plurality of cumulative multiplication values ​​as the hallucination evaluation characteristic value.

[0151] In a possible implementation, the above-mentioned determination submodule includes:

[0152] A product calculation unit, used to calculate the product of the length of the sliding window and the second largest cumulative value;

[0153] The hallucination judgment unit is used to determine whether the first multimodal large language model has a hallucination feature if the maximum cumulative multiplication value is greater than the product.

[0154] In a possible implementation, the first multimodal large language model includes a word segmenter, a visual feature extractor, and a large language model based on a self-attention mechanism, and the first processing module 81 includes:

[0155] A digital sequence determination submodule, used to obtain a digital sequence of the first text instruction using the word segmenter;

[0156] A first high-dimensional vector determination submodule, configured to convert the digital sequence of the first text instruction into a first high-dimensional vector;

[0157] An image feature extraction submodule, used to extract image features of the above picture based on the above visual feature extractor;

[0158] A second high-dimensional vector determination submodule, used for converting the above-mentioned image features into a second high-dimensional vector;

[0159] A feature fusion submodule, used for aligning the first high-dimensional vector and the second high-dimensional vector in the same feature space to obtain a fused feature, wherein the fused feature is used to represent the semantic relationship between the image and the first text instruction;

[0160] The initial reply text generation submodule is used to input the above fusion features into the above large language model to obtain the above initial reply text.

[0161] In a possible implementation, the above device further includes:

[0162] The reply text output module is used to output the initial reply text when it is determined that the first multimodal large language model does not have hallucination features in the process of generating the initial reply text.

[0163] In a possible implementation, the first multimodal large language model is deployed in a first processing module, and the first processing module and the second processing module are deployed in an electronic device; or,

[0164] The first processing module is deployed in the electronic device, and the second processing module is deployed in a server.

[0165] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0166] An embodiment of the present application provides a reply generation system, including: an electronic device and a server, wherein the electronic device is used to execute the method described in any of the above embodiments, and the server is used to provide a target reply text of input data to the above electronic device.

[0167] For example, the electronic device has a first processing module and a hallucination assessment module. The server is deployed with a second processing module. The first processing module uses a first multimodal large language model to infer input data, the input data includes a first text instruction and a picture, and the first multimodal large language model is used to generate an initial response text for the input data; and when it is determined that the first multimodal large language model does not produce hallucinations in the process of generating the initial response text, the initial response text is output;

[0168] The hallucination assessment module is used to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning on the input data;

[0169] The second processing module is used to process the input data to determine the target reply text of the input data and to provide the target reply text to the electronic device. The electronic device may also include a reply output module, which is used to output the target reply text.

[0170] Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 9 of this embodiment includes: at least one processor 90 ( Fig. 9Only one is shown in the figure), a memory 91, and a computer program 92 stored in the memory 91 and executable on the at least one processor 90, wherein the processor 90 implements the steps of any of the above-mentioned method embodiments when executing the computer program 92.

[0171] The electronic device 9 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud electronic device. The electronic device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will appreciate that Fig. 9 It is only an example of the electronic device 9 and does not constitute a limitation on the electronic device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0172] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0173] In some embodiments, the memory 91 may be an internal storage unit of the electronic device 9, such as a hard disk or memory of the electronic device 9. In other embodiments, the memory 91 may also be an external storage device of the electronic device 9, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 9. Further, the memory 91 may also include both an internal storage unit of the electronic device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0174] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0175] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0176] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A reply generation method, characterized in that: include: Using a first multimodal large language model to infer input data, the input data includes a first text instruction and a picture, and the first multimodal large language model is used to generate an initial response text for the input data; During the process of the first multimodal large language model inferring the input data, performing hallucination detection on the first multimodal large language model; If it is detected that the first multimodal large language model has hallucination features, the input data is processed by a second processing module to determine a target reply text of the input data; The target reply text is output.

2. The method according to claim 1, characterized in that The second processing module is deployed with a second multimodal large language model, and the second processing module is used to infer the input data to determine the target reply text of the input data, including: The input data is input into the second multimodal large language model to obtain the target reply text output by the second multimodal large language model, wherein the parameter scale of the second multimodal large language model is greater than the parameter scale of the first multimodal large language model.

3. The method according to claim 1, characterized in that The second processing module is deployed with an open set visual basis model, and the second processing module processes the input data to determine a target reply text of the input data, including: Determining auxiliary information of the target contained in the picture using the open set visual basis model; Determine a second text instruction according to the auxiliary information and the first text instruction; The second text instruction and the picture are input into the first multimodal large language model to obtain the target reply text output by the first multimodal large language model.

4. The method according to claim 1, characterized in that The second processing module is deployed with a second multimodal large language model, and the second processing module processes the input data to determine a target reply text of the input data, including: Sending the input data to a server; the second processing module is deployed in the server; Receive a target reply text of the input data fed back from the server.

5. The method according to any one of claims 1 to 4, characterized in that: The first multimodal large language model includes a self-attention layer, and the performing hallucination detection on the first multimodal large language model during the process of the first multimodal large language model reasoning the input data includes: In the process of the first multimodal large language model inferring the input data, extracting the self-attention matrix of the last layer in the self-attention layer; Based on the self-attention matrix, determining a hallucination evaluation feature value of the initial reply text, wherein the hallucination evaluation feature value is used to evaluate the influence of the summary word-element on the initial reply text; It is determined whether the first multimodal large language model has a hallucination feature according to the hallucination evaluation feature value.

6. The method according to claim 5, characterized in that The step of determining the hallucination evaluation feature value of the initial reply text based on the self-attention matrix includes: Based on a sliding window, determining a plurality of sliding matrices from the self-attention matrix; Performing column-by-column multiplication on the lower triangular matrix in each of the sliding matrices to obtain a plurality of cumulative multiplication values, wherein the cumulative multiplication values ​​are used to characterize the attention aggregation features in the sliding matrix; The maximum accumulated value and the second largest accumulated value among the plurality of accumulated values ​​are used as the hallucination evaluation feature values.

7. The method according to any one of claims 1 to 4 or 6, characterized in that: The first multimodal large language model is deployed in a first processing module, and the first processing module and the second processing module are deployed in an electronic device; or, The first processing module is deployed in the electronic device, and the second processing module is deployed in a server.

8. A reply generating device, characterized in that: include: A first processing module, configured to use a first multimodal large language model to infer input data, wherein the input data includes a first text instruction and a picture, and the first multimodal large language model is configured to generate an initial response text for the input data; a hallucination assessment module, configured to perform hallucination detection on the first multimodal large language model during the process of the first multimodal large language model inferring the input data, and to provide the input data to the second processing module when hallucination features are detected in the first multimodal large language model; The second processing module is used to process the input data to determine a target reply text of the input data; The reply output module is used to output the target reply text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images