Answer generation method and device

Abstract

The invention provides an answer generation method and device. The method comprises the steps of obtaining a query text and N documents; determining input data, the input data including N + 2 prompts, the N + 2 prompts including a first prompt generated based on the query text, a second prompt generated based on the query text and the N documents, and N third prompts respectively generated based on the query text and each of the N documents; the input data are input into the large language model, N + 2 logarithm probabilities are generated, and the N + 2 logarithm probabilities comprise the first logarithm probability generated by the first prompt of the large language model, the second logarithm probability generated based on the second prompt and the N third logarithm probabilities generated based on each third prompt in the N third prompts; obtaining target probability distribution based on N + 2 logarithmic probabilities; and decoding the target probability distribution to obtain an answer corresponding to the query text. According to the method, the ability of integrating various knowledge in the answer generation process of the large language model is enhanced, and the answer generation quality of the large language model is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a method and apparatus for generating answers. Background Technology

[0002] The emergence of large language models (LLMs) has significantly propelled the development of various natural language processing tasks. However, despite their extensive knowledge bases and linguistic capabilities, LLMs often struggle with processing new knowledge and are prone to generating outdated content and illusions. A direct solution is to continuously update the knowledge of LLMs through training, but this process typically requires substantial time and computational resources. Retrieval-augmented generation (RAG) offers an alternative solution, effectively mitigating the illusion problem by introducing external knowledge. Following document retrieval, RAG can be viewed as a multi-document question-answering (MDQA) task. Recent research indicates that variations in document quality can lead to interference and degrade generation quality. Furthermore, knowledge conflicts, such as differences between retrieved documents and contradictions between parameterized knowledge and external non-parameterized knowledge, can hinder the performance of LLMs. Therefore, effectively integrating diverse knowledge during the generation process remains a significant challenge for LLMs. Summary of the Invention

[0003] Embodiments of this application provide an answer generation method and apparatus that enhance the ability of a large language model to integrate multiple types of knowledge (including multiple external knowledge from multiple documents and internal model knowledge) during the answer generation process, thereby effectively improving the quality of answers generated by the large language model.

[0004] Firstly, this application provides an answer generation method, which includes obtaining a query text and N documents, wherein the N documents are retrieved based on the query text, and N is a positive integer greater than 1; determining input data based on the query text and the N documents, wherein the input data includes N+2 hints, wherein the N+2 hints include a first hint generated based on the query text, a second hint generated based on the query text and the N documents, and N third hints generated based on each of the query text and the N documents; inputting the input data into a large language model to generate N+2 logits, wherein the N+2 logits include a first logits generated by the large language model based on the first hint, a second logits generated based on the second hint, and N third logits generated based on each of the N third hints; obtaining a target probability distribution based on the N+2 logits; and decoding the target probability distribution to obtain the answer corresponding to the query text.

[0005] The answer generation method provided in this application processes the first logits obtained by the large language model based on its own parameter knowledge reasoning, the second logits obtained based on the knowledge reasoning provided by N external documents, and the N third logits obtained based on the knowledge reasoning provided by a single document among the N external documents to obtain a target probability distribution. Based on the target probability distribution, the method decodes the data to obtain the response output, thereby integrating multiple types of knowledge. This enhances the ability of the large language model to integrate multiple types of knowledge during the answer generation process and effectively improves the quality of the answers generated by the large language model.

[0006] In one possible implementation, a specific way to obtain the target probability distribution based on N+2 log odds is as follows: evaluate each of the N+2 log odds to obtain a score for each log odds; based on the score of each log odds, select the log odds with the highest score and the log odds with the lowest score from the N third log odds; obtain the target log odds based on the second log odds, the first log odds, the highest-scoring log odds, and the lowest-scoring log odds; and obtain the target probability distribution based on the target log odds.

[0007] By evaluating the log-odds ratio with the highest score and the log-odds ratio with the lowest score among N third log-odds ratios, the model not only considers the influence of individual documents during the answer generation process, but also enhances the knowledge of specific documents, thereby improving the answers generated by the model.

[0008] In another possible implementation, a specific way to obtain the target log-odds based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds is as follows: determine the fourth log-odds based on the difference between the second and first log-odds; determine the fifth log-odds based on the difference between the lowest-rated and highest-rated log-odds; and obtain the target log-odds by weighted summation of the second, fourth, and fifth log-odds.

[0009] By comparing and decoding logits, the logits that guide the generation of the final answer can be optimized, thereby improving the response quality of the large language model.

[0010] In another possible implementation, the score of the first log-odds is determined to be an order of magnitude smaller than the score of the second log-odds, and the weighting parameter of the fourth log-odds is set to 0. This allows the large language model to take into account internal knowledge during the answer generation process.

[0011] In another possible implementation, each log odds in N+2 probability distributions is evaluated to obtain a score for each log odds. One specific implementation is as follows: calculate the entropy for the top K tokens with the highest probability in each log odds to obtain the entropy calculation result corresponding to each log odds, where K is a positive integer; and obtain the score for each log odds based on the entropy calculation result corresponding to each log odds.

[0012] This application emphasizes the importance of the top tokens, calculating entropy only for the top K tokens by probability, thus avoiding the influence of meaningless probabilities on the entropy calculation results. Understandably, based on the characteristics of entropy, lower scores tend to indicate a better distribution, meaning a better evaluation of logits.

[0013] In another possible implementation, the weighted weight parameters of the fourth log-odds and the fifth log-odds are adjusted at each decoding time step. In other words, the weighted weight parameters change dynamically at different decoding time steps to ensure the decoding quality at each decoding time step.

[0014] In another possible implementation, the weighting parameter of the fourth log-odds is determined based on the confidence of the first and second log-odds in the current decoding time step; the weighting parameter of the fifth log-odds is determined based on the confidence of the lowest-scoring and highest-scoring log-odds in the current decoding time step; and the confidence of each log-odds is determined based on the difference between the two highest probabilities within each log-odds. This allows for the dynamic amplification of knowledge from selected documents during the integration of multiple knowledge sets in the large language model, further improving the quality of the model-generated answers.

[0015] Secondly, this application also provides an answer generation device, which includes an acquisition module, a prompting module, an inference module, and a decoding module. The acquisition module acquires a query text and N documents, where the N documents are retrieved based on the query text, and N is a positive integer greater than 1. The prompting module determines input data based on the query text and the N documents. The input data includes N+2 prompts, which include a first prompt generated based on the query text, a second prompt generated based on the query text and the N documents, and N third prompts generated based on the query text and each of the N documents. The inference module inputs the input data into a large language model to generate N+2 log-probabilities, which include a first log-probability generated by the large language model based on the first prompt, a second log-probability generated based on the second prompt, and N third log-probabilities generated based on each of the N third prompts. The decoding module obtains a target probability distribution based on the N+2 log-probabilities and decodes the target probability distribution to obtain the answer corresponding to the query text.

[0016] In one possible implementation, the decoding module is specifically used to: evaluate each of the N+2 log-odds to obtain a score for each log-odd; based on the score of each log-odd, select the log-odd with the highest score and the log-odd with the lowest score from the N third log-odds; based on the second log-odd, the first log-odd, the highest-scoring log-odd, and the lowest-scoring log-odd, obtain the target log-odd; and based on the target log-odd, obtain the target probability distribution.

[0017] In another possible implementation, a specific way to obtain the target log-odds based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds is as follows: determine the fourth log-odds based on the difference between the second and first log-odds; determine the fifth log-odds based on the difference between the lowest-rated and highest-rated log-odds; and obtain the target log-odds by weighted summation of the second, fourth, and fifth log-odds.

[0018] In another possible implementation, the score of the first log-odds is determined to be an order of magnitude smaller than the score of the second log-odds, and the weighting parameter of the fourth log-odds is set to 0.

[0019] In another possible implementation, each log odds in N+2 probability distributions is evaluated to obtain a score for each log odds. One specific implementation is as follows: calculate the entropy for the top K tokens with the highest probability in each log odds to obtain the entropy calculation result corresponding to each log odds, where K is a positive integer; and obtain the score for each log odds based on the entropy calculation result corresponding to each log odds.

[0020] In another possible implementation, the weighted weight parameters of the fourth log-odds and the fifth log-odds are adjusted at each decoding time step. In other words, the weighted weight parameters change dynamically at different decoding time steps to ensure the decoding quality at each decoding time step.

[0021] In another possible implementation, the weighting parameter of the fourth log-odds is determined based on the confidence of the first and second log-odds in the current decoding time step; the weighting parameter of the fifth log-odds is determined based on the confidence of the lowest-scoring and highest-scoring log-odds in the current decoding time step; and the confidence of each log-odds is determined based on the difference between the two highest probabilities within each log-odds. This allows for the dynamic amplification of knowledge from selected documents during the integration of multiple knowledge sets in the large language model, further improving the quality of the model-generated answers.

[0022] Thirdly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the method described in the first aspect or any possible implementation thereof to be implemented.

[0024] Fifthly, embodiments of this application also provide a computer program or computer program product, which includes instructions that, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0025] In a sixth aspect, embodiments of this application also provide a chip including at least one processor and a communication interface, the processor being configured to perform the method described in the first aspect or any possible implementation thereof.

[0026] It is understandable that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here for the sake of brevity. Attached Figure Description

[0027] Figure 1 A schematic diagram of a system architecture is shown;

[0028] Figure 2 A flowchart illustrating an answer generation method provided in an embodiment of this application;

[0029] Figure 3 This illustration shows a specific implementation architecture diagram of the answer generation method provided in an embodiment of this application;

[0030] Figure 4 A schematic diagram showing the performance comparison of the decoding scheme provided in this application embodiment with the Regular-full and CAD decoding schemes on the ASQA dataset;

[0031] Figure 5 This is a schematic diagram of the structure of an answer generation device provided in an embodiment of this application;

[0032] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0033] The term "and / or" used in this article describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0034] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same properties in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such processes, methods, systems, products, or apparatus.

[0035] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0036] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0037] The primary approach to injecting new knowledge into large language models is through supervised fine-tuning or continuous training, but this requires significant resources. Previous RAG research has proposed various improvements, such as enhancing retrieval quality, optimizing response generation through multiple iterations, using optimized cue words, and developing novel decoding strategies. However, these methods typically require retraining or multiple iterations.

[0038] To improve the response quality of large language models, the following solutions have been proposed in related technologies:

[0039] One relevant technical solution is contrastive decoding (CD). This solution addresses some issues in open-ended text generation, such as overly short, repetitive, or off-topic text. Contrastive decoding generates higher-quality text by optimizing a contrastive objective while satisfying a plausibility constraint. The specific technical solution includes the following key points:

[0040] The contrast objective is based on the difference between a large language model (called an expert model, such as OPT-13B) and a small language model (called an amateur model, such as OPT-125M). The contrast objective is achieved by calculating the difference in the log-likelihood of the two models for the same text segment; this difference highlights text that the expert model considers more reasonable, while the amateur model considers unreasonable. Specifically:

[0041] Expert Models (LMs): These are large-scale language models, such as OPT-13B. They possess high modeling capabilities, generating fluent, rich, and coherent text. Expert models represent the high standard we aspire to achieve in text generation.

[0042] Amateur Model (LM): This is a small language model, such as OPT-125M. Compared to expert models, it has weaker modeling capabilities and is more likely to generate repetitive, incoherent, or off-topic text.

[0043] Calculation of the contrast target: The contrast target is calculated by comparing the log-likelihood of the expert model and the amateur model for the same text sequence. The formula is as follows:

[0044] LCD(x cont x pre ) = logp EXP (x cont |x pre )-logp AMA (xcont |x pre )

[0045] This difference indicates which text sequences the expert model favors over the amateur model. The contrastive objective rewards text patterns that the expert model deems more likely and penalizes those that the amateur model considers more likely. During decoding, the contrastive objective encourages the selection of text sequences that maximize the log-likelihood of the expert model and minimize the log-likelihood of the amateur model. This way, the generated text will tend to avoid common failure patterns of amateur models (such as repetition and incoherence) while retaining the high-quality text features of the expert model.

[0046] Reasonableness constraints are a key component of the contrastive decoding scheme. Their role is to ensure that the generated text not only performs well under the contrast target but is also reasonable and credible in real-world language use scenarios. The following is a detailed explanation of reasonableness constraints:

[0047] The rationality constraint is a limiting condition that ensures the generated text sequence has a sufficiently high log-likelihood under the expert model during the contrastive decoding process. This means that the generated text must not only score high under the contrastive target, but also be considered reasonable and plausible by the expert model.

[0048] The purpose of rationality constraints is to avoid generating texts that, while scoring high on the comparison target, appear unreasonable or unbelievable in actual language use. This constraint helps improve the quality and credibility of the generated text.

[0049] In contrastive decoding, the rationality constraint is implemented through an adaptive threshold (Vhead). This threshold is based on the probability distribution of the expert model for each possible next word, allowing only words with a sufficiently high probability under the expert model to be selected. Specifically, this threshold is determined by a hyperparameter α, which truncates the next word distribution of the expert model, retaining only the most likely words as candidates.

[0050] Contrastive decoding uses beam search to optimize CD-score. First, words with low probability under the expert model are filtered out based on reasonableness constraints. Then, the remaining words are scored based on the comparison target. Finally, words with high probability under the expert model and large difference from the amateur model are selected.

[0051] This approach requires two large models—an expert model and an amateur model—for computation, consuming significant inference resources and incurring substantial costs. Furthermore, the contrastive decoding method was not studied within the framework of retrieval enhancement, potentially making it unsuitable for scenarios involving multiple documents as input within this paradigm.

[0052] The second related solution is context-aware decoding (CAD), which aims to improve the language model's attention to the input context and reduce illusions and infidelity in the generated text. Language models primarily rely on two sources of knowledge when generating text: prior knowledge learned during pre-training and contextual knowledge input as prefixed context. However, balancing these two sources of knowledge during generation remains an unresolved issue. Research shows that language models may fail to adequately consider new information introduced by contextual knowledge, potentially leading to illusions in tasks such as summarization, where the generated summary contains facts not present in the input document. This lack of attention to context is particularly severe when contextual knowledge contradicts prior knowledge. The specific technical solution is as follows:

[0053] Given a language model θ, an input query x, and a context c containing external knowledge that is unfamiliar with or conflicts with the model's prior knowledge, model θ generates a response y based on the query x and the context c.

[0054] y t ~p θ (y t |c, x, y <t )∝explogit θ (yt|c,x,y ＜t )

[0055] However, when the context contains knowledge that is inconsistent with the model's original output distribution, the model may struggle to focus effectively and instead become overly reliant on the prior knowledge encoded by the model.

[0056] To alleviate the above problems, prior knowledge is comparatively decomposed from the original output distribution of the model. The prior knowledge is modeled as p θ (y t |x, y ＜t The model's original output probability distribution is adjusted using point mutual information (PMI) between context and generation (condition x, y < t).

[0057] To construct a new output distribution, firstly, the output probability is the weighted product of the original output probability and the PMI (weighted by α), i.e.

[0058]

[0059] Essentially, when context is involved, the output that is more likely to occur is prioritized.

[0060] Then, the above expression is rearranged to obtain the final form.

[0061]

[0062] The larger α is, the greater the weight of the adjustment (when α = 0, it degenerates into regular decoding).

[0063] Essentially, context-aware decoding is p θ (y t |c, x, y ＜t ) and p θ (y t |x, y ＜t The comparison ensemble between logarithms of the models. Different models can also be used for distribution adjustment to suppress unwanted model behavior or extract the ability of expert models.

[0064] This approach primarily addresses situations where there are conflicts between the context in the input of a large model and the model's internal knowledge. However, in settings designed for knowledge-intensive question answering and retrieval enhancement, this specific type of conflict only accounts for a portion of the issues. There are also cases where the retrieved documents do not conflict with the model's internal knowledge, or where the model lacks the relevant knowledge.

[0065] The third related technical solution is dynamic contrastive decoding, a scheme that balances knowledge sources during the generation process. This scheme combines contrastive decoding with adversarial irrelevant paragraphs as negative samples to enhance robustness during generation. The dynamic contrastive decoding strategy is mainly used for retrieving enhanced large model generation to balance the use of parametric and non-parametric knowledge sources.

[0066] Large language models (MLMs) suffer from insufficient integration of input context during text generation, relying primarily on prior knowledge from model parameters. This can lead to factual inconsistencies or inaccurate contextual information in the generated text. MLMs derive their knowledge from two main sources: prior (parametric) knowledge from pre-training and contextual (non-parametric) knowledge derived from input prompts. Previous research has proposed solutions to address the knowledge integration problem in MLMs, such as retrieval enhancement mechanisms and context-aware decoding. However, these solutions have limitations. For instance, context-aware decoding primarily focuses on relevant non-parametric knowledge while neglecting irrelevant non-parametric information.

[0067] The principle of the dynamic comparison decoding scheme is to divide the context c into relevant contexts c. + and irrelevant context c - At each decoding time step t, the model's prediction (z) based on parameter knowledge is... t Prediction using relevant context And predictions without context Combining these, we obtain a new predicted probability distribution, the formula of which is:

[0068]

[0069] Here, α is a hyperparameter used to control the response of parameter (z). t The degree of modification.

[0070] c + You can select the top-ranked text in the search module, c - The search can select from lower-ranked search texts, random texts, or deliberately constructed adversarial texts. The dynamic contrastive decoding strategy excels in managing knowledge conflicts.

[0071] Dynamic contrastive decoding and context-aware decoding share similar goals: improving the robustness of large models when faced with conflicts between retrieved external and internal knowledge. However, neither addresses situations where the large model lacks relevant internal knowledge or where there are no conflicts between internal and external knowledge. Furthermore, dynamic contrastive decoding requires the deliberate construction of adversarial documents based on rules, which may introduce excessive noise, additional latency, and higher resource consumption in practical retrieval enhancement scenarios.

[0072] Contrastive decoding provides a training-free solution to illusion and has inspired much subsequent research, but these studies typically focus on single-document scenarios and resolving conflicts between internal and external knowledge, while neglecting the challenges of integrating multiple documents.

[0073] In view of this, this application provides an answer generation method and apparatus, which adopts the newly proposed decoding strategy to enhance the ability of a large language model to integrate multiple knowledge during the answer generation process, and dynamically amplifies the knowledge from the selected document during the integration process to improve the quality of the answer generated by the model.

[0074] The following describes in detail, with reference to the accompanying drawings, the specific implementation of the recommended explanation generation method and apparatus provided in the embodiments of this application.

[0075] Figure 1 A schematic diagram of a system architecture is shown. The system includes a terminal 110, a server 130, and a network 120 that communicatively connects the terminal 110 and the server 130. The server 130 may include one or more servers (…). Figure 1 (This example uses only one server) Server 130 can provide intelligent question-and-answer services (i.e., generate corresponding answers based on the questions asked by users) for one or more terminals 110, thereby improving the quality of the generated answers.

[0076] In some embodiments, server 130 may also provide other services or software applications, including both non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to end users under a Software as a Service (SaaS) model.

[0077] exist Figure 1 In the system shown, server 130 may include one or more components that implement the functions performed by server 130. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. A user of operating terminal 110 may sequentially interact with server 130 and utilize the services provided by these components using one or more client applications. It should be understood that... Figure 1 This is merely one example of a system for implementing the various methods described in the embodiments of this application; other different configurations are possible, and the embodiments of this application do not limit this.

[0078] Terminal 110 may have an intelligent question-and-answer application (such as an intelligent assistant) installed, or a webpage related to intelligent question-and-answer opened. The aforementioned application and webpage can provide an interactive interface, which includes a text box. Terminal 110 can receive the query text entered by the user in the text box and send the query to service 130. Server 130 can generate the answer corresponding to the query based on the received query using the answer generation method provided in this application embodiment, and return the generated answer to terminal 110 to be displayed to the user.

[0079] Terminal 110 may include various types of computer devices, such as portable handheld devices (e.g., smartphones), general-purpose computers (e.g., personal computers or laptops), workstation computers, wearable devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (e.g., Google Chrome OS), Harmony OS; or various mobile operating systems, such as Microsoft Windows Mobile OS, Windowsphone, Android, iPhone OS, HarmonyOS NEXT, etc. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and smartwatches, etc.

[0080] Network 120 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, 3G, 4G, 5G, etc., to support data communication. For example, one or more networks 120 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, the public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0081] Server 130 may include one or more general-purpose computers, special-purpose server computers (e.g., personal computer servers, UNIX servers, terminal servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement or combination. Server 330 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the server. In various embodiments, server 130 may run one or more services or software applications that provide the functionality described below.

[0082] In some implementations, server 130 can be a server for a distributed system or a server integrated with blockchain. Server 130 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and virtual private servers (VPS) services, such as high management difficulty and weak business scalability.

[0083] It should be noted that in some other examples, the terminal 110 can also generate the corresponding answer for the query input by the user by itself by executing the answer generation method provided in the embodiments of this application, without the need for the server to cooperate. This application does not make specific limitations on this.

[0084] Figure 2 This is a flowchart illustrating an answer generation method provided in an embodiment of this application. This method can be executed by any device, apparatus, platform, or cluster of devices with computing capabilities. This application does not specifically limit the specific computing device executing the method; a suitable computing device can be selected as needed. For example, it can be executed by… Figure 1 The implementation, shown on server 130, provides answer generation services to users. Alternatively, it can be implemented by a terminal device (such as a mobile phone). For ease of description, the form of the implementing entity will not be distinguished below, and all will be described as implementing devices. Figure 2 As shown, the answer generation method provided in this application embodiment includes at least steps S201 to S205.

[0085] In step S201, the query text and N documents are obtained.

[0086] Taking a mobile phone as an example, a large language model is deployed on the mobile phone, which can provide intelligent question and answer services to users in the form of a mobile phone intelligent assistant.

[0087] For example, when a user has a question that needs answering, they can activate their phone's smart assistant and manually enter a natural query as the search text in the assistant's question-and-answer input box. For instance, the user taps the question-and-answer input box on the phone's touchscreen to bring up the virtual keyboard, then enters the query text "Who played the Weasley brothers in Harry Potter?" in the input box using the virtual keyboard, and clicks the "OK" button to submit the query. In response, the phone retrieves the query text and displays it on the phone's screen.

[0088] In another example, users can also input query text via voice. For instance, after activating the phone's smart assistant, a user can directly ask a question through the microphone. The phone responds to the user's action, acquires the user's voice input, recognizes the voice content, and then converts it into query text. Specific methods for converting voice content into text can be found in existing technologies, and will not be elaborated upon in this application.

[0089] It is understood that the methods for obtaining query text by terminal devices described above are merely illustrative and do not constitute a limitation on the scope of protection of the embodiments of this application.

[0090] After obtaining the query text, the execution device can retrieve N documents related to the query text, where N is a positive integer greater than 2.

[0091] This application does not specifically limit the retrieval method. For example, vector retrieval can be used to retrieve N documents related to the query text from the Internet. Alternatively, inverted index retrieval can be used to retrieve N documents related to the query text from the Internet. Furthermore, vector retrieval or inverted index retrieval can be used to retrieve N documents related to the query text from a specific database.

[0092] It should be noted that the N documents related to the query text can be understood as the N documents related to the solution to the query text, or in other words, the N documents containing knowledge related to the solution to the query text are called the N documents related to the query text.

[0093] In step S202, based on the query text and N documents, the input data is determined, which includes N+2 prompts.

[0094] After obtaining the query and the retrieved N documents, the execution device constructs three types of prompts: prompts without a document, prompts with multiple documents, and prompts with a single document. In essence, prompts without a document are built based on the query; prompts with multiple documents are built based on the query and the retrieved N documents; and prompts with a single document are built based on the query and each of the N documents individually. For example, if the N documents include document 1, document 2, document 3, document 4, and document 5, the generated N+2 prompts include: prompt 1 based on the query; prompt 2 based on the query and documents 1, 2, 3, 4, and 5; prompt 3 based on the query and document 1; prompt 4 based on the query and document 2; prompt 5 based on the query and document 3; prompt 6 based on the query and document 4; and prompt 7 based on the query and document 5, for a total of seven prompts.

[0095] Figure 3 This diagram illustrates a specific implementation architecture of the answer generation method provided in an embodiment of this application. Figure 3 As shown, the retrieved query is: "Who played the Weasley brothers in Harry Potter?", and the retrieved documents include document 1, document 2 and document 3, a total of 3 documents, that is, N equals 3.

[0096] The constructed prompts are denoted by x. Five prompts are constructed for each query, including prompt x1, prompt x2, prompt x3, prompt x4, and prompt x5. Prompt x1 is constructed based on the query and includes an instruction and the query. The instruction indicates the task of the large language model and, optionally, also indicates the format of the answer generated by the large language model. For example, prompt x1 could be: "Give the answer to the following question: Who played the Weasley brothers in Harry Potter?". Prompt x2 is constructed based on the query and all retrieved documents, namely document 1, document 2, and document 3. It includes an instruction, the query, document 1, document 2, and document 3. For example, prompt x2 could be: "Give the answer to the following question: Who played the Weasley brothers in Harry Potter?" based on the following documents: document 1, document 2, and document 3. Hint x3 is built based on the query and document 1, including the instruction, query, and document 1. For example, hint x3 could be: "Based on the following document: Document 1, generate the answer to the question: Who played the Weasley brothers in Harry Potter?". Hint x4 is built based on the query and document 2, including the instruction, query, and document 2. For example, hint x4 could be: "Based on the following document: Document 2, generate the answer to the question: Who played the Weasley brothers in Harry Potter?". Hint x5 is built based on the query and document 3, including the instruction, query, and document 3. For example, hint x5 could be: "Based on the following document: Document 3, generate the answer to the question: Who played the Weasley brothers in Harry Potter?".

[0097] In one example, N+2 hints can be generated using three different types of hint templates. For instance, a no-document hint template could be: "Give an answer to the following question: ***", where the "***" part is filled with the retrieved query. A multi-document hint model could be: "Give an answer to the following question: *** based on the following documents: document1, ..., documentN", where the "document1, ..., documentN" part is filled with the N retrieved documents, and the "***" part is filled with the retrieved query. A single-document hint model could be: "Give an answer to the following question: *** based on the following document: documenti", where the "documenti" part is filled with any one of the N retrieved documents, and the "***" part is filled with the retrieved query.

[0098] In step S203, the input data is input into the large language model to generate N+2 logits.

[0099] After generating N+2 prompts through device construction, these N+2 prompts can be input as a batch into the large language model. The large language model generates one logit for each of the N+2 prompts, for a total of N+2 logits. In other words, the large language model uses each of the N+2 prompts as guidance to perform inference and generate a corresponding logit for each prompt, resulting in a total of N+2 logits.

[0100] For example, the query is represented by q. Assume the retrieved document set is D = {d1, ... d2}. N}, where d i Let q be a single document, and N be the total number of documents. The total number of retrieved documents N is no less than 2, forming a multi-document scenario. Given q and D, the task is to generate an answer for q based on the retrieved documents D. The quality of the documents may vary, and the large language model should combine its internal parameter knowledge with external knowledge from D to generate an accurate and comprehensive answer. Let x represent the input of the large language model, and y represent the output. The large language model, denoted as θ, generates each token in the answer v in an autoregressive manner.

[0101] This application's embodiments construct input x in a special way. Multiple inputs are considered simultaneously, and different prompt templates are applied to construct them. There are three types of inputs, corresponding to three templates. First, consider the input without a document, i.e., x1 = T1(q). Second, consider the input where all documents are concatenated together, i.e., x2 = T2(q). Finally, consider the individual input for each document, i.e., x3 = T3(q), x3 = T4(q), ..., x N+2 =T3(q). In summary, for a given query, N+2 inputs are constructed, where N is the number of documents. These inputs are then grouped into a batch and fed into the large language model. Let the batch be B = {x1, x2, ..., x...} N+2 The large language model generates N+2 logits for this batch of input, denoted as z, i.e., Z = {Z1, Z2, ... Z}. N+2 The formula for generating the formula is as follows:

[0102] Z = θ(B)

[0103] It should be explained that, in the embodiments of this application, logits refer to the unnormalized probabilities, the raw predicted values ​​before being processed by an activation function (such as softmax or sigmoid). Logits are typically passed to softmax or sigmoid functions, which transform logits into a probability distribution. In other words, logits are the values ​​before being transformed into a probability distribution. Only logits processed by the softmax function represent the true probability distribution.

[0104] Of course, in some other examples, the N+2 prompts can also be input into the large language model separately, so that the large language model generates a logits for each prompt.

[0105] In step S204, the target probability distribution is obtained based on N+2 logits.

[0106] In order to integrate various types of knowledge (including knowledge within the model and external knowledge from multiple documents), this application provides a new decoding strategy. By adjusting N+2 logits through the new decoding strategy, diverse knowledge is integrated, thereby optimizing the logits that guide the generation of the final answer.

[0107] For example, in order to combine internal and external knowledge and amplify or ignore knowledge in certain documents, a standard is needed to evaluate the quality of logits and make selections. Previous studies have typically calculated the entropy of each logit. However, large language models tend to assign probabilities to many tokens in the vocabulary after pre-training, which causes the overall entropy to be affected by many meaningless probabilities. Therefore, this application proposes to emphasize the importance of the head tokens and calculate the entropy only for the top K tokens in terms of probability. Then, a scoring function f is used to calculate the logits z in each batch B. i The following ratings i And obtain the score S = {s1, s2, ..., s} N+2 The corresponding formula is as follows:

[0108] s i =f(z) i )

[0109]

[0110] Where V topK It is the set of the top K tokens with the highest probabilities. Based on the properties of entropy, the lower the score, the more the distribution tends to be better.

[0111] Optionally, K can be set based on experiments. For example, if experiments show that K=10 yields the best results, then the value of K can be set to 10.

[0112] Scores s1 and s2 correspond to inputs with no document and inputs containing all retrieved documents, respectively, and are used to determine the importance of internal parameter knowledge. This application's embodiments assume that the large language model should prioritize the provided documents, but cannot completely ignore the influence of internal knowledge. The large language model can only maintain its dependence on internal knowledge if s1 is more than an order of magnitude lower than s2 (i.e., s1 ≤ s2 / 10). Otherwise, the large language model should rely on knowledge in the documents to answer questions and eliminate self-interference.

[0113] Rating from S2 to S N+2 Used to determine the importance of each document. The documents with the lowest and highest scores are selected to adjust logits and amplify the knowledge of specific documents, denoted as z, respectively. l and z h The specific formula is as follows:

[0114]

[0115] β and γ are hyperparameters, also known as weighting parameters. When the value of s1 is more than an order of magnitude lower than that of s2, β is set to 0.

[0116] The final result The target logits are the new decoding strategy provided in this application embodiment, which can be called a new comparative decoding scheme. By adjusting the logits through comparative decoding, the logits that guide the generation of the final answer are optimized, thereby improving the response quality of the large language model.

[0117] After obtaining the target logits, the probability distribution is calculated from the logits. For example, the logits can be used as a softmax function to calculate the probability distribution, i.e., probability = softmax(logits).

[0118] In step S205, the target probability distribution is decoded to obtain the answer corresponding to the query text.

[0119] After obtaining the target probability distribution, the answer is obtained by decoding on the probability distribution. That is, the answer token is obtained by decoding on the probability distribution generated on the adjusted logits, as shown in the following formula:

[0120]

[0121] Equivalent to,

[0122]

[0123] Among them, y tThis refers to the token obtained by decoding at the current decoding time step, p θ This refers to the target probability distribution at the current decoding time step. This refers to the target logits of the current decoding step, where x1 and x2 represent inputs with no documents and inputs containing all documents, respectively, and x... l and x h These correspond to the two inputs with the lowest and highest scores, respectively.

[0124] It is understandable that the target probability distribution refers to the probability distribution of the tokens that may be decoded at the current decoding time step.

[0125] In this embodiment of the application, when decoding on the target probability distribution, any decoding method can be used, such as greedy search, beam search, and sampling.

[0126] Understandably, greedy search selects the token with the highest probability at each step. This is the simplest and fastest method, but it can lead to suboptimal results. Bundle search retains multiple most probable sequences (called bundles) at each step and ultimately selects the sequence with the highest overall probability. This method strikes a balance between quality and speed. Sampling randomly selects tokens based on the probability distribution of the model's output. This method can increase the diversity of the output.

[0127] The decoding strategy provided in this application not only considers the impact of a single document, but also enhances the knowledge of a specific document, and designs a special index (i.e., the score of this application) to select the target document (i.e. the document corresponding to the lowest score z1) during the generation process.

[0128] In the scheme provided in this application embodiment, the hyperparameters β and γ remain unchanged in each decoding time step. The hyperparameters can be determined based on experimental results, human experience, or confidence level. The specific scheme for determining the confidence level is described below. In this case, the scheme can be called DvD-fixed decoding strategy.

[0129] In preliminary experiments, this application embodiment found that the settings of hyperparameters β and γ are crucial to downstream performance. Exploring the optimal weights for each dataset and large language model requires numerous experiments, which is inconvenient. Therefore, it is desirable to dynamically set these weights at each decoding time step. Previous work used the probability of the highest predicted token in the normalized dataset to measure the confidence of the large language model, but this approach was not effective in experiments. In this application embodiment, the probability difference between the two tokens with the highest probabilities is used as the confidence metric. The confidence calculation formula for each logits is as follows:

[0130]

[0131] The formulas for calculating hyperparameters β and γ at each decoding time step are shown below:

[0132]

[0133] γ=max(C l -C h ,0)

[0134] in, Represents the distribution z i The highest probability in, and Represents the distribution z i The second highest probability value in the range. Therefore, y is dynamic and determined by the confidence difference between the logits with the lowest and highest ratings, while β is jointly determined by the logits with and without documents.

[0135] Decoding schemes that dynamically change hyperparameters β and γ in each decoding time step can be called DvD-dynamic decoding strategies.

[0136] This application provides a novel decoding strategy that combines selection criteria and dynamic weights to integrate knowledge from all documents and enhance the knowledge extracted from selected documents.

[0137] To evaluate the effectiveness of the answer generation method provided in this application, zero-shot experiments were conducted on multiple datasets, including the commonly used RAG datasets ALCE-ASQA, Natural Questions, TriviaQA, and PopQA. The LLaMA2, Vicuna, Qwen2-7B, and Mistral-7B-v0.1 models were used in the experiments. This application proposes a novel decoding strategy; therefore, the proposed new decoding strategy is mainly compared with other decoding methods, such as conventional decoding, CAD, and Zhao's work. Conventional decoding has several variations, corresponding to decoding based on no document input, all documents, and a single document, denoted as "Regular-closed," "Regular-full," and "Regular-single" in this paper. A single document is selected from the retriever and ranked first. Zhao's work also has two variations, corresponding to fixed-weight decoding and dynamic-weight decoding, considering only dynamic weights, denoted as "Zdynamic."

[0138] The experimental results are shown in the table below:

[0139]

[0140]

[0141] The results show that the answer generation method provided in this application consistently performs well in terms of generation quality. This improvement is attributed to the method of dynamically amplifying the knowledge in selected documents when integrating different knowledge. Through in-depth analysis of selection criteria, weight calculation, and document quantity, the method proposed in this application demonstrates consistent performance improvement across all datasets.

[0142] The answer generation method provided in this application is applied to multi-document scenarios, and the above description explains how to construct input for each document. This application also verifies the effectiveness of the answer generation method in a wider range of situations using different N values. For simplicity, a dynamic weighting method is used to represent the solution of this application. The solution of this application is mainly compared with Regular-full and CAD, as they are applicable to different document scenarios and serve as strong benchmarks. The results are as follows... Figure 4 As shown.

[0143] The results show that the answer generation method provided in this application surpasses conventional decoding and CAD regardless of the number of documents. As the number of documents $N$ increases, the interference of large language models with irrelevant information also increases, while the solution provided in this application can always provide effective assistance by amplifying the knowledge of specific documents.

[0144] Based on the same concept as the aforementioned embodiment of the answer generation method, this application also provides an answer generation device 500, which can be deployed on a server or terminal device to provide high-quality question-and-answer services. The answer generation device 500 includes components for implementing... Figure 2 The units or modules of each step in the answer generation method shown.

[0145] Figure 5 This is a schematic diagram of an answer generation device provided in an embodiment of this application. Figure 5 As shown, the answer generation device 500 includes an acquisition module 501, a prompting module 502, a reasoning module 503, and a decoding module 504. The acquisition module 501 acquires a query text and N documents, where the N documents are retrieved based on the query text, and N is a positive integer greater than 1. The prompting module 502 determines the input data, which includes N+2 prompts. These N+2 prompts include a first prompt generated based on the query text, a second prompt generated based on the query text and the N documents, and N third prompts generated based on the query text and each of the N documents. The reasoning module 503 inputs the input data into a large language model to generate N+2 log-probabilities. These N+2 log-probabilities include a first log-probability generated by the large language model based on the first prompt, a second log-probability generated based on the second prompt, and N third log-probabilities generated based on each of the N third prompts. The decoding module 504 obtains a target probability distribution based on the N+2 log-probabilities and decodes the target probability distribution to obtain the answer corresponding to the query text.

[0146] In one possible implementation, the decoding module 504 is specifically used to: evaluate each of the N+2 log odds to obtain a score for each log odds; based on the score of each log odds, select the log odds with the highest score and the log odds with the lowest score from the N third log odds; obtain the target log odds based on the second log odds, the first log odds, the highest-scoring log odds, and the lowest-scoring log odds; and obtain the target probability distribution based on the target log odds.

[0147] In another possible implementation, a specific way to obtain the target log-odds based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds is as follows: determine the fourth log-odds based on the difference between the second and first log-odds; determine the fifth log-odds based on the difference between the lowest-rated and highest-rated log-odds; and obtain the target log-odds by weighted summation of the second, fourth, and fifth log-odds.

[0148] In another possible implementation, the score of the first log-odds is determined to be an order of magnitude smaller than the score of the second log-odds, and the weighting parameter of the fourth log-odds is set to 0.

[0149] In another possible implementation, each log odds in N+2 probability distributions is evaluated to obtain a score for each log odds. One specific implementation is as follows: calculate the entropy for the top K tokens with the highest probability in each log odds to obtain the entropy calculation result corresponding to each log odds, where K is a positive integer; and obtain the score for each log odds based on the entropy calculation result corresponding to each log odds.

[0150] In another possible implementation, the weighted weight parameters of the fourth log-odds and the fifth log-odds are adjusted at each decoding time step. In other words, the weighted weight parameters change dynamically at different decoding time steps to ensure the decoding quality at each decoding time step.

[0151] In another possible implementation, the weighting parameter of the fourth log-odds is determined based on the confidence of the first and second log-odds in the current decoding time step; the weighting parameter of the fifth log-odds is determined based on the confidence of the lowest-scoring and highest-scoring log-odds in the current decoding time step; and the confidence of each log-odds is determined based on the difference between the two highest probabilities within each log-odds. This allows for the dynamic amplification of knowledge from selected documents during the integration of multiple knowledge sets in the large language model, further improving the quality of the model-generated answers.

[0152] The answer generation apparatus 500 according to the embodiments of this application can correspond to the execution of the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the answer generation apparatus 500 are respectively for implementing Figure 2 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0153] This application embodiment also provides a computing device, including at least one processor, a memory, and a communication interface, wherein the processor is used to execute... Figure 2-4 The method described.

[0154] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0155] like Figure 6As shown, the computing device 600 includes at least one processor 601, a memory 602, and a communication interface 603. The processor 601, memory 602, and communication interface 603 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 603 is used to send and / or receive data from other devices. The memory 602 stores computer instructions, which the processor 601 executes to perform the methods described in the foregoing method embodiments, thereby enhancing the ability of the large language model to integrate multiple knowledge sources during answer generation and improving the quality of the answers generated by the large language model.

[0156] It should be understood that in the embodiments of this application, the processor 601 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0157] The memory 602 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 602 may also include non-volatile random access memory. Optionally, the random access memory may be, for example, high bandwidth memory (HBM).

[0158] The memory 602 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0159] It should be understood that the computing device 600 according to the embodiments of this application can perform the implementation of the embodiments of this application. Figure 2-4 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0160] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0161] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0162] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0163] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented using hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0165] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating answers, characterized in that, include: Obtain the query text and N documents, wherein the N documents are retrieved based on the query text, and N is a positive integer greater than 1; Based on the query text and the N documents, input data is determined, and the input data includes N+2 prompts. The N+2 prompts include a first prompt generated based on the query text, a second prompt generated based on the query text and the N documents, and N third prompts generated based on the query text and each of the N documents. The input data is input into a large language model to generate N+2 log probabilities. The N+2 log probabilities include the first log probabilities generated by the large language model based on the first prompt, the second log probabilities generated based on the second prompt, and the N third log probabilities generated based on each of the N third prompts. Based on the N+2 log probabilities, the target probability distribution is obtained; The target probability distribution is decoded to obtain the answer corresponding to the query text.

2. The method according to claim 1, characterized in that, The process of obtaining the target probability distribution based on the N+2 log probabilities includes: Each of the N+2 log odds is evaluated to obtain a score for each log odds; Based on the score of each log odds, select the log odds with the highest score and the log odds with the lowest score from the N third log odds; The target log-odds is obtained based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds. The target probability distribution is obtained based on the target log odds.

3. The method according to claim 2, characterized in that, The process of obtaining the target log-odds based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds includes: The fourth log-odds is determined based on the difference between the second log-odds and the first log-odds; The fifth log odds is determined based on the difference between the lowest log odds and the highest log odds. The target logarithmic probability is obtained by weighted summing of the second, fourth, and fifth logarithmic probabilities.

4. The method according to claim 3, characterized in that, If the score of the first log-odds ratio is determined to be less than one order of magnitude less than the score of the second log-odds ratio, the weighting parameter of the fourth log-odds ratio is set to 0.

5. The method according to any one of claims 2-4, characterized in that, Each log odds in the N+2 probability distributions is evaluated to obtain a score for each log odds, including: For each log-probability, calculate the entropy of the top K words with the highest probability to obtain the entropy calculation result corresponding to each log-probability, where K is a positive integer; Based on the entropy calculation result corresponding to each log odds, a score is obtained for each log odds.

6. The method according to any one of claims 3-5, characterized in that, The weighted weighting parameters of the fourth logarithmic odds and the fifth logarithmic odds are adjusted at each decoding time step.

7. The method according to claim 6, characterized in that, The weighting parameter of the fourth log-odds is determined based on the confidence levels of the first log-odds and the second log-odds in the current decoding time step; The weighting parameter of the fifth log-odds is determined based on the confidence level of the lowest-scoring log-odds and the highest-scoring log-odds in the current decoding time step. The confidence level of each log odds is determined based on the difference between the two highest probabilities in each log odds.

8. An answer generation device, characterized in that, include: The acquisition module is used to acquire query text and N documents, wherein the N documents are retrieved based on the query text, and N is a positive integer greater than 1; The prompting module, based on the query text and the N documents, is used to determine input data, which includes N+2 prompts. The N+2 prompts include a first prompt generated based on the query text, a second prompt generated based on the query text and the N documents, and N third prompts generated based on the query text and each of the N documents. The inference module is used to input the input data into the large language model and generate N+2 log probabilities. The N+2 log probabilities include the first log probabilities generated by the large language model based on the first prompt, the second log probabilities generated based on the second prompt, and the N third log probabilities generated based on each of the N third prompts. The decoding module is used to obtain the target probability distribution based on the N+2 log probabilities; And for decoding the target probability distribution to obtain the answer corresponding to the query text.

9. The apparatus according to claim 8, characterized in that, The decoding module is specifically used for: Each of the N+2 log odds is evaluated to obtain a score for each log odds; Based on the score of each log odds, select the log odds with the highest score and the log odds with the lowest score from the N third log odds; The target log-odds is obtained based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds. Based on the target log odds, the target probability distribution is obtained.

10. The apparatus according to claim 9, characterized in that, The process of obtaining the target log-odds based on the second log-odds, the first log-odds, the highest-rated log-odds, and the lowest-rated log-odds includes: The fourth log-odds is determined based on the difference between the second log-odds and the first log-odds; The fifth log odds is determined based on the difference between the lowest log odds and the highest log odds. The target logarithmic probability is obtained by weighted summing of the second, fourth, and fifth logarithmic probabilities.

11. The apparatus according to claim 10, characterized in that, The weighted weighting parameters of the fourth logarithmic odds and the fifth logarithmic odds are adjusted at each decoding time step.

12. The apparatus according to claim 11, characterized in that, The weighting parameter of the fourth log-odds is determined based on the confidence levels of the first log-odds and the second log-odds in the current decoding time step; The weighting parameter of the fifth log-odds is determined based on the confidence level of the lowest-scoring log-odds and the highest-scoring log-odds in the current decoding time step. The confidence level of each log odds is determined based on the difference between the two highest probabilities in each log odds.

13. A computing device, comprising a memory and a processor, characterized in that, The memory stores instructions that, when executed by a processor, cause the method described in any one of claims 1-7 to be implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it causes the method described in any one of claims 1-7 to be implemented.

Images