Retrieval enhancement generation method and device, electronic equipment and storage medium
By performing sentence-level verification and document evaluation on the initial answers generated during the question-and-answer process of the AI assistant for film and television production, illusions and errors are filtered out, ensuring the reliability and accuracy of the final answers. This solves the problem of inaccurate output from large language models in film and television production, and improves creative efficiency and credibility.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING QIYI CENTURY SCI & TECH CO LTD
- Filing Date
- 2026-01-15
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, the question-and-answer results output by large language models in AI question-and-answer scenarios for film and television production are inaccurate, containing noise, missing key information, or exceeding the scope of the knowledge base. They also fabricate plot backgrounds or historical details and cite inaccurate information, which reduces the usability and credibility of AI creation tools.
By retrieving a set of documents from a knowledge base based on the question text, an initial answer is generated and sentence-level verification is performed to filter out obvious illusions or citation errors and determine a set of reliable claims. The document set is then evaluated to extract a set of supporting evidence. Finally, a final answer with traceable citations is generated to ensure that each sentence has an accurate citation.
It improves the reliability and citation quality of Q&A results, ensures that every answer has a traceable document source, enhances the efficiency and quality of film and television creation, reduces the time users spend verifying information themselves, and lowers the risks of content errors and compliance.
Smart Images
Figure CN122019708A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of search enhancement generation technology, and in particular to a search enhancement generation method, apparatus, electronic device and storage medium. Background Technology
[0002] In the question-and-answer scenarios of AI assistants in film and television production, creators often need to frequently consult historical materials, professional literature, and original scripts, and hope that the system will clearly indicate the source of information in the answer, so that they can further verify, supplement, and explore. For example, when a screenwriter searches for "the architectural style characteristics of the Song Dynasty," they hope that the system can tell them which descriptions come from the *Yingzao Fashi* (Building Standards), which come from an academic paper or an existing documentary script, and allow them to trace the source. The current mainstream practice in the industry is based on a large language model application framework of retrieval-enhanced generation (RAG): First, internal documents (scripts, historical materials, academic papers, encyclopedia entries, etc.) are vectorized and indexed; then, several relevant document fragments are retrieved based on the user's question; finally, these document fragments and the question are input into the large language model, which generates a natural language answer. However, when the retrieved documents contain noise, lack key information, or the question itself is outside the scope of the knowledge base, the large language model will still "force an answer," fabricating plot background or historical details, resulting in inaccurate question-and-answer results. Summary of the Invention
[0003] This application provides a search enhancement generation method, apparatus, electronic device, and storage medium to solve the technical problem of how to ensure the reliability of search result output.
[0004] In a first aspect, this application provides a retrieval enhancement generation method, the method comprising: A collection of documents is retrieved from the knowledge base based on the question text; The initial answer corresponding to the question text is generated based on the document set, and the initial answer is validated at the sentence level to obtain a set of reliable statements; Each document in the document set is evaluated based on the question text to determine the set of supporting evidence in the document set; The final answer corresponding to the question text is generated based on the set of reliable statements and the set of supporting evidence.
[0005] Optionally, generating an initial answer corresponding to the question text based on the document set includes: The question text and each document in the document set are input into the large language model; wherein, each document carries the document number used for retrieval; Obtain the initial answer output by the large language model, which carries the document number.
[0006] Optionally, sentence-level validation is performed on the initial answer to obtain a set of reliable statements, including: The initial answer is split into sentence-level statements to obtain multiple sentence-level statements; each sentence-level statement carries at least one document number. Input the question text, the sentence-level declaration, and the target document corresponding to the document number carried by the sentence-level declaration into the large language model to obtain a judgment result on whether the sentence-level declaration can be supported by the target document; The set of reliable statements is determined based on all the sentence-level statements that are supported according to the judgment result.
[0007] Optionally, the method further includes: removing all sentence-level declarations whose judgment result is unsupported.
[0008] Optionally, each document in the document set is evaluated based on the question text to determine a set of supporting evidence in the document set, including: For each document in the document set, a first input question is constructed based on the question text and the document in the document set; The first input question is input into a large language model to evaluate the document's usefulness, and a first evaluation result is obtained. A first set of evidence is determined from the document set based on the first evaluation result; For each document in the first set of evidence, a second input question is constructed based on the question text and the document in the first set of evidence; The second input question is input into a large language model to evaluate the evidentiary usefulness of the document, and a second evaluation result is obtained. The supporting evidence set is determined from the first evidence set based on the second evaluation result.
[0009] Optionally, generating the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence includes: The claims with a similarity greater than a preset threshold in the reliable claim set are merged to obtain the target claim set; The final answer corresponding to the question text is generated based on the target statement set and the supporting evidence set; wherein each factual statement in the final answer corresponds to a document number at the time of retrieval.
[0010] Optionally, after generating the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, the method further includes: Establish a mapping relationship between the document number and the document metadata; Display the final answer and the document number corresponding to the factual statement; Obtain a view request for the document number; Based on the viewing request and the mapping relationship, display the target document metadata corresponding to the viewing request.
[0011] Secondly, this application provides a retrieval enhancement generation apparatus, the apparatus comprising: The preliminary retrieval module is used to retrieve a collection of documents from the knowledge base based on the question text; The sentence-level verification module is used to generate initial answers corresponding to the question text based on the document set, and to perform sentence-level verification on the initial answers to obtain a set of reliable statements; The evidence evaluation module is used to evaluate each document in the document set based on the question text, and determine the set of supporting evidence in the document set; The answer generation module is used to generate the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence.
[0012] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the retrieval enhancement generation method described in any embodiment of the first aspect.
[0013] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the retrieval enhancement generation method as described in any embodiment of the first aspect.
[0014] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application retrieves a document set from a knowledge base based on the question text; generates an initial answer corresponding to the question text based on the document set, and performs sentence-level verification on the initial answer to obtain a set of reliable statements; evaluates each document in the document set based on the question text to determine a set of supporting evidence in the document set; and generates the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence. This method first generates an initial answer corresponding to the question text based on the document set, and performs sentence-level verification on the initial answer to filter out obviously illusory or incorrectly cited parts to obtain a set of reliable statements. Then, it evaluates each document in the document set based on the question text to determine a set of supporting evidence in the document set. That is, it extracts evidence only from useful documents in the document set, ensuring that the extracted evidence is indeed from the original text and highly relevant to the question text. Finally, it generates the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, ensuring that each sentence in the final answer has a traceable citation, thus guaranteeing the reliability and citation quality of the output final answer. Attached Figure Description
[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0018] Figure 1 A system architecture diagram of a retrieval enhancement generation method provided in one embodiment of this application; Figure 2 A flowchart illustrating a search enhancement generation method provided in one embodiment of this application; Figure 3 A schematic diagram of a search enhancement generation device provided in one embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.
[0021] In the question-and-answer scenarios of AI assistants in film and television production, creators often need to frequently consult historical materials, professional literature, and original scripts, and hope that the system will clearly indicate the source of information in the answer, so that they can further verify, supplement, and explore. For example, when a screenwriter searches for "the architectural style characteristics of the Song Dynasty," they hope that the system can tell them which descriptions come from the *Yingzao Fashi* (Building Standards), which come from an academic paper or an existing documentary script, and allow them to trace the source. The current mainstream practice in the industry is based on a large language model application framework of retrieval-enhanced generation (RAG): First, internal documents (scripts, historical materials, academic papers, encyclopedia entries, etc.) are vectorized and indexed; then, several relevant document fragments are retrieved based on the user's question; finally, these document fragments and the question are input into the large language model, which generates a natural language answer. However, when the retrieved documents contain noise, lack key information, or the question itself is outside the scope of the knowledge base, the large language model will still "force an answer," fabricating plot background or historical details, resulting in inaccurate question-and-answer results. Alternatively, there may be instances where cited documents are only "vaguely related" to the sentence content and do not truly support the key facts; some core arguments lack any citations; or a single sentence is given only as a source when multiple documents actually support it—all issues of inaccurate or missing citations and poor source attribution. Furthermore, in script / historical questions, an answer often requires synthesizing multiple paragraphs and sources. Current solutions either only label a single "most similar" citation, ignoring other source information, or simply list all documents, making it difficult for users to determine which document corresponds to which sentence. This reduces the usability and credibility of AI-powered creation tools, forcing screenwriters and creators to spend considerable time verifying information themselves, significantly diminishing the value of AI assistants.
[0022] To address at least the technical problem of ensuring the reliability of retrieval results in existing technologies, this application provides a retrieval enhancement generation method, apparatus, electronic device, and storage medium. It first generates an initial answer corresponding to the question text based on a document set, and performs sentence-level verification on the initial answer to filter out obviously illusory or incorrectly cited parts, obtaining a set of reliable statements. Then, it evaluates each document in the document set based on the question text to determine the set of supporting evidence in the document set. This means that evidence is extracted only from useful documents in the document set, ensuring that the extracted evidence is indeed from the original text and highly relevant to the question text. Finally, it generates a final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, ensuring that every sentence in the final answer has a traceable citation, thus guaranteeing the reliability and citation quality of the output final answer.
[0023] The first embodiment of this application provides a retrieval enhancement generation method, which can be applied to, for example... Figure 1 The system architecture shown includes at least a terminal 101 and a server 102, which establish a communication connection.
[0024] This method can be applied to either terminal 101 or server 102 in this system architecture. Terminal 101 can be a desktop computer, tablet computer, laptop computer, supercomputer, or other similar device, while server 102 can be a local server, a cloud server, or a server cluster. For example, an AI assistant can be set up in terminal 101. The AI assistant can use a large language model to respond to user questions. Of course, the AI assistant's processing of user questions can also be performed in server 102 without restriction.
[0025] Next, based on this system architecture, the retrieval enhancement generation method will be described in detail, such as... Figure 2 The retrieval enhancement generation method includes: Step 201: Retrieve a set of documents from the knowledge base based on the question text.
[0026] The question text (Q) can be text input by the user or text generated after speech recognition of the user's input voice data; there are no restrictions. Based on this question text, a set of documents can be retrieved from a knowledge base. The knowledge base can be a database stored on a server or an internet knowledge base. Retrieving the question text from the knowledge base yields a set of documents (D), which can be represented as {d1, d2, ..., dn}, representing the set of retrieved documents (or document fragments). Here, d1, d2, etc., represent each document, and n represents the number of documents in the document set. The i-th document in the document set can be represented by di.
[0027] Step 202: Generate initial answers corresponding to the question text based on the document set, and perform sentence-level verification on the initial answers to obtain a set of reliable statements.
[0028] The initial answers to the questions are generated from the document set, and sentence-level verification is performed on the initial answers to filter out obviously illusory or incorrectly cited parts, thus obtaining a set of reliable statements.
[0029] In one embodiment, generating an initial answer corresponding to the question text based on a document set includes: inputting the question text and each document in the document set into a large language model; wherein the document carries the document number at the time of retrieval; and obtaining the initial answer carrying the document number output by the large language model.
[0030] In this embodiment, each document in the document set can be used as input to the large language model. Specifically, it can be input according to a certain preset format, such as "Document: [i](Title): content" along with the question text. The output of the large language model can be constrained by prompt words. For example, each factual sentence can be constrained to cite at least one document number, such as citing [1] (meaning citing document 1) or citing [1][3] (meaning citing documents 1 and 3). At the same time, the answer can be constrained to contain as much detailed information related to the question as possible. Then, the initial answer (A0) with document number carried by the output of the large language model can be obtained.
[0031] In one embodiment, sentence-level verification is performed on the initial answer to obtain a set of reliable statements, including: splitting the initial answer into sentences to obtain multiple sentence-level statements; wherein each sentence-level statement carries at least one document number; inputting the question text, sentence-level statements, and target documents corresponding to the document numbers carried by the sentence-level statements into a large language model to obtain a judgment result on whether the sentence-level statements can be supported by the target documents; and determining a set of reliable statements based on all sentence-level statements that are supported by the judgment result.
[0032] In this embodiment, the initial answer can be segmented into sentences according to punctuation or semantic boundaries. For example, the initial answer A0 can be segmented into several sentences, and each sentence can be regarded as a sentence-level declaration ci. The set of sentence-level declarations can be called C, C={c1, c2, ..., cm}, where m represents the number of sentence-level declarations and ci represents the i-th one in C. For each sentence-level declaration, the document number of its reference in the initial answer A0 is recorded (e.g., reference [1] or reference [1][3]). Next, the sentence-by-sentence support relationship can be verified for the sentence-level declarations.
[0033] For example, for each sentence-level declaration ci, the corresponding target document can be retrieved based on its document number, such as document 1 (d1) and document 3 (d3) in the document set [1][3]. The verification input is constructed based on the question text Q, the sentence-level declaration ci, and the corresponding document (such as document 1 and document 3). The large language model is called to judge whether the sentence-level declaration can be supported by the target document. For example, the large language model is asked to judge "Based on the given document content, can the declaration be fully supported?", and the judgment result of the large language model is obtained, such as whether it is supported or not. Sentence-level declarations that are judged to be supported can be added to the set of reliable declarations S, which refers to the set of reliable declarations that have passed the verification.
[0034] It should be understandable that all sentence-level declarations that are judged as unsupported can be directly removed.
[0035] By following the steps above, the quality of the answers can be controlled, preventing obviously unfounded content from entering the subsequent processes.
[0036] Step 203: Evaluate each document in the document set based on the question text to determine the set of supporting evidence in the document set.
[0037] Here, the system no longer processes all documents "mixed together," but evaluates the usefulness of each document to the problem one by one, extracting evidence only from useful documents to ensure that the extracted evidence does indeed come from the document set and is highly relevant to the problem.
[0038] In one embodiment, evaluating each document in a document set based on the question text to determine a set of supporting evidence in the document set includes: for each document in the document set, constructing a first input question based on the question text and the documents in the document set; inputting the first input question into a large language model to perform a document usefulness evaluation on the document, obtaining a first evaluation result; determining a first set of evidence from the document set based on the first evaluation result; for each document in the first set of evidence, constructing a second input question based on the question text and the documents in the first set of evidence; inputting the second input question into a large language model to perform an evidence usefulness evaluation on the document, obtaining a second evaluation result; and determining a set of supporting evidence from the first set of evidence based on the second evaluation result.
[0039] In this embodiment, for each document in the document set, a first input question is constructed based on the question text and the document in the document set. For example, the first input question is the question text Q + the content of document di. The first input question is input into a large language model to evaluate the document's usefulness. Prompts can be used to ask whether the document contains information that substantially helps in answering question Q. The response should only be "Yes / No" or "Yes / No," resulting in a first evaluation result. Based on the first evaluation result, a first evidence set is determined from the document set. Documents judged as "Yes" are included in the useful document set D′, while other documents are temporarily excluded from evidence extraction. For each document in the first evidence set D′, based on the question text and the content of document di, a first input question is constructed. The documents in the first evidence set are used to construct a second input question. For example, the second input question is the question text Q plus the content of document di. The second input question is input into a large language model to evaluate the evidence usefulness of the documents. The model is prompted to ignore content irrelevant to the question, extract all key information fragments directly related to Q, and retain details, data, sources, etc. for later citation, to obtain the second evaluation result. Based on the second evaluation result, a supporting evidence set is determined from the first evidence set. The supporting evidence set E={e1, e2, ..., ek} represents the set of supporting evidence fragments extracted from each document, where k represents the number of supporting evidences, and ei can be used to represent the i-th supporting evidence in the set. Each supporting evidence is associated with a corresponding document number.
[0040] Step 204: Generate the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence.
[0041] This method first generates an initial answer corresponding to the question text based on a document set, and performs sentence-level verification on the initial answer to filter out obviously illusory or incorrectly cited parts, thus obtaining a set of reliable statements. Then, it evaluates each document in the document set based on the question text to determine the set of supporting evidence in the document set. That is, it extracts evidence only from useful documents in the document set, ensuring that the extracted evidence is indeed from the original text and is highly relevant to the question text. Finally, it generates the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, ensuring that every sentence in the final answer has a traceable citation, thus guaranteeing the reliability and citation quality of the output final answer.
[0042] In one embodiment, generating the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence includes: merging statements in the set of reliable statements with a similarity greater than a preset threshold to obtain a target statement set; generating the final answer corresponding to the question text based on the target statement set and the set of supporting evidence; wherein each factual statement in the final answer corresponds to a document number at the time of retrieval.
[0043] In this embodiment, a clear and fluent final answer A can be generated based on the set of reliable statements and the set of supporting evidence. For each factual statement, an accurate and concise document number (which may include multiple sources) is attached to ensure traceability.
[0044] In this embodiment, the large language model can be guided to comprehensively summarize and reorganize the reliable statement set S and the supporting evidence set E based on the question text. First, statements in the reliable statement set S with a similarity greater than a preset threshold can be merged, that is, statements with semantic repetition or high similarity can be merged to obtain the target statement set. During the merging, if similar statements correspond to different citation sources (document numbers), the document numbers can be merged into a list such as [1][3]. Then, the final answer corresponding to the question text can be generated based on the target statement set and the supporting evidence set. When generating the final answer, the word order and paragraph structure can be adjusted to make the overall answer natural and fluent in language and clear in logic. At the same time, it should be ensured that each factual statement in the final answer corresponds to the document number at the time of retrieval. When the same factual statement is supported by multiple documents, a unified multi-citation format can be adopted. For example, if it is supported by document 1, document 3 and document 5, then the format such as [1][3][5] can be adopted.
[0045] In one embodiment, after generating the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, the method further includes: establishing a mapping relationship between document numbers and document metadata; displaying the final answer and the document numbers corresponding to the factual statements; obtaining a viewing request for the document number; and displaying the target document metadata corresponding to the viewing request based on the viewing request and the mapping relationship.
[0046] In this embodiment, the mapping relationship between document number and document meta-information (such as title, source, time, etc.) can be maintained inside the AI assistant. When displaying the final answer, the corresponding document number can be displayed at the end of each factual statement in the final answer, such as displaying [1][3]. When the user clicks on the document number, a viewing request is triggered. The target document meta-information corresponding to the viewing request can be displayed according to the viewing request and the mapping relationship, which facilitates the source tracing of the click.
[0047] In the embodiments described above, the traditional RAG "retrieval-generation" process is extended to a multi-stage framework of "retrieval-initial generation and sentence-level verification-support evidence selection and extraction-final refinement and citation integration." This reduces the influx of illusory content into the final answer, significantly improving the reliability and stability of the overall response, making it particularly suitable for scenarios with high factual requirements, such as film and television production. Furthermore, through sentence-level declaration verification and citation source binding mechanisms, after the initial answer is generated, the answer is broken down into sentence-level declarations. Utilizing the reasoning capabilities of the large language model, each statement is checked sentence by sentence to see if it can be supported by its tagged document source. Based on this, statements are filtered or marked, refining the traditional coarse-grained matching at the "overall answer level" to the granularity of "each sentence." This ensures that every factual statement is verified, fundamentally improving the consistency between citations and content. In the embodiments described above, each search document is individually judged as "useful," and evidence fragments highly relevant to the question are extracted only from the "useful documents." This further verifies the evidence, effectively avoids cross-interference and citation confusion caused by mixing multiple documents, makes the final citation structure clearer, and makes it easier for users to understand "which documents support this statement in the final answer."
[0048] The solutions described in the above embodiments of this application can improve the efficiency and quality of scriptwriting. For example, when screenwriters search for background information on history, geography, clothing, architecture, etc., they can directly obtain answers with precise citations, reducing the time spent searching and verifying multiple sources. In terms of world-building and explanation of professional terminology, the AI assistant can act as a reliable "data editor," providing a solid factual foundation for creation. Furthermore, it can enhance the credibility and traceability of knowledge-based question-and-answer scenarios, allowing users to not only see the answers but also clearly understand the supporting documentation behind them, facilitating in-depth reading and secondary creation. In addition, it can reduce the risks of content errors and compliance. For historical or contemporary content, if the basic information is incorrect, the subsequent filming and distribution costs will be enormous. This application significantly reduces the risk of "widespread use of erroneous knowledge" through multi-stage verification, and also helps to quickly trace the source in content review and legal compliance stages, proving which public information or internal documents a certain setting is based on.
[0049] Based on the same technical concept, the second embodiment of this application provides a retrieval enhancement generation device, such as... Figure 3 The device includes: The preliminary retrieval module 301 is used to retrieve a set of documents from the knowledge base based on the question text; The sentence-level verification module 302 is used to generate an initial answer corresponding to the question text based on the document set, and to perform sentence-level verification on the initial answer to obtain a reliable statement set; The evidence evaluation module 303 is used to evaluate each document in the document set based on the question text, and determine the set of supporting evidence in the document set; The answer generation module 304 is used to generate the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence.
[0050] The device first generates an initial answer to the question text based on a set of documents, and then performs sentence-level verification on the initial answer to filter out obviously illusory or incorrectly cited parts, resulting in a set of reliable statements. Next, it evaluates each document in the document set based on the question text to determine the set of supporting evidence in the document set. In other words, it extracts evidence only from useful documents in the document set, ensuring that the extracted evidence is indeed from the original text and is highly relevant to the question text. Finally, it generates the final answer to the question text based on the set of reliable statements and the set of supporting evidence, ensuring that every sentence in the final answer has a traceable citation, thus guaranteeing the reliability and citation quality of the output final answer.
[0051] like Figure 4 As shown in the figure, this application provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the retrieval enhancement generation method provided in any of the foregoing method embodiments.
[0052] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0053] The communication interface is used for communication between the aforementioned terminal and other devices.
[0054] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0055] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0056] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the retrieval enhancement generation method as provided in any of the foregoing method embodiments.
[0057] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0058] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0059] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.
[0060] It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. In the description, suffixes such as "module," "part," or "unit" used to denote elements are used solely for illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.
[0061] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A retrieval enhancement generation method, characterized in that, The method includes: A collection of documents is retrieved from the knowledge base based on the question text; The initial answer corresponding to the question text is generated based on the document set, and the initial answer is validated at the sentence level to obtain a set of reliable statements; Each document in the document set is evaluated based on the question text to determine the set of supporting evidence in the document set; The final answer corresponding to the question text is generated based on the set of reliable statements and the set of supporting evidence.
2. The method according to claim 1, characterized in that, Generate an initial answer to the question text based on the document set, including: The question text and each document in the document set are input into the large language model; wherein, each document carries the document number used for retrieval; Obtain the initial answer output by the large language model, which carries the document number.
3. The method according to claim 1, characterized in that, Sentence-level validation is performed on the initial answer to obtain a set of reliable statements, including: The initial answer is split into sentence-level statements to obtain multiple sentence-level statements; each sentence-level statement carries at least one document number. Input the question text, the sentence-level declaration, and the target document corresponding to the document number carried by the sentence-level declaration into the large language model to obtain a judgment result on whether the sentence-level declaration can be supported by the target document; The set of reliable statements is determined based on all the sentence-level statements that are supported according to the judgment result.
4. The method according to claim 3, characterized in that, The method further includes: removing all sentence-level declarations whose judgment result is unsupported.
5. The method according to claim 1, characterized in that, Each document in the document set is evaluated based on the question text to determine the set of supporting evidence in the document set, including: For each document in the document set, a first input question is constructed based on the question text and the document in the document set; The first input question is input into a large language model to evaluate the document's usefulness, and a first evaluation result is obtained. A first set of evidence is determined from the document set based on the first evaluation result; For each document in the first set of evidence, a second input question is constructed based on the question text and the document in the first set of evidence; The second input question is input into a large language model to evaluate the evidentiary usefulness of the document, and a second evaluation result is obtained. The supporting evidence set is determined from the first evidence set based on the second evaluation result.
6. The method according to claim 1, characterized in that, The final answer corresponding to the question text is generated based on the set of reliable statements and the set of supporting evidence, including: The claims with a similarity greater than a preset threshold in the reliable claim set are merged to obtain the target claim set; The final answer corresponding to the question text is generated based on the target statement set and the supporting evidence set; wherein each factual statement in the final answer corresponds to a document number at the time of retrieval.
7. The method according to claim 6, characterized in that, After generating the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence, the method further includes: Establish a mapping relationship between the document number and the document metadata; Display the final answer and the document number corresponding to the factual statement; Obtain a view request for the document number; Based on the viewing request and the mapping relationship, display the target document metadata corresponding to the viewing request.
8. A retrieval enhancement generation device, characterized in that, The device includes: The preliminary retrieval module is used to retrieve a collection of documents from the knowledge base based on the question text; The sentence-level verification module is used to generate initial answers corresponding to the question text based on the document set, and to perform sentence-level verification on the initial answers to obtain a set of reliable statements; The evidence evaluation module is used to evaluate each document in the document set based on the question text, and determine the set of supporting evidence in the document set; The answer generation module is used to generate the final answer corresponding to the question text based on the set of reliable statements and the set of supporting evidence.
9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the retrieval enhancement generation method according to any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the retrieval enhancement generation method as described in any one of claims 1-7.