Retrieval enhancement generation method and device based on large language model and electronic equipment

Abstract

The invention provides a retrieval enhancement generation method and device based on a large language model and electronic equipment, and the method comprises the steps: inputting question information by a user, and generating first abstract data based on a preset knowledge base; rewriting the question information input by the user according to the first abstract data and the question information input by the user, and performing multiple rounds of retrieval in the knowledge base based on the large language model to obtain second abstract data; and generating answer data corresponding to the question information input by the user by using the second abstract data. According to the embodiment of the invention, through multiple rounds of retrieval, the retrieved knowledge is adaptively abstracted and deeply integrated, and deep, stable and controllable exploration of external knowledge is realized through a large language model, so that the problems of knowledge splitting and retrieval decision deviation existing in a retrieval enhancement generation technology in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a retrieval enhancement generation method and apparatus based on a large language model, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] As a general-purpose model, large language models struggle to accurately and quickly adapt to the domain knowledge of various industries. Therefore, retrieval-enhanced generation methods can integrate the latest industry-specific knowledge data into the context of the large language model, enabling rapid adaptation to industry-specific domains and achieving better question-answering results. Under conditions of massive unsupervised data and large-scale parameters, large language models have developed rapidly and generated powerful generative capabilities. However, issues may exist regarding the correctness, timeliness, and completeness of the data. Furthermore, parameters are difficult to fully learn from the data, leading to errors or even "illusions" in the generation of answers to certain questions, resulting in seemingly reasonable but actually logically flawed or factually incorrect responses.

[0003] To address the issues with this information, existing technologies store relevant information in knowledge databases, retrieve and merge the knowledge data, and then generate better answers. This allows small models, combined with industry data, to achieve results comparable to large models. This ability to retrieve relevant information from knowledge databases to improve the quality of large language models is called Retrieval-Augmented Generation (RAG).

[0004] To address the issues of factual errors and illusions in large language models during complex question-answering tasks, traditional retrieval-enhanced generative techniques, while capable of incorporating external knowledge, also have limitations in their adaptive methods. For example, knowledge fragmentation arises, as fragments are generated immediately after each retrieval, leading to a lack of integration across different retrieval rounds; retrieval decision bias exists, relying on LLM to predict retrieval timing, creating a cognitive gap with actual information needs.

[0005] For example, the REALM method only retrieves information once per task, which may result in incomplete information coverage.

[0006] For example, the KNN-LM method uses a multi-round retrieval system, which performs multiple retrievals during the generation process, and executes the retrieval process once every k tokens. However, this method is difficult to avoid noise issues.

[0007] For example, the FLARE method, a current adaptive retrieval system, automatically determines whether a retrieval is needed during the generation process. However, this can lead to information fragmentation and a lack of interactive integration. Summary of the Invention

[0008] This application proposes a retrieval enhancement generation method and apparatus, electronic device, and non-transitory computer-readable storage medium based on a large language model, in order to solve the problems of knowledge fragmentation and retrieval decision bias in existing retrieval enhancement generation technologies.

[0009] According to one aspect of this application, a retrieval enhancement generation method based on a large language model is proposed, comprising: Using user-input question information, generate first-level summary data based on a pre-set knowledge base; Based on the first summary data, the knowledge base is searched multiple times using the large language model to obtain the second summary data; Using the second summary data, answer data corresponding to the user-input question information is generated.

[0010] According to some embodiments, for the first summary data, multiple rounds of retrieval are performed in the knowledge base based on the large language model to obtain second summary data, including: (1) Based on the first summary data, the user input question information is reconstructed using the large language model to obtain reconstructed question information; (2) Using the reconstructed problem information, generate retrieval result data through the knowledge base; (3) Evaluate whether the search results data show any information growth relative to the first summary data; (4) If the search result data shows an increase in information compared to the first summary data, generate second summary data based on the first summary data and the search result data; (5) Using the second summary data, repeat steps (1)-(4) until the obtained search result data no longer has any information growth relative to the second summary data.

[0011] According to some embodiments, step (3) assesses whether the retrieval result data has information growth relative to the first summary data, including: Evaluate the relevance between the search results data and the user-input question information; The relevance is used to assess whether there is information growth in the search results data relative to the first summary data.

[0012] According to some embodiments, in step (4), before generating second summary data based on the first summary data and the search result data when the search result data shows an increase in information compared to the first summary data, Based on the first summary data, multiple rounds of retrieval are performed in the knowledge base using the large language model to obtain the second summary data, which also includes: Evaluate whether there is redundant data in the search results data relative to the first summary data.

[0013] According to some embodiments, in step (4), before generating second summary data based on the first summary data and the search result data when the search result data shows an increase in information compared to the first summary data, Based on the first summary data, multiple rounds of retrieval are performed in the knowledge base using the large language model to obtain the second summary data, which also includes: If redundant data exists in the search results, the redundant data shall be deleted.

[0014] According to some embodiments, for the first summary data, multiple rounds of retrieval are performed in the knowledge base based on the large language model to obtain second summary data, which further includes: When the number of times steps (1)-(4) are repeated is not less than the preset number of iterations, stop executing steps (1)-(4).

[0015] According to some embodiments, using the second summary data, answer data corresponding to the user-input question information is generated, including: Based on the question information type corresponding to the user input question information, answer data corresponding to the user input question information is generated using the second summary data.

[0016] According to one aspect of this application, a retrieval enhancement generation device based on a large language model is proposed, comprising: The first summary data generation unit is used to generate first summary data based on a preset knowledge base by using user-input question information. The second summary data generation unit is used to perform multiple rounds of retrieval on the knowledge base based on the large language model to obtain the second summary data based on the first summary data. The answer data generation unit is used to generate answer data corresponding to the user-input question information using the second summary data.

[0017] According to one aspect of this application, an electronic device is provided, comprising: a processor; a memory for storing a computer program; wherein, when the computer program is executed by the processor, the processor causes the processor to perform the method as described in any of the preceding embodiments.

[0018] According to one aspect of this application, a non-transitory computer-readable storage medium is provided, having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform the method as described in any of the preceding embodiments.

[0019] According to the example embodiments of this application, through multiple rounds of retrieval, the retrieved knowledge is adaptively summarized and deeply integrated. Through a large language model, a deep, robust, and controllable exploration of external knowledge is achieved, thereby solving the problems of knowledge fragmentation and retrieval decision bias in existing retrieval enhancement generation technologies. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0021] Figure 1 A block diagram of a retrieval enhancement generation apparatus based on a large language model according to an example embodiment of this application is shown.

[0022] Figure 2 A flowchart of a retrieval enhancement generation method based on a large language model according to an example embodiment of this application is shown.

[0023] Figure 3 A flowchart illustrating a method for performing multi-round retrieval in the knowledge base based on the large language model according to an example embodiment of this application is shown.

[0024] Figure 4 The diagram illustrates an application example of a retrieval enhancement generation method based on a large language model, according to an example embodiment of this application.

[0025] Figure 5 A block diagram of a retrieval enhancement generation apparatus based on a large language model according to an example embodiment of this application is shown.

[0026] Figure 6 An electronic device is shown according to an exemplary embodiment of this application. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0028] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, apparatus, or operations may be employed. In these cases, well-known structures, methods, apparatuses, implementations, materials, or operations will not be shown or described in detail.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0030] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0031] Figure 1 A block diagram of a retrieval enhancement generation apparatus based on a large language model according to an example embodiment of this application is shown, such as... Figure 1 The retrieval generation device shown includes a summary initialization module 101, a summary adaptive retrieval module 103, and an answer generation module 105.

[0032] According to an embodiment of this application, the summary initialization module 101 uses a large language model to generate a structured initial summary strategy based on the user-input question information, serving as the starting point for subsequent exploration. This initial summary strategy does not simply accumulate knowledge, but rather integrates information from the original document, preserving key entities, relationships, and context to ensure knowledge integrity and scalability.

[0033] In some embodiments, the summary adaptive retrieval module 103 includes a query reconstruction subunit, a knowledge accumulation subunit, and an adaptive decision-making subunit. The query reconstruction subunit is configured to generate a new round of retrieval queries based on the current optimal summary content, obtaining new retrieval results while avoiding duplication and ambiguity with previous summary content. The knowledge accumulation subunit is configured to update the summary using the new retrieval results, obtaining optimal summary content, wherein the added information must meet the requirements of fidelity, relevance, and non-redundancy. The adaptive decision-making subunit dynamically decides whether to continue to the next round of retrieval through a multi-dimensional evaluation mechanism to determine whether the added information brings knowledge growth to the summary, and continuously maintains the "optimal summary" state during the process. This submodule effectively alleviates the knowledge fragmentation problem caused by "retrieval equals generation" in traditional methods, realizing cross-round information fusion and progressive knowledge construction.

[0034] In other embodiments, the answer generation module 105 generates an answer based on the final optimal summary after a preset stopping condition is met, such as the maximum number of iterations or the number of consecutive failures.

[0035] According to embodiments of this application, addressing the issues of illusion and limited knowledge boundaries faced by large language model question-answering systems, a multi-round retrieval and deep integration technology solution with adaptive summarization is proposed. Compared to the shortcomings of traditional retrieval enhancement generation methods, which directly answer after a single retrieval and are unable to handle complex multi-hop or long text questions, this application, by sequentially employing knowledge acquisition, knowledge verification, and knowledge organization, adaptively summarizes and deeply integrates the retrieved knowledge after multiple rounds of retrieval, thereby achieving in-depth, robust, and controllable exploration of external knowledge bases.

[0036] According to embodiments of this application, knowledge acquisition in complex question-answering tasks is achieved through adaptive summarization and deep integration technologies. This overcomes the bottlenecks of traditional retrieval enhancement systems in terms of knowledge depth, breadth, and stability, providing a new paradigm for building reliable, controllable, and scalable large-scale retrieval enhancement models. It exhibits significant advantages in question-answering tasks, especially knowledge-intensive tasks.

[0037] The specific embodiments according to this application will now be described in detail with reference to the accompanying drawings.

[0038] Figure 2 A flowchart illustrating a retrieval enhancement generation method based on a large language model according to an example embodiment of this application is shown, such as... Figure 2 The retrieval enhancement generation method shown includes steps S201, S203, and S205. The following uses... Figure 2 Taking an example, a retrieval enhancement generation method based on a large language model according to an example embodiment of this application will be described in detail.

[0039] like Figure 2As shown, in step S201, the first summary data is generated based on a preset knowledge base using the user-input question information.

[0040] According to an embodiment of this application, in step S201, the user-input question information is used to perform a search based on a preset knowledge base, and the search results are used to generate summary data as an initial summary strategy, which is defined as the current best summary as the knowledge starting point for subsequent exploration.

[0041] It should be noted that the preset knowledge base is a collection of knowledge related to the question information. In this application, the knowledge base can collect relevant knowledge in various ways, and this application does not impose any limitations. At the same time, this application does not impose any specific limitations on the method of generating relevant search results based on the knowledge base using the question information input by the user in step S201.

[0042] In step S203, for the first summary data, multiple rounds of retrieval are performed in the knowledge base based on the large language model to obtain the second summary data.

[0043] According to an embodiment of this application, in step S203, firstly, the upper limit of the number of cycles and the number of consecutive no-progress cycles are set, and then, a multi-round search is performed.

[0044] In a specific implementation, during each round of retrieval, firstly, the question is reorganized using the current best summary to avoid duplication and ambiguity; then, the new question is used to perform retrieval through a large language model to obtain new retrieval results; finally, the new retrieval results are merged with the current best summary, retaining only faithfully relevant and non-duplicative information, and after merging, it is evaluated whether this new summary brings knowledge growth; if there is new knowledge growth, the current best summary is replaced, and the no-progress count is reset to zero; otherwise, the content of the current best summary remains unchanged, and the no-progress count is incremented by one; finally, the loop count is incremented by one.

[0045] In step S205, the second summary data is used to generate answer data corresponding to the user-input question information.

[0046] According to an embodiment of this application, based on the question information type corresponding to the user-input question information, answer data corresponding to the user-input question information is generated using the second summary data.

[0047] In specific implementations, task-oriented generation instructions are designed for different task types (e.g., multi-hop question answering, long text question answering, common sense judgment), ensuring that the output style is highly aligned with the evaluation criteria. For example, corresponding prompts or training models are set for different tasks to meet the writing style of the corresponding task.

[0048] In some embodiments, multi-hop question answering refers to a question-answering task that requires synthesizing multiple pieces of information and using logical reasoning (such as causal, adversative, or progressive relationships) to arrive at the answer. The model cannot find the answer directly from a single sentence; it must "jump" through multiple related points to deduce it.

[0049] For example, the question is: Xiaoming's mother is 25 years older than Xiaoming. When Xiaoming was born, his mother's age was exactly five times his current age. How old is Xiaoming now? Answer: Step-by-step reasoning is required: 1. Let Xiaoming's current age be \(x\), then his mother's age is \(x+25\).

[0050] 2. "The year Xiaoming was born" means that his mother's age was (x + 25 - x = 25) years old.

[0051] 3. Based on the condition: \(25 = 5x\), we get \(x = 5\).

[0052] Answer: Xiaoming is 5 years old now.

[0053] In other embodiments, long text question answering refers to answer information scattered throughout a long text (such as thousands to tens of thousands of words), requiring the model to have the ability to integrate and locate global information and extract accurate answers from a large number of details.

[0054] For example, the text content: A 10,000-word summary of a world-class sporting event mentions the highlights of each event, athlete stories, changes in the medal standings, etc.

[0055] Question: Who won the first women's breakdancing gold medal at this Games? Answer: Locate the key details across the entire text and provide the specific name of the sports meet.

[0056] In other embodiments, common sense judgments are inferred based on everyday knowledge, social norms, or laws of the physical world that are universally possessed by humans, without the need for additional text, and rely on an internalized common sense base.

[0057] For example, the question is: What happens to an ice cube when it is put into hot water? Options: A. Explode B. Melt C. Grow larger D. Change color

[0058] Answer: B (based on the common physics principle that "ice melts when heated").

[0059] according to Figure 2The embodiment shown adopts knowledge acquisition, knowledge verification and knowledge organization in sequence. After multiple rounds of retrieval, it adaptively summarizes and deeply integrates the retrieved knowledge. Through a large language model, it realizes in-depth, robust and controllable exploration of external knowledge, and solves the problems of knowledge fragmentation and retrieval decision bias in existing retrieval enhancement generation technologies.

[0060] Figure 3 A flowchart illustrating a method for multi-round retrieval in the knowledge base based on the large language model according to an example embodiment of this application is shown, such as... Figure 3 The method shown is a further explanation of step S203. For example... Figure 3 As shown, step S203 includes sub-steps S2031, S2033, S2035, S2037, and S2039.

[0061] like Figure 3 As shown, in sub-step S2031, the user input question information is reconstructed using the large language model based on the first summary data to obtain reconstructed question information.

[0062] In sub-step S2031, a new question is generated using the large language model for the first summary data in order to reconstruct the user input question information.

[0063] In sub-step S2033, the reconstructed problem information is used to generate retrieval result data through the knowledge base.

[0064] In a specific embodiment, in sub-step S2033, the same retrieval method as in step S201 can be selected, and retrieval result data can be generated through the knowledge base by utilizing the reconstructed question information.

[0065] Sub-step S2035: Evaluate whether there is information growth in the search result data relative to the first summary data.

[0066] According to an embodiment of this application, in sub-step S2035, the search result data may be evaluated for information growth relative to the first summary data by assessing the fidelity, relevance, and / or non-redundancy checks of the search result data relative to the first summary data.

[0067] For example, in sub-step S2035, firstly, the relevance between the search result data and the user input question information is evaluated; then, based on the relevance, it is evaluated whether there is information growth in the search result data relative to the first summary data.

[0068] Sub-step S2037: If the search result data shows an increase in information compared to the first summary data, generate second summary data based on the first summary data and the search result data.

[0069] According to an embodiment of this application, before step S2037, it is also necessary to evaluate whether there is redundant data in the search result data relative to the first summary data, and if there is redundant data in the search result data, the redundant data is deleted.

[0070] Sub-step S2039: Using the second summary data, repeat sub-steps S2031-S2037 until the obtained search result data no longer shows any information growth relative to the second summary data.

[0071] In a specific embodiment, when the number of times sub-steps S2031-S2037 are repeatedly executed is not less than the preset number of iterations, the execution of steps S2031-S2037 is stopped.

[0072] Figure 4 The diagram illustrates an application example of a retrieval enhancement generation method based on a large language model, as shown in the example embodiment of this application. Figure 4 As shown, in step S401, the input question is obtained.

[0073] For example, a user might enter the question, "I want to know, in the Harry Potter series, who killed Dumbledore, and what was the fate of the murderer?"

[0074] In step S403, the first summary data is generated.

[0075] In step S403, based on the user's input question, the query keywords are obtained: Dumbledore, murdered, murderer, who is, and 3 public encyclopedia paragraphs are obtained through the preset knowledge base.

[0076] Search result 1: In June 1997, at the Hogwarts Tower of Astronomy, Severus Snape killed Albus Dumbledore with the "Avada Kedavra Curse".

[0077] Search result 2: The murder was actually a pre-arranged "euthanasia" by Dumbledore and Snape, with the aim of preserving Malfoy's soul and gaining Voldemort's trust.

[0078] Search result 3: Snape was bitten to death by Voldemort's snake Nagini during the Battle of Hogwarts. Voldemort mistakenly believed that killing Snape would allow him to inherit the Elder Wand.

[0079] The first summary data generated based on the search results is represented in JSON format in this embodiment. Here, `best_summary` represents the current best summary, and `no_progress` indicates whether knowledge growth has occurred.

[0080] { "best_summary": { The killer: Severus Snape "Murder Background": "The Astronomy Tower Incident, June 1997, pre-arranged with Dumbledore," Motive: "To preserve Draco Malfoy's soul and gain Voldemort's trust" The killer's fate: "He was bitten to death by Nagini, Voldemort's snake, during the Battle of Hogwarts." }, "turn": 0, "no_progress": 0 } In step S405, multiple rounds of retrieval are performed to obtain the second summary data.

[0081] In this embodiment, in the first round of retrieval In sub-step S4051, query reconstruction.

[0082] In some embodiments, query reconstruction also involves using a large model to generate a new question based on the original question and existing summaries.

[0083] For example, using the current best_summary, generate a new question: "After Severus Snape killed Dumbledore, what were the specific details of his death in the Battle of Hogwarts, and why did Voldemort have to kill him?" In sub-step S4053, the knowledge base is used for further retrieval, and new retrieval results are obtained.

[0084] Search result 4: Voldemort believed that he had to kill Snape first in order to make the Elder Wand loyal to him, so he ordered Nagini to attack Snape.

[0085] Search result 5: Before his death, Snape gave his memories to Harry, revealing that he had been secretly protecting Harry and deeply loved Lily Potter.

[0086] In sub-step S4055, the abstract fusion is performed.

[0087] Suppose the new information is: "Direct cause of death: Voldemort mistakenly believed that 'killing Snape' would lead to the Elder Wand's allegiance; dying act: giving his memories to Harry."

[0088] After fidelity, relevance, and / or non-redundancy checks, all new information can be added to the current best summary.

[0089] In sub-step S4057, assess whether there is knowledge growth in the fused summary.

[0090] Because the key fields "direct cause of death" and "end-of-life behavior" have been added, there is knowledge growth. Therefore, best_summary is updated and no_progress is set to zero. The updated latest summary is shown below.

[0091] { The killer: Severus Snape "Murder Background": "The Astronomy Tower Incident, June 1997, pre-arranged with Dumbledore," Motive: "To preserve Draco Malfoy's soul and gain Voldemort's trust" "The killer's fate": { "Time of death": "During the Battle of Hogwarts" "Direct cause": "Voldemort ordered Nagini to bite him to death in order to seize the Elder Wand." "Last Action": "Giving his memories to Harry, revealing the truth about protecting Harry." } } Since no_progress is 0, there is knowledge growth, so the second round of retrieval is executed, and sub-steps S4051-S4057 are repeated.

[0092] In substep S4051, the query is refactored, and a new question is generated using the current best summary: "Who was the true owner of the Elder Wand after Snape's death? Did Snape ever possess it?"

[0093] In sub-step S4053, the knowledge base is used for further retrieval, and new retrieval results are obtained.

[0094] Search result 6: The Elder Wand's true owner was Draco Malfoy (who disarmed Dumbledore at the Astronomy Tower), and Snape never owned it; Harry later took the wand from Draco.

[0095] In sub-step S4055, the abstract fusion is performed.

[0096] Suppose the new information is "Snape was never the master of the Elder Wand → related to "The Murderer's Ending", it can be inserted because it is directly related to the current best summary.

[0097] In sub-step S4057, assess whether there is knowledge growth in the fused summary. Due to the addition of the "Old Wand Master" knowledge point, which involves knowledge growth, the current best summary has been updated, and no_progress has been set to 0. The current best summary is (excerpt): "The killer's fate": { ... "Relationship with the Elder Wand": "Never owned it; its true owner was Draco Malfoy." } Since no_progress is 0, there is knowledge growth, so the third round of retrieval is executed, and sub-steps S4051-S4057 are repeated.

[0098] In substep S4051, the query is refactored, and the current best summary generates a new question: "Besides protecting Harry, what other key espionage activities did Snape undertake before the Battle of Hogwarts?".

[0099] In sub-step S4053, the knowledge base is used for further retrieval, and new retrieval results are obtained.

[0100] Assuming that all returned paragraphs are related to "espionage before the war," but the information points (such as secretly passing the Gryffindor sword and tipping off the Order of the Phoenix) have already been mentioned in search result 5, they are duplicates.

[0101] In sub-step S4055, the abstract fusion is performed.

[0102] No new entities or relationships have entered at present.

[0103] In sub-step S4057, assess whether there is knowledge growth in the fused summary.

[0104] Since knowledge growth = 0, setting no_progress = 1 (the preset limit of no continuous progress of 1 has been reached) triggers the stop condition and exits the multi-round retrieval loop.

[0105] In step S407, the best current summary is used to perform a retrieval using a large language model to generate answer data.

[0106] For example, based on the "best_summary" assembly command: "Please use two coherent paragraphs to first answer 'Who killed Dumbledore', then explain 'the final fate of the murderer', and point out the misconception about the Elder Wand's ownership." The answer data is: "Dumbledore's death was carried out by Severus Snape in June 1997 at the Hogwarts Astronomy Tower using the Avada Kedavra Curse; however, it was a pre-arranged 'euthanasia' to protect Draco Malfoy's soul and allow Snape to continue gaining Voldemort's trust. During the subsequent Battle of Hogwarts, Voldemort mistakenly believed that killing Snape would make the Elder Wand loyal to him, so he ordered Nagini to bite him to death; before his death, Snape passed on his memories to Harry, revealing that he had been secretly protecting Harry all along. It is worth noting that Snape never truly possessed the Elder Wand—its owner was first Draco Malfoy, and then Harry Potter."

[0107] The above description primarily focuses on the methodological aspects of the embodiments of this application. Those skilled in the art should readily recognize that, based on the operations or steps described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Those skilled in the art can implement the described functionality in different ways for each specific operation or method, and such implementations should not be considered beyond the scope of this application.

[0108] The apparatus embodiments of this application are described below. For details not described in the apparatus embodiments of this application, please refer to the method embodiments of this application.

[0109] Figure 5 A block diagram of a retrieval enhancement generation apparatus based on a large language model according to an example embodiment of this application is shown, such as... Figure 5 The retrieval enhancement generation device shown includes a first summary data generation unit 501, a second summary data generation unit 503, and an answer data generation unit 505. The first summary data generation unit 501 generates first summary data based on a preset knowledge base using user-input question information. The second summary data generation unit 503 performs multiple rounds of retrieval in the knowledge base based on the large language model using the first summary data to obtain second summary data. The answer data generation unit 505 uses the second summary data to generate answer data corresponding to the user-input question information.

[0110] Figure 6 An electronic device according to an exemplary embodiment of this application is shown. Reference is made below. Figure 6 To describe an electronic device 200 according to this embodiment of the present application. Figure 6 The electronic device 200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0111] like Figure 6 As shown, the electronic device 200 is presented in the form of a general-purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including storage unit 220 and processing unit 210), a display unit 240, etc.

[0112] The storage unit stores program code that can be executed by the processing unit 210, causing the processing unit 210 to perform the methods described in this specification according to various exemplary embodiments of this application. For example, the processing unit 210 can perform the methods described above.

[0113] Storage unit 220 may include readable media in the form of volatile storage units, such as random access memory (RAM) 2201 and / or cache memory 2202, and may further include read-only memory (ROM) 2203.

[0114] Storage unit 220 may also include a program / utility 2204 having a set (at least one) program module 2205, such program module 2205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0115] Bus 230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0116] Electronic device 200 can also communicate with one or more external devices 300 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 200, and / or with any device that enables electronic device 200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 250. Furthermore, electronic device 200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of electronic device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0117] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. The technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the methods described above according to the embodiments of this application.

[0118] Software products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0120] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0121] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the aforementioned functions.

[0122] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0123] According to an embodiment of this application, a computer program is proposed, including a computer program or instructions, which, when executed by a processor, can perform the methods described above.

[0124] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, and on the specific implementation methods and application scope of this application, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0125] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0126] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, and on the specific implementation methods and application scope of this application, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A large language model-based retrieval enhancement generation method, characterized in that, Comprise: Utilize user input question information, generate first summary data based on preset knowledge base; For the first summary data, based on the large language model in the knowledge base, multiple rounds of retrieval are carried out to obtain second summary data; Utilize the second summary data, generate answer data corresponding to the user input question information.

2. The search enhancement generation method of claim 1, wherein, For the first summary data, based on the large language model in the knowledge base, multiple rounds of retrieval are carried out to obtain second summary data, comprising: (1) For the first summary data, utilize the large language model to reconstruct the user input question information to obtain reconstructed question information; (2) Utilize the reconstructed question information to generate retrieval result data through the knowledge base; (3) Evaluate whether the retrieval result data has information growth relative to the first summary data; (4) In the case that the retrieval result data has information growth relative to the first summary data, generate second summary data according to the first summary data and the retrieval result data; (5) Utilize the second summary data, repeat steps (1)-(4) until the retrieval result data obtained no longer has information growth relative to the second summary data.

3. The search enhancement generation method of claim 2, wherein, Step (3) evaluates whether the retrieval result data has information growth relative to the first summary data, comprising: Evaluate the relevance of the retrieval result data and the user input question information; According to the relevance, evaluate whether the retrieval result data has information growth relative to the first summary data.

4. The search enhancement generation method of claim 3, wherein, Before step (4) generates second summary data according to the first summary data and the retrieval result data in the case that the retrieval result data has information growth relative to the first summary data, For the first summary data, based on the large language model in the knowledge base, multiple rounds of retrieval are carried out to obtain second summary data, further comprising: Evaluate whether the retrieval result data has redundant data relative to the first summary data.

5. The method of claim 4, wherein, Before step (4) generates second summary data according to the first summary data and the retrieval result data in the case that the retrieval result data has information growth relative to the first summary data, For the first summary data, based on the large language model in the knowledge base, multiple rounds of retrieval are carried out to obtain second summary data, further comprising: In the case that there is redundant data in the retrieval result data, delete the redundant data.

6. The search enhancement generation method of claim 2, wherein, For the first summary data, based on the large language model in the knowledge base, multiple rounds of retrieval are carried out to obtain second summary data, further comprising: When the number of repeated execution of steps (1)-(4) is not less than the preset iteration number, stop executing steps (1)-(4).

7. The search enhancement generation method of claim 1, wherein, Utilize the second summary data, generate answer data corresponding to the user input question information, comprising: Based on the question information type corresponding to the user input question information, utilize the second summary data to generate answer data corresponding to the user input question information.

8. A large language model-based retrieval augmentation generation apparatus, characterized by comprising: Comprise: First summary data generation unit, for utilizing user input question information, generating first summary data based on preset knowledge base; The second abstract data generation unit is configured to, for the first abstract data, perform multi-round retrieval in the knowledge base based on the large language model to obtain second abstract data. The answer data generation unit is configured to generate answer data corresponding to the user input question information by using the second abstract data.

9. An electronic device, comprising: The method comprises: a processor; a memory configured to store a computer program; when the computer program is executed by the processor, the processor implements the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-7.

Images