Retrieval enhancement generation method, system and equipment based on two-stage retrieval optimization
By adopting a two-stage search optimization method on a large language model or a hybrid expert model, splitting the user query text into a structured entity relationship triple and searching, the limitations of the knowledge understanding and application of large language models in vertical fields are solved. , improves the generation efficiency and accuracy of answer texts and realizes efficient utilization of large-scale documents.
Patent Information
- Application Number
- CN202410935352.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-12
AI Technical Summary
Large language models or hybrid expert models have limitations in domain-specific knowledge understanding and application on specific tasks in vertical fields, and it is very challenging to fully utilize search and generate search and generation in large-scale document knowledge bases, resulting in answers Questions about text generation efficiency and accuracy.
Using a search enhancement generation method based on two-stage search optimization, by obtaining user query text and knowledge documents, it is split into structured subquery entity relationship triplets, and searching using abstract entity relationship triplets and sentence vectors, and adjusting Retrieval granularity, improve retrieval accuracy and efficiency, and ultimately build knowledge Q&A for specific tasks in vertical fields.
It expands the knowledge application capabilities and scenarios of the document knowledge base, improves the generation efficiency and accuracy of answer text, reduces the search time, and realizes the efficient use of large-scale knowledge documents.
Smart Images

Figure CN118939764A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of natural language processing, and in particular to a retrieval enhancement generation method, system and device based on two-stage retrieval optimization. Background Art
[0002] In recent years, generative artificial intelligence technology has developed rapidly, and many organizations and institutions have released large language models (LLMs) and mixed expert models (MoEs). These models use massive amounts of open data and general knowledge, and are trained based on deep learning algorithms with huge parameters. They have strong generalization and general capabilities. However, the application of these models to specific tasks in vertical fields still faces the limitations of understanding and applying knowledge in specific fields, and retraining requires huge computing power costs.
[0003] In response to the above challenges, there are currently two main technical directions for applying these models to downstream question-answering tasks, namely supervised fine-tuning (SFT) and retrieval augmented generation (RAG). Compared with supervised fine-tuning, retrieval augmented generation technology can also provide effective services under limited computing power. In addition, by decomposing the problem into two stages, retrieval and generation, retrieval augmented generation technology can make full use of database knowledge and provide greater application flexibility. However, in actual enterprise applications, it is very challenging to make full use of retrieval augmented generation technology for retrieval and generation in large-scale document knowledge bases.
[0004] In summary, there is an urgent need to address the limitations of large language models or mixed expert models in understanding and applying knowledge in specific fields, as well as the efficiency and accuracy of generating answer texts. Summary of the invention
[0005] The purpose of this application is to provide a retrieval enhancement generation method, system and device based on two-stage retrieval optimization, which can expand the knowledge application capabilities and scenarios of the document knowledge base and improve the generation efficiency and accuracy of the answer text.
[0006] To achieve the above objectives, this application provides the following solutions:
[0007] In a first aspect, the present application provides a retrieval enhancement generation method based on two-stage retrieval optimization, comprising:
[0008] Obtain user query text and multiple knowledge documents; each knowledge document includes a document summary and a full document;
[0009] For any knowledge document, determine the summary entity relationship triple set of the knowledge document according to the document summary of the knowledge document; determine the sentence vector set of the knowledge document according to the full text of the knowledge document;
[0010] Based on a pre-built query rewriting hint template, split the user query text into multiple sub-query entity relationship triples;
[0011] Calculate the summary retrieval score of each sub-query entity relationship triple and the summary entity relationship triple set of each knowledge document respectively, and determine the first stage retrieval result according to the summary retrieval score; the first stage retrieval result is empty or the most matching knowledge document;
[0012] If the search result of the first stage is empty, the answer text is determined based on the pre-built answer generation prompt word template and the sentence vector set of all sub-query entity relationship triples and all knowledge documents;
[0013] If the first-stage retrieval result is the best matching knowledge document, then the sentence retrieval scores of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document are calculated respectively, and the best matching sentence set of each sub-query entity relationship triple is determined according to the sentence retrieval scores;
[0014] Based on the pre-built answer generation prompt word template, the answer text is determined according to all sub-query entity-relation triples and the best matching sentence set of each sub-query entity-relation triple.
[0015] In a second aspect, the present application provides a retrieval enhancement generation system based on two-stage retrieval optimization, comprising:
[0016] A data preprocessing and storage module is used to obtain multiple knowledge documents, and for any knowledge document, determine the summary entity relationship triple set of the knowledge document according to the document summary of the knowledge document, and determine the sentence vector set of the knowledge document according to the full text of the knowledge document; each knowledge document includes a document summary and a full text;
[0017] A query rewriting module, used to obtain a user query text and split the user query text into a plurality of sub-query entity relationship triples based on a pre-built query rewriting prompt word template;
[0018] A two-stage retrieval module, connected to the data preprocessing and storage module and the query rewriting module, respectively, for respectively calculating the summary retrieval score of each sub-query entity relationship triple and the summary entity relationship triple set of each knowledge document, and determining the first-stage retrieval result according to the summary retrieval score; if the first-stage retrieval result is empty or the best matching knowledge document, when the first-stage retrieval result is the best matching knowledge document, respectively calculating the sentence retrieval score of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document, and determining the best matching sentence set of each sub-query entity relationship triple according to the sentence retrieval score;
[0019] The answer generation module is connected to the data preprocessing and storage module and the two-stage retrieval module respectively, and is used to generate a prompt word template based on a pre-built answer when the first-stage retrieval result is empty, and determine the answer text according to the sentence vector set of all sub-query entity relationship triples and all knowledge documents; when the first-stage retrieval result is the most matching knowledge document, the prompt word template is generated based on the pre-built answer, and determine the answer text according to the most matching sentence set of all sub-query entity relationship triples and each sub-query entity relationship triple.
[0020] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned retrieval enhancement generation method based on two-stage retrieval optimization.
[0021] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0022] This application is aimed at knowledge application scenarios in vertical fields, optimizes the storage content and storage structure of large-scale knowledge documents, and uses summary entity relationship triples and sentence vectors to store knowledge documents, providing data support for subsequent knowledge retrieval and application, and further constructs special query rewriting prompts to split the unstructured and complex query text entered by the user into structured sub-query entity relationship triples, thereby improving the retrieval accuracy of the retrieval generation technology. In the two-stage retrieval process, it can be determined whether to perform a full-text retrieval of the document based on the summary retrieval results, and the retrieval granularity can be efficiently adjusted to reduce the time spent on retrieval. Finally, a special answer generation prompt is constructed to perform knowledge questions and answers for specific tasks in vertical fields, expanding the knowledge application capabilities and scenarios of the document knowledge base and improving the generation efficiency and accuracy of the answer text. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0024] Figure 1 This is an application environment diagram of a retrieval enhancement generation method based on two-stage retrieval optimization in an embodiment of the present application;
[0025] Figure 2 A flowchart of a retrieval enhancement generation method based on two-stage retrieval optimization provided in an embodiment of the present application;
[0026] Figure 3 A basic architecture diagram of a two-stage search optimization search enhancement generation method provided in an embodiment of the present application;
[0027] Figure 4 A schematic diagram of a query rewriting hint template;
[0028] Figure 5 Schematic diagram of generating a prompt word template for answering;
[0029] Figure 6 A schematic diagram of functional modules of a retrieval enhancement generation system based on two-stage retrieval optimization provided by another embodiment of the present application;
[0030] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0032] Entity-relationship triples are a type of data that expresses and stores knowledge in a structured manner. They are usually represented in the form of (entity 1, relationship, entity 2). They can provide structured knowledge representation and semantic association information for retrieval enhancement generation, helping it to more accurately understand and process user questions during knowledge retrieval and answer generation. On this basis, combined with scientific retrieval solutions, the efficiency and accuracy of retrieval enhancement generation technology can be improved.
[0033] Retrieval-enhanced generation technology is a method that combines information retrieval and information generation. It uses the powerful information generation capabilities of large language models or hybrid expert models to handle information retrieval and generation problems in intelligent question-answering tasks.
[0034] The purpose of this application is to provide a retrieval enhancement generation method, system and device based on two-stage retrieval optimization to solve the problem of limitations of large language models or hybrid expert models in understanding and applying knowledge in specific fields, while maintaining dynamic updating of knowledge, dynamically adjusting retrieval content and generating personalized answers based on queries of different users.
[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0036] The retrieval enhancement generation method based on two-stage retrieval optimization provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the user query text and knowledge document to be processed to the server 104. After the server 104 receives the user query text and knowledge document, for the user query text and knowledge document to be processed, the server 104 performs retrieval enhancement generation processing on the user query text and knowledge document to determine the answer text. The server 104 can feedback the obtained answer text to the terminal 102. In addition, in some embodiments, the retrieval enhancement generation method based on the two-stage retrieval optimization can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly perform retrieval enhancement generation processing on the user query text and knowledge document, or the server 104 can obtain the user query text and knowledge document to be processed from the data storage system, and perform retrieval enhancement generation processing on the user query text and knowledge document to be processed.
[0037] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.
[0038] In an exemplary embodiment, Figure 2 and Figure 3As shown, a method for generating retrieval enhancement based on two-stage retrieval optimization is provided. The method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 1 to 7. Among them:
[0039] Step 1: Obtain user query text and multiple knowledge documents. Each knowledge document includes a document summary and a full document.
[0040] Step 2: For any knowledge document, determine the summary entity relationship triple set of the knowledge document based on the document summary of the knowledge document; and determine the sentence vector set of the knowledge document based on the full text of the knowledge document.
[0041] Specifically, the knowledge documents are first preprocessed by data cleaning, outlier processing, document summary extraction, and summary entity relationship triple extraction. In order to achieve efficient data storage and utilization, the full text of the document is segmented and embedded, and then the preprocessed document summary and full text are stored in different databases using data hierarchical storage. Specifically, it includes document summary database, content database, vocabulary database, etc., and the document content summary information and full text are stored in different databases at different levels.
[0042] The process of slicing and embedding the full text of the document is as follows: the full text of the document is sliced into sentences, each sentence is divided into one slice, the sliced data is input into the embedding model (such as sentence-transformer or text-embedding-ada-002, etc.), and then the vector representation of the sentence is output.
[0043] In this application, the document summary database stores data such as document title, summary entity relationship triples, and document length. The document full-text database stores the full text of the document segmented by sentence, the corresponding document number, sentence number, etc. The vocabulary database stores data such as word vectors, TF-IDF and other retrieval algorithm scores.
[0044] Step 3: Based on the pre-built query rewriting prompt word template, split the user query text into multiple sub-query entity relationship triples Q = {q1, q2, ..., q I}. Among them, Q is the set of subquery entity relationship triples, q i is the i-th subquery entity-relationship triple, i.e., {entity 1, relationship, entity 2}, i=1,...,I, where I is the number of subquery entity-relationship triples.
[0045] Specifically, query rewriting prompt words are constructed according to a pre-constructed query rewriting prompt word template and the user query text, and the query rewriting prompt words are input into a large language model or a hybrid expert model to obtain a plurality of sub-query entity triples.
[0046] like Figure 4 As shown, the query rewriting prompt words include defining system roles, system instructions, sample dialogue instructions and question instructions. Among them, the system instructions are used to lay the tone of the dialogue. The sample dialogue instructions are used to enhance the model's context understanding ability. The question instructions are filled with the user query text.
[0047] For example, the "role" in the system instruction is "system", and the "content" is "please play the role of a mathematical expert who extracts text entity triples". There are two roles in the example dialogue instruction, namely user and assistant: the "content" corresponding to "user" is "the following is a user query text <May I ask what kind of investigation category does the situation where major accident hazards in the project have not been rectified and reviewed on time and there is no written record belong to?>, please strictly output all entity triples in the query text in json format"; the "content" corresponding to "assistant" is "[{"entity 1":"project","relationship":"rectification","entity 2":"major accident hazard"},{"entity 1":"project","relationship":"review","entity 2":"major accident hazard"},{"entity 1":"no written record","relationship":"belongs to","entity 2":"investigation category"}]". The "role" in the question instruction is "user" and the "content" is "The following is a user query text <user query specific text>. Please output all entity triples in the query text strictly in JSON format."
[0048] In actual application, the system instructions and sample dialogue instructions remain unchanged, and the user query text is filled into the <user query specific text> of the corresponding value of the "content" key of the question instruction.
[0049] The hybrid expert model can complete the task corresponding to the instruction according to the input prompt word and output the result. To make the hybrid expert model output more accurate results, how to construct the prompt word is the key. Different prompt words for the same task will have different accuracy of the output results. In this embodiment, a query rewriting prompt word template is constructed, and query rewriting prompt words are constructed according to the template. The query rewriting prompt words are input into the hybrid expert model to output the query rewriting result, and the unstructured user query text is converted into a structured entity triple.
[0050] The present application constructs a hint word template for extracting query entity relationship triples based on a large language model or a hybrid expert model, namely, a query rewriting hint word template. By using this hint word template, the powerful text understanding and generation capabilities of the large language model or the hybrid expert model are fully utilized to extract structured entity relationship triples in unstructured query texts, provide structured knowledge representation and semantic association information for retrieval enhancement generation, and effectively improve the accuracy of retrieval enhancement generation.
[0051] Furthermore, the present application searches for knowledge documents in the database in two stages. The first stage is a document summary hybrid search. If relevant information is retrieved, the second stage is a document full-text context window search. This can achieve faster and more accurate searches in large-scale document databases. Step 4 corresponds to a summary hybrid search, and step 5 corresponds to a document full-text context window search.
[0052] Step 4: Calculate the summary retrieval scores of each subquery entity relation triple and each summary entity relation triple respectively, and determine the first stage retrieval result according to the summary retrieval score. The first stage retrieval result is empty or the best matching knowledge document.
[0053] Further, step 4 includes:
[0054] (1) For any sub-query entity relationship triple and any summary entity relationship triple set, a plurality of retrieval algorithms are respectively used to calculate the retrieval scores of the sub-query entity relationship triple and the summary entity relationship triple set.
[0055] The various retrieval algorithms include the TF-IDF algorithm, the BM25 retrieval algorithm, and the vector retrieval algorithm based on semantic similarity. They can also be expanded to support more algorithms that support the reciprocal rank fusion (RRF) algorithm to unify the rearrangement results, not limited to the above three.
[0056] (2) According to the retrieval scores of the sub-query entity relationship triples and the set of summary entity relationship triples calculated by each retrieval algorithm, a reciprocal ranking fusion algorithm is used to determine the summary retrieval scores of the sub-query entity relationship triples and the set of summary entity relationship triples.
[0057] Specifically, for the document summary entity relationship triple set A={a1,a2,...,a J}, traverse the summary entity relationship triple set a of each knowledge document j , j = 1, 2, ..., J, J is the number of knowledge documents, calculate a j Subquery entity relation triple q in subquery entity relation triple set Q i Abstract search score of:
[0058] S RRF (q i ,a j )=W TF-IDF *S TF-IDF (q i ,a j )+..+W Semantic *S Semantic (q i ,a j );
[0059] Among them, S RRF (q i ,a j ) is the subquery entity relationship triple q i A set of triples of relations between entities and summary entities j Abstract retrieval score, W TF-IDF , W Semantic Represent the weights of each retrieval algorithm, satisfying W TF-IDF +...+W Semantic =1, S TF-IDF (q i ,a j ), S Semantic (q i ,a j ) represent the q calculated by each retrieval algorithm i with a j The search score of .
[0060] (3) Determine the maximum summary retrieval score of all subquery entity relationship triples and the set of all summary entity relationship triples.
[0061] Record the best matching document summary, that is, max{S RRF (q i ,a j )}Corresponding document number max{S RRF (q i ,a j )} represents the maximum value of the abstract retrieval score.
[0062] (4) Determine whether the maximum value of the abstract retrieval score is less than a valid threshold. If so, the first-stage retrieval result is empty. Otherwise, the knowledge document to which the abstract entity relationship triple set corresponding to the maximum value of the abstract retrieval score belongs is taken as the most matching knowledge document.
[0063] Set the validity threshold used to determine whether the search is valid And determine the validity of the search, if It means that relevant data can be retrieved in the database, and the knowledge document to which the set of abstract entity relationship triples corresponding to the maximum abstract retrieval score belongs is taken as the most matching knowledge document like This means that no relevant data is found in the database, and the first-stage search result is empty. k is the kth sentence vector that best matches the knowledge document, and K is the number of sentence vectors that best match the knowledge document.
[0064] Based on the output results of the document summary hybrid retrieval in the first stage, determine whether it is necessary to perform a document full-text context window retrieval. If the document full-text context window retrieval is not necessary, directly proceed to the question and answer generation step, that is, enter step 5; otherwise, perform the document full-text context window retrieval step, that is, enter step 6.
[0065] Step 5: If the search results of the first stage are empty, generate a prompt word template based on the pre-built answer, and determine the answer text based on all sub-query entity relationship triples and all sentence vectors of all knowledge documents.
[0066] Step 6: If the retrieval result of the first stage is the best matching knowledge document, the sentence retrieval score of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document is calculated respectively, and the best matching sentence set of each sub-query entity relationship triple is determined according to the sentence retrieval score.
[0067] In this embodiment, the sentence retrieval score of the sub-query entity relationship triple and the sentence vector is the cosine similarity between the sub-query entity relationship triple and the sentence vector.
[0068] For each subquery entity relation triple q in the subquery entity relation triple set Q i , calculate q i and Each of k The cosine similarity score S Semantic (q i ,s k ), and record each q i The sentence number with the highest cosine similarity score
[0069] In step 6, the best matching sentence set of each sub-query entity relationship triple is determined according to the sentence retrieval score, specifically including:
[0070] For any sub-query entity relationship triple, determine the maximum sentence retrieval score of all sentence vectors of the sub-query entity relationship triple and the most matching knowledge document, and take the sentence vector corresponding to the maximum sentence retrieval score as the most matching sentence.
[0071] In the best matching knowledge document, sentence vectors within a set window before and after the best matching sentence are used as a best matching sentence set of the sub-query entity relationship triple.
[0072] Specifically, for the subquery entity relationship triple q i The most matching sentence Select the front and back expanded sentence windows n, and output the expanded sentence set, i.e., the subquery entity relationship triple q i The most matching sentence set
[0073] In order to solve the problem of low retrieval efficiency of huge knowledge documents in databases, this application proposes a two-stage retrieval method. The first stage uses document summary hybrid retrieval to greatly save retrieval time and improve the retrieval efficiency and accuracy of retrieval enhancement generation. The second stage uses document full-text context window retrieval to provide detailed context information of database retrieval results to large language models or hybrid expert models, thereby improving the comprehensiveness of generated answers.
[0074] Step 7: Generate a prompt word template based on the pre-built answer, and determine the answer text according to the best matching sentence set of all sub-query entity relationship triples and each sub-query entity relationship triple.
[0075] Specifically, according to the pre-constructed answer generation prompt word template, all sub-query entity relationship triples and the best matching sentence set of each sub-query entity relationship triple, the answer generation prompt word is constructed, and the answer generation prompt word is input into the large language model or the hybrid expert model to obtain the answer text.
[0076] like Figure 5 As shown in the figure, the answer generation prompt words include defining system roles, system instructions, background introduction instructions and question instructions. Among them, system instructions are used to lay the tone of the conversation, and background introduction instructions are used to enhance the model's context understanding ability. The question instructions are filled with all sub-query entity relationship triples and the best matching sentence set for each sub-query entity relationship triple.
[0077] For example, the "role" in the system instruction is "system" and the "content" is "please play the role of a consulting expert". There are two roles in the background import instruction, namely user and assistant: the "content" corresponding to "user" is "the following is a set of entity-relationship triples queried by the user <[{"entity 1":"engineering","relationship":"rectification","entity 2":"major accident hazards"}, {"entity 1":"engineering","relationship":"re-inspection","entity 2":"major accident hazards"}, {"entity 1":"no written records","relationship":"belongs to","entity 2":"investigation category"}]>, please determine what field this query belongs to?"; the "content" corresponding to "assistant" is "the entity triples of the above query involve the field of hidden danger investigation generated by railway safety". The "role" in the question instruction is "user" and the "content" is "The following is a set of entity-relationship triples queried by users <[{"entity 1":"project","relationship":"rectification","entity 2":"major accident hazards"}, {"entity 1":"project","relationship":"review","entity 2":"major accident hazards"}…]>, and the corresponding search results of the database are <search result 1>, <search result 2>… Please analyze and generate answers based on the above queries and search results."
[0078] In actual application, the system instructions and background import instructions remain unchanged, and all sub-query entity relationship triples are filled into the entity relationship triples <> of the user query in the question instruction, and the most matching sentence set of each sub-query entity relationship triple is filled into <retrieval result 1> and <retrieval result 2> in the question instruction.
[0079] In order to address the problem of insufficient linkage between database retrieval results and large language models or hybrid expert models, this application constructs an answer generation prompt word template that can fully utilize the database retrieval results and the model's powerful understanding and generation capabilities to generate more professional and accurate answer texts in downstream vertical fields.
[0080] In summary, this application is aimed at knowledge application scenarios in vertical fields, optimizes the storage content and storage structure of large-scale knowledge documents, and provides data support for subsequent knowledge retrieval and application. Further construct special query rewriting prompts, and use the capabilities of large language models or hybrid expert models to extract unstructured and complex queries input by users into structured entity relationship triples, thereby improving the retrieval accuracy of retrieval generation technology. In the two-stage retrieval process, it is possible to determine whether to perform a context window retrieval of the full text of the document based on the retrieval results of the hybrid retrieval of the document summary, and the retrieval granularity can be efficiently adjusted to reduce the time spent on retrieval. Finally, construct special answer generation prompts, and use the capabilities of large language models or hybrid expert models to conduct knowledge questions and answers for specific tasks in vertical fields, expanding the knowledge application capabilities and scenarios of the document knowledge base.
[0081] The present application also provides an application scenario, which applies the above-mentioned retrieval enhancement generation method based on two-stage retrieval optimization. Specifically: the retrieval enhancement generation method based on two-stage retrieval optimization provided in this embodiment can be applied in the knowledge base intelligent retrieval scenario. The knowledge documents in the knowledge base and the query text input by the user enter the content processing link from the dialogue link, and the answer text corresponding to the query text is obtained through human-computer collaboration, and enters the next round of dialogue link. The retrieval enhancement generation method based on two-stage retrieval optimization provided in this embodiment belongs to the dialogue link in the process of knowledge base intelligent retrieval.
[0082] Based on the same inventive concept, the embodiment of the present application also provides a two-stage retrieval optimization-based retrieval enhancement generation system for implementing the above-mentioned two-stage retrieval optimization-based retrieval enhancement generation method. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the retrieval enhancement generation system based on two-stage retrieval optimization provided below can refer to the limitations of the retrieval enhancement generation method based on two-stage retrieval optimization above, and will not be repeated here.
[0083] In an exemplary embodiment, Figure 6 As shown, a retrieval enhancement generation system based on two-stage retrieval optimization is provided, including: a data preprocessing and storage module 21, a query rewriting module 22, a two-stage retrieval module 23 and an answer generation module 24.
[0084] The data preprocessing and storage module 21 is used to obtain multiple knowledge documents, and for any knowledge document, determine the summary entity relationship triple set of the knowledge document according to the document summary of the knowledge document, and determine the sentence vector set of the knowledge document according to the full text of the knowledge document. Each knowledge document includes a document summary and a full text.
[0085] The query rewriting module 22 is used to obtain a user query text and split the user query text into a plurality of sub-query entity relationship triples based on a pre-built query rewriting prompt word template.
[0086] The two-stage retrieval module 23 is connected to the data preprocessing and storage module 21 and the query rewriting module 22 respectively. The two-stage retrieval module is used to respectively calculate the summary retrieval score of each sub-query entity relationship triple and the summary entity relationship triple set of each knowledge document, and determine the first-stage retrieval result according to the summary retrieval score; the first-stage retrieval result is empty or the best matching knowledge document. When the first-stage retrieval result is the best matching knowledge document, the sentence retrieval score of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document is calculated respectively, and the best matching sentence set of each sub-query entity relationship triple is determined according to the sentence retrieval score.
[0087] The answer generation module 24 is connected to the data preprocessing and storage module 21 and the two-stage retrieval module 23 respectively. The answer generation module is used to generate a prompt word template based on a pre-built answer when the first-stage retrieval result is empty, and determine the answer text according to the sentence vector set of all sub-query entity relationship triples and all knowledge documents. When the first-stage retrieval result is the most matching knowledge document, the answer generation prompt word template is based on the pre-built answer, and the answer text is determined according to the most matching sentence set of all sub-query entity relationship triples and each sub-query entity relationship triple.
[0088] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store user query texts and knowledge documents. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a retrieval enhancement generation method based on two-stage retrieval optimization is implemented.
[0089] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0090] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0091] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0093] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0094] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0095] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0096] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A retrieval enhancement generation method based on two-stage retrieval optimization, characterized in that: The retrieval enhancement generation method based on two-stage retrieval optimization includes: Obtain user query text and multiple knowledge documents; each knowledge document includes a document summary and a full document; For any knowledge document, determine the summary entity relationship triple set of the knowledge document according to the document summary of the knowledge document; determine the sentence vector set of the knowledge document according to the full text of the knowledge document; Based on a pre-built query rewriting hint template, split the user query text into multiple sub-query entity relationship triples; Calculate the summary retrieval score of each sub-query entity relationship triple and the summary entity relationship triple set of each knowledge document respectively, and determine the first stage retrieval result according to the summary retrieval score; the first stage retrieval result is empty or the most matching knowledge document; If the search result of the first stage is empty, the answer text is determined based on the pre-built answer generation prompt word template and the sentence vector set of all sub-query entity relationship triples and all knowledge documents; If the first-stage retrieval result is the best matching knowledge document, then the sentence retrieval scores of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document are calculated respectively, and the best matching sentence set of each sub-query entity relationship triple is determined according to the sentence retrieval scores; Based on the pre-built answer generation prompt word template, the answer text is determined according to all sub-query entity relationship triples and the best matching sentence set of each sub-query entity relationship triple.
2. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: Based on the pre-built query rewriting hint template, the user query text is split into multiple sub-query entity relationship triples, specifically including: Constructing query rewriting prompt words according to a pre-constructed query rewriting prompt word template and the user query text; The query rewriting hint words are input into a large language model or a hybrid expert model to obtain a plurality of sub-query entity triples.
3. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: The query rewriting prompt words include defining system roles, system instructions, sample dialogue instructions and question instructions; the question instructions are filled with the user query text.
4. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: The summary retrieval scores of each subquery entity relation triple and the summary entity relation triple of each knowledge document are calculated respectively, and the first-stage retrieval results are determined according to the summary retrieval scores, including: For any sub-query entity relationship triple and any summary entity relationship triple set, a plurality of retrieval algorithms are respectively used to calculate the retrieval scores of the sub-query entity relationship triple and the summary entity relationship triple set; According to the retrieval scores of the sub-query entity relationship triples and the set of summary entity relationship triples calculated by each retrieval algorithm, a reciprocal ranking fusion algorithm is used to determine the summary retrieval scores of the sub-query entity relationship triples and the set of summary entity relationship triples; Determine the maximum summary retrieval score of all subquery entity relationship triples and all summary entity relationship triples; Determine whether the maximum value of the abstract retrieval score is less than the effective threshold. If so, the first-stage retrieval result is empty. Otherwise, the knowledge document to which the abstract entity relationship triple set corresponding to the maximum value of the abstract retrieval score belongs is taken as the most matching knowledge document.
5. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 4 is characterized in that: Various retrieval algorithms include TF-IDF algorithm, BM25 retrieval algorithm and vector retrieval algorithm based on semantic similarity.
6. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: The sentence retrieval score of the subquery entity relation triple and the sentence vector is the cosine similarity between the subquery entity relation triple and the sentence vector.
7. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: The best matching sentence set for each subquery entity relation triple is determined based on the sentence retrieval score, including: For any sub-query entity relationship triple, determine the maximum sentence retrieval score of all sentence vectors of the sub-query entity relationship triple and the most matching knowledge document, and take the sentence vector corresponding to the maximum sentence retrieval score as the most matching sentence; In the best matching knowledge document, sentence vectors within a set window before and after the best matching sentence are used as the best matching sentence set of the sub-query entity relationship triple.
8. The retrieval enhancement generation method based on two-stage retrieval optimization according to claim 1 is characterized in that: Based on the pre-built answer generation prompt word template, the answer text is determined according to all sub-query entity relationship triples and the best matching sentence set of each sub-query entity relationship triple, including: Construct answer generation prompt words according to a pre-constructed answer generation prompt word template, all sub-query entity relationship triples and the best matching sentence set of each sub-query entity relationship triple; The answer generation prompt words are input into a large language model or a mixed expert model to obtain an answer text.
9. A retrieval enhancement generation system based on two-stage retrieval optimization, characterized in that: The retrieval enhancement generation system based on two-stage retrieval optimization includes: A data preprocessing and storage module is used to obtain multiple knowledge documents, and for any knowledge document, determine the summary entity relationship triple set of the knowledge document according to the document summary of the knowledge document, and determine the sentence vector set of the knowledge document according to the full text of the knowledge document; each knowledge document includes a document summary and a full text; A query rewriting module, used to obtain a user query text and split the user query text into a plurality of sub-query entity relationship triples based on a pre-built query rewriting prompt word template; A two-stage retrieval module, connected to the data preprocessing and storage module and the query rewriting module, respectively, for respectively calculating the summary retrieval score of each sub-query entity relationship triple and the summary entity relationship triple set of each knowledge document, and determining the first-stage retrieval result according to the summary retrieval score; if the first-stage retrieval result is empty or the best matching knowledge document, when the first-stage retrieval result is the best matching knowledge document, respectively calculating the sentence retrieval score of each sub-query entity relationship triple and each sentence vector of the best matching knowledge document, and determining the best matching sentence set of each sub-query entity relationship triple according to the sentence retrieval score; The answer generation module is connected to the data preprocessing and storage module and the two-stage retrieval module respectively, and is used to generate a prompt word template based on a pre-constructed answer when the first-stage retrieval result is empty, and determine the answer text according to the sentence vector set of all sub-query entity relationship triples and all knowledge documents; when the first-stage retrieval result is the most matching knowledge document, the prompt word template is generated based on the pre-constructed answer, and determine the answer text according to the most matching sentence set of all sub-query entity relationship triples and each sub-query entity relationship triple.
10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the retrieval enhancement generation method based on two-stage retrieval optimization described in any one of claims 1 to 8.
Citation Information
Cited By
Retrieval problem optimization method based on multilayer knowledge base, medium and system
CN119494411A
Retrieval report generation method and device based on local language model, equipment and medium
CN120407935A
Method, device and medium for generating a search report based on a local language model
CN120407935B
Double-granularity text retrieval method based on atomic sentences
CN120821825A
Spaceflight knowledge field-oriented large model retrieval enhancement generation enhancement method
CN121188148A
