Knowledge graph based search enhancement generation system and method

Abstract

The application discloses a kind of retrieval enhancement generation system and method based on knowledge graph, solve the problem that retrieval candidate set is too large and is easily influenced by scattered paragraph and cannot consider accuracy and generalization ability.The system includes: data import module, retrieval database, retrieval module, question and answer module, knowledge graph database is additionally arranged between data import and retrieval module, retrieval module is multilayer mixed retriever, and retrieval database is corresponding three-layer architecture.The method includes: constructing system;Constructing graph structure information retrieval index;Receive question;Mixed level retrieval obtains enhanced information;Generate answer in combination with enhanced information.The application is on the multilayer retrieval index of the directory structure of knowledge graph graph structure semantic information, combined with graph structure and semantic information mixed level retrieval, reduce the size of candidate set, reduce the misleading of scattered paragraph, consider accuracy and generalization ability.Application in text generation technical field, such as intelligent question and answer, code generation etc.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, mainly related to information retrieval and text generation, specifically a retrieval enhancement generation system and method based on a knowledge graph. It is applied to intelligent question answering, sentiment analysis, code generation and other scenarios. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art or prior art.

[0003] With the rapid development of artificial intelligence technology represented by deep learning, information retrieval and text generation technology plays a key role in modern society. The continuous integration of these two fields promotes the deeper understanding, interpretation and generation of computer language. The introduction of retrieval enhancement generation and deep learning technology enables text generation models to better capture semantic relationships and contextual information, thus participating more deeply in human social production and life in various fields. Typical examples include intelligent assistants, chat robots, sentiment analysis, machine translation, etc. In modern society, information retrieval and text generation are of great significance to improving work efficiency, expanding human-computer interaction methods, and promoting global information flow.

[0004] Large language generation models are the latest achievements in the field of text generation, which are based on deep learning technology and have a large number of parameters and complex structures. These models learn high-level semantic information in language by training on super-large text datasets and achieve excellent results in performing various language understanding and text generation tasks. However, due to the extremely large number of parameters, the update of large language generation models is expensive and time-consuming, making it difficult to update the knowledge base of these models in a timely manner. Ultimately, these models, despite their strong semantic understanding capabilities, are prone to hallucinations when encountering input scenarios with insufficient or outdated training data.

[0005] Retrieval enhancement generation is a natural language processing paradigm that combines retrieval and generation methods, which includes two core modules, a retrieval model and a generation model, as well as an external knowledge base. The retrieval model is responsible for retrieving possible helpful external auxiliary information based on the input information, and the generation model is responsible for text understanding and generation based on the input information and the enhancement information provided by the external knowledge base. Retrieval enhancement generation for large language generation models aims to use retrieval models to input questions and enhancement information to the generation model through prompting methods to supplement the knowledge base and enhance the performance of the generation model, ensuring that the generated text is based on relevant and accurate additional knowledge. This technology has been proven to effectively improve hallucinations and other problems caused by insufficient or outdated training data.

[0006] There are two technical points in the search enhanced generation of the search model, one is the index architecture, and the other is the search method.

[0007] The conventional index architecture is to split the external knowledge which is usually long text according to a certain length threshold or according to the sentence structure, convert the long text set into a text block set, embed or perform word segmentation analysis on the text block unit semantic vector, and construct a corresponding flat search index based on the embedding or word segmentation analysis result. When searching, the entire text block set is directly used as a candidate set, and the most relevant information is searched at one time.

[0008] The search method is usually divided into sparse search and dense search, wherein the sparse search is generally based on the bag-of-words model and uses a sparse vector with most elements being zero for semantic representation. Its characteristics are fast reasoning speed, simple model and robustness and adaptability, but since the model only considers the semantic information related to the word frequency, it ignores the context relationship, word order and other semantic information, so its performance is not good in complex situations; the dense search is generally based on deep learning text embedding technology, and the semantic representation is performed through a dense vector. Since the embedding model learns more context semantic information based on the attention mechanism, the dense search often has better performance, but it still has two key weaknesses: the text embedding model is usually limited by the context length, and in most cases it cannot be applied to the entire document, but can only be split according to the threshold upper limit and then embedded, which will cause the loss of context semantic information; the performance of the dense search depends on the semantic space of the text embedding model, and it performs poorly on out-of-domain data and has weak generalization ability.

[0009] At present, the traditional scheme of the search enhanced generation system has the following limitations: the flat index directly constructed on the split text segment needs to traverse all the text segments for searching, resulting in a large number of candidate sets, which has a negative impact on the system throughput; there is no directory structure to determine the search range, and the relevance calculation is easily affected by misleading information in the scattered paragraphs, which contain seemingly relevant local context but are not the correct answer; a single search engine cannot balance the search accuracy and the generalization ability of out-of-domain information. SUMMARY

[0010] In view of the deficiencies and problems in the prior art, the present application provides a knowledge graph-based search enhanced generation system and method with a mixed search of a multi-layer search index with a graph structure semantic information as a search directory structure, to overcome the technical problems of the prior art that the accuracy is reduced due to misleading in scattered paragraphs, the size of the search candidate set after text segment splitting is too large, and it is difficult to balance the search accuracy and generalization ability.

[0011] The application is a retrieval enhancement generation system based on a knowledge graph, which is sequentially connected in the direction of information flow and comprises a data import module, a retrieval database, a retrieval module, and a question and answer module; the question and answer module receives user questions and returns answers to the user by giving the retrieval results and the user questions to a generation model in the question and answer module to generate answers; the feature lies in that a knowledge graph database is connected in parallel with the retrieval database between the data import module and the retrieval module, the module receives import information flow to construct a knowledge graph identical to the retrieval theme, which contains a graph directory structure suitable for the retrieval module; the retrieval database is a three-layer retrieval index architecture suitable for the retrieval module, and the input is import information flow; the knowledge graph database outputs graph structure semantic information to the retrieval module, and the retrieval database outputs three-layer retrieval index information to the retrieval module, which are logically combined in the retrieval module to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as a directory structure;

[0012] The retrieval module is a multi-layer hybrid retriever, which is a hierarchical hybrid retrieval architecture retrieval model based on a knowledge graph; the retrieval is performed on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as a directory structure, and the structure of the multi-layer hybrid retriever is sequentially an index entity retrieval layer, an ordinary entity retrieval layer, an attribute document retrieval layer, and an information selection layer for selection according to an output length threshold, and finally enhanced information is obtained and given to the question and answer module.

[0013] The application is also a retrieval enhancement generation method based on a knowledge graph, which is run on the above-mentioned retrieval enhancement generation system based on a knowledge graph; the feature lies in that a knowledge graph database is additionally provided, the retrieval module is a multi-layer hybrid retrieval architecture retrieval model based on a knowledge graph, and the method comprises the following steps:

[0014] S1: constructing a retrieval enhancement generation system based on a knowledge graph: sequentially connecting in the direction of information flow and comprising a data import module, a retrieval database, a retrieval module, and a question and answer module; the question and answer module receives user questions and returns answers to the user by giving the retrieval results and the user questions to a generation model in the question and answer module to generate answers; a knowledge graph database is connected in parallel with the retrieval database between the data import module and the retrieval module, the module receives import information flow to construct a knowledge graph identical to the retrieval theme, which contains a graph directory structure suitable for the retrieval module; the retrieval database is a three-layer retrieval index architecture suitable for the retrieval module, and the input is import information flow; the knowledge graph database outputs graph structure semantic information to the retrieval module, and the retrieval database outputs three-layer retrieval index information to the retrieval module, which are logically combined in the retrieval module to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as a directory structure;

[0015] The retrieval module is a multi-layer hybrid retriever, which is a hierarchical hybrid retrieval architecture retrieval model based on a knowledge graph; retrieval is performed on a multi-layer retrieval index with a directory structure of graph structure semantic information of the knowledge graph, and the structure of the multi-layer hybrid retriever is, in turn, an index entity retrieval layer, a general entity retrieval layer, an attribute document retrieval layer, and an information selection layer for selection according to an output length threshold, and finally enhanced information is obtained and given to a question and answer module.

[0016] S2: Constructing a retrieval index containing graph structure information: performing ontology analysis on existing structured information corresponding to the graph structure and importing the knowledge graph database to construct a knowledge graph to form a three-layer architecture composed of an index entity, a general entity, and an attribute document graph directory structure, and then layering the graph structure information in the knowledge graph database into a retrieval database to construct a corresponding three-layer retrieval index containing graph structure information;

[0017] S3: The question and answer module receives a question: the question and answer module receives a user question through user interaction and gives the user question to the retrieval module.

[0018] S4: Perform hybrid hierarchical retrieval to obtain enhanced information: according to the user question obtained in step S3, generate a sparse bag-of-words vector and a dense semantic vector of the user question as the basis for retrieval, perform hybrid hierarchical retrieval according to the graph structure information and the semantic information on the multi-layer retrieval index with the directory structure of the graph structure semantic information of the knowledge graph, and perform dense retrieval for the index entity through the hybrid hierarchical retriever.

[0019] S5: Generate an answer by combining enhanced information with stronger relevance: input the enhanced information with stronger relevance obtained in step S4 into the large language generation model in the question and answer module by merging it with the user question through a prompt method, generate a corresponding answer and return it to the user.

[0020] The present application solves the problem of excessive size of the candidate set of the flat retrieval architecture and the problem of being easily affected by scattered paragraphs, and also solves the problem of the single retrieval method being unable to balance retrieval accuracy and generalization ability.

[0021] Compared with the prior art, the technical advantages of the present application are:

[0022] The added graph directory structure improves the retrieval accuracy: the present application uses a knowledge graph as a retrieval directory structure, adds graph structure semantic information to the traditional retrieval index based on text semantic information, and adds a graph directory structure, which enriches the semantic information and index structure of the retrieval index and improves the retrieval accuracy.

[0023] The three-layer architecture hierarchical retrieval greatly reduces the influence of retrieval data volume and misleading information: the three-layer retrieval index architecture based on the knowledge graph is adopted, the index entity, the general entity and the attribute document in the knowledge graph are corresponded, the retrieval index architecture is divided into three layers composed of the index entity retrieval index, the general entity retrieval index and the attribute document retrieval index, the retrieval is performed from top to bottom in layers, the candidate set range is trimmed through the graph structure semantic information of the knowledge graph, the candidate set scale of each layer and the total candidate set scale are greatly reduced, the influence of the scattered paragraph is reduced, and the retrieval efficiency and accuracy are improved.

[0024] The mixed retrieval is adopted to consider the generalization ability without reducing the retrieval accuracy: the mixed retrieval combining the sparse retrieval and the dense retrieval is adopted, the sparse retrieval and the dense retrieval are respectively performed for the general entity layer and the attribute document layer with complex retrieval environment, and the final result is obtained through the deduplication fusion of the results in the form of the dense retrieval result in front and the sparse retrieval result behind the zipper type merging method, so that the advantages of the two retrieval methods are complementary, the generalization ability to the out-of-domain information is improved without affecting the accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 the method flowchart of the present application is shown in the figure;

[0026] Figure 2 the system architecture schematic diagram of the present application is shown in the figure;

[0027] Figure 3 the index entity retrieval index construction method schematic diagram of the present application is shown in the figure;

[0028] Figure 4 the sparse retriever architecture principle schematic diagram of the present application is shown in the figure;

[0029] Figure 5 the dense retriever architecture principle schematic diagram of the present application is shown in the figure;

[0030] Figure 6 the enhanced information selection algorithm principle schematic diagram of the present application is shown in the figure. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0032] Embodiment 1:

[0033] Artificial intelligence technology is rapidly integrated into various fields, injecting unprecedented vitality into human production and life. With the development of advanced technologies such as deep learning, artificial intelligence can receive, analyze, and process various types and modalities of information and generate value, thus playing a key role in various aspects. Information retrieval and text generation technologies continue to evolve and integrate, enabling better understanding and generation of various types of text. Based on this, intelligent assistants, chat robots, sentiment analysis, and machine translation have emerged, providing new possibilities for human production and life.

[0034] Currently, the traditional scheme of retrieval augmented generation is as follows: first, the plan is taken as the external knowledge source, and the text segments are blocked according to the input threshold of the semantic embedding model and the syntactic structure, forming an external knowledge set with text blocks as the basic unit. After text embedding processing of these knowledge, the external knowledge set is imported into the database, and a retrieval index is established on the embedding vectors of all text blocks to form an external knowledge retrieval database for retrieval. In practical application, due to the text splitting forming a quite large external knowledge base, all these data need to be traversed and the most relevant content is retrieved, which often leads to the need for sharding operation or data compression of the retrieval database to ensure that the retrieval database can support such retrieval operations. However, this way, on the one hand, it increases the storage cost, and on the other hand, it also leads to a decrease in retrieval speed, thereby causing a bottleneck in the throughput of the entire retrieval augmented generation system. For some applications that require strict response speed, such as intelligent customer service, code completion, and machine translation, the system may not be able to perform well, greatly damaging the applicable field of the system. In addition, the simple text segmentation strategy and one-time retrieval architecture also make the text content lose the directory structure and context relationship, leading to the fact that in some cases, although a certain segment accurately mentions the retrieval target and looks very similar, the meaningful content for the retrieval target may exist in another segment around it. Since the text blocks lose the directory structure and context after text segment splitting, it is impossible to determine these relationships during retrieval, thus leading to the fact that the retrieval model is easily misled by scattered paragraphs, and then selects the segment that looks more relevant but does not actually contain the answer, reducing the accuracy of retrieval. In addition, the single retrieval method also has a big defect, and sparse retrieval performs poorly in complex retrieval environments, while dense retrieval cannot adapt to out-of-domain knowledge and lacks generalization ability. Therefore, once the text is complex and there is a large amount of out-of-domain knowledge that is less touched by embedding models, such as some cutting-edge knowledge base, the retrieval model may confuse the content and be misled by scattered paragraphs, greatly reducing the accuracy of retrieval, making the current mainstream retrieval augmented generation model unable to be applied to some very specialized fields.

[0035] In view of the above problems, the present application proposes a knowledge graph-based retrieval enhancement generation system and method with a multi-layer hybrid architecture with graph structure semantic information as a retrieval directory structure through research and practice.

[0036] The present application is first a knowledge graph-based retrieval enhancement generation system, which is sequentially connected in the direction of information flow: a data import module, a retrieval database, a retrieval module, and a question and answer module. The data import module receives structured external knowledge and outputs import information flow, the retrieval database receives the import information flow and provides retrieval indexes, the retrieval module receives retrieval basis, i.e., user questions, and outputs retrieval results, and the question and answer module accepts user questions and returns answers to users by giving the retrieval results and user questions to a generation model in the question and answer module to generate answers. The generation model in the question and answer module includes a large language generation model and a traditional small-scale generation model. Figure 2 As shown in the figure, Figure 2 is a system architecture diagram of the present application, and between the data import module and the retrieval module, a knowledge graph database is connected in parallel with the retrieval database. This module receives import information flow to construct and form a knowledge graph with the same topic as retrieval, which contains a graph directory structure suitable for the retrieval module. The retrieval database of the present application is a three-layer retrieval index architecture suitable for the retrieval module, the input is import information flow, the knowledge graph database outputs graph structure semantic information to the retrieval module, and the retrieval database outputs three-layer retrieval index information to the retrieval module. In the retrieval module, the two are logically combined to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as a directory structure for the retrieval module to perform retrieval thereon, thereby reducing the influence of scattered paragraphs and reducing the number of candidate sets, improving accuracy and throughput.

[0037] The retrieval module of the present application is a multi-layer hybrid retriever, which is a knowledge graph-based hierarchical hybrid retrieval architecture retrieval model. It performs retrieval on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as a directory structure. The structure of the multi-layer hybrid retriever is, in order, an index entity retrieval layer, a general entity retrieval layer, an attribute document retrieval layer, and an information selection layer that selects according to an output length threshold, and finally obtains enhanced information and transmits it to the question and answer module to enhance the generation model.

[0038] The traditional scheme directly constructs a tiled retrieval index on the split text segments, lacks a directory structure, and based on the index, all contents can only be retrieved at one time, thereby causing the retrieval candidate set to be too large and susceptible to scattered paragraphs. The application adds a graph directory structure to the retrieval index by constructing a knowledge graph, so that each text segment adds the context relationship provided by the graph directory structure, retains more semantic information and connections for reference during retrieval. Based on the graph directory structure, the application uses a three-layer retrieval architecture, each layer corresponding to a level in the directory, each layer of retrieval being performed only on the information of the corresponding directory level, and each layer of retrieval delimiting the range according to the retrieval result of the previous layer, and the candidate set range being trimmed layer by layer, greatly reducing the candidate set size of each layer of retrieval and the total candidate set size, and ensuring that the retrieval range of the next layer does not deviate from the relevant range delimited by the retrieval result of the previous layer, thereby reducing the influence of scattered paragraphs and improving retrieval efficiency and accuracy.

[0039] Embodiment 2:

[0040] The retrieval enhancement generation system based on the knowledge graph is the same as in Embodiment 1, and the knowledge graph database of the application provides a graph directory structure formed by a three-layer architecture of index entities, ordinary entities, and attribute documents for the retrieval module to perform hierarchical retrieval. The ordinary entity is a non-index entity, and the attribute document is an attribute of the ordinary entity.

[0041] The knowledge graph database of the application contains entities, relationships, and corresponding attributes corresponding to structured information, wherein the entities include index entities and ordinary entities, the relationships are the relationships between the index entities and the ordinary entities, and the attributes are the attached information of the entities. Referring to Figure 2 Among them, there are index entities, which are entities containing only names and no other attributes, and all non-index entities in the knowledge graph database must have a direct relationship with at least one index entity.

[0042] The relationship between the index entity and the ordinary entity in the application is an abstraction of a certain feature of the ordinary entity. Taking the case of an article as an ordinary entity, the index entities related to the ordinary entity can be the publication time, keywords, author, publisher, and word count. The purpose is to represent part of the features of the ordinary entity related to the index entity through the index entity, and to form the topmost structure of the graph directory structure based on this to facilitate retrieval.

[0043] A knowledge graph is a structured knowledge representation form used to describe the relationships between entities. It is a graph data model composed of nodes and edges, where nodes represent entities and edges represent relationships between entities. Based on the characteristics of the knowledge graph, referring to Figure 3 , Figure 3The index entity retrieval index construction method is used for retrieving the index entity of the application, the upper left of the figure is a knowledge graph, which is used as a graphical directory structure in the application, which has more dimensions than the common two-dimensional directory, and can store and organize information in a form that machines can understand to assist retrieval.

[0044] Embodiment 3:

[0045] The retrieval enhancement generation system based on the knowledge graph is the same as embodiments 1-2, the retrieval module of the application is a multi-layer hybrid retriever, see Figure 2 , which is sequentially connected with four layers: index entity retrieval layer, ordinary entity retrieval layer, attribute document retrieval layer, and information selection layer.

[0046] The index entity retrieval layer receives the user question and generates the corresponding dense semantic vector as the retrieval basis and outputs the relevant subgraph: on the index entity layer of the retrieval database, the dense semantic vector is retrieved to obtain the relevant index entity, and the relevant subgraph formed by the ordinary entity related to the index entity is further queried from the knowledge graph. The content of the index entity in the application is short and highly abstract, the retrieval of the index entity can first define the approximate range of the retrieval, reduce the number of candidate sets of the next layer of retrieval, and exclude irrelevant content, thereby improving the retrieval efficiency and accuracy.

[0047] The ordinary entity retrieval layer receives the relevant ordinary entity subgraph retrieved by the index entity retrieval layer and the user question and outputs the relevant subgraph: the dense semantic vector and the sparse bag-of-words vector generated by the user question are used as the retrieval basis, and after the retrieval range is defined on the ordinary entity layer of the retrieval database according to the input relevant ordinary entity subgraph, further relevant ordinary entity subgraphs are obtained through hybrid retrieval. After the exclusion of the index entity related subgraph in the application, the retrieval of the relevant subgraph is performed again according to the title or name of the ordinary entity, so as to further define the retrieval range in the attribute document with the most number and the most complex content, reduce the number of candidate sets of the next layer of retrieval, and exclude irrelevant content, thereby improving the retrieval efficiency and accuracy.

[0048] The attribute document retrieval layer receives the relevant ordinary entity subgraph retrieved by the ordinary entity retrieval layer, i.e., the output of the ordinary entity retrieval layer, and the corresponding dense semantic vector and sparse bag-of-words vector of the user question, and outputs the final retrieval result: the corresponding dense semantic vector and sparse bag-of-words vector of the input user question are used as the retrieval basis, and after the retrieval range is defined on the attribute document layer of the retrieval database according to the input relevant ordinary entity subgraph, the relevant specific attribute document retrieval result is obtained through hybrid retrieval. After the previous two layers of retrieval, the number of attribute documents to be retrieved in this layer is greatly reduced, thereby greatly improving the retrieval efficiency, and since a large number of scattered paragraphs are excluded, there is a greater probability of noticing the paragraph containing relevant information in the retrieval process, thereby improving the retrieval efficiency and accuracy.

[0049] The information selection layer receives the retrieval results of the attribute document retrieval layer, selects and outputs the final enhanced information: using a knapsack algorithm, according to the length threshold of the enhanced information of the system question answering module, the retrieval results are selected and discarded in order of relevance ranking from high to low, if the remaining length is sufficient to accommodate the information, it is retained, otherwise it is discarded, until all the retrieval results are selected and discarded, and the final enhanced information is obtained. Since the generation model has a length threshold for input content, once the threshold is exceeded, the generation model will cause errors or performance degradation, and all retrieval results need to be screened to ensure that the input content does not exceed the threshold.

[0050] The present application builds a multi-layer hybrid retrieval architecture, which focuses on the most relevant information and excludes less relevant information. In the complex retrieval environment of the ordinary entity layer and the attribute document layer, the hybrid retrieval method is used to ensure a certain generalization ability without reducing the retrieval accuracy, so that the retrieved enhanced information is more accurate and has a greater probability of containing relevant information, which is beneficial to the generation model in the subsequent question answering module to generate more valuable answers. The present application provides a three-layer retrieval environment division to focus on the complex retrieval environment of the ordinary entity and the attribute document, and provides an architectural basis for the hybrid retrieval.

[0051] Embodiment 4:

[0052] The retrieval enhancement generation system based on the knowledge graph is the same as embodiments 1-3, see Figure 2 The retrieval database of the present application corresponds to the retrieval levels of the multi-layer hybrid retriever and is divided into three layers, including:

[0053] The index entity retrieval index layer, see Figure 3 , only contains retrieval information of index entities, including index entity names, name embedding vectors, and unique primary keys of the entities in the knowledge graph, wherein vector similarity index, i.e. dense index, is established on the name embedding vectors.

[0054] The ordinary entity retrieval index layer only contains retrieval information of ordinary entities, including ordinary entity names, name embedding vectors, and unique primary keys of the entities in the knowledge graph, wherein full-text index, i.e. sparse index, is established on the entity names, vector similarity index, i.e. dense index, is established on the name embedding vectors, and simple matching index is established on the unique primary keys for range division.

[0055] The attribute document retrieval index layer only contains retrieval information of attribute documents, including unique primary keys of the attribute documents in the knowledge graph, attribute names, specific contents, and specific content embedding vectors, wherein full-text index, i.e. sparse index, is established on the specific contents, vector similarity index, i.e. dense index, is established on the specific content embedding vectors, and simple matching index is established on the unique primary keys of the attribute documents for range division.

[0056] To adapt to the retrieval model with three-layer architecture, the retrieval database of the present application is also divided into corresponding three layers according to the retrieval model to store the information of each layer and build corresponding indexes on the information for the retrieval model to use according to the needs of the retrieval model.

[0057] Embodiment 5:

[0058] The present application is also a knowledge graph-based retrieval enhancement generation method running on the above-mentioned knowledge graph-based retrieval enhancement generation system. Referring to Figure 1 , Figure 1 The present application is a method flowchart, including the following steps:

[0059] S1: Construct a knowledge graph-based retrieval enhancement generation system: sequentially connected in the information flow direction are a data import module, a retrieval database, a retrieval module, and a question and answer module. The question and answer module accepts user questions and gives the retrieval results and user questions to a generation model to generate answers and return to the user, as shown in Figure 2 The knowledge graph database is connected in parallel with the retrieval database between the data import module and the retrieval module. This module receives the imported information flow to construct and form a knowledge graph with the same theme as the retrieval, which contains a graph directory structure suitable for the retrieval module. The retrieval database of the present application is a three-layer retrieval index architecture suitable for the retrieval module. The input is the imported information flow, and the knowledge graph database outputs the graph structure semantic information to the retrieval module. The retrieval database outputs the three-layer retrieval index information to the retrieval module. In the retrieval module, the two are logically combined to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure for the retrieval module to perform retrieval thereon. The retrieval module of the present application is a multi-layer hybrid retriever, which is a hierarchical hybrid retrieval architecture retrieval model based on the knowledge graph. It performs retrieval on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure. The structure of the multi-layer hybrid retriever is, in order, an index entity retrieval layer, a general entity retrieval layer, an attribute document retrieval layer, and an information selection layer that selects according to the output length threshold. Finally, enhanced information is obtained and transmitted to the question and answer module to enhance the generation model.

[0060] S2: Construct a retrieval index containing graph structure information: perform ontology analysis on the existing structured information corresponding to the graph structure and import it into the knowledge graph database to construct the knowledge graph, form a graph directory structure composed of an index entity, a general entity, and an attribute document three-layer architecture, and then layer-by-layer import the graph structure information in the knowledge graph database into the retrieval database to construct the corresponding three-layer retrieval index containing graph structure information.

[0061] Further, the knowledge graph database of the present application contains entities, relations and corresponding attributes corresponding to structured information, wherein there are index entities, which are entities containing only names and no other attributes, and all non-index entities in the knowledge graph database must have a direct relationship with at least one index entity.

[0062] The S3 question and answer module receives a question: the question and answer module receives a user question to be answered, i.e., a user question, through user interaction, and submits the user question to the retrieval module.

[0063] S4 performs hybrid hierarchical retrieval to obtain enhanced information: according to the user question obtained in step S3, a sparse bag-of-words vector and a dense semantic vector of the user question are generated as the basis for retrieval, and a hybrid hierarchical retrieval according to the graph structure information and the semantic information is performed on a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure, and through the hybrid hierarchical retrieval according to the graph structure information and the semantic information, a dense retrieval for index entities, a hybrid retrieval for ordinary entities, and a hybrid retrieval for attribute documents are sequentially performed by a hybrid hierarchical retriever, and finally more relevant enhanced information is obtained.

[0064] Further, the hybrid retrieval of the present application is a retrieval method that uses sparse retrieval and dense retrieval to retrieve on corresponding sparse index or dense index respectively, and then uses a zip-like merging method to remove and fuse the final retrieval result with the dense retrieval result first and the sparse retrieval result second. After completing the retrieval, see Figure 6 , Figure 6 The principle of the enhanced information selection algorithm of the present application is shown in the figure, the retrieval result needs to be selected according to the generation model input threshold value, and the knapsack algorithm is used for selection, the retrieval result is traversed in order from high to low relevance, if the remaining length is sufficient to accommodate the result, it is adopted, otherwise it is discarded, and the processed result is the enhanced information.

[0065] S5 combines the more relevant enhanced information to generate an answer: the more relevant enhanced information obtained in step S4 is combined with the user question through a prompt method and input into the large language generation model in the question and answer module, a corresponding answer is generated and returned to the user.

[0066] The traditional scheme directly retrieves the final result in the entire text segment at one time, which results in a large retrieval candidate set and is easily affected by scattered paragraphs. The present application introduces a knowledge graph-based graph directory, based on the graph directory structure, the present application adopts a three-layer retrieval method, each layer only retrieves the content of the corresponding directory level, and the retrieval range of each layer is the next level content related to the retrieval result of the previous layer in the directory structure, which greatly reduces the size of the retrieval candidate set of each layer and the total candidate set, and ensures that the retrieval range of the next layer will not deviate from the relevant range defined by the retrieval result of the previous layer, thereby reducing the influence of scattered paragraphs and improving the retrieval efficiency and accuracy.

[0067] The traditional scheme only adopts one of dense retrieval or sparse retrieval for retrieval, both of which have defects and cannot balance accuracy and generalization ability in a complex retrieval environment, the mixed retrieval method of the present application adopts sparse retrieval and dense retrieval simultaneously in the general entity layer and attribute document layer in a complex retrieval environment, and obtains the final result by fusing and de-duplicating the results in a zippered merging method of dense retrieval results first and sparse retrieval results last, so that the advantages of the two retrieval methods are complementary, and the generalization ability to out-of-domain information is improved without affecting the accuracy.

[0068] Embodiment 6:

[0069] The retrieval enhancement generation system and method based on the knowledge graph of embodiments 1-5, in step S4, the mixed hierarchical retrieval method comprises the following steps:

[0070] S41: obtaining a plurality of semantic representation vectors of the user question: performing sentence vector embedding and word segmentation processing on the user question to obtain a sparse bag-of-words vector and a dense semantic vector of the user question.

[0071] The present application can use any corresponding algorithm or model for sentence vector embedding and word segmentation processing, but the algorithm or model used for sentence vector embedding and word segmentation processing must be globally consistent to avoid semantic deviation caused by different algorithms or models and affect the final effect.

[0072] S42: performing dense retrieval on the index entity retrieval layer to obtain a related subgraph: in the index entity retrieval layer of the retrieval module and the index entity retrieval index layer of the retrieval database, taking the dense semantic vector of the user question as the retrieval basis, performing dense retrieval to obtain the top N relevant index entities, where N is usually set to 30, and according to these index entities, further querying all general entities related to these index entities in the knowledge graph database to form a related subgraph.

[0073] S43: performing mixed retrieval containing graph structure information on the general entity retrieval layer to obtain a related subgraph: according to the graph structure information of the related subgraph obtained in step S42, after defining the retrieval range in the general entity retrieval layer of the retrieval database, taking the sparse bag-of-words vector and the dense semantic vector of the user question as the basis, performing mixed retrieval of the related general entities to obtain the top 2K relevant general entities, where K is usually set to 10, and exporting these relevant entities to obtain a related subgraph.

[0074] S44 performs mixed retrieval containing graph structure information in the attribute document retrieval layer to obtain enhanced information: according to the graph structure information of the relevant subgraph obtained in step S43, the mixed retrieval of the relevant specific attribute documents is performed in the attribute document retrieval layer of the retrieval database after the retrieval range is delimited, based on the sparse bag-of-words vector and the dense semantic vector of the user question, to obtain the top 2M specific attribute documents in terms of relevance. Here, M is used to adjust the number of retrieved documents, and M is usually set to 5 because of the input length threshold of the generation model. The specific contents of all the retrieved attribute documents are further derived and integrated into an information list in order of relevance to obtain enhanced information.

[0075] It is worth noting that the cosine similarity algorithm is used to calculate the similarity between the candidate and the question in the dense retrieval used in the above steps, and the BM25 algorithm is used to calculate the similarity between the candidate and the question in the sparse retrieval. As long as the algorithm or model used for sentence vector embedding and word segmentation processing is globally consistent, the retrieval of each step can be adjusted according to the situation to optimize the parameters of the retrieval algorithm.

[0076] S45 selects enhanced information according to the input threshold of the large language model: referring to Figure 6 As shown in the figure, the retrieval results obtained in step S44 are selected by the knapsack algorithm according to the input threshold of the large language generation model. The retrieval results are traversed in order of relevance, and the remaining threshold is sufficient to accommodate the retrieval results. If it cannot be accommodated, it will be discarded. Finally, more relevant enhanced information is obtained.

[0077] In the traditional scheme, one-time retrieval of all text segments will result in a large candidate set, which will further affect the retrieval efficiency. The mixed hierarchical retrieval method of the present application is based on the graph directory structure of the knowledge graph, and all text segments are divided into three layers for retrieval. Each layer is divided into relevant subgraphs in the corresponding retrieval index of the layer according to the retrieval results of the previous layer and the graph directory structure, and retrieval is performed thereon. This greatly reduces the number of candidate sets for each retrieval, and the impact caused by this reduction will further affect the next layer. The most massive candidate set and the heaviest retrieval burden are the lowest layer, i.e. the specific attribute document layer, which is reduced the most. Although it is divided into three layers for retrieval, the total number of candidate sets is still much smaller than that of the traditional scheme under normal circumstances, improving the retrieval efficiency. Since the retrieval of each layer is based on the retrieval results of the previous layer, the influence of a large number of scattered paragraphs is excluded, and the accuracy of the retrieval is improved.

[0078] Example 7:

[0079] The retrieval enhancement generation system and method based on the knowledge graph are the same as embodiments 1-6. The mixed retrieval in steps S43 and S44 is a mixed retrieval method that uses a dense retrieval method for dense semantic vectors and a sparse retrieval method for sparse bag-of-words vectors respectively, and aggregates the retrieval results after simple deduplication. Specifically, the method comprises the following steps:

[0080] A1 performs sparse retrieval based on the input sparse bag-of-words vector to obtain retrieval results: based on the input sparse bag-of-words vector, sparse retrieval based on the BM25 algorithm is performed on the full-text index of the retrieval target, i.e., the sparse index, to obtain sparse retrieval results with a relevance score ranking of k, where k needs to be given according to the use scenario in application, and the default is 20.

[0081] Further, please refer to Figure 4 , Figure 4 which is a schematic diagram of the principle of the sparse retriever architecture of the present application. The sparse retriever of the present application uses the BM25 algorithm for similarity calculation. The BM25 algorithm is improved based on the probabilistic retrieval algorithm and performs well in processing long documents and short queries. Its formula is:

[0082]

[0083] where Score(D, Q) refers to the relevance score of document D and question Q, q i is the i-th word in the question sentence, f(q i , D) is the frequency of word q i in document D, IDF(q i ) is the inverse document frequency of word q i , i.e., the inverse of the document frequency, |D| is the length of document D, avgdl is the average length of all indexed documents, k1 and b are adjustable parameters, where k1 is a positive coefficient for controlling the saturation degree of word frequency, a higher k1 value means that word frequency has a greater impact on the score, and k1 is usually between 1.2 and 2. b is used to control the impact of document length on the score, a higher b value means that the document length has a greater impact, and b ranges from 0 to 1, and b is usually set to 0.75.

[0084] A2 performs dense retrieval based on the input dense semantic vector to obtain retrieval results: based on the input dense embedding vector, dense retrieval based on cosine similarity is performed on the vector index of the retrieval target, i.e., the dense index, to obtain dense retrieval results with a relevance score ranking of k, where k needs to be given according to the use scenario in application, and the default is 20.

[0085] Further, please refer to Figure 5 , Figure 5For the schematic diagram of the principle of the dense retriever architecture of the present application, the dense retriever of the present application uses a cosine similarity algorithm to calculate the similarity, the cosine similarity algorithm is used to measure the semantic similarity degree of two texts a and b, and the formula is:

[0086]

[0087] Wherein, A is the semantic vector of the text a, B is the semantic vector of the text b, A i is the i-th element of the vector A, B i is the i-th element of the vector B.

[0088] A3 Aggregates the results of sparse retrieval and dense retrieval to obtain a hybrid retrieval result: the sparse retrieval result and the dense retrieval result obtained in steps A1 and A2 are combined in a staggered interlocking method with the dense retrieval result in front and the sparse retrieval result behind, and the two retrieval results are de-duplicated and fused according to the relevance ranking order to aggregate the final hybrid retrieval result, the number of which is between k and 2k.

[0089] Since a single retrieval method cannot overcome its own defects, the present application uses two methods for retrieval and aggregates the results, which can balance the advantages and disadvantages of the two methods to a great extent, the interlocking combination method with the dense retrieval result in front and the sparse retrieval result behind makes the more accurate dense retrieval result still have a greater weight, but the sparse retrieval with strong generalization ability and excellent robustness can also provide better performance on out-of-domain data, so that the generalization ability is considered without reducing the retrieval accuracy.

[0090] The present application mainly relates to the field of information retrieval and text generation, and is used to solve the problems of low retrieval efficiency and being easily misled by scattered paragraphs caused by the large scale of the retrieval candidate set of the existing retrieval enhancement generation method, and the problem that a single retrieval method cannot balance the retrieval accuracy and generalization ability. Specifically, the retrieval enhancement generation system and method based on a knowledge graph sequentially connect the data import module, the retrieval database, the retrieval module and the question and answer module according to the information flow direction, and the knowledge graph database is connected in parallel with the retrieval database between the data import module and the retrieval module. The present application performs a hybrid hierarchical retrieval combining the graph structure information and the semantic information on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure, greatly reduces the size of the candidate set, reduces the influence of scattered paragraphs, and balances the retrieval accuracy and generalization ability.

[0091] Next, an example of simulation is given to further illustrate the present application in combination with the technical effects.

[0092] Example 8:

[0093] The retrieval enhancement generation system and method based on the knowledge graph are the same as examples 1-7.

[0094] The application is a knowledge graph-based retrieval enhancement generation system. Referring to Figure 2 , comprising a data import module, a knowledge graph database, a retrieval database, a retrieval module and a question and answer module. The data import module is used for parsing and splitting structured data, and respectively outputting import data streams for constructing a knowledge graph and constructing a retrieval index. The knowledge graph database is used for storing knowledge graph information and providing a graph directory structure for the retrieval module to perform hierarchical mixed retrieval. The retrieval database is used for storing information at each level and constructing layered indexes thereon for the retrieval module to perform hierarchical mixed retrieval. The retrieval module is used for performing mixed hierarchical retrieval of enhanced information on the multi-layer retrieval index formed by logically combining the graph directory structure and the layered index according to the user question. The question and answer module is used to accept user questions and give the enhanced information and the user question to a generation model to generate an answer and return it to the user.

[0095] The application is also a knowledge graph-based retrieval enhancement generation method, which runs on the above-mentioned knowledge graph-based retrieval enhancement generation system. Referring to Figure 1 , comprising the following steps:

[0096] S1 constructs a knowledge graph-based retrieval enhancement generation system.

[0097] S2 constructs a retrieval index containing graph structure information: the existing structured information is subjected to graph structure ontology construction and imported into the knowledge graph database to construct a knowledge graph, forming a graph directory structure composed of three layers of index entities, general entities and attribute documents, and then the graph structure information in the knowledge graph database is layered imported into the retrieval database to construct the corresponding three-layer retrieval index.

[0098] Further, the knowledge graph database contains entities, relationships and corresponding attributes corresponding to structured information, wherein there are index entities, and the index entity is an entity containing only a name and no other attributes, and all non-index entities in the knowledge graph database must have a direct relationship with at least one index entity.

[0099] Specifically, the knowledge graph construction includes three steps:

[0100] Ontology construction is performed: first, entities are distinguished according to content, which is usually a text chapter or a topic, and then the internal attribute types of various entities are analyzed, including small sections or small word entries under the topic. This embodiment takes the Beijing City word entry information of Baidu Encyclopedia as an example, the entity types include Beijing history, Beijing geography, Beijing climate, etc., and the internal attributes take Beijing history as an example, including establishment evolution and capital dynasty.

[0101] Analysis of index entities: Index entities are entities that only contain names and do not contain other attributes. All non-index entities in the knowledge graph must have a direct relationship with at least one index entity. Index entities can be specified by storing meta-information, i.e. information describing attributes, during the construction of the knowledge graph, or can be obtained by abstracting and stripping from original entities after the construction of the knowledge graph. In this embodiment, for entities such as Beijing history, Beijing geography, and Beijing climate, index entities such as Baidu Baike and city information can be obtained according to the meta-information, and index entities such as Beijing, history, geography, and climate can be obtained by abstraction.

[0102] Importing database: The structured data corresponding to the ontology structure is imported into the graph database to construct the knowledge graph. In this embodiment, the above index entities are imported into the graph database in the form of nodes containing only names, and the remaining ordinary entities are imported into the graph database in the form of nodes with names as corresponding name information and attributes as internal information. When importing ordinary entities, nodes and relationship connections between nodes are created according to the specific relationship between the index entities, such as "Beijing history-keyword-Beijing", "Beijing history-keyword-history", and "Beijing history-information source-Baidu Baike".

[0103] Specifically, the construction of the retrieval index includes three steps: constructing an index entity retrieval index, constructing an ordinary entity retrieval index, and constructing an attribute document retrieval index.

[0104] Constructing an index entity retrieval index, referring to Figure 3 The index contains the following three types of information: index entity name, embedded vector of index entity retrieval semantic name, and unique primary key of index entity in the knowledge graph. The construction steps are as follows: exporting index entity information, semantic embedding, importing database, and creating index:

[0105] Exporting index entity information: From the constructed knowledge graph, the specific information of all index entities is exported through the graph database interface. In this embodiment, it includes Baidu Baike, city information, Beijing, history, geography, and climate. These entities only contain their names.

[0106] Semantic embedding: Using a semantic representation learning model, the name of the index entity is vector-embedded.

[0107] Importing database: Importing the name and semantic vector of the index entity into the retrieval database.

[0108] Creating index: Generating a vector index, i.e. a dense index, for the embedded vector of the index entity retrieval semantic name, and generating a simple matching index for the unique primary key of the index entity in the knowledge graph.

[0109] Build a common entity retrieval index, which contains the following three kinds of information: common entity name, the embedding vector of the common entity name, and the unique primary key of the common entity in the knowledge graph. The construction steps are as follows: common entity information, semantic embedding, database import, index creation:

[0110] Export common entity information, and export the detailed information of all common entities from the constructed knowledge graph through the graph database interface.

[0111] Semantic embedding: use semantic representation learning model to perform vector embedding on the name of the common entity.

[0112] Import database: import the unique primary key of the common entity knowledge graph, the name, and the embedding vector of the name into the retrieval database.

[0113] Create index: generate full-text index for common entity name, i.e. sparse index, generate vector index for embedding vector of common entity name, i.e. dense index, and generate simple matching index for unique primary key of common entity in knowledge graph.

[0114] Build attribute document retrieval index, which contains the following four kinds of information: the unique primary key of the attribute document belonging to the common entity in the knowledge graph, the attribute name of the attribute document, the specific content of the attribute document, and the embedding vector of the specific content of the attribute document. The construction steps are as follows: export attribute document information, text segment splitting, semantic embedding, database import, index creation

[0115] Export attribute document information, and export the detailed information of all common entities from the constructed knowledge graph through the graph database interface.

[0116] Text segment splitting: check whether the length of all attribute documents is within the input threshold of the semantic representation learning model. If it exceeds, perform text segment splitting with the threshold length as the maximum limit while maintaining the syntactic structure.

[0117] Semantic embedding: use semantic representation learning model to perform vector embedding on the specific attribute document.

[0118] Import database: import the unique primary key of the attribute belonging to the common entity in the knowledge graph, the attribute name, the specific document content, and the semantic embedding vector of the specific document content into the retrieval database.

[0119] Create index: generate full-text index for the specific content of the attribute document, i.e. sparse index, generate vector index for the embedding vector of the specific content of the attribute document, i.e. dense index, and generate simple matching index for the unique primary key of the common entity to which the attribute document belongs in the knowledge graph.

[0120] The database supporting the construction of the full-text index and the vector index can be used for construction, or the vector database and the full-text index database can be used for construction respectively, and the steps of the two schemes are not different, but the scheme of separate construction needs to be operated in different databases, and in the embodiment, the ElasticSearch supporting the two indexes is used as the retrieval database.

[0121] The semantic embedding can be performed by using any semantic representation learning model, but the same semantic representation learning model must be used globally to avoid semantic deviation caused by different semantic representation learning models, and in the embodiment, the Microsoft E5 semantic learning model is used for semantic embedding.

[0122] S3 receives the question: receiving the user question by the user interaction function of the question and answer module, and delivering the question to the retrieval module.

[0123] Specifically, step S3 can have multiple receiving methods, and in the embodiment, a conventional scheme is used, that is, a web service is built for the whole retrieval enhancement generation system, an input box or a chat box is used to collect the user question input, and the user question input is collected to the server where the retrieval enhancement generation system is located through the network, and then the user question is obtained.

[0124] S4 performs the mixed hierarchical retrieval according to the graph structure information and the semantic information to obtain the enhanced information: according to the user question obtained in S3, the sparse bag-of-words vector and the dense semantic vector of the user question are generated as the basis for retrieval, and on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure, the dense retrieval of the index entity, the mixed retrieval of the general entity and the mixed retrieval of the attribute document are sequentially performed by the multi-layer mixed retriever, and finally the enhanced information is obtained.

[0125] Further, the mixed retrieval is a retrieval method in which the sparse retrieval and the dense retrieval are used to retrieve on the corresponding sparse index or dense index respectively, and then the final retrieval result is obtained by using the zippered merging method of the dense retrieval result first and the sparse retrieval result last.

[0126] Referring to Figure 2 , the mixed hierarchical retrieval sequentially includes semantic representation acquisition, related index entity retrieval, related general entity retrieval, related attribute document retrieval and information selection.

[0127] Semantic representation acquisition: performing sentence vector embedding and word segmentation on the user question to obtain the word segmentation list and the sentence semantic vector of the user input question.

[0128] Related index entity retrieval: according to the sentence vector of the user question, a dense retriever based on the cosine similarity algorithm is used to retrieve the top N entities in terms of entity name relevance score on the index entity retrieval index, and all ordinary entities related to these entities in the knowledge graph are further retrieved to form a related subgraph. N represents the number of keywords that are expected to be retrieved according to the question, which can be set to 20, and is set to 5 in this embodiment for ease of description.

[0129] Since the index entity only contains the name, the retrieval overhead for these entities is low. However, the index entity represents part of the semantics of the ordinary entity related to these entities, and the range of ordinary indexes that need to be retrieved can be determined through retrieval of the entity index, thereby reducing the number of candidate sets in the next layer and excluding the influence of irrelevant entities, thereby improving retrieval efficiency and accuracy. In this embodiment, the semantic vector of the user question "Where is Beijing?" is densely retrieved in the index entity retrieval layer, and the top N=5 related index entities are obtained, including "Beijing", "geography", etc. Further, related ordinary entities related to these index entities are obtained from the graph database, including "Beijing geography", "Beijing scenic spot", "Beijing history", "Beijing climate", etc. These ordinary entities form a related subgraph at this layer.

[0130] Referring to Figure 5 In the dense retrieval process of the present application, the cosine similarity algorithm is used to calculate the similarity of each candidate set to the question. This algorithm can be used to measure the semantic similarity between two texts, and its formula is the same as formula 2 above.

[0131] The same type of retriever at different stages in the present application can be directly reused, or parameters can be set separately. In this embodiment, all sparse retrievers and dense retrievers are directly reused according to the category.

[0132] Related ordinary entity retrieval: This step combines the related subgraph obtained from the index entity, and performs related entity sparse retrieval and related entity dense retrieval on the user question, and aggregates the two retrieval results to obtain the final retrieval result.

[0133] The related ordinary entity sparse retrieval of the present application is to first input the related subgraph retrieved in the second part as an input on the ordinary entity retrieval index, and filter out the corresponding ordinary entity candidate set through simple unique primary key matching. Then, through full-text indexing on the candidate set, sparse retrieval is performed to obtain the top k entities in terms of entity name relevance score.

[0134] The related general entity intensive retrieval of the application is that, on the general entity retrieval index, the relevant subgraph retrieved in the second part is taken as input, and through simple unique primary key matching, the corresponding general entity candidate set is screened out. Then, on the candidate set, the vector index is used, and intensive retrieval is adopted to obtain the entity name relevance score top-k entity.

[0135] The sparse retrieval result and the intensive retrieval result of the related general entity are de-duplicated and merged by using the zippered merging method of intensive retrieval first and sparse retrieval later, to obtain the related general entity set. In this embodiment, the parameter k=5, and no more than 2k=10 related entities are retrieved. Since the sparse retrieval and the intensive retrieval may have duplicate results, the final result quantity is k to 2k, that is, 5 to 10. For example, the intensive retrieval result is ["Beijing geography", "Beijing division"], the sparse retrieval result is ["Beijing hydrology", "Beijing division"], and the merged result is ["Beijing geography", "Beijing hydrology", "Beijing division"]. These general entities form the relevant subgraph of the current layer.

[0136] Referring to Figure 4 In the sparse retrieval process, the BM25 algorithm is used to calculate the similarity of each candidate set and the question. The algorithm is improved on the basis of the probabilistic retrieval framework and performs well when processing long documents and short queries. The formula is the same as formula 1 above. In this example, the parameters of all sparse retrievers are set as follows: k1 is set to 1.5, and b is set to 0.75.

[0137] Related attribute document retrieval: This step combines the relevant subgraph obtained in the third part, and respectively performs sparse retrieval of related specific attribute documents and intensive retrieval of related specific attribute documents according to the word segmentation list and sentence vector embedding of the user question. The results are aggregated by simple fusion to obtain the final retrieval result.

[0138] The related attribute document sparse retrieval of the application is that, on the attribute document retrieval index, the relevant subgraph retrieved in the third part is taken as input, and through simple unique primary key matching, the corresponding specific document candidate set is screened out. Then, on the candidate set, the full-text index is used, and sparse retrieval is adopted to obtain the specific document relevance score top-m attribute document, that is, the sparse retrieval result.

[0139] The related attribute document intensive retrieval of the application is that, on the attribute document retrieval index, the relevant subgraph retrieved in the third part is taken as input, and through simple unique primary key matching, the corresponding specific document candidate set is screened out. Then, on the candidate set, the vector index is used, and intensive retrieval is adopted to obtain the specific document relevance score top-m attribute document, that is, the intensive retrieval result.

[0140] The sparse retrieval result of the relevant attribute document is combined with the dense retrieval result in a zip-in-zip combination method with the dense retrieval in front and the sparse retrieval in back to obtain a relevant specific attribute document set, that is, a retrieval result. In the embodiment, the parameter m=5, and no more than 2m=10 retrieval results are retrieved. Since the sparse retrieval and the dense retrieval can be repeated, the final result quantity is m to 2m, that is, 5 to 10.

[0141] Information selection: The retrieval result obtained by retrieving the relevant attribute document is subjected to information selection by a knapsack algorithm according to the input threshold of the large language generation model, and the relevant text exceeding the remaining threshold is discarded to finally form enhanced text.

[0142] The information selection of the application is to calculate the available character quantity according to the text quantity of the user question and the set large model input text quantity threshold, and to traverse and process the input retrieval result according to the relevance from high to low. The processing method is to calculate whether the current remaining available character quantity is sufficient to accommodate the information. If it is sufficient, the information is retained. If it is insufficient, the information is discarded. Finally, enhanced information is obtained.

[0143] In the embodiment, referring to Figure 6 It can be seen that the top 4 relevant attributes are “Beijing geography”, “Beijing hydrology”, “Beijing division”, and “Beijing history”. When processing in order, “Beijing geography” and “Beijing hydrology” are adopted due to sufficient remaining length, while “Beijing division” is directly discarded due to insufficient remaining space. Finally, “Beijing history” can be accommodated and adopted. The remaining enhanced information is selected and spliced according to the above rules to finally obtain enhanced information.

[0144] S5 combines the enhanced information with stronger relevance to generate an answer: the enhanced information with stronger relevance obtained in step S4 is input into a large language generation model in a question and answer module after being combined with the user question through a prompt method, and a corresponding answer is generated and returned to the user.

[0145] In the embodiment, referring to Figure 6 Finally, a total of 1936 characters of relevant information composed of “Beijing geography”, “Beijing hydrology”, and “Beijing history” are obtained. These information is integrated with the question by a prompt template to form the final input. As can be seen from the simulation example, the application focuses on target information through a hierarchical retrieval architecture based on a knowledge graph, and excludes a large amount of irrelevant information, thereby improving the retrieval accuracy, greatly reducing the candidate set size, and reducing the retrieval delay. In addition, the application has stronger robustness and better generalization ability while considering the retrieval accuracy through hybrid retrieval.

[0146] In summary, the present application is a kind of retrieval enhancement generation system and method based on knowledge graph, solve the problem of the size of retrieval candidate set is too large and easy to be affected by scattered paragraph under the flat retrieval architecture and the problem of single retrieval method cannot balance retrieval accuracy and generalization ability.The system includes: data import module, retrieval database, retrieval module, question and answer module, the present application is connected with knowledge graph database in parallel between data import module and retrieval module, retrieval database is a three-layer architecture corresponding to retrieval module, retrieval module is a multi-layer hybrid retriever.The present application adds a graph directory structure to assist retrieval, based on the graph directory structure, a multi-layer hybrid retriever with three-layer retrieval architecture is used for retrieval.The method of the present application includes: constructing a retrieval enhancement generation system based on knowledge graph, constructing a retrieval index containing graph structure information, the question and answer module receives a question, performs a hybrid hierarchical retrieval to obtain enhanced information, and generates an answer by combining the enhanced information with stronger relevance.The present application performs hybrid hierarchical retrieval combining graph structure information and semantic information on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure, greatly reduces the size of the candidate set, reduces the influence of scattered paragraphs, and balances the accuracy and generalization ability of retrieval.The present application can be applied to the field of text generation, such as intelligent question answering, sentiment analysis, code generation.

Claims

1. A knowledge graph-based retrieval enhancement generation system, sequentially connected in the direction of information flow: a data import module, a retrieval database, a retrieval module, and a question and answer module; the question and answer module accepts user questions and delivers the retrieval results and the user questions to a generation model in the question and answer module to generate answers and return them to the user, characterized in that, Between the data import module and the retrieval module, a knowledge graph database is connected in parallel with the retrieval database. The module receives the import information stream to construct a knowledge graph with the same topic as the retrieval topic, which contains a graph directory structure suitable for the retrieval module; the retrieval database is a three-layer retrieval index architecture suitable for the retrieval module, and the input is the import information stream; the knowledge graph database outputs the graph structure semantic information to the retrieval module, and the retrieval database outputs the three-layer retrieval index information to the retrieval module. In the retrieval module, the two are logically combined to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure; The retrieval module is a multi-layer hybrid retriever, which is a hierarchical hybrid retrieval architecture retrieval model based on the knowledge graph. The multi-layer hybrid retriever is used for retrieval on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure. The structure of the multi-layer hybrid retriever is, in turn, an index entity retrieval layer, a general entity retrieval layer, an attribute document retrieval layer, and an information selection layer that selects according to the output length threshold. The final enhanced information is obtained and given to the question and answer module; The retrieval module is a multi-layer hybrid retriever, and the multi-layer hybrid retrieval architecture is connected with four layers in turn: an index entity retrieval layer, a general entity retrieval layer, an attribute document retrieval layer, and an information selection layer; The index entity retrieval layer receives the user's problem to be solved and generates the corresponding dense semantic vector as the retrieval basis and outputs the relevant subgraph: on the index entity layer of the retrieval database, the dense semantic vector is used to obtain the relevant index entity through dense retrieval, and the relevant subgraph formed by the general entity related to these index entities is obtained by querying the knowledge graph; The general entity retrieval layer receives the relevant general entity subgraph retrieved by the index entity retrieval layer and the user's problem to be solved and outputs the relevant subgraph: the dense semantic vector and the sparse bag-of-words vector generated by the problem to be solved are used as the retrieval basis. After the retrieval range is defined on the general entity layer of the retrieval database according to the input relevant general entity subgraph, further relevant general entity subgraphs are obtained through hybrid retrieval; The attribute document retrieval layer receives the relevant general entity subgraph retrieved by the general entity retrieval layer and the corresponding dense semantic vector and sparse bag-of-words vector of the user's problem to be solved and outputs the final retrieval result: the corresponding dense semantic vector and sparse bag-of-words vector of the input user's problem to be solved are used as the retrieval basis. After the retrieval range is defined on the attribute document layer of the retrieval database according to the input relevant general entity subgraph, the relevant specific attribute document retrieval result is obtained through hybrid retrieval; The information selection layer receives the retrieval result of the attribute document retrieval layer for selection and outputs the final enhanced information: a knapsack algorithm is used to select according to the ranking of the relevance of the retrieval result from high to low, and if the remaining length is sufficient to accommodate the information, it is retained, otherwise it is discarded. Until all the selection is completed, the final enhanced information is obtained. 2.The knowledge graph based retrieval augmentation generation system of claim 1, wherein, The knowledge graph database provides the retrieval module with a graph directory structure formed by the index entity, general entity, and attribute document three-layer architecture for hierarchical retrieval; The knowledge graph database contains entities, relations and corresponding attributes corresponding to structured information, wherein there are index entities, which are entities containing only names and no other attributes, and all non-index entities in the knowledge graph database must have a direct relationship with at least one index entity. The ordinary entity is a non-index entity. 3.The knowledge graph based retrieval augmentation generation system of claim 1, wherein, The retrieval database corresponds to three layers of retrieval levels of a multi-layer hybrid retriever, including: The index entity retrieval index layer only contains retrieval information of index entities, including index entity names, name embedding vectors, and the unique primary key of the entity in the knowledge graph, wherein vector similarity index, i.e. dense index, is established on the name embedding vector; The ordinary entity retrieval index layer only contains retrieval information of ordinary entities, including ordinary entity names, name embedding vectors, and the unique primary key of the entity in the knowledge graph, wherein full-text index, i.e. sparse index, is established on the entity name, vector similarity index, i.e. dense index, is established on the name embedding vector, and simple matching index is established on the unique primary key for range division; The attribute document retrieval index layer only contains retrieval information of attribute documents, including the unique primary key of the belonging ordinary entity in the knowledge graph, attribute names, specific contents, and specific content embedding vectors, wherein full-text index, i.e. sparse index, is established on the specific content, vector similarity index, i.e. dense index, is established on the specific content embedding vector, and simple matching index is established on the unique primary key of the belonging entity for range division.

4. A knowledge graph-based retrieval augmentation generation method, running on any of the knowledge graph-based retrieval augmentation generation systems of claims 1 to 3, characterized in that, A knowledge graph database is added, and the retrieval module is a multi-layer hybrid retrieval architecture retrieval model based on a knowledge graph, including the following steps: S1 Construct a knowledge graph-based retrieval enhancement generation system: sequentially connected in the order of information flow direction: data import module, retrieval database, retrieval module, and question answering module; the question answering module accepts user questions and delivers the retrieval results and user questions to the generation model in the question answering module to generate answers and return them to the user. Between the data import module and the retrieval module, the knowledge graph database is connected in parallel with the retrieval database. This module receives the import information flow to construct a knowledge graph with the same retrieval theme, which contains a graph directory structure suitable for the retrieval module. The retrieval database is a three-layer retrieval index architecture suitable for the retrieval module, and the input is the import information flow. The knowledge graph database outputs graph structure semantic information to the retrieval module, and the retrieval database outputs three-layer retrieval index information to the retrieval module. In the retrieval module, the two are logically combined to form a multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure; The retrieval module is a multi-layer hybrid retriever, which is a hierarchical hybrid retrieval architecture retrieval model based on a knowledge graph. The retrieval is performed on the multi-layer retrieval index with the graph structure semantic information of the knowledge graph as the directory structure. The structure of the multi-layer hybrid retriever is, in order, the index entity retrieval layer, the ordinary entity retrieval layer, the attribute document retrieval layer, and the information selection layer based on the output length threshold for selection. Finally, the enhanced information is obtained and delivered to the question answering module. S2: Constructing a search index containing graph structure information: The existing structured information is analyzed and imported into a knowledge graph database to construct a knowledge graph, forming a three-layer architecture of index entities, common entities, and attribute documents, and then the graph structure information in the knowledge graph database is layered and imported into a search database to construct a corresponding three-layer search index containing graph structure information; S3: The question and answer module receives a question: The question and answer module receives the user's question through user interaction and passes it to the search module; S4: Perform hybrid hierarchical search to obtain enhanced information: According to the user's question obtained in step S3, generate a sparse bag-of-words vector and a dense semantic vector of the user's question as the basis for searching, and perform hybrid hierarchical search based on graph structure information and semantic information on the multi-layer search index with the graph structure semantic information of the knowledge graph as the directory structure, and perform dense search on the index entities through the hybrid hierarchical search based on graph structure information and semantic information. S5: Combine the enhanced information with stronger relevance to generate an answer: The enhanced information with stronger relevance obtained in step S4 is combined with the user's question through a prompt method and input into the large language generation model in the question and answer module to generate a corresponding answer and return it to the user.

5. The search enhancement generation method of claim 4, wherein, In step S4, the hybrid hierarchical search method includes the following steps: S41: Obtain multiple semantic representation vectors of the input question: Perform sentence vector embedding and word segmentation on the user's question to obtain a sparse bag-of-words vector and a dense semantic vector of the user's question; S42: Perform dense search on the index entity search layer to obtain a related subgraph: In the index entity search layer of the search module and the index entity search index layer of the search database, use the dense semantic vector of the user's question as the basis for searching, and perform dense search to obtain the top N index entities in terms of relevance, where N is usually set to 30. According to these index entities, further query all common entities related to these index entities in the knowledge graph database, and export these common entities to form a related subgraph; S43: Perform hybrid search containing graph structure information on the common entity search layer to obtain a related subgraph: According to the graph structure information of the related subgraph obtained in step S42, perform hybrid search on the common entity search layer of the search database after defining the search range, and use the sparse bag-of-words vector and the dense semantic vector of the user's question as the basis for searching to obtain the top 2K common entities in terms of relevance, where K is usually set to 10. Export these related entities to obtain a related subgraph. S44 performs mixed retrieval containing graph structure information in the attribute document retrieval layer to obtain enhanced information: according to the graph structure information of the relevant subgraph obtained in step S43, the mixed retrieval of the relevant specific attribute documents is performed in the attribute document retrieval layer of the retrieval database after the retrieval range is delimited, based on the sparse bag-of-words vector and the dense semantic vector of the user question, to obtain the specific attribute documents with the top 2M relevance, where M is usually set to 5, and the specific contents of all the retrieved attribute documents are further derived and integrated into an information list to obtain the retrieval result; S45 selects the enhanced information according to the input threshold of the large language model: for the retrieval result obtained in step S44, the input threshold of the large language generation model is used as a standard to select by using a knapsack algorithm, the retrieval results that can be accommodated are included in order of relevance, and the retrieval results that cannot be accommodated are discarded, and finally the enhanced information with stronger relevance is obtained.

6. The search enhancement generation method of claim 5, wherein, The mixed retrieval in steps S43 and S44 is a mixed retrieval method that uses a dense retrieval method for dense semantic vectors and a sparse retrieval method for sparse bag-of-words vectors to retrieve and aggregate the two results after simple deduplication to obtain the retrieval result, and specifically includes: A1 performs sparse retrieval based on the input sparse bag-of-words vector to obtain retrieval results: based on the input sparse bag-of-words vector, sparse retrieval based on the BM25 algorithm is performed on the full-text index, i.e., the sparse index, of the retrieval target to obtain the sparse retrieval results with the top k relevance scores; A2 performs dense retrieval based on the input dense semantic vector to obtain retrieval results: based on the input dense embedding vector, dense retrieval based on the cosine similarity algorithm is performed on the vector index, i.e., the dense index, of the retrieval target to obtain the dense retrieval results with the top k relevance scores; A3 aggregates the results of sparse retrieval and dense retrieval to obtain mixed retrieval results: the sparse retrieval results and the dense retrieval results retrieved in steps A1 and A2 are interlaced and zipped in the order of the dense retrieval results first and the sparse retrieval results second, the two retrieval results are deduplicated and fused according to the relevance ranking order, and the final mixed retrieval results are aggregated, the number of which is between k and 2k.

Images