Retrieval generation method and device based on large language model and knowledge graph

Abstract

The embodiment of the invention discloses a retrieval generation method and device based on a large language model and a knowledge graph. The problems that retrieval semantic relevance is weak, and recalled data lacks global knowledge can be solved. The method comprises the following steps: performing vector analysis on a plurality of preset domain documents, and constructing a vector database; constructing a knowledge graph according to the entity description information and the relationship description information extracted from the plurality of field documents, and storing the knowledge graph into a graph database; performing feature extraction on a target question input by a user to obtain a multi-dimensional keyword corresponding to the target question; retrieving the multi-dimensional keywords through a preset search engine, a vector database and a graph database to obtain a target retrieval result; and determining target input information according to the target retrieval result, and inputting the target input information into the target language model to obtain target output response information.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of natural language processing technology, and in particular to a retrieval generation method and device based on a large language model and a knowledge graph. Background Art

[0002] With the vigorous development and application of Large Language Model (LLM), LLM has demonstrated powerful capabilities in natural language processing and image processing, but it is slightly insufficient when facing practical application tasks such as hallucinations, new knowledge, and specific private domain knowledge. At present, there is a way to combine LLM with external specific domain information sources, so that LLM can obtain unseen domain context information about specific industries, which improves the accuracy and relevance of LLM responses. However, there are still problems such as unclear understanding of user intentions, weak relevance and correlation between semantic text blocks and query questions, and lack of globality, and inability to understand the complex associations between knowledge points. Summary of the invention

[0003] Based on this, it is necessary to provide a retrieval generation method and device based on a large language model and knowledge graph to address the above technical problems.

[0004] In a first aspect, an embodiment of the present application provides a retrieval generation method based on a large language model and a knowledge graph, wherein the retrieval generation method based on a large language model and a knowledge graph comprises: performing vector parsing on a plurality of preset domain documents and constructing a vector database;

[0005] Constructing the knowledge graph according to the entity description information and the relationship description information extracted from the multiple domain documents, and storing the knowledge graph in a graph database;

[0006] Extract features of the target question input by the user to obtain multi-dimensional keywords corresponding to the target question;

[0007] The multi-dimensional keywords are searched by using a preset search engine, the vector database, and the graph database to obtain target search results;

[0008] According to the target search result, target input information is determined, and the target input information is input into a target language model to obtain target output response information.

[0009] As an optional implementation, in the first aspect of the embodiment of the present application, the performing vector parsing on the preset multiple field documents and constructing the vector database includes:

[0010] Obtaining the multiple field documents;

[0011] According to the file type of each domain document, the plurality of domain documents are converted to obtain a plurality of initial text contents;

[0012] Converting the multiple initial text contents into multiple initial text vectors by using a preset text vector model;

[0013] The vector database is constructed by using the multiple initial text contents and the multiple initial text vectors.

[0014] As an optional implementation, in the first aspect of the embodiment of the present application, before constructing the graph database according to the entity description information and relationship description information extracted from the multiple domain documents, the method further includes:

[0015] Obtaining domain seed data from the plurality of domain documents through a random sampling method, and embedding the data into a prompt instruction template to obtain an initial extraction rule;

[0016] Based on the initial extraction rule, the entity description information and the relationship description information are extracted from the multiple domain documents.

[0017] As an optional implementation, in the first aspect of the embodiment of the present application, after constructing the graph database according to the entity description information and the relationship description information extracted from the multiple domain documents, the method further includes:

[0018] Mining explicit information and implicit information of an entity node according to the entity description information and the relationship description information to obtain summary information corresponding to the entity node;

[0019] The summary information is associated with the attribute of the entity node, and the entity node is vector parsed to obtain an entity node vector, and the entity node vector is stored in the vector database.

[0020] As an optional implementation manner, in the first aspect of the embodiment of the present application, before extracting features of the target question input by the user to obtain the multi-dimensional keywords corresponding to the target question, the method further includes:

[0021] Obtaining an initial question input by the user;

[0022] The initial problem is rewritten and error-corrected to obtain the target problem.

[0023] As an optional implementation, in the first aspect of the embodiment of the present application, searching the multi-dimensional keywords through the preset search engine, the vector database, and the graph database to obtain the target search results includes:

[0024] Searching the multi-dimensional keyword through the preset search engine to obtain a first search result;

[0025] Searching the low-dimensional search keywords in the multi-dimensional keywords through the vector database to obtain a second search result;

[0026] The high-dimensional search keywords in the multi-dimensional keywords are searched through the vector database and the graph database to obtain a third search result.

[0027] As an optional implementation manner, in the first aspect of the embodiment of the present application, determining the target input information according to the target search result includes:

[0028] According to the input text restriction condition of the target language model, the first search result, the second search result and the third search result are cut and spliced ​​to obtain the target input information.

[0029] In a second aspect, an embodiment of the present application provides a retrieval generation device based on a large language model and a knowledge graph, wherein the retrieval generation device based on a large language model and a knowledge graph comprises: a construction module for performing vector parsing on a plurality of preset domain documents and constructing a vector database;

[0030] The construction module is further used to construct the knowledge graph according to the entity description information and relationship description information extracted from the multiple domain documents, and store the knowledge graph in a graph database;

[0031] A processing module, used to extract features of a target question input by a user, and obtain multi-dimensional keywords corresponding to the target question;

[0032] The processing module is further used to search the multi-dimensional keywords through a preset search engine, the vector database and the graph database to obtain a target search result;

[0033] The processing module is further used to determine target input information according to the target search result, and input the target input information into the target language model to obtain target output response information.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0035] A memory storing executable program code;

[0036] a processor coupled to the memory;

[0037] The processor calls the executable program code stored in the memory to execute the retrieval generation method based on a large language model and a knowledge graph in the first aspect of the embodiment of the present application.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the retrieval generation method based on a large language model and a knowledge graph in the first aspect of the embodiment of the present application. The computer-readable storage medium includes a ROM / RAM, a disk or an optical disk, etc.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute part or all of the steps of any one of the methods of the first aspect.

[0040] In a sixth aspect, an embodiment of the present application provides an application publishing platform, which is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of any one method of the first aspect.

[0041] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0042] The embodiment of the present application provides a retrieval generation method and device based on a large language model and a knowledge graph, which performs vector analysis on preset multiple field documents and constructs a vector database; constructs a knowledge graph based on entity description information and relationship description information extracted from multiple field documents, and stores the knowledge graph in a graph database; performs feature extraction on a target question input by a user to obtain multi-dimensional keywords corresponding to the target question; searches for multi-dimensional keywords through a preset search engine, a vector database, and a graph database to obtain target retrieval results; determines target input information based on the target retrieval results, and inputs the target input information into a target language model to obtain target output response information. In this scheme, the text content of multiple types of private domain knowledge documents is parsed, and extraction rules are obtained by sampling domain data, which can avoid the deviation between pre-defined extraction rules and domain data, affecting the accuracy of knowledge extraction and the quality of domain knowledge graph construction; in addition, query keywords of user questions are extracted from multiple dimensions for user query questions, which can fully understand the purpose of user queries; and a hybrid retrieval mode is adopted to recall relevant contextual information from multiple sources of search engines, vector libraries and graph databases by retrieving similar text blocks, similar entity node information, and by querying the association relationship of similar nodes in the graph database, so as to effectively improve the reliability and richness of retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 This is a schematic diagram of a retrieval generation method based on a large language model and a knowledge graph provided in an embodiment of the present application. Figure 1 ;

[0046] Figure 2 This is a schematic diagram of a retrieval generation method based on a large language model and a knowledge graph provided in an embodiment of the present application. Figure 2 ;

[0047] Figure 3 This is a schematic diagram of a retrieval generation method based on a large language model and a knowledge graph provided in an embodiment of the present application. Figure 3 ;

[0048] Figure 4 It is a structural schematic diagram of a retrieval generation device based on a large language model and a knowledge graph provided in an embodiment of the present application;

[0049] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] The terms “first”, “second” and the like in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.

[0052] The terms "including" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0053] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] With the vigorous development and application of large language models (LLMs), LLMs have demonstrated powerful capabilities in natural language processing and image processing, but they are slightly insufficient in practical application tasks such as hallucinations, new knowledge, and specific private domain knowledge. The Retrieval-Augmented Generation (RAG) system can combine LLM with external domain-specific information sources to enable LLM to obtain unseen domain context information about specific industries, effectively eliminate hallucinations, and improve the accuracy and relevance of LLM responses. However, the existing RAG system still has certain limitations. On the one hand, the RAG system search only uses the similarity search between text blocks and question vectors to recall text blocks, which is prone to problems such as unclear understanding of user intentions, weak relevance and correlation between semantic text blocks and query questions. On the other hand, the RAG system relies on local text blocks as context information for retrieval recall, lacks globality, and cannot understand the complex associations between knowledge points.

[0055] At present, there are some solutions that improve the retrieval-related information of the RAG system and enhance the retrieval accuracy by integrating structured knowledge bases. For example: (1) Using text vectors and text keywords to fuse to improve the accuracy and relevance of RAG retrieval (Zhang Jinlin. A RAG knowledge question-answering method and device based on fusion vector and keyword retrieval: 202410121013[P][2024-11-24].). This method is limited to text block vectors and Chinese word segmentation based on text blocks to obtain keywords for knowledge base construction and retrieval fusion. It ignores global information and does not consider the entities contained in the text blocks and the application of related knowledge points between text blocks; (2) Enhanced retrieval generation based on graph database (Qu Jinwen. A method and system for enhanced retrieval generation based on graph database: 202411020071[P][2024-11-24].). This method requires the construction of a domain knowledge graph in advance and stores it in the graph database. It only uses structured information for enhanced retrieval generation, which increases the difficulty of data preprocessing, and the efficiency of knowledge graph construction is low. It is time-consuming and labor-intensive and is only based on the graph database. Precise search, not effectively using vector search to improve the recall rate and relevance of search; (3) Enhance search generation based on graph structure (Zhao Yufan, Cai Xiaochen, Ding Liwen. A method for building a RAG system based on Graph: 202410646362[P][2024-11-24].), using knowledge graph to improve the effect of RAG system on generalization problems and complex connection problems. However, on the one hand, this method does not use adaptive schema for knowledge extraction when building domain knowledge graph. It needs to predefine the extraction schema for different data domains, and after building the graph, it needs to generate summaries based on the graph community, which requires frequent calls to the large model to generate additional token consumption, reducing the efficiency of knowledge construction; on the other hand, when querying, the user needs to select the query type to determine the number of community summaries to use. This process requires an additional call to the large model to filter and obtain relevant community information, which generates additional overhead and reduces the efficiency of query. At the same time, this method only retrieves from highly summarized community summary information, ignoring the detailed information of the original text information, reducing the accuracy of detailed query problems.

[0056] When existing methods improve retrieval enhancement generation by integrating structured knowledge graphs, firstly, they fail to effectively implement the construction of domain knowledge graphs based on private domain data, which is convenient and quick from unstructured data based on domain-adaptive schemas to improve the accuracy and relevance of enhanced retrieval generation; secondly, in the process of integrating knowledge graph retrieval, they fail to fully integrate and utilize the detailed information of the original unstructured text and the structured knowledge graph associations and global information for hybrid retrieval, and the retrieval information is not comprehensive enough.

[0057] In order to solve some or all of the above-mentioned technical problems, the embodiments of the present application provide a retrieval generation method and device based on a large language model and a knowledge graph, which performs vector analysis on preset multiple field documents and constructs a vector database; constructs a knowledge graph based on entity description information and relationship description information extracted from multiple field documents, and stores the knowledge graph in a graph database; performs feature extraction on the target question input by the user to obtain multi-dimensional keywords corresponding to the target question; searches for multi-dimensional keywords through a preset search engine, a vector database, and a graph database to obtain target retrieval results; determines target input information based on the target retrieval results, and inputs the target input information into the target language model to obtain target output response information. In this scheme, the text content of multiple types of private domain knowledge documents is parsed, and extraction rules are obtained by sampling domain data, which can avoid the deviation between pre-defined extraction rules and domain data, affecting the accuracy of knowledge extraction and the quality of domain knowledge graph construction; in addition, query keywords of user questions are extracted from multiple dimensions for user query questions, which can fully understand the purpose of user queries; and a hybrid retrieval mode is adopted to recall relevant contextual information from multiple sources of search engines, vector libraries and graph databases by retrieving similar text blocks, similar entity node information, and by querying the association relationship of similar nodes in the graph database, so as to effectively improve the reliability and richness of retrieval.

[0058] like Figure 1 As shown, Figure 1 A flowchart of a retrieval generation method based on a large language model and a knowledge graph provided in an embodiment of the present application, the method may include the following steps:

[0059] 101. Perform vector analysis on documents in multiple preset fields and build a vector database.

[0060] In the embodiments of the present application, the domain document can be considered as a text collection of content in some private or rare domains. The multiple domain documents can be content uploaded by users themselves. The multiple domain documents can be content in multiple formats such as PDF, Word, Excel, PPT, Markdown, TXT, etc. Since the subsequent model application and text processing are all based on the vector mode, the multiple domain documents can be vector parsed and the text can be vectorized to construct a vector database.

[0061] 102. Construct a knowledge graph based on the entity description information and relationship description information extracted from documents in multiple fields, and store the knowledge graph in a graph database.

[0062] In an embodiment of the present application, entities, relationships, and attribute-related elements can be identified and extracted from multiple domain documents in a Schema adaptive domain manner to form instances of graph nodes and edges, thereby constructing a knowledge graph and storing the knowledge graph in a graph database.

[0063] It should be noted that in the knowledge graph, an entity can refer to an object or concept with clear boundaries and distinguishability. These objects or concepts can be physical (such as people, places, objects) or abstract concepts (such as events, concepts, ideas). Each entity has a unique identifier in the knowledge graph. The entity has attributes that describe its characteristics, such as a person's name, age, gender, etc. An entity is an instantiation of a class (or type), that is, each entity belongs to one or more classes, which define the common attributes and behaviors of the entity.

[0064] It should be noted that a relationship can refer to a certain connection or interaction between entities. In the knowledge graph, relationships are used to describe the association and interaction between entities. Relationships are usually directional, indicating the direction of interaction between entities. For example, "A is the father of B" and "B is the son of A" represent the same relationship, but in opposite directions. Relationships have types, which define the semantics of the relationship. For example, "born" represents a relationship in which an entity is born at a certain time and place. In some cases, relationships may have weights, indicating the strength or importance of the relationship.

[0065] In the knowledge graph, entities and relationships are interdependent. Entities are the basic units of the knowledge graph, while relationships connect these units to form a meaningful network. By defining entities and relationships, the knowledge graph can represent complex knowledge structures, including facts, rules, conceptual hierarchies, etc. The combination of entities and relationships can represent various types of knowledge, such as common sense knowledge, domain knowledge, personal knowledge, etc. In practical applications, knowledge graphs can be used in intelligent question and answer, recommendation systems, semantic search and other fields to provide users with more intelligent and personalized services.

[0066] Therefore, in an embodiment of the present application, a knowledge graph can be constructed by mining the description information of entities and relationships included in multiple domain documents.

[0067] 103. Perform feature extraction on the target question input by the user to obtain multi-dimensional keywords corresponding to the target question.

[0068] In an embodiment of the present application, the target question may be a question that a query model needs to answer and that is input by a user. In order to avoid unclear understanding of the user's intention and ambiguity that affects the relevance and accuracy of the LLM answer generation, it is necessary to vectorize the target question that the user inputs and extract multi-dimensional keywords for the target question. Specifically, it may include high-dimensional retrieval keywords and low-dimensional retrieval keywords to describe the target question from different angles.

[0069] 104. Through the preset search engine, vector database and graph database, multi-dimensional keywords are searched to obtain target search results.

[0070] In the embodiment of the present application, the vector database and the graph database may be databases pre-created through multiple field documents, and the preset search engine may be currently commonly used internal and external search engines, but is not limited. A hybrid mode search is adopted to search the vector database, the graph database, and the search engine to recall information related to the target question, thereby improving the effectiveness of the query information, ensuring the comprehensiveness of the query information, and not losing detailed information, while increasing the reliability of the information.

[0071] 105. According to the target search result, target input information is determined, and the target input information is input into the target language model to obtain target output response information.

[0072] In an embodiment of the present application, the retrieval results obtained according to the hybrid retrieval mode can be synthesized to obtain final target input information, and the target input information is input into the target language model to obtain target output response information.

[0073] The embodiment of the present application provides a retrieval generation method based on a large language model and a knowledge graph, which parses the text content of multiple types of private domain knowledge documents and obtains extraction rules by sampling domain data, thereby avoiding the deviation between pre-defined extraction rules and domain data, which affects the accuracy of knowledge extraction and the quality of domain knowledge graph construction; in addition, query keywords of user questions are extracted from multiple dimensions for user query questions, so as to fully understand the purpose of user queries; and, a hybrid retrieval mode is adopted to recall relevant context information from multiple sources of search engines, vector libraries and graph databases by retrieving and recalling similar text blocks, similar entity node information, and by querying the association relationship between similar nodes in the graph database, thereby effectively improving the reliability and richness of the retrieval.

[0074] like Figure 2 As shown, Figure 2 A flowchart of a retrieval generation method based on a large language model and a knowledge graph provided in an embodiment of the present application, the method may also include the following steps:

[0075] 201. Get multiple domain documents.

[0076] In the embodiment of the present application, the multiple domain documents may be domain documents uploaded by the user himself.

[0077] 202. Perform document conversion on multiple domain documents according to the file type of each domain document to obtain multiple initial text contents.

[0078] In the embodiment of the present application, since the file types of domain documents may include multiple types, document conversion can be performed for different types of domain documents to facilitate subsequent text processing.

[0079] In some embodiments, domain documents may include PDF, Word, Excel, PPT, Markdown, TXT and other types. For different types of documents, the following operations are performed respectively:

[0080] (1) For PDF / PPT documents:

[0081] S20211. If the document raw_document ends with .ppt / .pptx, convert it to PDF document document pdf To carry out unified processing;

[0082] document pdf =convert(raw_document)

[0083] S20212, for PDF document pdf First, through document layout analysis, two deep learning algorithm models, fully convolutional neural network (FCN) and graph neural network (GNN), are introduced into document layout analysis. The text reading order is predicted by combining multiple modal information such as geometric coordinates, visual features, and text semantics of the text area. Each element in the document is detected, including titles, texts, headers, footers, formulas, images, tables, etc., and the regional coordinates (x, y) of the text elements are obtained. text ;

[0084] (x, y) text = panel_analysis(document pdf )

[0085] S20213, according to the text area coordinates (x, y) text , after extracting the text content of multiple regions and merging paragraphs through heuristic rules, all the text content text in the document is obtained.

[0086] The specific principle is as follows: multiple left-aligned lines with the same line spacing are considered as the same paragraph, and the beginning of the first line is allowed to have two full-width spaces. If the line spacing of two lines exceeds a certain threshold, they are considered to belong to different paragraphs.

[0087] text=∑merge(extract((x,y) text ))

[0088] (2) For Word / Markdown / TXT documents:

[0089] S20221. If the document raw_document ends with .doc / .docx, convert it to a Markdown document document markdown To carry out unified processing;

[0090] document markdown =convert(raw_document)

[0091] S20222, directly read Markdown / TXT document markdown / txt The text content in text.

[0092] text=read(document markdown / txt )

[0093] (3) For Excel documents:

[0094] For the Excel document raw_document ending with .xls / .xlsx, get the text content text by getting the content of each Sheet and each cell and concatenating them.

[0095] text=read&joint(raw_document)

[0096] It can be seen that after performing corresponding operations on documents of different formats, multiple initial text contents can be obtained.

[0097] 203. Convert multiple initial text contents into multiple initial text vectors through a preset text vector model.

[0098] In the embodiment of the present application, in the process of vectorizing text content, the initial text content can be converted into an initial text vector representation based on the multi-language BGE-M3 text vector model.

[0099] In some embodiments, before vectorization, in order to adapt to the LLM window size, the text content can also be segmented, and a recursive segmentation strategy is adopted to split the text text into text blocks text_chunk. First, it is split according to larger paragraphs; if a unit exceeds the block size limit, the splitting will be further split according to specific punctuation marks and sentences; if further splitting is still needed, it will continue to the word level.

[0100] text_chunk = split(text)

[0101] Therefore, the text block text_chunk can be converted into a text vector text_chunk_embedding representation using the multi-language BGE-M3 text vector model;

[0102] text_chunk_embedding=embedding(text_chunk)

[0103] =[CLS, token1, token2,...,SEP]

[0104] Among them, CLS represents the classification mark of the sentence, SEP represents the separation mark of the sentence, and token i Represents the i-th token in the input text.

[0105] 204. Construct a vector database using multiple initial text contents and multiple initial text vectors.

[0106] In an embodiment of the present application, multiple initial text contents and multiple initial text vectors may be stored correspondingly, thereby constructing a vector database.

[0107] In some embodiments, Elasticsearch may also be used as a storage vector library to store text chunk contents text_chunk (ie, multiple initial text contents) and text chunk vectors text_chunk_embedding (ie, multiple initial text vectors) into the library.

[0108] text_chunk_db=save(text_chunk+text_chunk_embedding)

[0109] 205. Through a random sampling method, domain seed data is obtained from multiple domain documents and embedded into the prompt instruction template to obtain the initial extraction rules.

[0110] In some embodiments, the domain seed data text_chunk of the data to be extracted is obtained from the manageable text block text_chunk by random sampling.sample , embedded in the prompt instruction template prompt for generating schema schema_generate As the input of LLM, it outputs all potential extraction rule schemas related to the data domain. The expression is as follows:

[0111] schema=generate(LLM, prompt schema_generate +text_chunk sample )

[0112] 206. Based on the initial extraction rules, entity description information and relationship description information are extracted from multiple domain documents.

[0113] In the embodiment of the present application, the schema is extracted based on the initially acquired domain candidate , input the text_chunk obtained after text segmentation into LLM, and extract the entities and relations contained in the text block.

[0114] In some embodiments, when extracting entities and relationships, you can also set the maximum number of rounds max_turns to allow LLM to self-audit the results of the previous round of extraction entities candidate , relations candidate Whether all entities and relations in the data are included, the missing entities and relations are extracted again according to the feedback of the audit content. At the same time, the extraction schema is updated to make it fully adaptive to the domain data until the maximum number of rounds is reached, and finally the extracted entities and relations are determined to improve the accuracy of entity and relationship extraction. The expression is as follows:

[0115] entities candidate , relations candidate

[0116] =generate(LLM, prompt extract +schema candidate +text_chunk)

[0117] entities,relations=check n≤max_turns (LLM, prompt check +schema candidate +entities candidate +relations candidate )+generate n≤max_turns (LLM, prompt extract +schema+text_chunk)

[0118] 207. Based on the entity description information and relationship description information extracted from documents in multiple fields, a knowledge graph is constructed and stored in a graph database.

[0119] In an embodiment of the present application, entity description information and relationship description information can be constructed into a knowledge graph, and the knowledge graph can be stored in a graph database.

[0120] In some embodiments, Nebula Graph can be used as a high-concurrency, low-latency graph database to implement storage and query support, and store entities and relations in the graph database.

[0121] graph_db=save(entities+relations)

[0122] 208. According to the entity description information and the relationship description information, the explicit information and implicit information of the entity node are mined to obtain the summary information corresponding to the entity node.

[0123] In the embodiment of the present application, according to the extracted entity description information and relationship description information, the explicit information and implicit information related to the entity node are mined through LLM, so as to obtain summary information about the entity node, and the summary information is associated with the entity node to improve the summary summarization ability of the entity node, and further reflect the global information of the entity node. The expression is as follows:

[0124] entity summary =generate(LLM,Prompt summary +entity+relations)

[0125] 209. Associate the summary information with the attributes of the entity node, perform vector analysis on the entity node to obtain an entity node vector, and store the entity node vector in a vector database.

[0126] In the embodiment of the present application, the summary information entitysummar y The entity node entity is parsed by the multi-language BGE-M3 text vector model and converted into an entity node vector entity_embedding for representation, which is used for subsequent vector retrieval;

[0127] entity_embedding=embedding(entity)

[0128] In some embodiments, Elasticsearch may also be used as a storage vector library to store entity node content entity and entity node vector entity_embedding in the library.

[0129] entity_db=save(entity+entity_embedding)

[0130] 210. Get the initial question input by the user.

[0131] 211. Rewrite and correct the initial problem to obtain the target problem.

[0132] In the embodiment of the present application, for the initial question raw_query input by the user, the question is first rewritten and corrected to obtain an accurate target question query.

[0133] Question rewriting is to expand the search scope by adjusting the structure of the user's query, adding or deleting keywords, and replacing synonyms. Question correction is to correct possible spelling or grammatical errors in the user's query, or to decompose unclear or ambiguous content in the user's query, so that the system can better understand the user's query intention;

[0134] query=rewrite&revise(raw_query)

[0135] 212. Perform feature extraction on the target question input by the user to obtain multi-dimensional keywords corresponding to the target question.

[0136] In the embodiment of the present application, for the rewritten and revised target question query, the fundamental intention or goal of the user's query question is understood, and the user's query question is marked with one or more of the most relevant labels query_type in the problem types such as "summary query", "knowledge query", "search engine", "other tasks", etc. At the same time, for the query question, high-dimensional search keywords high_dimentional_keywords and low-dimensional search keywords low_dimentional_keywords in the question are extracted respectively for use in a hybrid search mode of high-dimensional, low-dimensional and multi-source integration;

[0137] query_type, high_dimentional_keywords, low_dimentional_keywords

[0138] =intent_detection&keywords_extraction(query)

[0139] In some embodiments, the user query question query is converted into a question vector query_embedding representation through a multi-language BGE-M3 text vector model for subsequent vector retrieval;

[0140] query_embedding=embedding(query)

[0141] 213. Search the multi-dimensional keywords through a preset search engine to obtain a first search result.

[0142] In an embodiment of the present application, based on the target question query identified by intent understanding, combined with multi-dimensional keywords and question type query_type, it is determined whether it is necessary to search for information in the "search engine". If a search engine retrieval is required, the search recall information is returned, otherwise the search engine search process is directly skipped.

[0143] Optionally, the search engine can be an open source or internal trusted search source to ensure the reliability of the information source;

[0144] browse_information=search_internet(query)

[0145] 214. Search low-dimensional search keywords in the multi-dimensional keywords through the vector database to obtain a second search result.

[0146] In the embodiment of the present application, based on the question vector query_embedding and combined with low-dimensional search keywords low_dimentional_keywords, low-dimensional detail information related to the question is retrieved in the vector database to obtain multiple similar text block data, namely the second search result.

[0147] In some embodiments, based on the retrieval of the vector knowledge base, the vector database is first searched and sorted by vector similarity retrieval and BM25 retrieval, and then the inverse sorting fusion algorithm is used for fusion sorting, and the top n block data are selected as the final similar text block data. The specific steps are as follows:

[0148] S2141. Vector similarity-based retrieval is a method that uses vector representation and similarity measurement to retrieve information from a knowledge base. This method is commonly used in natural language processing and information retrieval tasks, where text data is mapped to a vector space so that semantically similar texts are closer in the vector space. This solution measures their proximity in the vector space by calculating the cosine similarity between vectors. The similarity value is between -1 and 1, with 1 indicating complete similarity. The formula is as follows:

[0149]

[0150] Among them, A·B is the vector inner product, ||A|| and ||B|| are the norms of the vectors respectively.

[0151] According to the user query question vector query_embedding, through the cosine similarity to the text block index text_chunk_db, recall the low-dimensional similar text block text_chunk that is most relevant to the question embedding .

[0152] S2142, based on BM25 retrieval, calculates the relevance score of a document by considering the statistical characteristics of document length and query word frequency. It is a scoring function used for information retrieval, which measures the relevance between the query and the text block. The formula is as follows:

[0153]

[0154] Among them, Q represents a user query, q i represents the i-th query of query, d represents the document to be matched, ω i Indicates its own importance, and its formula is as follows:

[0155]

[0156] Among them, N represents the total number of documents to be matched, df i To include q i The total number of documents. i , including q i The more documents there are, the more likely the q i The less important, i To some extent, it can be used to characterize q i Relevance to the document.

[0157] According to the user query query, the BM25 retrieval method is used to retrieve the text chunk index text_chunk_db, and the low-dimensional similar text chunk text_chunk that is most relevant to the question is retrieved. BM25 .

[0158] S2143. Based on the inverse sorting fusion algorithm, the similar text blocks recalled by the vector similarity retrieval and BM25 retrieval methods are fused and sorted. The formula is as follows:

[0159]

[0160] Among them, Score represents the inverse sorting fusion result of vector similarity retrieval and BM25 retrieval method, k is 50, which is used to adjust the ranking influence, rank i Represents the ranking of document data in the i-th retrieval method, Represents the inverse ranking score of each document in the i-th retrieval method.

[0161] Similar text chunks retrieved based on vector similarity text_chunk embedding , and similar text chunks retrieved by BM25 text_chunk BM25 , after sorting, obtain the similar original text block recall information source_information related to the user's query question.

[0162] source__information

[0163] =text_chunk_db_query(query+query_embedding)

[0164] 215. Search the high-dimensional search keywords in the multi-dimensional keywords through the vector database and the graph database to obtain a third search result.

[0165] In the embodiment of the present application, according to the question vector query_embedding, combined with high-dimensional search keywords high_dimentional_keywords, high-dimensional global information retrieval related to the question is performed in the vector database and the graph database. This step uses high-dimensional search keywords to query and summarize global entity and relationship information related to the question, providing higher-dimensional concepts and summary information instead of specific details. The specific steps are as follows:

[0166] S2151, according to the question vector query_embedding and high-dimensional search keywords high_dimentional_keywords into the vector database entity index entities_db, according to the vector similarity search, BM25 search and fusion reciprocal sorting algorithm mentioned in step 214, recall the entity node information containing the node summary similar to the target question;

[0167] entities_information=entities_db_query(query_embedding)

[0168] S2152. According to the recalled entities_information, the graph database constructed by schema adaptiveness is sorted according to the node association degree to obtain the association relationship information related to the entity node.

[0169] relations_information=graph_db_query(entities_information)

[0170] 216. According to the input text restriction condition of the target language model, the first search result, the second search result, and the third search result are cut and spliced ​​to obtain target input information.

[0171] In an embodiment of the present application, according to the search engine information browse_information, original text block information source_information, entity node information entities_information, and relationship information relations_information recalled by the hybrid retrieval mode, the recalled information is cropped and spliced ​​and synthesized through appropriate LLM window size input restrictions to obtain the target input information.

[0172] context=concat&joint(browse_information+source_information+entities_information+relations_information)

[0173] 217. Input the target input information into the target language model to obtain the target output response information.

[0174] In the embodiment of the present application, the target input information is input into the LLM to generate a final answer.

[0175] answer=generate(LLM, prompt qa +context)

[0176] The embodiment of the present application provides a retrieval generation method based on a large language model and a knowledge graph, which uses natural language processing technology to process text content analysis of multiple types of documents, and supports users to upload private domain knowledge documents including PDF, Word, Excel, PPT, Markdown, TXT and other types; extracts entities and relationships in a schema-adaptive manner to construct a domain knowledge graph, avoids the mismatch between the pre-defined inherent schema model and the domain data, and reduces the accuracy of knowledge extraction and the quality of domain knowledge graph construction; based on the description of entity nodes and their association relationships, generates summary information of entity nodes and associates it with the description attributes of entity nodes, fully mines explicit and implicit information of entity nodes, and improves the global description capability of entity nodes; adopts a hybrid retrieval mode to retrieve and recall relevant information from multiple sources such as search engines, vector databases and graph databases, through a vector library to retrieve similar text blocks, similar entity nodes combined with graph databases to query the association relationship of nodes, and other methods, thereby improving the reliability and richness of retrieval.

[0177] Rewrite and correct user query questions, identify user query intentions, and extract query keywords from user questions from both high-dimensional and low-dimensional aspects to fully understand the purpose of user queries; based on the question vector, retrieve similar text vectors and similar entity vectors at the same time to ensure that the information has both detailed information based on text blocks and global association information based on entity nodes, thereby improving the comprehensiveness of the retrieval enhancement generation system; when retrieving entity nodes and association relationship information, first retrieve similar entity nodes from the vector library through vector similarity, and then search for associated edges of nodes in the graph database based on similar entity nodes, thereby alleviating the problems of low recall rate and missing semantic relevance when performing precise entity searches directly from the graph database.

[0178] In some embodiments, Figure 3 The whole process of the retrieval generation method based on the large language model and knowledge graph provided in the embodiment of the present application is shown as follows: Figure 3 As shown, domain documents are uploaded, and then document data processing is performed. After that, a schema-adaptive domain knowledge graph is built to obtain a vector database and a graph database together. Data processing is performed on user query questions, and the search engine, vector database, and graph database are combined to synthesize context. The synthesized text is input into a large language model to generate answers.

[0179] like Figure 4 As shown, the embodiment of the present application provides a retrieval generation device based on a large language model and a knowledge graph, and the retrieval generation device based on a large language model and a knowledge graph may include: a construction module 401, which is used to perform vector parsing on a plurality of preset domain documents and construct a vector database;

[0180] The construction module 401 is further used to construct a knowledge graph based on the entity description information and relationship description information extracted from the multiple domain documents, and store the knowledge graph in the graph database;

[0181] Processing module 402, used to extract features of the target question input by the user to obtain multi-dimensional keywords corresponding to the target question;

[0182] The processing module 402 is further used to search for multi-dimensional keywords through a preset search engine, a vector database, and a graph database to obtain a target search result;

[0183] The processing module 402 is further used to determine target input information according to the target search result, and input the target input information into the target language model to obtain target output response information.

[0184] In some embodiments, the construction module 401 is specifically used to obtain multiple domain documents;

[0185] The construction module 401 is specifically used to convert multiple domain documents according to the file type of each domain document to obtain multiple initial text contents;

[0186] A construction module 401 is specifically used to convert a plurality of initial text contents into a plurality of initial text vectors by using a preset text vector model;

[0187] The construction module 401 is specifically used to construct a vector database through multiple initial text contents and multiple initial text vectors.

[0188] In some embodiments, the construction module 401 is further used to obtain domain seed data from multiple domain documents through a random sampling method, and embed it into the prompt instruction template to obtain an initial extraction rule;

[0189] The construction module 401 is also used to extract entity description information and relationship description information from multiple domain documents based on the initial extraction rules, construct a knowledge graph, and store the knowledge graph in a graph database.

[0190] In some embodiments, the construction module 401 is further used to mine explicit information and implicit information of the entity node according to the entity description information and the relationship description information to obtain summary information corresponding to the entity node;

[0191] The construction module 401 is further used to associate the summary information with the attributes of the entity node, perform vector analysis on the entity node to obtain the entity node vector, and store the entity node vector in the vector database.

[0192] In some embodiments, the processing module 402 is used to obtain an initial question input by a user;

[0193] The processing module 402 is also used to rewrite and correct the initial question to obtain the target question.

[0194] In some embodiments, the processing module 402 is specifically configured to search for the multi-dimensional keywords through a preset search engine to obtain a first search result;

[0195] The processing module 402 is specifically configured to search the low-dimensional search keywords in the multi-dimensional keywords through the vector database to obtain a second search result;

[0196] The processing module 402 is specifically configured to search for high-dimensional search keywords in the multi-dimensional keywords through the vector database and the graph database to obtain a third search result.

[0197] In some embodiments, the processing module 402 is specifically used to cut and splice the first search result, the second search result, and the third search result according to the input text restriction condition of the target language model to obtain the target input information.

[0198] In the embodiments of the present application, each module can implement the retrieval generation method based on a large language model and a knowledge graph provided in the above method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0199] like Figure 5 As shown, an embodiment of the present application further provides an electronic device, which may include:

[0200] A memory 501 storing executable program codes;

[0201] a processor 502 coupled to the memory 501;

[0202] The processor 502 calls the executable program code stored in the memory 501 to execute the retrieval generation method based on the large language model and the knowledge graph executed by the electronic device in the above-mentioned method embodiments.

[0203] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the retrieval generation method based on a large language model and a knowledge graph in the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0204] An embodiment of the present application also provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the various processes of the retrieval generation method based on a large language model and a knowledge graph in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0205] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0206] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] In the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0208] In this application, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0209] In this application, a person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by controlling the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the computer-readable medium includes permanent and non-permanent, removable and non-removable storage media. The storage medium can implement information storage by any method or technology, and the information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, Parallel Random Access Memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), other types of Random Access Memory (RAM), Read-Only Memory (ROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0210] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0211] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application. The above-mentioned multiple embodiments are not necessarily multiple independent embodiments. Dividing into multiple embodiments is only used to highlight the different technical features in different embodiments. Those skilled in the art should be aware that the above-mentioned multiple embodiments can also be combined in any way.

[0212] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0213] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0215] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0216] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A retrieval generation method based on a large language model and a knowledge graph, characterized in that: The method comprises: Perform vector analysis on documents in multiple preset fields and build a vector database; Constructing the knowledge graph according to the entity description information and the relationship description information extracted from the multiple domain documents, and storing the knowledge graph in a graph database; Extract features of the target question input by the user to obtain multi-dimensional keywords corresponding to the target question; The multi-dimensional keywords are searched by using a preset search engine, the vector database, and the graph database to obtain target search results; According to the target search result, target input information is determined, and the target input information is input into a target language model to obtain target output response information.

2. The method according to claim 1, characterized in that The vector parsing of the preset multiple domain documents and building a vector database includes: Obtaining the multiple field documents; According to the file type of each domain document, the plurality of domain documents are converted to obtain a plurality of initial text contents; Converting the multiple initial text contents into multiple initial text vectors by using a preset text vector model; The vector database is constructed by using the multiple initial text contents and the multiple initial text vectors.

3. The method according to claim 1, characterized in that Before constructing a knowledge graph based on the entity description information and relationship description information extracted from the multiple domain documents and storing the knowledge graph in a graph database, the method further includes: Obtaining domain seed data from the plurality of domain documents through a random sampling method, and embedding the data into a prompt instruction template to obtain an initial extraction rule; Based on the initial extraction rule, the entity description information and the relationship description information are extracted from the multiple domain documents.

4. The method according to claim 3, characterized in that After constructing a knowledge graph based on the entity description information and relationship description information extracted from the multiple domain documents and storing the knowledge graph in a graph database, the method further includes: Mining explicit information and implicit information of an entity node according to the entity description information and the relationship description information to obtain summary information corresponding to the entity node; The summary information is associated with the attribute of the entity node, and the entity node is vector parsed to obtain an entity node vector, and the entity node vector is stored in the vector database.

5. The method according to claim 1, characterized in that Before extracting features of the target question input by the user to obtain multi-dimensional keywords corresponding to the target question, the method further includes: Obtaining an initial question input by the user; The initial problem is rewritten and error-corrected to obtain the target problem.

6. The method according to claim 1, characterized in that The method of searching the multi-dimensional keywords through the preset search engine, the vector database, and the graph database to obtain target search results includes: Searching the multi-dimensional keyword through the preset search engine to obtain a first search result; Searching the low-dimensional search keywords in the multi-dimensional keywords through the vector database to obtain a second search result; The high-dimensional search keywords in the multi-dimensional keywords are searched through the vector database and the graph database to obtain a third search result.

7. The method according to claim 6, characterized in that Determining target input information according to the target search result includes: According to the input text restriction condition of the target language model, the first search result, the second search result and the third search result are cut and spliced ​​to obtain the target input information.

8. A retrieval generation device based on a large language model and a knowledge graph, characterized in that: The retrieval generation device based on the large language model and the knowledge graph includes: The construction module is used to perform vector analysis on documents in multiple preset fields and build a vector database; The construction module is further used to construct the knowledge graph according to the entity description information and relationship description information extracted from the multiple domain documents, and store the knowledge graph in a graph database; A processing module, used to extract features of a target question input by a user, and obtain multi-dimensional keywords corresponding to the target question; The processing module is further used to search the multi-dimensional keywords through a preset search engine, the vector database and the graph database to obtain a target search result; The processing module is further used to determine target input information according to the target search result, and input the target input information into the target language model to obtain target output response information.

9. An electronic device, characterized in that: The electronic device comprises: A memory storing executable program code; and a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the retrieval generation method based on a large language model and a knowledge graph as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by the processor, the retrieval generation method based on a large language model and a knowledge graph is implemented as described in any one of claims 1 to 7.

Images