Large model retrieval enhancement method based on knowledge graph and knowledge representation

Abstract

The invention discloses a large model retrieval enhancement method based on a knowledge graph and knowledge representation. The method comprises the following steps: analyzing a document and other unstructured data uploaded by a user; using a preset template and a large model to extract entities, relationships and attributes in the document, and constructing a knowledge graph; performing multi-level embedding on the extracted data, and performing mapping storage on the extracted data and original data of a knowledge graph to serve as knowledge representation; user query is optimized, query statements are supplemented and rewritten, and query clauses and keywords are extracted; and carrying out multi-level embedding on the processed query, carrying out similarity calculation retrieval in the knowledge graph database, carrying out text restoration on a retrieval result, fusing the retrieval result with the original query, and inputting the fused query into a large model to generate an answer. According to the method disclosed by the invention, the multi-embedded model is represented by using the clear annotation entity and relationship in the knowledge graph data, so that the effect of faster and more accurate generative large model retrieval enhancement is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and specifically to a large model retrieval enhancement method based on knowledge graph and knowledge representation. Background Art

[0002] The large models (LLM) we often refer to usually refer to general language models that are trained on a large amount of text data and can be used to generate content, such as GPT-3. Although these models perform well on a variety of language tasks, they may lack expertise in specific fields, generate text that is not professional enough, and may produce hallucinations. In order to improve the performance of the model on a specific task, fine-tuning methods are usually used.

[0003] The fine-tuning process involves further training the model with task-specific data so that the model can better understand and perform the task. However, fine-tuning is similar to the training process, and often requires a lot of computing resources and takes too long to train. In addition, when data is limited, fine-tuning may also cause the model to be over-trained and perform poorly on new data sets. In addition, in scenarios where data information is constantly changing, the cost of fine-tuning each time to update data is too high, and the time is too long, which means that fine-tuning can only be applied to some scenarios.

[0004] In order to enable LLM to update data in real time and deal with knowledge problems in different fields, the retrieval enhancement (RAG) method is usually adopted. It is a technology that combines information retrieval and text generation. It enhances the responsiveness of traditional language models by retrieving relevant information from knowledge sources and integrating it into the generated text. However, the existing RAG technology is difficult to handle problems with multiple languages ​​and multi-layer logic. At the same time, it is too dependent on query recall data. For some complex parallel or serial question queries, it is difficult to use RAG to obtain comprehensive knowledge points at one time, which is what we call low recall rate. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] Therefore, the purpose of the present invention is to provide a large-model retrieval enhancement method based on knowledge graph and knowledge representation. The present invention uses the clear labeled entities and relationship representations in the knowledge graph data to retrieve relevant structured information to provide more accurate answers. The knowledge graph database is used to retrieve elements such as nodes, triples, paths or subgraphs related to the query, and these elements are used to generate the context of the answer, so as to extract the background knowledge required for the question faster and more accurately.

[0007] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0008] A large model retrieval enhancement method based on knowledge graph and knowledge representation, the steps are as follows:

[0009] S1. Parse documents and other unstructured data uploaded by users;

[0010] S2. Using preset templates, the big model extracts entities, relationships, and attributes from documents and builds a knowledge graph;

[0011] S3, perform multi-level embedding on the extracted data, and map and store it with the original data of the knowledge graph as knowledge representation;

[0012] S4. Optimize user queries, supplement and rewrite query statements, and extract query clauses and keywords;

[0013] S5, embed the processed query in multiple levels, and perform similarity calculation and retrieval in the knowledge graph database;

[0014] S6. Perform text restoration on the search results, merge them with the original query, and input them into the big model to generate answers.

[0015] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S1, the specific steps of parsing the documents and other unstructured data uploaded by the user are as follows:

[0016] The source documents uploaded by users are stored in object storage servers or other file management services, and then text and images are extracted from the uploaded files through parsing libraries with different types of file content and stored in association with the documents.

[0017] As a preferred solution of the large-model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S2, using a preset template, the large model extracts entities, relationships and attributes in the document, and the specific steps of constructing the knowledge graph are as follows: through the text semantic understanding ability of LLM, the text and pictures processed in step S1 are extracted as "entity-relationship-entity" structured data, and attribute values ​​are marked for all entities, and then according to the different document types uploaded by users, corresponding graph templates are designed, and the extracted entities are filled into the corresponding positions of the templates to generate a preliminary knowledge graph, and then the preliminary knowledge graph is expanded into a richer knowledge graph.

[0018] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, the LLM model and self-built entity vocabulary are used to mark the attribute values ​​of the extracted entities.

[0019] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S3, the extracted data is multi-level embedded and mapped and stored with the original data of the knowledge graph. The specific steps of knowledge representation are as follows:

[0020] For the extracted entities, relations, and attributes, different embedding models are used to generate sentence-level and word-level embeddings according to their different lengths, accurately representing the knowledge graph from vectors of different dimensions.

[0021] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S4, the specific steps of optimizing user queries, supplementing and rewriting query statements, and extracting query clauses and keywords are as follows:

[0022] Get the user's input query, use the large language model to complete the query according to the context, split the complex query into "sub-queries", represented by Q = [q1, q2, ... qn], and extract "keywords", represented by K = [k1, k2, ..., km].

[0023] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S5, the processed query is multi-level embedded, and the specific steps of similarity calculation and retrieval in the knowledge graph database are as follows:

[0024] Keywords are embedded at the word level, sub-query statements are embedded at the sentence level, and then the embeddings of keywords and query clauses are entered into the graph database for vector calculation. Similarity is calculated using cosine. Word embedding is used to query attributes and entity nouns, and sentence embedding is used to query long-text knowledge. All queries are concurrent queries. Finally, they are sorted by similarity, and based on the limited query amount of required knowledge, multiple entity-relationship-entity knowledge graph subgraphs are returned, and the intersecting subgraphs are fused.

[0025] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S6, the specific steps of restoring the text of the retrieval results, fusing them with the original query and inputting them into the large model to generate the answer are as follows:

[0026] The LLM model is used to restore the entities, relationships, attributes, etc. obtained from the knowledge graph into the form of natural language text, and after being integrated with the original query, it is input into the large model to generate answers.

[0027] As a preferred solution of the large model retrieval enhancement method based on knowledge graph and knowledge representation described in the present invention, in step S6, if the content returned by the retrieval contains image information, the multimodal model is called to identify the image content and the information summarized as text is returned to the LLM model.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. In the construction of the knowledge graph, the patent of this invention presets corresponding templates based on the documents uploaded by different users, uses a large language model to automatically extract entities and their relationship attributes, and fills them into the pre-designed template, thereby realizing the automatic construction of the vertical scene knowledge graph.

[0030] 2. Different from the general RAG retrieval model, the patent of the present invention not only provides accurate matching of keywords through the knowledge graph, but also provides semantic matching by embedding keywords, sentences, etc. at different levels. This processing feature can improve the retrieval accuracy and recall rate of the patent of the present invention.

[0031] 3. In actual applications, user questions often contain multiple sub-questions and their complex relationships. If the traditional search enhancement model is directly applied to Q&A, only a relatively shallow answer can be obtained, and the problem cannot be broken down. However, the knowledge graph database of this patent establishes a relationship network for each entity and its attributes. When searching for key entities, the entities, attributes, and relationships adjacent to the entity can also be obtained, providing rich supplementary information, which facilitates the large language model to extract multi-layer knowledge, and thus can solve more complex problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0033] Figure 1 A basic step diagram of a large model retrieval enhancement method based on knowledge graph and knowledge representation according to the present invention;

[0034] Figure 2 A schematic diagram comparing the application scenarios of the technical solution of the present invention and the prior art solution;

[0035] Figure 3 A schematic diagram of embedding the extracted data at different levels provided by the present invention;

[0036] Figure 4 A schematic diagram of an adaptive template provided by the present invention;

[0037] Figure 5 A practical application demonstration diagram of the knowledge graph for resume file generation provided by the present invention. DETAILED DESCRIPTION

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0039] Secondly, the present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0041] The present invention provides a large-model retrieval enhancement method based on knowledge graph and knowledge representation. The present invention uses the clearly labeled entities and relationship representations in the knowledge graph data to retrieve relevant structured information to provide more accurate answers. The knowledge graph database is used to retrieve elements such as nodes, triples, paths or subgraphs related to the query, and these elements are used to generate the context of the answer, so as to extract the background knowledge required for the question faster and more accurately.

[0042] The large-model retrieval enhancement method based on knowledge graph and knowledge representation can find matching texts according to user queries and generate answers enhanced by the large model after data processing to establish the knowledge graph database. For the specific process, please refer to Figure 1 .

[0043] Step 1: Document file parsing

[0044] In the first step, in order to deal with different types of files uploaded by users, such as PDF, Word, txt, pictures, etc., a unified processing interface is designed to process user documents: first, the source documents uploaded by the user are stored. In the application scenario of the present invention, they are stored in the object storage server (OSS) and other file management services; then, through the parsing library of different types of file contents, the text and pictures are extracted from the uploaded files and stored in association with the files.

[0045] Step 2: Use preset templates to extract entity relationships and build a knowledge graph

[0046] In this step, we first use the powerful text semantic understanding ability of the large language model to extract entities and their relationships from the text preprocessed in the "first step". Specifically, the extraction is in the form of a triple structure of "entity------relationship------entity"; at the same time, we use the large language model and the self-built entity vocabulary to add attribute labels to the extracted entities. For example, if the extracted entity is "Peking University", we add the attributes "985, 211, Double First-Class". Finally, Figure 4 As shown, according to the different document types uploaded by users, corresponding graph templates are designed, and the extracted entities are filled into the corresponding positions of the templates to generate a preliminary knowledge graph. Then, according to the extracted "entity------relationship------entity" structuring, the preliminary knowledge graph is expanded into a richer knowledge graph. For example, in actual applications, if the document uploaded by the user is a personal resume, four modules related to the character subject, namely "personal basic information", "internship experience", "project experience", and "education" are pre-designed. The extracted entity content is divided into these four models step by step, and then the entities are connected to more detailed relationships according to the "entity------relationship------entity" correspondence. Finally, the user can manually modify the automatically generated knowledge graph to generate the final knowledge graph. Specific examples are Figure 5 As shown, Figure 5 A practical case study on knowledge graph generation for resume documents was implemented.

[0047] Step 3: Data embedding and mapping to the knowledge graph

[0048] The extracted data is embedded at different levels, that is, the representation of information is generated, mainly vectors, which are expressed as the extracted entities, relationships, and attributes. According to their different lengths, different embedding models are used to generate sentence-level and word-level embeddings, accurately representing knowledge from different vector dimensions. In particular, the present invention still has special embedding models for long texts and pictures to deal with different types of data. The specific process is as follows Figure 3 As shown, Figure 3 It shows different types of embedding methods, including word-level embedding for entities / attributes, relational embedding for relations, and sentence / paragraph-level embedding for long texts. They are always mapped to the original data. All data on the knowledge graph has corresponding embedding vectors for similarity queries.

[0049] Step 4: Optimize user query

[0050] The original user query often does not produce a good retrieval effect, and needs to be optimized to achieve better retrieval matching results. First, the user's input query is obtained; then, the large language model is used to complete the query according to the context; finally, the complex query is split into "sub-queries", represented by Q = [q1, q2, ... qn], and the "keywords" are extracted, represented by K = [k1, k2, ..., km].

[0051] Step 5: Multi-way search matching

[0052] In this step, after preprocessing, the user query has extracted the keyword sequence K and the subquery sentence sequence Q. All elements of K are embedded at the word level, and all elements of Q are embedded at the sentence level. Then the embeddings in K and Q are matched one by one in the knowledge graph according to the cosine similarity, and the top p matched entities, relations, and attributes are retained. Finally, p knowledge graph subgraphs are extracted from p elements, and each subgraph extracts information about q layers of relations. The above p and q are variables determined by the actual scenario. In practice, we usually extract subgraphs with 3 layers of relations. Figure 3 An example with q = 2 and p = 1 is shown. It is worth noting that all embedding vectors are queried and similarity is calculated concurrently. If different embeddings query the same subgraph or intersecting subgraphs (attributes, relationships, and entities may intersect), the graph needs to be merged.

[0053] It is worth mentioning that query algorithms are mainly divided into serial and parallel queries, both of which are query tasks that need to be executed multiple times. Serial query means that there is a progressive query relationship in Q from q1->q2->q3...->qn, and the latter sub-question depends on the result of the previous sub-question, for example, "What other works does the author of Romance of the Three Kingdoms have?"; Parallel tasks mean that q1, q2..., qn in Q are independent of each other, but each one still needs to be queried once, for example, "Who are the authors of Romance of the Three Kingdoms and Dream of the Red Chamber?". In more complex cases, serial and parallel queries may exist in a query at the same time, and both correspond to multiple embedded elements of Q and K.

[0054] according to Figure 2 It can be seen that no matter it is a serial or parallel query task, or even a more complex mixed cross-query task of the two, the patent of the present invention can locate the nodes of the retrieved entity and its associated other entities in a single search in the knowledge graph. However, when using the traditional RAG algorithm query, the program logic will involve recursive queries. When encountering complex queries, the algorithm complexity will increase exponentially, and the algorithm lacks flexibility and is difficult to iterate and maintain.

[0055] Step 6: Restore search result text and search enhancements

[0056] This step first uses the big model to restore the entities, relationships, attributes, etc. obtained from the knowledge graph to the form of natural language text (unstructured); then the context of the RAG and the user's prompt are integrated, and the prompts given to the big model are designed according to the scenario to achieve the effect of retrieval enhancement. In particular, if the content returned by the search contains image information, the multimodal model will be called to identify the image content and summarize it into text information and return it to the big language model; if the user needs the original document, the source file and source image can also be returned to the user at the same time.

[0057] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A large model retrieval enhancement method based on knowledge graph and knowledge representation, characterized in that: Here are the steps: S1. Parse documents and other unstructured data uploaded by users; S2, using the preset template, large model (LLM) to extract entities, relations and attributes in the document and build a knowledge graph; S3, perform multi-level embedding on the extracted data, and map and store it with the original data of the knowledge graph as knowledge representation; S4. Optimize user queries, supplement and rewrite query statements, and extract query clauses and keywords; S5, embed the processed query in multiple levels, and perform similarity calculation and retrieval in the knowledge graph database; S6. Perform text restoration on the search results, merge them with the original query, and input them into LLM to generate answers.

2. According to claim 1, a large model retrieval enhancement method based on knowledge graph and knowledge representation is characterized in that: In step S1, the specific steps of parsing the documents and other unstructured data uploaded by the user are as follows: The source documents uploaded by users are stored in object storage servers or other file management services, and then text and images are extracted from the uploaded files through parsing libraries with different types of file content and stored in association with the documents.

3. According to claim 1, a large model retrieval enhancement method based on knowledge graph and knowledge representation is characterized in that: In step S2, using the preset template, the big model extracts entities, relationships and attributes in the document. The specific steps for constructing the knowledge graph are as follows: through the big model's ability to understand text semantics, the text and pictures processed in step S1 are extracted as "entity-relationship-entity" structured data, and attribute values ​​are marked for all entities. Then, according to the different document types uploaded by the user, the corresponding graph template is designed, and the extracted entities are filled into the corresponding positions of the template to form a knowledge graph.

4. According to claim 3, a large model retrieval enhancement method based on knowledge graph and knowledge representation is characterized in that: Use a large model and self-built entity vocabulary to label the attribute values ​​of the extracted entities.

5. According to claim 1, a large model retrieval enhancement method based on knowledge graph and knowledge representation is characterized in that: In step S3, the extracted data is embedded in multiple levels and mapped with the original data of the knowledge graph for storage. The specific steps of knowledge representation are as follows: For the extracted entities, relations, and attributes, different embedding models are used to generate sentence-level and word-level embeddings according to their different lengths, accurately representing the knowledge graph from vectors of different dimensions.

6. The large model retrieval enhancement method based on knowledge graph and knowledge representation according to claim 1 is characterized in that: In step S4, the specific steps of optimizing the user query, supplementing and rewriting the query statement, and extracting the query clauses and keywords are as follows: Get the user's input query, use the big model to complete the query according to the context, split the complex query into "sub-queries", represented by Q = [q1, q2, ... qn], and extract "keywords", represented by K = [k1, k2, ..., km].

7. The large model retrieval enhancement method based on knowledge graph and knowledge representation according to claim 1 is characterized in that: In step S5, the processed query is embedded in multiple levels, and the specific steps of similarity calculation and retrieval in the knowledge graph database are as follows: Keywords are embedded at the word level, sub-query statements are embedded at the sentence level, and then the embeddings of keywords and query clauses are entered into the graph database for vector calculation. Similarity is calculated using cosine. Word embedding is used to query attributes and entity nouns, and sentence embedding is used to query long-text knowledge. All queries are concurrent queries. Finally, they are sorted by similarity, and based on the limited query amount of required knowledge, multiple entity-relationship-entity knowledge graph subgraphs are returned, and the intersecting subgraphs are fused.

8. The large model retrieval enhancement method based on knowledge graph and knowledge representation according to claim 1 is characterized in that: In step S6, the specific steps of restoring the text of the search results, fusing them with the original query and inputting them into the big model to generate the answer are as follows: LLM is used to restore the entities, relationships, attributes, etc. obtained from the knowledge graph into the form of natural language text, and after being integrated with the original query, it is input into the large model to generate answers.

9. The large model retrieval enhancement method based on knowledge graph and knowledge representation according to claim 8 is characterized in that: In step S6, if the retrieved content contains image information, the multimodal model is called to identify the image content and then answer the question in combination with the text information.

Images