Retrieval enhancement search method based on abstract entity relationship
By fine-tuning a large language model with a small number of parameters to extract entities and relationships, and constructing a knowledge graph and vector database, the problem of handling abstract entities and pronoun references in traditional methods is solved, thereby improving the accuracy and robustness of the question answering system.
Patent Information
- Application Number
- CN202511647112.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-11
- Publication Date
- 2026-02-13
AI Technical Summary
Traditional entity relationship extraction methods cannot effectively handle abstract entities, pronoun references, and contextual implicit information, and suffer from problems such as error propagation, reliance on predefined types, and high local deployment costs.
By fine-tuning an open-source large language model with relatively few parameters, it specializes in extracting all entity types and their complex relationships from text, including regular named entities, abstract entities, and pronoun references. It also constructs a knowledge graph and vector database, and combines the first and second retrieval paths to improve the relevance and completeness of the retrieval.
While ensuring the cost-effectiveness of local deployment, it significantly improves the ability to understand complex linguistic phenomena, enhances the accuracy and robustness of the question-answering system, and solves the shortcomings of traditional methods.
Smart Images
Figure CN121525855A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge graph technology, and more specifically to an enhanced search method based on abstract entity relationships. Background Technology
[0002] Traditional entity relation extraction relies on unique named entities. However, many nouns in documents are not unique named entities, but rather abstract sets of scopes, which traditional entity relation extraction cannot effectively extract. Furthermore, considering people's note-taking habits, such as objects mentioned above being referred to by personal pronouns in the following text, traditional NLP models cannot grasp the relationship between entities and personal pronouns well, nor can they grasp the information hidden in the textual context.
[0003] Current relation extraction relies on the results of named entity extraction, which can lead to errors propagating from the named entity extraction stage to relation extraction, resulting in decreased accuracy. Furthermore, since traditional NLP models are largely probabilistic in named entity and relation extraction, entity types and relationships need to be predefined before extraction, leading to increased manual intervention, higher costs, and more complex processing. Large models can perform these tasks well. However, data privacy dictates that large model service providers cannot be used; large models need to be deployed locally, requiring the use of weighted open-source models, which are costly to deploy. Therefore, deploying smaller large models can save on hardware costs, but smaller large models perform worse than larger ones. Summary of the Invention
[0004] Traditional entity relationship extraction methods, as mentioned above, cannot effectively handle abstract entities, pronoun references, and contextually implicit information. They also suffer from technical problems such as error propagation, reliance on predefined types, and high local deployment costs. Therefore, this invention provides a retrieval enhancement method based on abstract entity relationships. This invention, through instruction fine-tuning, specializes an open-source large language model with relatively few parameters in extracting all entity types, including regular named entities, abstract entities, and pronoun references, and their complex relationships from text. This achieves significant improvements in understanding complex linguistic phenomena, enhancing the relevance and completeness of the retrieval context, and ultimately improving the accuracy and robustness of the question-answering system, while ensuring cost-effective local deployment.
[0005] The technical means employed in this invention are as follows: The retrieval enhancement search method based on abstract entity relationships includes the following steps: The first major language model is fine-tuned, and the fine-tuned major language model has the ability to extract entities and relationships between entities from unstructured text data; Using a fine-tuned large language model, text data in the target knowledge base is processed to extract all entities and relationships between entities, and a knowledge graph is constructed based on all entities and relationships between entities. The knowledge graph is stored in a graph database. Upon receiving a user's query request, the system executes the first search path and the second search path in parallel based on the query request. The steps of the first retrieval path include: identifying entities from the query request, querying the graph database, and obtaining subgraph information associated with the entities as the first context. The steps of the second retrieval path include: calculating the similarity between the vector representation of the query request and the text fragment vectors pre-stored in the vector database, and obtaining the most relevant text fragment as the second context; The first and second contexts are concatenated with the query request to form enhanced prompts. These enhanced prompts are then input into the second language model to generate a reliable answer.
[0006] Furthermore, the steps for fine-tuning a large language model specifically include: Prepare a training dataset that includes the original text and the entities and relationships annotated by humans or large models corresponding to the original text; Organize the training dataset into an instruction fine-tuning format; Based on the instructions, the format is fine-tuned, and low-rank adaptive techniques are used to fine-tune the large language model.
[0007] Furthermore, the steps for constructing the knowledge graph specifically include: Each text or document in the target knowledge base is processed through the fine-tuned large language model to extract entities and relationships; Each extracted entity is treated as a node, and each relationship is treated as an edge. A graph structure is constructed based on the nodes and edges. The graph structure is stored in a graph database.
[0008] Furthermore, the target knowledge base contains structured scale data, and when extracting entities and relations, specific fields from the structured data are selected and input into the large language model.
[0009] Furthermore, the first retrieval path specifically includes: The finely tuned large language model is used to extract entities from user query requests and generate extracted entities. The graph database is used to query the relationships directly connected to the extracted entities and other entities to form a subgraph. The structured information of the subgraph is converted into a natural language description, which serves as the first context.
[0010] Furthermore, the second retrieval path specifically includes: Divide the raw text data in the target knowledge base into blocks; Each text block is embedded, a vector is generated, and the vector is stored in a vector database; Calculate the embedding vector of the user's query request; By calculating cosine similarity, the K vectors most similar to the embedded vector are retrieved from the vector database, and the set of text blocks corresponding to the K vectors is used as the second context.
[0011] Furthermore, the method also includes a model testing step: The performance of the fine-tuned large language model was evaluated on a publicly available entity relation extraction dataset. The ablation experiments were used to evaluate the contribution of subgraph information provided by the first retrieval path and text fragment information provided by the second retrieval path to the accuracy of the retrieval results.
[0012] Furthermore, when fine-tuning or using the large language model for entity relation extraction, structured prompt words are employed, including: The task objective emphasizes the need to identify all entities and relationships of a given type in the text, taking into account the sentence components implied in the context. The task steps are clearly defined, including entity recognition and normalization, and relationship discovery and naming. Output format example.
[0013] Compared with the prior art, the present invention has the following advantages: This invention addresses the shortcomings of traditional NLP models in this regard by fine-tuning a large model to optimize its output accuracy and format. The fine-tuned model is then used for abstract entity relationship extraction, and the resulting relationships are stored in a graph database. The system can then retrieve entities related to the query and obtain the relationships between entities, ensuring the system's grasp of the overall knowledge base. Simultaneously, the original data is embedded, and the resulting vectors are stored in a vector database. The retrieved information ensures the system's grasp of the details of the knowledge base. Through the mutual supplementation of global and local detailed information, the accuracy of the system's responses can be improved.
[0014] Based on the above reasons, this invention can be widely applied in fields such as knowledge graphs. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of the enhanced search method based on abstract entity relationships of the present invention.
[0017] Figure 2 This is the entity relationship graph established in the embodiments of the present invention. Detailed Implementation
[0018] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0020] like Figure 1 As shown, this invention provides an enhanced search method based on abstract entity relationships, comprising the following steps: S1. Fine-tune the first large language model. The fine-tuned large language model has the ability to extract entities and relationships between entities from unstructured text data.
[0021] The specific steps for fine-tuning are as follows: S11. Prepare a training dataset that includes the original text and the entities and relationships annotated by humans or large models corresponding to the original text.
[0022] A portion of the data is set aside for fine-tuning the large model to extract entity relationships. This portion is either manually labeled or labeled by a larger model with more parameters. 80% of this data serves as the training dataset, and 20% as the test dataset.
[0023] S12. Organize the training dataset into the instruction fine-tuning (sharegpt) format.
[0024] S13. Fine-tune the format according to the instructions, and use low-rank adaptive techniques to fine-tune the large language model.
[0025] Some sentence components in documents are implied in the text's context and do not appear directly on the document, requiring the restriction of prompt words. Using a finely tuned large model, all data was labeled.
[0026] When fine-tuning or using large language models for entity relation extraction, structured cue words are employed. These cue words include: The task objective is to identify all entities and relationships of a given type in the text, taking into account implicit sentence components within the context. The task steps are clearly defined, including entity identification and normalization, and relationship discovery and naming. An example output format is provided.
[0027] Specifically, based on the prompt word engineering, this invention fine-tunes the prompt words for extracting entity relationships from the large model as follows: Prompt words: { -Target- You are an experienced entity relation extraction expert. Given a text document related to this activity and a set of entity types, identify all entities belonging to those types and all relationships between them from the text. Sentences may lack subjects, but these omissions are implied in the context; please do not ignore them. *If you do not identify entities and relationships from the text, do not fabricate entities and relationships. *Note: If you extract relationships, *check the head and tail entities of the relationship to see if they appear in your previously extracted entities. If the entities extracted in the two steps are inconsistent, prioritize the entities extracted in the first step. -step- 1. Identify all entities in a document or text. An entity may have multiple names, which you need to merge into a unique one.
[0028] 2. From the entities identified in step 1, find all *obviously related* (source entity, target entity) pairs and name the relationships between them, based on the context.
[0029] 3. After completion, output ###Example### Example 1: {Text Example} Output 1: {[entity list], [relationship list]} ###Real Data### Real data: {real text or document} Output: } Marking different parts with special symbols helps large models better understand prompt words, and standardized output examples can also improve the recognition accuracy of large models.
[0030] S2. Using the fine-tuned large language model, the text data in the target knowledge base is processed to extract all entities and relationships between entities, and a knowledge graph is constructed based on all entities and relationships between entities. The knowledge graph is stored in a graph database.
[0031] The specific steps involved in constructing a knowledge graph include: First, each text or document in the target knowledge base is processed using a fine-tuned large language model to extract entities and relationships.
[0032] Secondly, each extracted entity is treated as a node, and each relationship as an edge, and a graph structure is constructed based on the nodes and edges.
[0033] Finally, the graph structure is stored in a graph database.
[0034] The fine-tuned large model can better understand the task and output a more controllable format, because most large models with a small number of parameters do not understand instructions very well. By fine-tuning the output format and understanding ability of the large model, real data is input into the large model through the above prompts. Entities and their relationships are extracted for each text or document, and each entity and its relationship are connected and stored in a graph database, such as Neo4j.
[0035] The target knowledge base contains structured scale data. When extracting entities and relations, specific fields from the structured data are selected and input into the large language model.
[0036] S3. Receive the user's query request and execute the first search path and the second search path in parallel according to the query request.
[0037] The steps of the first retrieval path include: identifying entities from the query request, querying the graph database, and obtaining subgraph information associated with the entities as the first context.
[0038] Specifically, the first search path is: The first step is to use the fine-tuned large language model to extract entities from the user query request and generate the extracted entities.
[0039] The second step is to query the graph database for the relationships directly connected to the extracted entities and other entities to form a subgraph.
[0040] The third step is to convert the structured information of the subgraph into a natural language description, which serves as the first context.
[0041] For example, the user first enters a query question, or simply query. The large model then extracts existing entities. The prompts mentioned above can be used for this step. By querying the graph database, related subgraphs are retrieved, such as (entity 1, relation 1, entity 2), (entity 3, relation 2, entity 4), and so on. In this way, all relevant information about an entity in the knowledge graph is obtained.
[0042] The steps of the second retrieval path include: calculating the similarity between the vector representation of the query request and the text fragment vectors pre-stored in the vector database, and obtaining the most relevant text fragment as the second context.
[0043] Specifically, the second search path is: The first step is to divide the original text data in the target knowledge base into blocks.
[0044] The second step is to embed each text block, generate vectors, and store them in a vector database.
[0045] The third step is to calculate the embedding vector of the user query request.
[0046] The fourth step involves calculating cosine similarity and retrieving the K vectors most similar to the embedded vector from the vector database. The set of text blocks corresponding to these K vectors serves as the second context.
[0047] S4. Concatenate the first and second contexts with the query request to form enhanced prompts. Input the enhanced prompts into the second language model to generate a reliable answer.
[0048] Specifically, the query and the context obtained from the two processes mentioned above are concatenated and passed to the large model, allowing the large model to respond to the query within this context. Knowledge graph data is based on a global graph, containing the relationships between all entities in a document. Graph retrieval grasps the macro-level information of the knowledge base, while retrieval in a vector database involves querying specific related documents within the knowledge base, allowing for a grasp of the data's details. Through the context composed of macro and micro information, the large model can provide a more accurate response.
[0049] S5. Evaluate the performance of the fine-tuned large language model on a publicly available entity relation extraction dataset.
[0050] S6. Through ablation experiments, evaluate the contribution of subgraph information provided by the first retrieval path and text fragment information provided by the second retrieval path to the accuracy of the retrieval results.
[0051] Example This embodiment provides a search-enhanced method for assessing autism in children, the method specifically including: The data used is an autism assessment scale, sourced from the Hong Kong Heep Hong Society's Children's Learning Framework, and is used to assist in the assessment of childhood autism. The data is in tabular form and includes test items, ability categories, success criteria, etc.
[0052] S1. Fine-tuning the first major language model, the fine-tuned major oracle model has the ability to extract entities and relationships between entities from unstructured text data.
[0053] S11. Prepare a training dataset that includes the original text and the entities and relationships annotated by humans or large models corresponding to the original text.
[0054] A portion of the data is used to fine-tune the entity relationship extraction of the larger model. This portion is either manually labeled or labeled by a larger model with more parameters. 80% of this data serves as the training dataset, and 20% as the test dataset. For example, in the evaluation scale, one test item is named: "Toss a small plastic ball downwards while standing." The extracted abstract entities and their relationships are: (child, toss, small plastic ball).
[0055] S12. Organize the training dataset into the instruction fine-tuning (sharegpt) format.
[0056] S13. Fine-tune the format according to the instructions, and use low-rank adaptive techniques to fine-tune the large language model.
[0057] Specifically, in this embodiment, based on the prompt word engineering, the prompt words for extracting entity relationships from the large model are fine-tuned as follows: Prompt words: { -Target- You are an experienced entity and relation extraction expert. Given an assessment test text related to autism assessment, identify all entities belonging to these types and all relationships between them. Sentences may lack subjects, but these omissions are implied in the context; please do not ignore them. The test subjects are children; please be aware of this. *If you do not identify entities and relationships from the text, do not fabricate entities and relationships. *Note that if you extract relationships, *please check the head and tail entities of the relationship to see if they appear in the entities you extracted previously. If the entities extracted in the two steps are inconsistent, prioritize the entities extracted in the first step. -step- 1. Identify all entities in a document or text. An entity may have multiple names, which you need to merge into a unique one.
[0058] 2. From the entities identified in step 1, find all *obviously related* (source entity, target entity) pairs and name the relationships between them, based on the context.
[0059] 3. After completion, output ###Example### Example 1: {Text Example} Output 1: {[entity list], [relationship list]} ###Real Data### Real data: {real text or document} Output: } S2. Using the fine-tuned large language model, the text data in the target knowledge base is processed to extract all entities and relationships between entities, and a knowledge graph is constructed based on all entities and relationships between entities. The knowledge graph is stored in a graph database.
[0060] like Figure 2 The image shows the application of knowledge graph component methods to scale-type data. The scale data involves autism assessment in children, and the data is stored in XLSX and CSV formats. In this method, each item is a complete test item. For this data, only a few fields are selected and the aforementioned prompt words are added. This image shows a partial data example.
[0061] S3. Receive the user's query request and execute the first search path and the second search path in parallel according to the query request.
[0062] The first retrieval path is as follows: The user first enters their query question, or simply query, and the large model extracts existing entities. The prompts mentioned above can be used for this step. By querying the graph database, relevant subgraphs are retrieved, such as (children, throwing, small plastic balls)... In this way, all relevant information about the entity in the knowledge graph is obtained.
[0063] The second retrieval path is as follows: divide the xlsx and csv files into blocks by row, embed each block of data, store the original data and its embedding vector in a vector database, calculate the embedding vector of the user query, and then calculate the cosine similarity with all vectors in the vector database. Find the top k vectors with the highest similarity, and then find the corresponding original data fragments according to the data index. These data fragments can be regarded as the context related to the user query.
[0064] Specifically, the second search path includes: The first step is to divide the original document data D into individual text blocks. Then use the embedded model Convert each text block into a feature vector (These vectors form a set F), and finally the original text block and the corresponding embedding vector are stored together in the vector database.
[0065] The second step is to use the embedding model to convert the query into an embedding vector Q when the user initiates a query.
[0066] Step 3: Calculate the query vector and the vector f of all text blocks in the vector database. i The cosine similarity is used to measure the directional similarity between two vectors. The formula for cosine similarity is: [1][2]
[0067] Here, α and β represent two embedding vectors [3].
[0068] S4. Concatenate the first and second contexts with the query request to form enhanced prompts. Input the enhanced prompts into the second language model to generate a reliable answer.
[0069] The final result is a comprehensive answer obtained by the large model based on the user's query, similar text blocks, entities, and their relationships.
[0070] S5. Evaluate the performance of the fine-tuned large language model on a publicly available entity relation extraction dataset.
[0071] S6. Through ablation experiments, evaluate the contribution of subgraph information provided by the first retrieval path and text fragment information provided by the second retrieval path to the accuracy of the retrieval results.
[0072] The specific ablation experiment in this embodiment is as follows: 1. Determine the validity of the first search path.
[0073] Regarding the context provision, only the subgraph information of the entities related to the user query is provided, and the context information obtained from the second search path is not included.
[0074] 2. Determine the validity of the second search path.
[0075] In terms of context provision, only document block information related to the user query is provided, excluding entity subgraph information obtained from the first search path.
[0076] The reference materials for this embodiment include: [1] Edge D, Trinh H, Cheng N, Bradley J, Chao A, Mody A, Truitt S,Larson J. From local to global: A graph RAG approach to query-focusedsummarization[J / OL]. arXiv, 2024, : [2025-11-07]. [2] Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih Wt, Rocktäschel T, Riedel S, Kiela D. Retrieval-augmented generation for knowledge-intensive NLP tasks[C] / / Advances in Neural Information Processing Systems (NeurIPS). 2020: 9459-9474. [3] Douze M, Guzhva A, Deng C, Johnson J, Szilvasy G, Mazaré PE,Lomeli M, Hosseini L, Jégou H. The Faiss library[J / OL]. arXiv, 2024, : [2025-11-07]. [4] Hu EJ, Shen Y, Wallis P, et al. LoRA: Low-rank adaptation oflarge language models[J / OL]. arXiv, 2021, : [2025-11-07]. [5] White J, Fu Q, Hays S, Sandborn M, Olea C, Gilbert H, Elnashar A, Spencer-Smith J, Schmidt D C. A prompt pattern catalog to enhance promptengineering with ChatGPT[J / OL]. arXiv, 2023, : [2025-11-07]. [6] Vasiliev Y. Natural language processing with Python and spaCy: Apractical introduction[M]. No Starch Press, 2020. The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An enhanced search method based on abstract entity relationships, characterized in that, Includes the following steps: The first major language model is fine-tuned, and the fine-tuned major language model has the ability to extract entities and relationships between entities from unstructured text data; Using a fine-tuned large language model, text data in the target knowledge base is processed to extract all entities and relationships between entities, and a knowledge graph is constructed based on all entities and relationships between entities. The knowledge graph is stored in a graph database. Upon receiving a user's query request, the system executes the first search path and the second search path in parallel based on the query request. The steps of the first retrieval path include: identifying entities from the query request, querying the graph database, and obtaining subgraph information associated with the entities as the first context. The steps of the second retrieval path include: calculating the similarity between the vector representation of the query request and the text fragment vectors pre-stored in the vector database, and obtaining the most relevant text fragment as the second context; The first and second contexts are concatenated with the query request to form enhanced prompts. These enhanced prompts are then input into the second language model to generate a reliable answer.
2. The enhanced search method based on abstract entity relationships according to claim 1, characterized in that, The steps for fine-tuning a large language model specifically include: Prepare a training dataset that includes the original text and the entities and relationships annotated by humans or large models corresponding to the original text; Organize the training dataset into an instruction fine-tuning format; Based on the instructions, the format is fine-tuned, and low-rank adaptive techniques are used to fine-tune the large language model.
3. The enhanced search method based on abstract entity relationships according to claim 1, characterized in that, The specific steps for constructing the knowledge graph include: Each text or document in the target knowledge base is processed through the fine-tuned large language model to extract entities and relationships; Each extracted entity is treated as a node, and each relationship is treated as an edge. A graph structure is constructed based on the nodes and edges. The graph structure is stored in a graph database.
4. The enhanced search method based on abstract entity relationships according to claim 3, characterized in that, The target knowledge base contains structured scale data. When extracting entities and relations, specific fields from the structured data are selected and input into the large language model.
5. The enhanced search method based on abstract entity relationships according to claim 1, characterized in that, The first search path specifically includes: The finely tuned large language model is used to extract entities from user query requests and generate extracted entities. The graph database is used to query the relationships directly connected to the extracted entities and other entities to form a subgraph. The structured information of the subgraph is converted into a natural language description, which serves as the first context.
6. The enhanced search method based on abstract entity relationships according to claim 1, characterized in that, The second search path specifically includes: Divide the raw text data in the target knowledge base into blocks; Each text block is embedded, a vector is generated, and the vector is stored in a vector database; Calculate the embedding vector of the user's query request; By calculating cosine similarity, the K vectors most similar to the embedded vector are retrieved from the vector database, and the set of text blocks corresponding to the K vectors is used as the second context.
7. The enhanced search method based on abstract entity relationships according to claim 1, characterized in that, The method also includes a model testing step: The performance of the fine-tuned large language model was evaluated on a publicly available entity relation extraction dataset. The ablation experiments were used to evaluate the contribution of subgraph information provided by the first retrieval path and text fragment information provided by the second retrieval path to the accuracy of the retrieval results.
8. The enhanced search method based on abstract entity relationships according to claim 2, characterized in that, When fine-tuning or using the large language model for entity relation extraction, structured prompt words are employed, including: The task objective emphasizes the need to identify all entities and relationships of a given type in the text, taking into account the sentence components implied in the context. The task steps are clearly defined, including entity recognition and normalization, and relationship discovery and naming. Output format example.