Method and system for relieving large model illusion through knowledge graph retrieval

Abstract

The invention discloses a method and system for relieving large model illusion through knowledge graph retrieval, and belongs to the technical field of natural language processing. The method comprises the following steps: constructing a knowledge graph; performing document semantic dicing based on the knowledge graph; constructing a community according to the blocks; determining a corresponding community to be retrieved from the communities according to the entity of the query question; and retrieving from the to-be-retrieved community to obtain an answer. The knowledge graph provided by the invention has a clear structure and certainty, and can clearly represent the relationship between entities, so that key fact details can be clearly presented. Through entity disambiguation, knowledge dispersed in a plurality of documents can be connected through entities and relationships to form a net-shaped knowledge structure instead of keeping the documents isolated. According to the method, the accuracy and correlation of knowledge retrieval can be improved, so that the overall performance of the question-answering system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a method and system for alleviating large model hallucinations through knowledge graph retrieval. Background Art

[0002] The traditional Retrieval-Augmented Generation (RAG) technology is a method for enhancing large language models at the document level, aiming to solve the hallucination problem that may occur when large models generate answers, that is, fabricating facts based on the internal knowledge of the model, thus producing incorrect answers. This technology constructs an unstructured or semi-structured knowledge base by performing layout analysis and chunking on documents, so as to retrieve relevant knowledge in the question-and-answer session, supplement the knowledge blind spots of large models, and effectively reduce the hallucination problem, especially when the retrieval results are accurate.

[0003] However, traditional RAG mainly relies on two knowledge retrieval strategies: semantic similarity and keyword matching between document slices and user queries. Although this method focuses on overall relevance, it is prone to ignoring local details, lacking precise grasp of key information in the question, and sometimes may even be interfered by some irrelevant local information. In addition, after document chunking, the relative position information between chunks is lost, and it is difficult to directly establish connections between documents.

[0004] In dealing with complex queries, precise retrieval positioning is particularly crucial. However, traditional RAG technology does not perform well in this regard. The similarity metric it relies on is difficult to accurately evaluate the relevance of documents, resulting in difficulty in precisely locating document fragments that meet the conditions. In addition, irrelevant texts may be omitted or retrieved incorrectly during the retrieval process, which not only increases the noise in the model's knowledge but also may affect the ability of large models to accurately answer query questions. Take a question involving multi-condition retrieval as an example: "Among the mobile phones newly released in the market in 2024, recommend several mobile phones equipped with Snapdragon 8 Gen 3 or Dimensity 9300 chips and with a price range of 3000-6000 yuan." This question contains three conditions: release year, specific chip, and price range. Retrieval based on document chunks may find text blocks that only meet one or two conditions, thus introducing a lot of knowledge that does not meet the conditions, causing the large model to have hallucinations, misinterpreting the condition "and" as "or", and then generating incorrect responses.

[0005] Therefore, it is particularly important to introduce a knowledge graph to enhance the precision of retrieval. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for alleviating large model hallucinations through knowledge graph retrieval. The knowledge graph of the present invention has a clear structure and certainty, and can clearly represent the relationships between entities, thus clearly presenting key factual details. Through entity disambiguation, the knowledge scattered in multiple documents can be connected by entities and relationships to form a networked knowledge structure, rather than keeping the documents isolated from each other. This method helps to improve the accuracy and relevance of knowledge retrieval, thereby enhancing the overall performance of the question-answering system.

[0007] To solve the above technical problems, the present invention provides a method for alleviating large model hallucinations through knowledge graph retrieval, including:

[0008] Construct a knowledge graph;

[0009] Perform document semantic chunking based on the knowledge graph;

[0010] Construct communities according to the chunks;

[0011] Determine the corresponding communities to be retrieved from the communities according to the entities in the query question;

[0012] Retrieve from the communities to be retrieved to obtain answers.

[0013] Preferably, constructing the knowledge graph specifically includes the following steps:

[0014] Extract entities and attributes from the documents;

[0015] According to the entity context, the large model extracts the relationships between entities from the documents;

[0016] Perform entity disambiguation on the entities and the relationships between entities;

[0017] Based on the entities, attributes and relationships, construct triples to form a knowledge graph.

[0018] Preferably, performing entity disambiguation on the entities and the relationships between entities specifically includes the following steps:

[0019] In combination with new documents, the large model merges the existing graph and the extracted subgraph; or, use the method of similarity comparison to process entities and relationships respectively.

[0020] Preferably, performing document semantic chunking based on the knowledge graph specifically includes the following steps:

[0021] Statistically calculate the starting positions of each triple in the document and sort them;

[0022] Calculate the similarity between the current triple and the next triple;

[0023] If the similarity between the current triple and the next triple is greater than the first threshold, the start position of the current triple is used as the starting point of the document slice, and the end position of the next triple is used as the end point of the document slice to perform document semantic chunking to obtain a slice;

[0024] If the similarity between the current triple and the next triple is less than the second threshold, the start position of the current triple is used as the starting point of the document slice, and the end position of the current triple is used as the end point of the document slice to perform document semantic chunking to obtain a slice;

[0025] Compare the lengths of each slice with the preset length, and merge the slices whose lengths are less than 60% of the preset length into the adjacent slices with high similarity.

[0026] Preferably, constructing communities according to the chunks specifically includes the following steps:

[0027] Based on the PageRank algorithm, calculate the pagerank value of the nodes in the knowledge graph, and select the central points according to the pagerank value;

[0028] Based on the Louvain community discovery algorithm, traverse all neighbor nodes of the central point, and calculate the modularity increment after moving the neighbor nodes to the community where the core node is located;

[0029] According to the modularity increment, select the optimal move to complete the merger of communities.

[0030] Preferably, it further includes the following steps:

[0031] For a new document, the large model constructs a subgraph;

[0032] Based on the pagerank algorithm, calculate the pagerank value of the nodes in the subgraph, and judge whether there is a core node according to the pagerank value;

[0033] If there is no core node, it is incorporated into the community of the existing knowledge graph according to its connection relationship, and no new community is added;

[0034] If there is a core node, the connected communities need to re - perform the Louvain community discovery algorithm.

[0035] Preferably, according to the entity of the query problem, determining the corresponding community to be retrieved from the communities specifically includes the following steps:

[0036] Identify the entity from the query problem;

[0037] Based on similarity calculation, obtain the initial nodes from the entities in the knowledge graph according to the entity in the query problem;

[0038] Based on beam search, starting from the initial node, traverse the adjacent nodes, and select the adjacent triples that exceed the third threshold from the top k results with the highest similarity to obtain the retrieval result path and the retrieval result triples;

[0039] According to the retrieval result path and the retrieval result triples, determine the corresponding community to be retrieved from the community.

[0040] Preferably, according to the retrieval result path and the retrieval result triples, determine the corresponding community to be retrieved from the community, and perform retrieval from the community to be retrieved to obtain the answer, which specifically includes the following steps:

[0041] Locate the community where the entity queried according to the retrieval result path and the retrieval result triples as the community to be retrieved;

[0042] Based on vector retrieval and keyword retrieval, perform retrieval from the slices of the community to be retrieved to obtain the answer.

[0043] Preferably, it further includes the following steps:

[0044] Query the k-layer neighbor nodes of the initial node, calculate the similarity between the triples and the query question, and use the triples with similarity exceeding the fourth threshold as the rewritten triples;

[0045] Rewrite the query question according to the rewritten triples.

[0046] The present invention also provides a system for alleviating the hallucination of large models through knowledge graph retrieval, including:

[0047] A construction module for constructing a knowledge graph;

[0048] A document semantic slicing module for performing document semantic slicing based on the knowledge graph;

[0049] A community construction module for constructing a community according to the slices;

[0050] A community location module for determining the corresponding community to be retrieved from the community according to the entity of the query question;

[0051] A retrieval module for performing retrieval from the community to be retrieved to obtain the answer.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] (1) Aiming at the inaccuracy problem of traditional retrieval, by introducing a knowledge graph, enhance the accuracy of knowledge retrieval and alleviate the hallucination problem of large model question answering;

[0054] Specifically, the document chunk similarity retrieval provides the ability of relevance retrieval at the global level, while the knowledge graph is incomplete compared to the knowledge of the complete document. Using only the graph may only retrieve the facts stated in the query question, ignoring the content of the original document slices, which is necessary for the injection of the large model. Therefore, it is crucial to establish the connection between the knowledge graph and the document.

[0055] (2) To ensure the integrity of extraction, the document should be triple-extracted chapter by chapter. If the document lacks a clear structure, the full text should be used (if it is too long, it should be split into whole sentences according to the set length). Using the extracted graph to guide document chunking can make the document chunking more reasonable and avoid cutting off logical relationships. Document semantic chunking is performed based on the positional relationship, similarity of the extracted triples, and the preset text length, so as to establish the correspondence between entities and document slices.

[0056] (3) Drawing on the idea of GraphRAG, community discovery is performed on the constructed knowledge graph, and the entire graph is divided into multiple basically independent communities (with close internal connections and sparse external connections) according to the connection relationship. Slices from different documents are aggregated into a community. During retrieval, first use the knowledge graph retrieval to determine the community where the answer may exist, and then perform precise retrieval within the community, which can narrow the search space, speed up the search, and reduce the recall of irrelevant documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The following further elaborates the specific implementation manners of the present invention in conjunction with the accompanying drawings.

[0058] Figure 1 It is a schematic diagram of a knowledge graph and community discovery example. DETAILED DESCRIPTION OF THE INVENTION

[0059] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0060] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term " / and / " used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more of the associated listed items.

[0061] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0062] The following further describes the present invention in detail with reference to the accompanying drawings:

[0063] The present invention provides a method for alleviating large model hallucinations through knowledge graph retrieval, including:

[0064] Construct a knowledge graph;

[0065] Perform document semantic chunking based on the knowledge graph;

[0066] Construct communities according to the chunks;

[0067] Determine the corresponding communities to be retrieved from the communities according to the entities of the query question;

[0068] Retrieve from the communities to be retrieved to obtain answers.

[0069] Preferably, constructing a knowledge graph specifically includes the following steps:

[0070] Extract entities and attributes from the document;

[0071] According to the entity context, the large model extracts the relationships between entities from the document;

[0072] Perform entity disambiguation on the entities and the relationships between entities;

[0073] Based on the entities, attributes and relationships, construct triples to form a knowledge graph.

[0074] Preferably, performing entity disambiguation on the entities and the relationships between entities specifically includes the following steps:

[0075] In combination with the new document, the large model merges the existing graph and the extracted subgraph; or, uses the method of similarity comparison to process entities and relationships respectively.

[0076] Preferably, performing document semantic chunking based on the knowledge graph specifically includes the following steps:

[0077] Statistically analyze the starting positions of each triple in the document and sort them;

[0078] Calculate the similarity between the current triple and the next triple;

[0079] If the similarity between the current triple and the next triple is greater than the first threshold, take the start position of the current triple as the starting point of the document slice and the end position of the next triple as the ending point of the document slice, perform document semantic chunking, and obtain the slice;

[0080] If the similarity between the current triple and the next triple is less than the second threshold, take the start position of the current triple as the starting point of the document slice and the end position of the current triple as the ending point of the document slice, perform document semantic chunking, and obtain the slice;

[0081] Compare the lengths of each slice with the preset length, and merge the slices whose lengths are less than 60% of the preset length into the adjacent slices with high similarity.

[0082] Preferably, construct communities according to the chunking, which specifically includes the following steps:

[0083] Based on the PageRank algorithm, calculate the pagerank value of the nodes in the knowledge graph, and select the central points according to the pagerank value;

[0084] Based on the Louvain community discovery algorithm, traverse all neighbor nodes of the central point, and calculate the modularity increment after moving the neighbor nodes to the community where the core node is located;

[0085] According to the modularity increment, select the optimal move to complete the merger of communities.

[0086] Preferably, it further includes the following steps:

[0087] For a new document, the large model constructs a subgraph;

[0088] Based on the pagerank algorithm, calculate the pagerank value of the nodes in the subgraph, and judge whether there is a core node according to the pagerank value;

[0089] If there is no core node, incorporate it into the community of the existing knowledge graph according to its connection relationship, and no new community will be added;

[0090] If there is a core node, the connected communities need to perform the Louvain community discovery algorithm again.

[0091] Preferably, according to the entity of the query problem, determine the corresponding community to be retrieved from the community, which specifically includes the following steps:

[0092] Identify the entity from the query problem;

[0093] Based on similarity calculation, obtain the initial nodes from the entities in the knowledge graph according to the entity in the query problem;

[0094] Starting from the initial node, traverse the adjacent nodes based on beam search, and select the adjacent triples that exceed the third threshold from the top k results with the highest similarity to obtain the retrieval result path and the retrieval result triples;

[0095] Based on the retrieval result path and the retrieval result triples, determine the corresponding community to be retrieved from the community.

[0096] Preferably, based on the retrieval result path and the retrieval result triples, determine the corresponding community to be retrieved from the community, and perform a retrieval in the community to obtain the answer, which specifically includes the following steps:

[0097] Locate the community where the entity queried according to the retrieval result path and the retrieval result triples is located as the community to be retrieved;

[0098] Based on vector retrieval and keyword retrieval, perform a retrieval in the slice of the community to be retrieved to obtain the answer.

[0099] Preferably, it further includes the following steps:

[0100] Query the k-layer neighbor nodes of the initial node, calculate the similarity between the triples and the query question, and use the triples with similarity exceeding the fourth threshold as the rewritten triples;

[0101] Rewrite the query question according to the rewritten triples.

[0102] The present invention also provides a system for alleviating large model hallucinations through knowledge graph retrieval, including:

[0103] A construction module for constructing a knowledge graph;

[0104] A document semantic slicing module for performing document semantic slicing based on the knowledge graph;

[0105] A community construction module for constructing communities according to the slices;

[0106] A community location module for determining the corresponding community to be retrieved from the community according to the entity of the query question;

[0107] A retrieval module for performing a retrieval in the community to be retrieved to obtain the answer.

[0108] The knowledge graph of the present invention has a clear structure and certainty, can clearly represent the relationships between entities, and thus clearly presents the key fact details. Through entity disambiguation, the knowledge scattered in multiple documents can be connected through entities and relationships to form a networked knowledge structure, rather than keeping the documents isolated. This method helps to improve the accuracy and relevance of knowledge retrieval, thereby enhancing the overall performance of the question-answering system.

[0109] To better illustrate the technical effects of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0110] Embodiment 1. A method for alleviating large model hallucinations through knowledge graph retrieval. Starting from three aspects: the construction, retrieval, and enhancement of the knowledge graph, an in-depth analysis and exploration are carried out on how to use the knowledge graph to enhance the question-answering ability of large models and alleviate the problem of model hallucinations, and a set of solutions is proposed:

[0111] 1. Graph construction: The present invention divides according to the inherent structure of the document, follows the predefined schema, uses a large language model to extract information, constructs a knowledge graph, and ensures semantic coherence. The extracted knowledge graph triples can in turn guide the semantic segmentation of the document, establishing the connection between entity nodes and document fragments; through the community discovery algorithm, the knowledge graph is divided into multiple subgraphs, aggregating relevant knowledge points scattered in different documents to form an organic whole.

[0112] 2. Knowledge graph retrieval: Retrieval is divided into two categories: one is divergent retrieval, such as querying which diseases a certain drug can treat; the other is path retrieval, such as querying the treatment plan required for a certain symptom (symptom-disease-treatment method-medication). We use the beam search method to handle both retrieval methods simultaneously.

[0113] 3. Retrieval enhancement: First, the adjacent triples of the entities involved in the query question can provide more professional information, which can be used to standardize the user's query question; in the retrieval stage, we first perform knowledge graph retrieval, locate the corresponding community according to the retrieval result, and further retrieve the document fragment, and input the result of the hybrid retrieval into the large language model to finally generate an accurate Q&A reply. Knowledge graph and community discovery examples are as Figure 1 shown.

[0114] S1. Knowledge graph construction

[0115] S1.1 Knowledge graph extraction: Before performing knowledge graph extraction, it is first necessary to define the schema:

[0116] EntityType:

[0117] -Theorem(Theorem):

[0118] properties:

[0119] desc: Description

[0120] -Definition(Mathematical definition):

[0121] properties:

[0122] desc: Description

[0123] money: Reward amount

[0124] -KnowledgePoint (Knowledge point):

[0125] properties:

[0126] desc: Description

[0127] -Formula (Mathematical formula)

[0128] -Question (Question):

[0129] properties:

[0130] desc: Description

[0131] RelationType:

[0132] -Derived (Derivation)

[0133] -SolveProblem (Solve the problem)

[0134] S1.1.1 Entity and attribute extraction

[0135] Divide the paragraphs according to the document structure, use the language large model to extract the knowledge graph entities and their positions in the original text, and at the same time determine the entity attributes to be extracted according to the entity type, and extract based on the entity context.

[0136] S1.1.2 Relationship extraction, based on the extracted entities, extract the relationships between the entities according to the context. In order to ensure the interpretability of the relationships, guide the large model to find a paragraph of text from the document to explain the rationality of the establishment of the triple.

[0137] S1.2 Entity disambiguation: mainly to solve the problems of synonymous entities and conflicting triples (different relationships between entities), and there are two solutions:

[0138] (1) Combine the new document, and the large model merges the existing knowledge graph and the extracted sub-graph;

[0139] The advantage of method (1) is that it considers the enhanced representation of the entity and relationship semantics by the graph structure, but the disadvantage is that the volume of the existing knowledge graph is very large, and this strategy is almost infeasible;

[0140] (2) Use the method of similarity comparison to process entities and relationships respectively.

[0141] Method (2) is faster, but it separates entities and relationships and represents them separately, losing the enhancement of the graph structure on the representation.

[0142] S1.3 Document Semantic Chunking Based on Knowledge Graph

[0143] S1.3.1 First, count the starting positions (head entity and tail entity) of each triple in the document and sort them according to their starting positions. Define the starting point of the initial document slice as 0, preset the document chunk length, and calculate the similarity or perplexity between adjacent triples:

[0144] The similarity can be calculated by the reranker model, specifically as follows:

[0145] (1) If the similarity of adjacent triples is very high (the similarity is greater than the first threshold, and the first threshold can be set to 0.7), then move the end point of the document slice to the end position of the latter triple. Compare the current length with the preset length. If it is less than the preset length, continue to compare the next triple; otherwise, it is necessary to judge whether the current length exceeds 50% of the preset length (high similarity can weaken the limit of the exceeded length). If it exceeds, move the slice end point back to the end position of the previous triple, otherwise no modification is required, and it can be directly output, and define the starting point of the next document slice;

[0146] (2) If the similarity of adjacent triples is very low (the similarity is less than the second threshold, and the second threshold can be set to 0.3), the end point of the document slice is at the end position of the previous triple. Compare the current length with the preset length. If it is less than the preset length, continue to compare the next triple; otherwise, it is necessary to judge whether the current length exceeds the preset length. If it exceeds, move the slice end point back to the end position of the previous triple (special case: if the current document slice only contains one triple, but the length far exceeds the preset length, then directly take the preset length of units backward from the slice starting point and output the document slice), otherwise no modification is required, and it can be directly output, and define the starting point of the next document slice.

[0147] It can also be judged by calculating the perplexity of the text sequence. The specific calculation method is as follows:

[0148]

[0149] In the formula: Perplexity is the perplexity, P() is the conditional probability of each word in the sequence calculated by the language model, and N is the total number of documents;

[0150] Perform the above comparison process according to the perplexity;

[0151] S1.3.2 In the previous step, the initial chunking was completed. Due to various restrictive conditions, the length differences between different chunks may be very large. Therefore, further merging is required. If the slice length is less than 60% of the preset length, it will be merged into the adjacent slice, and the merging is based on which adjacent triple similarity is higher. Through secondary processing, document semantic chunks that basically conform to semantics and preset length can be obtained. (The above similarity refers to the cosine similarity calculated by embedding / the score calculated by the re-ranking model)

[0152] S1.4 Community discovery: Relationships and entities such as the symptoms, treatment methods, and medications of diabetes will be aggregated into a community, and the corresponding documents will be associated

[0153] S1.4.1 PageRank algorithm: The PageRank algorithm is a graph ranking algorithm based on the random walk model. It mainly calculates scores based on the mutual links between web pages to measure the importance of web pages. In a knowledge graph, the PageRank algorithm can evaluate the importance of nodes by calculating the PageRank values of nodes, find key entities or concepts, and these nodes can be used as the core nodes, i.e., the central points, of community discovery

[0154] S1.4.2 Community discovery algorithm: The Louvain community discovery algorithm is a hierarchical clustering algorithm based on modularity optimization. The core lies in identifying the community structure by iteratively optimizing the modularity. Its advantage is that it can handle large-scale networks and can discover hierarchical community structures. Modularity is an index used in network analysis to measure the quality of community structure division. The value range of modularity is from 0 to 1. A high modularity value indicates that the connections within the community are closer than those in the random case, while the connections between communities are relatively fewer. For a given network partition, the modularity Q is defined as follows

[0155]

[0156] Among them, Aij represents the weight of the edge between nodes i and j; ki is the degree of node i, that is, the sum of the weights of all edges connected to node i; m is the sum of the weights of all edges in the graph; I(ci,cj) is an indicator function, which is 1 when i and j belong to the same community, and 0 otherwise

[0157] First, regard each node in the network as a separate community. For the central points identified by the pagerank algorithm, traverse all its neighbors, calculate the increment of modularity after moving the neighbor nodes to the community where the core node is located, and select an optimal move according to the modularity increment to maximize the modularity. Repeat traversing the above operations until there is no better move, that is, the community merging has been completed

[0158] S1.4.3 Incremental processing: For new documents, the large model constructs subgraphs. Within the subgraphs, it determines whether there are core nodes (central nodes) by calculating pagerank scores. If no core nodes are included, they are incorporated into the communities of the existing knowledge graph according to their connection relationships, and no new communities are added; otherwise, new communities are added, and the connected communities need to be rediscovered using the community discovery algorithm.

[0159] S1.4.4 Post-processing operations: The document nodes corresponding to the entity nodes in the community can also be incorporated into the community to establish the clustering relationship between documents. In addition, if a community contains slices with document numbers 2, 3, and 5 at the same time, then slice 4 of this document can be directly incorporated into the community to ensure the integrity of the context.

[0160] S2. Knowledge graph retrieval

[0161] S2.1 Initial node: Identify entities from the query question and link them to the entity nodes in the knowledge graph through similarity as the starting point for knowledge graph queries.

[0162] S2.2 Beam search process of the knowledge graph: Beam search is an efficient heuristic graph search algorithm that reduces the search space and computational cost by retaining the most likely paths at each step of the expansion process. Starting from the initial node, it traverses the adjacent nodes, filters out the triples with similarity scores exceeding the similarity threshold from the top k results with the highest similarity, and so on. Each time, at most top k paths are selected from at most top k * top k sub-paths. The limit of top k helps control the search width and avoid resource exhaustion. To ensure the comprehensiveness of the divergent retrieval, the returned results will also include adjacent triples whose similarity between the start and end entities of the path exceeds the third threshold (the third threshold can be set to 0.7 - 0.8), which means that even if some paths do not directly reach the final goal, they will be considered if they have a high similarity to the query entity. This process is iterated until the termination condition is met, such as reaching the preset search depth, finding a sufficient number of high-quality paths, or resource exhaustion. By balancing the breadth and depth of the search, beam search helps discover the most relevant information in large-scale and complex graph structures while effectively controlling the use of computational resources.

[0163] S3. Enhanced knowledge graph retrieval

[0164] S3.1 Query Rewriting: Since there are many professional terms in vertical domain Q&A, and the user's query questions may be more colloquial and less professional, some professional knowledge triples can be taken out from the knowledge graph to help standardize the questions. First, identify the entities from the query question and link them to the entity nodes in the knowledge graph through similarity; then query the k-layer (to control resource consumption, k should not be too large, k = 1 or 2 is fine) neighbor nodes of the node, and calculate the similarity between the triples and the query question, and set a relatively low fourth threshold (the fourth threshold can be set to 0.5); finally, inject the question and the retrieved triples into the large model, and use the triples greater than the fourth threshold as the rewritten triples, and rewrite the query according to the rewritten triples; to ensure that the basic semantics of the question are not changed, the large model also needs to perform a semantic judgment to reflect on whether it has changed the query question. If the semantics are basically the same, output the rewritten question, otherwise output the original question.

[0165] S3.2 Knowledge Graph Enhanced Document Chunk Retrieval: First, perform knowledge graph retrieval, select the k paths and retrieval result triples obtained in step S2.2, and locate the community where the retrieved entity is located. Once the community where the target entity is located is determined, the system will retrieve in the document semantic chunks corresponding to these communities. These document slices are pre-divided according to the document structure, each slice contains a certain amount of information, and is associated with a specific community or topic. In this way, the system can quickly locate the part of the document that may contain the answer, rather than blindly searching in the entire document collection. After determining the retrieval range of the document slices, a more traditional hybrid retrieval strategy of vector retrieval + keyword retrieval is adopted. The system will use the large model to comprehensively judge the existing document retrieval results and knowledge graph retrieval results. The large model will evaluate whether these results are sufficient to answer the user's query. If the large model believes that the current results are not sufficient to provide a complete answer, or some key information is missing in the knowledge graph, the system will automatically trigger the retrieval of other document slices not covered by the knowledge graph. This supplementary retrieval step ensures that even when the knowledge graph is incomplete, a comprehensive answer can be found as much as possible. Finally, the system will integrate all the retrieved information, including the triples from the knowledge graph and the content from the document slices, to form a comprehensive answer.

[0166] S3.3 Other Uses of the Graph Community: The structure and topic tightness of the community enable the system to understand the context of the query and recommend subsequent questions accordingly. For example, if the user queries the treatment drugs for diabetes, the system can recommend questions such as "What are the side effects of the drug?" or "How to use this drug correctly?" based on other entities and relationships in the community where the drug is located. In particular, according to the termination node of the query result, more appropriate recommended questions can be generated more effectively.

[0167] In addition, the close connection within the community helps to explore deeper knowledge. In the query of diabetes treatment drugs, the system can not only find drug information, but also, through the relationship chain within the community, find information related to complications, preventive measures, health management, etc. Through in-depth analysis of the community, the system can provide personalized information push based on the user's historical queries and preferences. For example, if a user often queries diabetes-related information, the system can push the latest research progress on diabetes treatment or drug updates.

[0168] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules, modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0169] The unit may or may not be physically separated. The components shown as units may be a physical unit or multiple physical units, that is, they may be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, each functional unit in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0171] Specifically, according to the embodiments disclosed by the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium. The computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0173] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for alleviating large model hallucination through knowledge graph retrieval, characterized in that: include: Build a knowledge graph; Semantic segmentation of documents based on knowledge graph; Building communities based on the cuts; According to the entity of the query question, determine the corresponding community to be retrieved from the community; Search from the community to be searched and get the answer.

2. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 1, characterized in that: Building a knowledge graph includes the following steps: Extract entities and attributes from documents; Based on the entity context, the large model extracts the relationships between entities from the document; Perform entity disambiguation on entities and relationships between them; Based on entities, attributes and relationships, triples are constructed to form a knowledge graph.

3. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 2, characterized in that: Entity disambiguation is performed on entities and relationships between them, including the following steps: In combination with new documents, the large model merges the existing graph and the extracted sub-graph; or, uses similarity comparison methods to process entities and relationships separately.

4. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 3, characterized in that: Document semantic segmentation based on knowledge graph includes the following steps: Count the starting positions of each triple in the document and sort them; Calculate the similarity between the current triplet and the next triplet; If the similarity between the current triplet and the next triplet is greater than the first threshold, the start position of the current triplet is used as the starting point of the document slice, and the end position of the next triplet is used as the end point of the document slice, and the document is semantically sliced ​​to obtain a slice; If the similarity between the current triplet and the next triplet is less than the second threshold, the start position of the current triplet is used as the starting point of the document slice, and the end position of the current triplet is used as the end point of the document slice, and the document is semantically sliced ​​to obtain a slice; The length of each slice is compared with the preset length, and the slices whose length is less than 60% of the preset length are merged into adjacent slices with high similarity.

5. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 4, characterized in that: Building a community based on the blocks includes the following steps: Based on the PageRank algorithm, calculate the pagerank value of the nodes in the knowledge graph and select the center point according to the pagerank value; Based on the Louvain community discovery algorithm, all neighbor nodes of the center point are traversed, and the modularity increment after the neighbor nodes are moved to the community where the core node is located is calculated; According to the modularity increment, the optimal move is selected to complete the merger of the community.

6. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 5, characterized in that: The following steps are also included: For new documents, the large model builds a subgraph; Based on the PageRank algorithm, the PageRank value of the nodes in the subgraph is calculated, and the presence or absence of core nodes is determined according to the PageRank value; If it does not contain core nodes, it will be incorporated into the community of the existing knowledge graph according to its connection relationship, and no new community will be added; If core nodes are included, the connected communities need to re-perform the Louvain community discovery algorithm.

7. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 6, characterized in that: According to the entity of the query question, the corresponding community to be retrieved is determined from the community, which specifically includes the following steps: Identify entities from the query; Based on similarity calculation, the initial nodes are obtained from the entities in the knowledge graph according to the entities in the query question; Based on beam search, we traverse the adjacent nodes from the first node, select the adjacent triples exceeding the third threshold from the topk results with the highest similarity, and obtain the search result path and search result triples; According to the retrieval result path and the retrieval result triples, the corresponding community to be retrieved is determined from the community.

8. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 7, characterized in that: According to the search result path and the search result triple, the corresponding community to be searched is determined from the community, and the search is performed from the community to be searched to obtain the answer, which specifically includes the following steps: According to the entity queried by the search result path and the search result triple, the community where the entity is located is located as the community to be searched; Based on vector search and keyword search, search from slices of the community to be searched to get the answer.

9. The method for alleviating large model hallucination through knowledge graph retrieval according to claim 8, characterized in that: The following steps are also included: Query the k-layer neighbor nodes of the initial node, and calculate the similarity between the triples and the query question, and use the triples whose similarity exceeds the fourth threshold as rewritten triples; Rewrite the query question based on the rewritten triples.

10. A system for alleviating large model hallucinations through knowledge graph retrieval, used to implement the method for alleviating large model hallucinations through knowledge graph retrieval as described in any one of claims 1 to 9, characterized in that: include: Building modules, used to build knowledge graphs; Document semantic segmentation module, used to perform document semantic segmentation based on knowledge graph; Community building module, used to build communities based on blocks; The community positioning module is used to determine the corresponding community to be retrieved from the community according to the entity of the query question; The retrieval module is used to search from the community to be searched and obtain answers.

Images