Search enhancement generation method and device fusing spatiotemporal correlation knowledge graph

Abstract

The application provides a retrieval enhancement generation method of a spatio-temporal correlation knowledge graph, which can be applied to the technical fields of artificial intelligence and natural language processing. The retrieval enhancement generation method of the spatio-temporal correlation knowledge graph comprises the following steps: receiving a user question, retrieving a relevant document set related to the user question from a preset document database; querying and obtaining relevant graph knowledge from a pre-constructed spatio-temporal correlation knowledge graph according to the user question and the relevant document set; performing reflection analysis on each document in the relevant document set based on the graph knowledge to obtain a reflection enhancement document set, wherein the reflection analysis comprises fact consistency reflection, knowledge missing reflection and knowledge time effectiveness reflection; inputting the user question and the reflection enhancement document set into a large language model to generate an answer result for the user question. The application also provides a retrieval enhancement generation device of a spatio-temporal correlation knowledge graph, equipment, a storage medium and a program product.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and specifically to a retrieval enhancement generation method, apparatus, device, medium, and program product that integrates spatiotemporal related knowledge graphs. Background Technology

[0002] Retrieval Augmented Generation (RAG) technology significantly improves the credibility of generated content by combining the knowledge reasoning capabilities of Large Language Models (LLMs) with external document retrieval mechanisms, and is widely used in open-domain question answering and knowledge quiz tasks. RAG systems first retrieve relevant documents from external knowledge bases using semantic similarity matching, and then input this information into the LLM to generate the answer. However, RAG systems face several challenges in practical applications, especially in spatiotemporal knowledge question answering scenarios. For example, retrieving relevant documents based on vector similarity easily recalls ambiguous statements or outdated data; spatiotemporal data is updated frequently, but existing RAGs lack timeliness verification mechanisms, potentially leading to frequent citation of outdated or invalid information when processing spatiotemporal data; retrieved documents are only used as isolated evidence input, without establishing cross-document / cross-information source correlation analysis, resulting in overly one-sided information. These problems make it difficult to guarantee the output quality of RAG systems.

[0003] Existing improvement solutions mainly fall into two categories, but both have significant drawbacks:

[0004] (1) Limitations of static knowledge verification: For example, the Self-Reflective Retrieval Augmentation (Self-RAG) framework relies on LLM introspection to score documents, but its verification basis is limited by the static nature of pre-trained knowledge and cannot detect factual content that changes dynamically over time.

[0005] (2) Limitations of Knowledge Graph Applications: Existing knowledge graph methods mainly focus on solving the problem of static knowledge data fusion and have not yet been designed for dynamically updated spatiotemporal knowledge. For example, the Knowledge Graph-Augmented RAG (KG-RAG) framework only calls the knowledge graph unidirectionally for entity linking during the retrieval stage, without forming a "retrieval-verification-enhancement" closed loop. The Knowledge-Augmented RAG (KAR) framework, although introducing graph retrieval, adopts a static fusion mode and cannot specifically identify knowledge defects in documents, resulting in a lack of accuracy in knowledge compensation.

[0006] Therefore, in the context of spatiotemporal knowledge question answering, a more efficient retrieval enhancement generation method is needed to solve the problem of factual bias caused by document noise and timeliness defects. Summary of the Invention

[0007] In view of the above problems, this application provides a retrieval enhancement generation method, apparatus, device, medium and program product that integrates spatiotemporal related knowledge graphs to improve the accuracy and comprehensiveness of responses.

[0008] According to the first aspect of this application, a retrieval enhancement generation method integrating spatiotemporal relational knowledge graphs is provided, comprising: receiving a user question; retrieving a set of relevant documents related to the user question from a pre-set document database; querying and obtaining relevant graph knowledge from a pre-constructed spatiotemporal relational knowledge graph based on the user question and the set of relevant documents; performing reflective analysis on each document in the set of relevant documents based on the graph knowledge to obtain a reflective enhanced document set, wherein the reflective analysis includes: factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection; and inputting the user question and the reflective enhanced document set into a large language model to generate an answer result for the user question.

[0009] According to an embodiment of this application, fact-consistent reflection includes: determining whether there are factual contradictions or errors in the document content based on graph knowledge; and outputting a fact-consistent reflection result in response to detecting factual contradictions or errors in the document content, wherein the fact-consistent reflection result includes at least the reflection type, the identified factual contradictions or errors, and their corresponding correct content in the graph knowledge.

[0010] According to an embodiment of this application, knowledge gap reflection includes: determining whether there is knowledge gap in the document content based on graph knowledge; and outputting a knowledge gap reflection result in response to detecting that the document content is missing related entity knowledge in the graph knowledge, wherein the knowledge gap reflection result includes at least a reflection type and related entity knowledge.

[0011] According to an embodiment of this application, the knowledge timeliness reflection includes: determining whether the timeliness of the document content is consistent with that of the graph knowledge based on graph knowledge; and outputting the knowledge timeliness reflection result in response to the detection of outdated content in the document, wherein the knowledge timeliness reflection result includes at least the reflection type, the identified outdated content, and the latest content corresponding to it in the graph knowledge.

[0012] According to an embodiment of this application, performing reflective analysis on each document in a relevant document set based on graph knowledge to obtain a reflective-enhanced document set includes: performing reflective analysis on the document content and knowledge in the graph knowledge to obtain analysis results; generating reflective key-value pairs based on the analysis results, wherein the key of the reflective key-value pair is the reflective analysis rule identifier, and the value of the reflective key-value pair is the result data of the document under the corresponding reflective analysis; and performing reflective tagging on each document based on the reflective key-value pairs to obtain a reflective-enhanced document set.

[0013] According to embodiments of this application, user questions and a set of reflective enhanced documents are input into a large language model to generate answers to user questions. This includes: processing user questions using a traditional retrieval enhancement generation method to obtain traditional answer results, wherein the traditional retrieval enhancement generation method directly retrieves original documents related to user questions from a preset document database without performing reflective enhancement processing on the original documents; inputting user questions and a set of reflective enhanced documents into a large language model to generate reflective enhanced answer results to user questions; and performing parallel visualization comparison and display of traditional answer results and reflective enhanced answer results.

[0014] According to an embodiment of this application, based on a user question and a set of related documents, querying and obtaining relevant graph knowledge from a pre-built spatiotemporal knowledge graph includes: using a large language model to extract entity, relation, and attribute information related to the user question from the content of each document in the set of related documents; generating a graph query statement based on the entity, relation, and attribute information; and executing the graph query statement to query and obtain relevant graph knowledge from the pre-built spatiotemporal knowledge graph.

[0015] According to an embodiment of this application, receiving a user question and retrieving a set of relevant documents related to the user question from a preset document database includes: receiving the user question and converting the user question into a user question vector; calculating the similarity value between the user question vector and each document vector in the preset document database; retrieving a first set of documents matching the user question from the preset document database based on the similarity value; inputting the user question and the first set of documents into a large language model for semantic similarity analysis, and outputting a second set of documents related to the user question.

[0016] The second aspect of this application provides a retrieval enhancement generation device that integrates a spatiotemporal knowledge graph, comprising: a document retrieval module for receiving a user question and retrieving a set of relevant documents related to the user question from a preset document database; a graph knowledge retrieval module for querying and obtaining relevant graph knowledge from a pre-constructed spatiotemporal knowledge graph based on the user question and the set of relevant documents; a document reflection module for performing reflection analysis on each document in the set of relevant documents based on graph knowledge to obtain a reflection-enhanced document set, wherein the reflection analysis includes: factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection; and an answer result generation module for inputting the user question and the reflection-enhanced document set into a large language model to generate an answer result for the user question.

[0017] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0018] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0019] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0020] The above-mentioned contents, as well as other objects, features, and advantages of this application, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 The illustrations depict application scenarios of the retrieval enhancement generation method, apparatus, device, medium, and program product based on embodiments of this application, which integrate spatiotemporal knowledge graphs.

[0022] Figure 2 A flowchart illustrating a retrieval enhancement generation method for fused spatiotemporal relational knowledge graphs according to an embodiment of this application is shown schematically.

[0023] Figure 3 The illustration shows a schematic diagram of a document retrieval process according to an embodiment of this application;

[0024] Figure 4 The illustration shows a schematic diagram of a spatiotemporal knowledge graph query process according to an embodiment of the present application;

[0025] Figure 5A schematic diagram illustrating a multidimensional reflection process of a method according to an embodiment of this application is provided.

[0026] Figure 6 A schematic diagram illustrating a dual-channel visual comparison of a method according to an embodiment of this application is shown.

[0027] Figure 7 The illustration shows an overall flowchart of a retrieval enhancement generation method for fused spatiotemporal knowledge graphs according to an embodiment of this application;

[0028] Figure 8 This schematically illustrates a structural block diagram of a retrieval enhancement generation apparatus for fusion of spatiotemporal relational knowledge graphs according to an embodiment of this application; and

[0029] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a retrieval enhancement generation method for fused spatiotemporal knowledge graphs, according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0033] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0034] Embodiments of this application provide a retrieval enhancement generation method that integrates a spatiotemporal knowledge graph. The method includes: receiving a user question; retrieving a set of relevant documents related to the user question from a pre-defined document database; querying and obtaining relevant graph knowledge from a pre-constructed spatiotemporal knowledge graph based on the user question and the set of relevant documents; performing reflective analysis on each document in the set of relevant documents based on the graph knowledge to obtain a reflective enhanced document set, wherein the reflective analysis includes: factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection; and inputting the user question and the reflective enhanced document set into a large language model to generate an answer to the user question.

[0035] According to the embodiments of this application, by introducing structured knowledge from a spatiotemporal knowledge graph, the content of documents in a set of related documents with correlations is cross-validated from three aspects: factual consistency, knowledge gaps, and knowledge timeliness. This forms an iterative mechanism of retrieval-reflection-enhancement, which can not only effectively suppress the interference of outdated content, erroneous facts, or biased viewpoints in the document content, but also significantly improve the relevance of the retrieval content to the user's query intent, thereby improving the retrieval answer accuracy of the large retrieval generation model.

[0036] Figure 1 The illustrations depict application scenarios of a retrieval enhancement generation method, apparatus, device, medium, and program product based on embodiments of this application for fusion spatiotemporal relational knowledge graphs.

[0037] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0041] It should be noted that the retrieval enhancement generation method for fused spatiotemporal relational knowledge graphs provided in the embodiments of this application can generally be executed by server 105. Correspondingly, the retrieval enhancement generation device for fused spatiotemporal relational knowledge graphs provided in the embodiments of this application can generally be located in server 105. The retrieval enhancement generation method for fused spatiotemporal relational knowledge graphs provided in the embodiments of this application can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the retrieval enhancement generation device for fused spatiotemporal relational knowledge graphs provided in the embodiments of this application can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] The following will be based on Figure 1 The described scene, through Figures 2-7 A detailed description is provided of a retrieval enhancement generation method for a fused spatiotemporal knowledge graph according to embodiments of this application.

[0044] Figure 2 A flowchart illustrating a retrieval enhancement generation method for a fused spatiotemporal relational knowledge graph according to an embodiment of this application is shown.

[0045] like Figure 2 As shown, the retrieval enhancement generation method of the fused spatiotemporal related knowledge graph in this embodiment includes operations S210 to S240, which can be executed by the server 105.

[0046] In operation S210, the user's question is received, and a set of relevant documents that are related to the user's question are retrieved from the preset document database.

[0047] In embodiments of this application, the preset document database may include multiple knowledge fragments, each of which can be considered an independent document. For example, raw knowledge data from various sources can be collected, cleaned to remove noise, duplicates, and irrelevant information, yielding intermediate knowledge data. This intermediate knowledge data can be segmented into smaller chunks, and these chunks can be converted into document vectors (Dvecs) using an embedding model. These document vectors are then stored in the preset document database.

[0048] Optionally, the specific generation method of the preset document database and the selection of the embedding model can be determined according to actual needs, and are not restricted here.

[0049] The following will be combined with the appendix Figure 3 The operation S210 will be described in detail.

[0050] Figure 3 The illustration shows a schematic diagram of a document retrieval process according to an embodiment of this application.

[0051] like Figure 3 As shown, this embodiment is one embodiment of operation S210, including operations S211 to S214.

[0052] In operation S211, the user question is received and converted into a user question vector.

[0053] In the embodiments of this application, an embedding model completely identical to that used in the preset document database can be employed to accurately convert user-submitted questions into corresponding user question vectors (Query Vector, Qvec). It should be noted that the Qvec and Dvec must be generated from the same embedding model; otherwise, their vector distributions will differ, making it impossible to correctly calculate similarity.

[0054] In operation S212, the similarity value between the user's question vector and each document vector in the preset document database is calculated.

[0055] In the embodiments of this application, a vector similarity calculation method can be used to compare the user question vector Qvec with each document vector Dvec in a preset document database one by one, thereby obtaining the similarity value between Qvec and each Dvec. These similarity values ​​can reflect the degree of relevance between the user question and each document, providing a key basis for accurately retrieving documents closely related to the user question from the document database.

[0056] For example, methods for calculating vector similarity may include cosine similarity, Euclidean distance, Manhattan distance, etc., without specific limitations here.

[0057] In operation S213, a first set of documents matching the user's question is retrieved from a preset document database based on the similarity value.

[0058] In the embodiments of this application, a similarity threshold can be preset to filter relevant documents with similarity values ​​greater than the threshold, forming a first set of documents matching the user's question (i.e., the initial screening document set), thereby reducing the burden of subsequent processing and improving response speed. Optionally, the top-K relevant documents can be filtered by sorting by similarity value, for example, sorting by similarity value from largest to smallest and selecting the top 10 documents. The specific filtering method can be determined according to actual needs and is not limited here.

[0059] In operation S214, the user question and the first document set are input into the large language model for semantic similarity analysis, and the output is a second document set related to the user question.

[0060] In the embodiments of this application, the user question and a first document set are input into a large language model. The large language model performs deep semantic analysis on the user question and each document in the first document set, thereby accurately determining the semantic similarity between the user question and each document in the first document set. In this way, documents that are highly semantically related to the user question can be identified, thus forming a second document set, which is the final set of relevant documents that are related to the user question. For example, the second document set may include a document identification (ID), a similarity score with the user question, and the document itself.

[0061] In the embodiments of this application, firstly, a vector similarity calculation method is used to calculate the similarity between the user's question and documents in a preset document database, and a preliminary screening (i.e., coarse screening) is performed based on the calculation results. Then, based on the coarse screening, a large language model is used to analyze the semantic similarity between the selected documents and the user's question in order to find the set of documents most relevant to the user's question, thereby providing accurate contextual information for the large language model and improving the accuracy, relevance and reliability of the answer.

[0062] When operating S220, based on the user's question and related document set, relevant graph knowledge is queried and obtained from the pre-built spatiotemporal relational knowledge graph.

[0063] Spatiotemporal knowledge graphs are knowledge representation models that integrate temporal and spatial dimensions on the basis of traditional knowledge graphs. They not only include entities and the relationships between them, but also the occurrence, evolution, or distribution of these entities and relationships within a specific temporal and spatial context. By embedding temporal information (such as the time of event occurrence, duration, and chronological order) and spatial information (such as geographical location, geographical scope, spatial orientation, and distance) into the structure of the knowledge graph, spatiotemporal knowledge graphs can more accurately and meticulously depict the dynamic, regional, and evolutionary knowledge structures of the real world.

[0064] For example, a spatiotemporal knowledge graph can be pre-built and generated using the Neo4j graph database. The core data structure of the Neo4j graph database is based on a triple structure of nodes (entities), relations (associations), and attributes (metadata). Nodes represent various entities in the spatiotemporal knowledge graph; these entities can be specific points in time, spatial locations, or related people, events, objects, etc. Relationships describe the connections between these entities. Attributes further describe the nodes and relations, providing them with rich metadata information. Through this triple structure, the Neo4j graph database can clearly present the complex entity relationships and spatiotemporal features in the spatiotemporal knowledge graph, providing strong support for the analysis, mining, and application of spatiotemporal data.

[0065] In the embodiments of this application, LLM (Limited Learning Model) is used to perform deep analysis on user questions and a set of relevant documents that are filtered and related to the user questions. Based on the deep analysis results and using pre-customized prompt templates, entities, relationships between entities, and entity attribute information are identified in each document. Prompts can be understood as a sequence of tokens, such as text and symbols, input to the LLM. Prompts can guide the LLM to generate specific outputs or perform specific tasks. For example, prompts can be a question, the beginning of a sentence, an instruction, or any text that can elicit a desired response from the model.

[0066] In the embodiments of this application, operation S220 specifically includes operations S221 to S223.

[0067] In operation S221, a large language model is used to extract entity, relation, and attribute information related to the user's question from the content of each document in the relevant document set.

[0068] Each document in a related document collection typically contains a massive amount of entity-related information, which is often complex and diverse. Some of this information is closely related to the user's question, providing key clues, data support, or background information for answering the question; however, a considerable portion of the information is not directly related to the user's question and is considered redundant content.

[0069] In the embodiments of this application, based on the prompt word template, a large language model is used to identify and analyze the content of each document in the relevant document set, effectively filtering out a large amount of redundant and interfering information that is irrelevant to the user's question, thereby extracting content such as entity, relationship, and attribute information related to the user's question.

[0070] In operation S222, a graph query statement is generated based on entity, relationship, and attribute information.

[0071] In the embodiments of this application, a graph query statement is generated based on the filtered entity, relationship, and attribute information.

[0072] In operation S223, a graph query statement is executed to query and obtain relevant graph knowledge from the pre-built spatiotemporal relational knowledge graph.

[0073] The following will be combined with the appendix Figure 4 The operation S220 is explained.

[0074] Figure 4 The illustration shows a schematic diagram of a spatiotemporal knowledge graph query process according to an embodiment of the present application.

[0075] like Figure 4 As shown, knowledge graph queries can be implemented through the following process: First, using the user's question and a set of related documents as input, a Large Language Model (LLM) is invoked as the entity extraction engine to accurately identify and extract key entity information, including entity, relation, and attribute information. Then, based on the extracted entity information, a graph query language generator dynamically constructs an adapted graph query statement. Finally, the graph query statement executor searches the spatiotemporal relational knowledge graph to efficiently obtain related knowledge about the entities. The following will combine... Figure 4 , Figure 5 Let's illustrate with examples.

[0076] In the embodiments of this application, based on the user's question, relevant entity information of each document is extracted from the relevant document set and filtered, so that the filtered entity information is highly targeted and has a direct logical relationship with the user's question, which can provide accurate support for LLM problem analysis and effective solution, thereby improving the quality and efficiency of information processing.

[0077] In operation S230, reflective analysis is performed on each document in the relevant document set based on graph knowledge to obtain a reflective enhancement document set. The reflective analysis includes: fact consistency reflection, knowledge gap reflection, and knowledge timeliness reflection.

[0078] In the embodiments of this application, the document content in the relevant document set and the knowledge in the graph knowledge are subjected to reflective analysis to obtain analysis results; reflective key-value pairs are generated based on the analysis results, wherein the key of the reflective key-value pair is the reflective analysis rule identifier, and the value of the reflective key-value pair is the result data of the document under the corresponding reflective analysis; based on the reflective key-value pairs, each document is reflected and marked to obtain a reflective enhanced document set. Optionally, the reflective key-value pairs can adopt JSON format (JavaScript Object Notation).

[0079] In the embodiments of this application, machine-resolvable reflection result injection can be achieved by converting the analysis results into reflection key-value pairs.

[0080] In the embodiments of this application, factual consistency refers to whether the factual information expressed by two different sources is consistent. Factual consistency reflection includes: determining whether there are factual contradictions or errors in the document content based on graph knowledge; and outputting a factual consistency reflection result in response to detecting factual contradictions or errors in the document content. The factual consistency reflection result includes at least the reflection type, the identified factual contradictions or errors, and their corresponding correct content in the graph knowledge.

[0081] For example, in fact consistency reflection, a contradiction detection prompt template is set, the relevant document set and graph knowledge are used as input data for LLM, the contradiction detection prompt is used to instruct LLM to use graph knowledge as high-priority knowledge to perform fact consistency check and analysis on each document in the relevant document set, and a reflection key-value pair in JSON format is generated and output based on the analysis result, that is, the fact consistency reflection result. For example, the fact consistency reflection result in JSON format can be expressed as: {"original text": "Expected login location City A", "graph data": "<Typhoon_Swordfish><Central point location><2025-08-23 City B>", "reflection type": "[fact contradiction]"}), that is, "original text", "graph data", and "reflection type" are used as the keys in the reflection key-value pair, and the factually contradictory or erroneous content obtained by the document under the fact consistency reflection analysis, the corresponding correct content in the graph knowledge, and the current reflection type are used as the values in the reflection key-value pair. Then, a structured fact consistency reflection tag is generated based on the output reflection result, and marked in the document. For example, a Python annotation engine can be used to first perform text positioning in the original document based on the JSON-format fact consistency reflection result, determine the position of the content with fact contradiction or error, and use a relevant string processing algorithm to mark the fact consistency reflection tag to the original document. For example, the fact consistency reflection tag marked in the original document can be: "<fact contradiction>Expected login location City A< / fact contradiction>["<Typhoon_Swordfish><Central point location><2025-08-23 City B"]".

[0082] Fact consistency reflection, whose function is similar to a professional fact auditor. Since the document content in the preset document database may come from multiple channels, including manual writing, automatic generation, multi-round editing or cross-platform migration, etc., it is inevitable that there are fact deviations, data inconsistencies or logical errors. By introducing an efficient fact consistency reflection mechanism, these potential problems can be effectively identified and corrected, thereby improving the credibility and accuracy of documents and avoiding the large model from generating wrong answers based on the wrong content of documents.

[0083] In the embodiments of the present application, knowledge missing refers to comparing information from one source with information from another source to determine whether there is important knowledge that is obviously not mentioned. Knowledge missing reflection includes: judging whether there is knowledge missing in the document content based on graph knowledge; outputting a knowledge missing reflection result in response to detecting that the associated entity knowledge in the graph knowledge is missing in the document content, wherein the knowledge missing reflection result includes at least the reflection type and the associated entity knowledge.

[0084] For example, in knowledge gap reflection, a gap detection prompt template is set up. The relevant document set and graph knowledge are used as input data for the LLM. The gap detection prompt specifies that the LLM uses graph knowledge as high-priority knowledge to perform knowledge gap verification analysis on each document in the relevant document set. This mainly checks whether the document lacks related entity knowledge contained in the graph knowledge, treating it as an important supplement to the document information in the spatiotemporal related knowledge graph. The result is output as a JSON-formatted reflection key-value pair, i.e., the knowledge gap reflection result. For example, the JSON-formatted knowledge gap reflection result can be represented as: {"Graph Data": "<Typhoon_Swordfish><Formation Cause><Climate_Tropical Depression>", "Reflection Type": "Knowledge Gap"}), where "Graph Data" and "Reflection Type" are used as the keys in the reflection key-value pair, and the related entity knowledge in the graph knowledge obtained from the knowledge gap reflection analysis of the document and the current reflection type are used as the values. Then, a structured knowledge gap reflection tag is generated for this JSON-formatted output result and annotated in the document. Optionally, an appended note can be added to the end of the original document. For example, the appended knowledge gap reflection mark could be: "[Supplementary map knowledge: "<Typhoon_Kujira><Forming cause><Climate_Tropical depression>"]".

[0085] Knowledge gap reflection functions similarly to a professional integrity reviewer. It uses a spatiotemporal knowledge graph as a detection benchmark. When the document content lacks related entity knowledge contained in the spatiotemporal knowledge graph, it fills in the missing content, thereby ensuring the integrity of the document at the structural, content, and logical levels. This helps the large language model provide more comprehensive answers.

[0086] In the embodiments of this application, knowledge timeliness refers to whether the dynamic information described by information from one source lags behind the latest information when compared with the most up-to-date information. Knowledge timeliness reflection includes: determining whether the timeliness of document content is consistent with that of graph knowledge based on graph knowledge; and outputting a knowledge timeliness reflection result in response to the detection of outdated content in the document. The knowledge timeliness reflection result includes at least the reflection type, the identified outdated content, and its corresponding latest content in the graph knowledge.

[0087] For example, in knowledge timeliness reflection, a timeliness detection prompt template is set, and the timeliness detection prompt is used to instruct an LLM to perform knowledge timeliness verification and analysis on each document in the relevant document set by using graph knowledge as high-priority knowledge. It mainly checks whether the relevant information in the document is consistent with the latest graph knowledge, corrects outdated information in the document, and outputs reflection key-value pairs in JSON format, that is, the knowledge timeliness reflection result. For example, the knowledge timeliness reflection result in JSON format can be expressed as: {"Original text fragment": "The current wind speed of Typhoon Swordfish is grade 8, the wind force is relatively strong, and attention should be paid to prevention", "Graph data": "<Typhoon_Swordfish><Wind speed><2025-08-23 Grade 9>", "Reflection type": "Timeliness lag"}), that is, "Original text fragment", "Graph data", and "Reflection type" are used as the keys in the reflection key-value pairs, and the lag content obtained by the document under the knowledge timeliness reflection analysis, the corresponding latest content in the graph knowledge, and the current reflection type are used as the values in the reflection key-value pairs. After that, a structured knowledge timeliness reflection mark is generated for the output result and annotated in the document. A relevant string processing algorithm can be used to annotate the timeliness verification result to the original text. For example, the knowledge timeliness reflection mark annotated in the original text of the document can be: "<timeliness lag>Typhoon Swordfish currently has a wind speed of 8, which is relatively strong, and needs to be prevented< / timeliness lag>[<Typhoon_Swordfish><wind speed><2025-08-23 9>]").

[0088] Optionally, a parallel processing mechanism can be used to synchronously perform the three tasks of fact consistency reflection, knowledge deficiency reflection and knowledge timeliness reflection. After outputting each reflection detection result (including fact consistency detection result, knowledge deficiency detection result, and timeliness detection result) respectively, multi-source result fusion analysis is performed on the three types of reflection detection results, and finally cross-reflection type collaborative annotation is realized on the original document. The execution order (parallel or serial) of the three types of reflection can be flexibly selected according to actual business requirements, system resources and performance requirements, and is not limited herein.

[0089] Optionally, the priority of knowledge timeliness reflection can be set higher than that of fact consistency reflection. If different reflection results are detected for the same document content in the two reflections, the reflection result of knowledge timeliness reflection is taken as the final result.

[0090] Knowledge timeliness reflection, whose function is similar to that of a timeliness verifier, takes the spatio-temporal associated knowledge graph as the detection benchmark, systematically identifies and studies the timeliness status of the content of each document in the relevant document set, locates the lagging information with significant time deviation or has become invalid between the document content and the current actual situation through multi-dimensional comparative analysis, and uses the knowledge graph to correct it. This can ensure that the output content of the large language model is strictly based on the latest information, output the answer that best fits the current development status, and effectively guarantee the accuracy and timeliness of the answer.

[0091] The following will combine Figure 4 , Figure 5 Specific embodiments are used for illustration.

[0092] For example, according to Figure 4 As shown, the user's question is: "The latest information on Typhoon Kujira". Based on the user's question, a relevant document set is retrieved from the preset document database ({ID: 1024; similarity: 0.92; document name: Typhoon Kujira_X website}, {ID: 2017; similarity: 0.87; document name: Typhoon Kujira_Y website}). Then, the entity extraction prompt template can be set to: "Focus on extracting entity or attribute names in the documents that are related to the question. For example, for natural phenomena, extract related entities or attributes such as their location, time of occurrence, associated phenomena, and related people." Based on this prompt template, a large language model is used to extract entity information from each document in the relevant document set. For example, the extracted entity information may include the location of occurrence, the expected landing location, etc. (e.g., {"name": "Kujira", "Location of occurrence": "Sea A", "Wind speed": ["2025-08-21 Level 8", "2025-08-22 Level 9", "2025-08-23 Level 9"; "2025-08-23 Level 9"; "2025-08-24 Level 9"; "2025-08-25 ... [Level 9], "Center Location": ["2025-08-21 A Sea", "2025-08-22 A Sea", "2025-08-23 C City"]}, where wind speed and center location are real-time changing dynamic spatiotemporal data, and the location of occurrence is entity relation data). Based on this entity information, a graph query language generator is used to generate graph query statements. For example, the graph query language generator can be used to map entity information to Neo4j graph database Cypher statements. Cypher statements are the query language of Neo4j graph database, which can perform efficient queries on graphs. For example, a Cypher statement can be: MATCH(p:Event{name:'Swordfish'})-[r]-(related_node)RETURN p,r,related_node). Optionally, LLM can be used as a graph query language generator. Input the prompt words for the graph query statement and use LLM to generate the graph query statement. Finally, graph query statements are executed in the spatiotemporal association knowledge graph through a graph query statement executor (such as the Cypher statement executor) to obtain relevant knowledge in the spatiotemporal association knowledge graph related to the entity. For example, relevant knowledge may include entity name, location of occurrence, wind speed at different time points, and center point location at different time points.

[0093] Figure 5 A schematic diagram illustrating a multidimensional reflection process of a method according to an embodiment of this application is shown.

[0094] like Figure 5 As shown, based on relevant document sets and related knowledge in spatiotemporal knowledge graphs, a large language model is used to perform reflections on fact consistency, knowledge gaps, and knowledge timeliness. Based on the reflection results, documents in the relevant document set are marked with reflection tags, resulting in a reflection-enhanced document set. For example, a Python annotation engine can be used for annotation. First, the Python annotation engine can locate the text in the original document based on the reflection results, thus determining the positions that need to be annotated. Then, reflection tags are generated based on the reflection results, and relevant string processing algorithms are used to annotate the original document with these reflection tags.

[0095] Alternatively, for spatiotemporal knowledge graph queries, relatively mature solutions such as NL2SQL (Natural Language to SQL) can be used, provided that the user's question and the initial screening documents are converted into graph query statements to be queried.

[0096] Alternatively, the Neo4j graph database can be replaced with other types of graph engines, but must be consistent with the graph query generator.

[0097] It should be noted that the reflective enhancement document set includes both documents with reflective tags and documents that, after reflective analysis, did not reveal any problems and were therefore not marked with reflective tags.

[0098] In operation S240, user questions and a set of reflective enhancement documents are input into the large language model to generate answers to the user questions.

[0099] In the embodiments of this application, the user question and the relevant document set after reflection analysis are input into the large language model, and finally the answer result for the user question can be generated. The content of the reflection mark can be used as the priority reference information for the question when the large language model answers the question.

[0100] In the embodiments of this application, a traditional retrieval enhancement generation method can be used to process the user question and obtain a traditional answer result. This traditional retrieval enhancement generation method directly retrieves the original documents related to the user question from a preset document database without performing reflective enhancement processing on the original documents. The user question and the set of reflective enhanced documents are input into a large language model to generate a reflective enhanced answer result for the user question. The traditional answer result and the reflective enhanced answer result are then displayed in parallel through visualization. The following will be combined with the appendix... Figure 6 Please provide a detailed explanation.

[0101] Figure 6 A schematic diagram illustrating a dual-channel visual comparison of a method according to an embodiment of this application is shown.

[0102] like Figure 6 As shown, a dual-channel visualization comparison of the traditional RAG channel and the reflection-enhanced RAG channel can be generated by splitting the data into left and right columns.

[0103] The traditional RAG channel employs conventional retrieval-enhanced generation methods, directly retrieving a set of original documents related to the user's question from a pre-defined document database. For example, it can calculate the similarity score between the user's question and documents in the database, filtering the most relevant original documents in descending order of similarity score. The user's question and the original document set are then input into the traditional RAG channel's large language model to obtain a conventional answer. For instance, the traditional RAG channel can accept the original document set and prompt instructions as input. For example, the prompt instructions could be: "Answer the question based on the following documents: {question content}, reference documents: {original document}".

[0104] For the Reflective Enhancement RAG channel: The retrieval enhancement generation method based on the fusion of spatiotemporal relational knowledge graphs, as described in the embodiments of this application, inputs the same question raised by the same user as in the traditional RAG channel, along with a set of reflective enhancement documents, into the large language model of the Reflective Enhancement RAG channel to obtain reflective enhancement answer results. For example, the Reflective Enhancement RAG channel can input tagged documents and prompt words. For instance, the prompt word could be "Answer the question based on the following documents (graph data is the primary reference): {question content}, reference document: {reflective enhancement document}".

[0105] The traditional response results and the reflection-enhanced response results are displayed in parallel and visually compared on a visualization interface.

[0106] In this embodiment of the application, by displaying the answer results of the traditional RAG channel and the enhanced RAG channel side by side, users can directly compare and observe the significant improvement in answer accuracy brought about by the enhanced RAG channel. This intuitive display of improvement can effectively enhance the user's confidence in the decision.

[0107] The "illusion" phenomenon in large language models typically refers to the model generating unrealistic or inaccurate content. For example, when generating text, large language models may introduce unverifiable, factually incorrect, or irrelevant elements, such as fictitious facts, incorrect details, or responses unrelated to the input.

[0108] In the embodiments of this application, by introducing structured knowledge from a spatiotemporal knowledge graph, documents are cross-validated from three dimensions: factual consistency, knowledge gaps, and knowledge timeliness. This forms an iterative mechanism of retrieval-reflection-enhancement, which can effectively solve the problem of insufficient verification of the authenticity and timeliness of retrieved documents in traditional RAG methods. It suppresses the "illusion" phenomenon caused by large language models relying on low-quality document retrieval results from the source, significantly reduces the error rate of generated content, and ultimately improves the overall accuracy and reliability of the question answering system.

[0109] Figure 7 The illustration shows an overall flowchart of a retrieval enhancement generation method for a fused spatiotemporal knowledge graph according to an embodiment of this application.

[0110] like Figure 7 As shown, the retrieval enhancement generation method for fused spatiotemporal knowledge graphs in this application mainly includes four steps:

[0111] The first major step is document retrieval and filtering: First, an embedding model is used to transform the user's question into a user question vector. For example, the embedding model can be a word-to-vector (Word2Vec) model. Then, a vector similarity measurement method is used to calculate the similarity value between the user question vector and the document vectors in the preset document database to perform a preliminary retrieval of relevant documents and obtain a preliminary set of documents. After that, the preliminary set of documents and the user's question are input into a large language model. The large language model uses semantic similarity analysis based on the relevance between the question and the document to further filter out the set of relevant documents most related to the user's question.

[0112] The second major step is spatiotemporal graph query: First, using a large language model, entity recognition is performed on the relevant document set and user questions selected in the first major step to obtain entity information; then, based on this entity information, a query is performed in the spatiotemporal association knowledge graph to filter out relevant spatiotemporal association knowledge graph data.

[0113] The third major step, multidimensional reflection: Based on user questions, the relevant document set selected in the first major step, and the graph knowledge (i.e., spatiotemporal relational knowledge graph data) obtained in the second major step, a large language model is used to conduct triple reflection on the document content in the relevant document set (including factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection), resulting in factual consistency reflection results, knowledge gap reflection results, and knowledge timeliness reflection results; then, reflection tags are added to one or more document contents in the document set based on these reflection results, resulting in a reflection-enhanced document set.

[0114] The fourth major step is response generation: The user's question and the resulting reflection document are input into the large language model to obtain the response results. Simultaneously, a visual interface can be used to display the response results from both the traditional and enhanced RAG channels, increasing the user's confidence in their decision.

[0115] According to the embodiments of this application, by performing triple reflection (fact consistency reflection, knowledge gap reflection, and knowledge timeliness reflection) on document content through spatiotemporal related knowledge graph structured data, the defects of noise and timeliness of traditional RAG documents can be solved, and the accuracy of large language models in spatiotemporal knowledge question answering scenarios can be effectively improved.

[0116] Based on the aforementioned retrieval enhancement generation method that integrates spatiotemporal related knowledge graphs, this application also provides a retrieval enhancement generation device that integrates spatiotemporal related knowledge graphs. The following will combine... Figure 8 The device is described in detail.

[0117] Figure 8 A schematic diagram of a retrieval enhancement generation apparatus for fused spatiotemporal knowledge graphs according to an embodiment of this application is shown.

[0118] like Figure 8 As shown, the retrieval enhancement generation device 800 of the fusion spatiotemporal knowledge graph in this embodiment includes a document retrieval module 810, a graph knowledge retrieval module 820, a document reflection module 830, and an answer result generation module 840.

[0119] The document retrieval module 810 is used to receive user questions and retrieve a set of relevant documents related to the user questions from a preset document database. In one embodiment, the document retrieval module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0120] The graph knowledge retrieval module 820 is used to query and retrieve relevant graph knowledge from a pre-built spatiotemporal relational knowledge graph based on the user's question and a set of related documents. In one embodiment, the graph knowledge retrieval module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0121] The document reflection module 830 is used to perform reflective analysis on each document in the relevant document set based on graph knowledge, to obtain a reflective enhanced document set. The reflective analysis includes: factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection. In one embodiment, the document reflection module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0122] The answer generation module 840 is used to input user questions and a set of reflection-enhanced documents into a large language model to generate answer results for the user questions. In one embodiment, the answer generation module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0123] According to an embodiment of this application, the document retrieval module 810 may include: a first document set filtering module and a second document set filtering module.

[0124] The first document set filtering module is used to receive user questions, convert user questions into user question vectors, calculate the similarity value between the user question vectors and the document vectors in the preset document database, and retrieve the first document set that matches the user questions from the preset document database based on the similarity value.

[0125] The second document set filtering module is used to input the user question and the first document set into the large language model for semantic similarity analysis, and output a second document set related to the user question.

[0126] According to an embodiment of this application, the graph knowledge retrieval module 820 may include: an entity information extraction module and a graph knowledge query module.

[0127] The entity information extraction module is used to extract entity, relationship, and attribute information related to the user's question from the content of each document in the relevant document collection using a large language model.

[0128] The graph knowledge query module is used to generate graph query statements based on entity, relationship, and attribute information; execute the graph query statements to query and obtain relevant graph knowledge from the pre-built spatiotemporal relational knowledge graph.

[0129] According to embodiments of this application, the document reflection module 830 may include: a fact consistency reflection module, a knowledge gap reflection module, and a knowledge timeliness reflection module.

[0130] The Fact Consistency Reflection Module is used to determine whether there are factual contradictions or errors in the document content based on graph knowledge. In response to the detection of factual contradictions or errors in the document content, it outputs the Fact Consistency Reflection Result, which includes at least the reflection type, the identified factual contradictions or errors, and their corresponding correct content in the graph knowledge.

[0131] The knowledge gap reflection module is used to determine whether there are knowledge gaps in the document content based on graph knowledge.

[0132] In response to the detection of missing related entity knowledge in the graph knowledge of the document content, the knowledge missing reflection result is output, wherein the knowledge missing reflection result includes at least the reflection type and related entity knowledge.

[0133] The knowledge timeliness reflection module is used to determine whether the timeliness of document content is consistent with that of graph knowledge based on graph knowledge. In response to the detection of outdated content in the document, it outputs the knowledge timeliness reflection result, which includes at least the reflection type, the identified outdated content, and its corresponding latest content in the graph knowledge.

[0134] According to embodiments of this application, any multiple modules among the document retrieval module 810, the graph knowledge retrieval module 820, the document reflection module 830, and the answer result generation module 840 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the document retrieval module 810, the graph knowledge retrieval module 820, the document reflection module 830, and the answer result generation module 840 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the document retrieval module 810, the graph knowledge retrieval module 820, the document reflection module 830, and the answer result generation module 840 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0135] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a retrieval enhancement generation method for fused spatiotemporal knowledge graphs, according to an embodiment of this application.

[0136] like Figure 9 As shown, an electronic device 900 according to an embodiment of this application includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0137] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0138] According to embodiments of this application, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0139] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0140] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0141] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the retrieval enhancement generation method for fused spatiotemporal relational knowledge graphs provided in the embodiments of this application.

[0142] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of the embodiments of this application. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0143] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0144] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of the embodiments of this application. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0145] According to embodiments of this application, program code for executing the computer programs provided in embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0147] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A retrieval enhancement generation method integrating spatiotemporal knowledge graphs, characterized in that, The method includes: Receive user questions and retrieve a set of relevant documents that are related to the user questions from a preset document database; Based on the user question and the relevant document set, query and obtain relevant graph knowledge from the pre-built spatiotemporal association knowledge graph; The contradiction detection prompts, missing detection prompts, and timeliness detection prompts are input into the large language model. The large language model is then used to perform factual consistency reflection, knowledge missing reflection, and knowledge timeliness reflection on each document in the relevant document set based on the graph knowledge, resulting in a reflection-enhanced document set. The factual consistency reflection is used to identify factual contradictions or errors in the document content, the knowledge missing reflection is used to identify knowledge missing in the document content, and the knowledge timeliness reflection is used to identify timeliness differences between the document content and the graph knowledge. The user question and the set of reflection-enhanced documents are input into the large language model, which generates an answer to the user question based primarily on the content of the reflection tags. The steps for generating the reflection-enhanced document set include: The document content is analyzed in conjunction with the knowledge in the aforementioned atlas, and the analysis results are obtained. Based on the analysis results, reflective key-value pairs are generated, wherein the key of the reflective key-value pair is the reflective analysis rule identifier, and the value of the reflective key-value pair is the result data of the document under the corresponding reflective analysis; Based on the reflexive key-value pairs, the original document text is located to determine the positions that need to be annotated. At least one of the following is marked in the original document text: content with factual contradictions or errors and its corresponding correct content in the graph, related entity knowledge with knowledge gaps, content with time lags, and the latest content in the graph. This results in a set of reflexive enhanced documents. The reflexive tags are used to annotate the result data under the corresponding reflexive analysis in the original document text.

2. The method according to claim 1, characterized in that, The consistent reflections on the aforementioned facts include: Based on the aforementioned graph knowledge, determine whether there are factual contradictions or errors in the document content; In response to the detection of factual contradictions or errors in the document content, a factual consistency reflection result is output, wherein the factual consistency reflection result includes at least the reflection type, the identified factual contradictions or errors, and their corresponding correct content in the graph knowledge.

3. The method according to claim 1, characterized in that, The reflections on the knowledge gaps mentioned include: Based on the knowledge graph, determine whether there are any knowledge gaps in the document content; In response to the detection that the document content is missing related entity knowledge in the graph knowledge, a knowledge missing reflection result is output, wherein the knowledge missing reflection result includes at least the reflection type and the related entity knowledge.

4. The method according to claim 1, characterized in that, The reflection on the timeliness of knowledge includes: Based on the graph knowledge, determine whether the timeliness of the document content is consistent with the graph knowledge; In response to the detection of outdated content in the document, a knowledge timeliness reflection result is output, wherein the knowledge timeliness reflection result includes at least the reflection type, the identified outdated content, and the latest content corresponding to it in the knowledge graph.

5. The method according to claim 1, characterized in that, The step of inputting the user question and the set of reflective augmented documents into a large language model to generate an answer to the user question includes: The user question is processed using a traditional retrieval enhancement generation method to obtain a traditional answer result. The traditional retrieval enhancement generation method directly retrieves the original document related to the user question from a preset document database, and does not perform reflective enhancement processing on the original document. The user question and the set of reflective and enhanced documents are input into the large language model to generate reflective and enhanced answers to the user question. The traditional response results and the reflection-enhanced response results are displayed in parallel through visualization.

6. The method according to claim 1, characterized in that, The step of querying and retrieving relevant knowledge graph information from a pre-built spatiotemporal relational knowledge graph based on the user question and the relevant document set includes: Using a large language model, entity, relation, and attribute information related to the user's question are extracted from the content of each document in the relevant document set; Based on the entity, relationship, and attribute information, a graph query statement is generated; Execute the graph query statement to query and obtain relevant graph knowledge from the pre-constructed spatiotemporal relational knowledge graph.

7. The method according to claim 1, characterized in that, The process of receiving a user question and retrieving a set of relevant documents from a preset document database that are related to the user question includes: Receive user questions and convert them into user question vectors; Calculate the similarity value between the user question vector and each document vector in the preset document database; Based on the similarity score, a first set of documents matching the user's question is retrieved from the preset document database; The user question and the first document set are input into a large language model for semantic similarity analysis, and a second document set related to the user question is output.

8. A retrieval enhancement generation device integrating spatiotemporal knowledge graphs, characterized in that, The device includes: The document retrieval module is used to receive user questions and retrieve a set of relevant documents that are related to the user questions from a preset document database. The graph knowledge retrieval module is used to query and obtain relevant graph knowledge from a pre-built spatiotemporal association knowledge graph based on the user question and the relevant document set. The document reflection module is used to input contradiction detection prompts, missing information detection prompts, and timeliness detection prompts into a large language model. Based on graph knowledge, the large language model performs factual consistency reflection, knowledge gap reflection, and knowledge timeliness reflection on each document in the relevant document set, resulting in a reflection-enhanced document set. Specifically, the factual consistency reflection identifies factual contradictions or errors in the document content, the knowledge gap reflection identifies knowledge gaps in the document content, and the knowledge timeliness reflection identifies timeliness differences between the document content and graph knowledge. The answer generation module is used to input user questions and a set of reflection-enhanced documents into a large language model. The large language model generates answer results for user questions based primarily on the content of the reflection tags. The steps for generating the reflection-enhanced document set include: The document content is analyzed in conjunction with the knowledge in the aforementioned atlas, and the analysis results are obtained. Based on the analysis results, reflective key-value pairs are generated, wherein the key of the reflective key-value pair is the reflective analysis rule identifier, and the value of the reflective key-value pair is the result data of the document under the corresponding reflective analysis; Based on the reflexive key-value pairs, the original document text is located to determine the positions that need to be annotated. At least one of the following is marked in the original document text: content with factual contradictions or errors and its corresponding correct content in the graph, related entity knowledge with knowledge gaps, content with time lags, and the latest content in the graph. This results in a set of reflexive enhanced documents. The reflexive tags are used to annotate the result data under the corresponding reflexive analysis in the original document text.

9. An electronic device, comprising: One or more processors; A memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images