An intelligent question-answering system based on natural language processing

Abstract

The application discloses an intelligent question and answer system based on natural language processing and relates to the technical field of artificial intelligence and natural language processing.The system comprises a routing composer, a dynamic confidence awareness routing module, a hierarchical semantic memory graph module, a large language model reasoning engine and a knowledge storage layer.In the large language model streaming generation process, the dynamic confidence awareness routing module calculates the word unit uncertainty score in real time, performs entity boundary detection in combination with named entity recognition, triggers a reasoning interception process when the entity node aggregated uncertainty score exceeds the dynamic safety threshold, generates a directional fact verification subquery and routes the subquery to the knowledge storage layer for retrieval, injects the retrieved fact into the prompt word and then restores and generates, the hierarchical semantic memory graph module manages the multi-round dialogue history in a three-layer architecture of an immediate memory layer, a short-term abstract layer and a long-term theme graph layer, and retains the explicit relationship structure between entities through a graph database.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, specifically to an intelligent question-answering system based on natural language processing. Background Technology

[0002] With the widespread deployment of large language models in enterprise-level knowledge services, medical consultation, legal inquiry and government affairs Q&A scenarios, the reliability and timeliness of knowledge of intelligent question answering systems have become increasingly prominent issues, becoming key bottlenecks restricting the application of such systems.

[0003] Existing intelligent question answering systems generally adopt a retrieval-enhanced generation architecture. After a user submits a query, this architecture first retrieves semantically similar document fragments from an external knowledge vector database, then concatenates the retrieval results into a prompt word template, and calls a large language model to generate the final answer in one go. Although this approach has some effect in single-round stateless question answering scenarios, it has the following fundamental flaws.

[0004] First, the generation-end illusion cannot be perceived and intercepted in real time. Traditional retrieval-enhanced generation systems adopt a black-box, single-call strategy for large language models after entering the generation stage. The generation process itself is completely closed to external knowledge, making it impossible to perceive and intercept high-uncertainty fact nodes in real time during the model's word-by-word generation process. When generating factual content such as proper nouns, numbers, and organization names, the model relies entirely on its own parameter memory. However, parameter memory is essentially statistical, and there is a systematic bias in the memory of low-frequency knowledge points. This inevitably leads to the incorrect generation of high-uncertainty fact nodes in a confident manner, i.e., structural illusion. Existing illusion suppression schemes either rely on post-event detection (high latency, requiring a large amount of labeled data) or on traditional retrieval-enhanced generation (decoupling of retrieval and generation semantics, limiting retrieval hit rate), neither of which can fundamentally eliminate the illusion at the generation end.

[0005] Second, the multi-turn dialogue memory management mechanism is crude, resulting in the loss of long-range dependencies. Existing systems generally use sliding window truncation or coarse-grained single-layer summarization to manage the history of multi-turn dialogues. The sliding window truncation method physically deletes early dialogues when the number of dialogue turns exceeds the window boundary, resulting in the irrecoverable loss of the entity reference relationships and business constraints established in the early stage. Although the coarse-grained summarization method compresses historical information at the semantic level, the flat text summarization cannot retain the explicit association relationships between entities. Neither of them can support the long-range semantic coherence in multi-turn, multi-hop business dialogue scenarios, resulting in cross-turn entity confusion and logical inconsistencies in complex dialogue scenarios involving multiple related entities.

[0006] Third, the knowledge base is static and cannot be updated incrementally online as the dialogue evolves. The knowledge base of the existing retrieval enhancement generation system is static after offline batch import, and cannot continuously learn new knowledge and relationships from the dialogue process. When faced with frequently updated business knowledge such as medical, legal, and government affairs, the timeliness of the knowledge rapidly decays. At the same time, the vector text library only supports block-level retrieval based on semantic similarity and lacks an explicit relational structure, which cannot support multi-hop reasoning queries that require jumping along relational edges, thus limiting the deep application of the system in high-value scenarios. To address this, we propose an intelligent question answering system based on natural language processing. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent question-answering system based on natural language processing, which solves three core technical problems in existing systems: the inability to intercept generation-end illusions in real time, the loss of long-range dependencies in multi-turn dialogues, and the static aging of the knowledge base.

[0008] This invention provides an intelligent question-answering system based on natural language processing. The system includes a routing orchestrator, a dynamic confidence-aware routing module, a hierarchical semantic memory graph module, a large language model inference engine, and a knowledge storage layer.

[0009] The large language model inference engine supports streaming lexical output and log probability exposure of each lexical, and accepts prompt word assembly and segmentation generation control instructions from the routing orchestrator.

[0010] The dynamic confidence-aware routing module receives streaming lexical units and their log probabilities from the large language model inference engine in real time. It calculates the local uncertainty score of each lexical unit based on a sliding window, and runs the named entity recognition streaming model in parallel to perform named entity boundary detection on the accumulated lexical unit sequence. When the named entity span ends, it aggregates the mean value of the uncertainty scores of all lexical units within that span to obtain the aggregated uncertainty score of the entity node. It compares the aggregated uncertainty score of the entity node with a dynamic security threshold. When the aggregated uncertainty score of the entity node exceeds the dynamic security threshold, it sends an interception signal to the routing orchestrator.

[0011] The routing orchestrator is the central control module of the system, responsible for receiving user queries, reading the three-layer memory context of the hierarchical semantic memory graph module, driving the large language model inference engine to generate answers in a streaming manner, and listening to the interception signals of the dynamic confidence perception routing module.

[0012] When an interception signal is received, the generation call of the large language model inference engine is paused. Based on the semantic context of the generated content and the category of the intercepted entity node, a targeted fact verification subquery is generated. The subquery is routed to the knowledge storage layer for retrieval. The retrieved fact text fragments are injected with the context prefix of the prompt word and then the generation is resumed.

[0013] After each round of answer generation is completed, the hierarchical semantic memory graph module is driven to perform memory updates and knowledge rewriting.

[0014] The hierarchical semantic memory graph module maintains a three-layer memory structure consisting of an immediate memory layer, a short-term summary layer, and a long-term topic graph layer, and is responsible for memory reading and writing and cross-layer update scheduling after each round of dialogue.

[0015] The knowledge storage layer consists of three parts: an immediate memory cache, a short-term summary vector library, and a long-term topic knowledge graph, which respectively correspond to the persistent backend of the three-layer memory architecture of the hierarchical semantic memory graph module.

[0016] Furthermore, the dynamic confidence-aware routing module for the first... The local uncertainty score is defined by each lexical position. Its calculation method is as follows: based on the model in context Generating lexical elements under conditions conditional probability Take the natural logarithm and then the inverse, i.e.

[0017] ;

[0018] in, A higher value indicates that the model is less certain about that lexical term;

[0019] For the first Text representation of each word element;

[0020] For the model in context Generating lexical elements under conditions The conditional probability;

[0021] For the first The sequence of all generated words before the step;

[0022] Take the natural logarithm.

[0023] Length is The arithmetic mean of the local uncertainty scores of all tokens within the sliding window constitutes the window average uncertainty. :

[0024] ;

[0025] in, This is the length of the sliding window, a system-preset hyperparameter with a default value of 8.

[0026] This is the index of the position of the word within the window.

[0027] When Named Entity Span At the end, the entity nodes aggregate uncertainty scores. for:

[0028] ;

[0029] in, This is the starting lexical position index for the current named entity span;

[0030] This is the index of the end term position for the current named entity span;

[0031] This represents the number of lexical units within the entity's span.

[0032] Furthermore, the dynamic security threshold The domain type coefficient and overall context confidence are dynamically adjusted, and the calculation formula is as follows:

[0033] ;

[0034] in, The base threshold is used and calibrated in experiments based on the model size.

[0035] The domain risk coefficient has a range of values. In high-risk fields such as medicine and law, the value is approximately 0.6, while in general question-and-answer fields, the value is approximately 1.0.

[0036] This is the context adjustment coefficient, which controls the weight of the overall confidence of the generated content on the threshold. The default value is 0.3.

[0037] This is the mean of the average uncertainty scores of all entity nodes in the currently generated content;

[0038] , , The units are all negative logarithmic probability units below the natural logarithm.

[0039] Furthermore, upon receiving the interception signal, the routing orchestrator pauses the generation and invocation of the large language model inference engine and rolls back to the starting position of the intercepting entity node.

[0040] Based on the semantic context of the generated content and the category of the intercepted entity node, a targeted fact verification subquery is automatically generated. After the subquery is vectorized and encoded, it is preferentially executed in the long-term topic knowledge graph for precise entity matching and one-hop relation query. If a match is found, the fact triple stored in the graph is directly returned.

[0041] If no match is found, the process degenerates to performing an approximate nearest neighbor semantic search in the short-term summary vector library, returning the segment with the highest semantic relevance as the factual text segment;

[0042] The routing orchestrator injects factual text fragments into the context prefix of the cue words and constrains the generation results of the large language model inference engine to be consistent with the fact with explicit instructions. Then, it resumes generation from the starting word position of the intercepted entity node, and the dynamic confidence-aware routing module continues to monitor subsequent words in real time.

[0043] Furthermore, the instant memory layer stores several rounds of original dialogue records in the memory cache in the form of key-value pairs. Each record contains the round number, the user query text, the system answer text, and the entity set obtained by named entity recognition in that round, without any compression.

[0044] When the total number of records exceeds the preset capacity, the earliest record is removed from the immediate memory layer, triggering the incremental update process of the short summary layer. The short summary layer, based on a finely tuned sequence-to-sequence intent summarization model, compresses the dialogue records removed from the immediate memory layer into semantic summary text, encodes them through a text vectorization model to obtain a summary semantic vector, and writes it into the short summary vector library along with the round number, conversation identifier, summary text and the set of entities involved.

[0045] During retrieval, the routing orchestrator performs vector encoding on the current query and then performs an approximate nearest neighbor search in the short-term summary vector library. Several of the most relevant summary records are then concatenated to the context of the prompt words. The long-term topic graph layer uses information extraction technology to dynamically write the entities and their relationships extracted from multiple rounds of conversations into the graph database. The graph database uses nodes to represent entities and directed edges to represent semantic relationships between entities, forming a triplet structure that supports multi-hop reasoning queries.

[0046] After each round of dialogue, the relation extraction model is used to identify the relation pairs of the current round, outputting a set of relation triples. After performing an existence check on each triple, it is written to the graph database in an incremental manner. At the same time, the source round, timestamp, and confidence score are recorded in the relation edge attributes.

[0047] The system periodically performs soft deletion on relation edges whose confidence scores are below the minimum retention threshold and whose last update time exceeds the aging window, in order to control the graph size and remove outdated, low-quality relations.

[0048] Furthermore, the system also includes a knowledge graph hot update and write-back mechanism. This mechanism operates in two scenarios: user-initiated feedback and automatic discovery of implicit relationships via multi-hop logical inference. It requires no parameter updates to the underlying large language model. In the user-initiated feedback scenario, the routing orchestrator receives the user feedback signal and determines the feedback type.

[0049] If it is a positive confirmation, the confidence of triples whose confidence is below the intermediate threshold in this round will be raised to the intermediate threshold and the corresponding relation edge attributes in the graph database will be updated.

[0050] If the text is for error correction, the error correction text is input into the relation extraction model to extract a set of corrected triples. This set is then compared with the original extraction results of this round. For deleted old triples, the confidence level is lowered to trigger soft deletion. For newly added correct triples, an incremental write operation is performed with full confidence. In multi-hop logic inference triggering scenarios, the routing orchestrator starts from the newly added node in this round in a depth-first traversal manner and performs a path search of up to 3 hops along the relation edges to identify the implicit triple candidate set that satisfies the predefined inference rules. The large language model inference engine performs a reasonableness verification and scoring for each candidate triple in a zero-shot hint manner. Only when the verification confidence level is not lower than the intermediate threshold is the inferred triple written to the graph database incrementally, and the summary vectors corresponding to the affected session paragraphs in the short-term summary vector library are updated simultaneously.

[0051] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows:

[0052] 1. This invention uses a dynamic confidence-aware routing module to monitor the uncertainty of each lexical unit in real time during the streaming generation of a large language model. It intercepts high-uncertainty fact nodes through reasoning and routes them to an external knowledge base for verification before resuming generation. This advances hallucination suppression from post-event detection to real-time intervention during generation, fundamentally eliminating structural hallucinations at the generation end. This provides the technical foundation for the system to be reliably implemented in zero-hallucination-tolerance scenarios such as medicine and law.

[0053] 2. This invention uses a hierarchical semantic memory graph module, with a three-layer architecture of immediate memory layer, short-term summary layer and long-term topic graph layer, to retain historical semantic information of different granularities. It retains the explicit relationship structure between entities through a graph database, and replaces flat summary with structured graph, effectively eliminating cross-turn entity confusion and long-range dependency loss, and supporting the coherence of complex multi-turn and multi-hop business dialogues.

[0054] 3. This invention uses a knowledge graph hot update and write-back mechanism to automatically and incrementally write newly discovered entity relationships into the knowledge graph during multi-round interactions. This achieves dynamic hot updates of knowledge without retraining the underlying large language model, ensuring the continuous timeliness of the knowledge base. Furthermore, the knowledge graph structure significantly enhances the system's multi-hop reasoning ability, realizing a new paradigm of dialogue-driven continuous evolution of the knowledge base.

[0055] 4. This invention uses an adaptive calculation mechanism for dynamic security thresholds, combining domain risk coefficients and overall contextual confidence to dynamically adjust the thresholds, enabling the system to maintain a reasonable balance between hallucination suppression accuracy and system response latency in different business domains and dialogue states. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the overall architecture of the intelligent question-answering system in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the core algorithm flow of the dynamic confidence-aware routing module in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the knowledge graph hot update and write-back mechanism in an embodiment of the present invention. Detailed Implementation

[0059] The following is in conjunction with the appendix Figure 1-3 The specific embodiments of the present invention will be further described below. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0060] System Overall Architecture and Implementation Method

[0061] The intelligent question-answering system described in this invention adopts an architecture that combines vertical layering with horizontal module parallelism. From top to bottom, it is divided into four main layers: user interaction layer, routing orchestration layer, reasoning and memory layer, and knowledge and storage layer.

[0062] The user interaction layer includes a user query input interface and an answer output interface, which are responsible for receiving user natural language queries and displaying system-generated answers.

[0063] The core module of the routing orchestration layer is the routing orchestrator, which is responsible for coordinating multiple processing links such as the generation stream, retrieval stream, and memory read / write stream. After receiving a user query, the routing orchestrator first triggers the memory reading process of the hierarchical semantic memory graph module: reading the original dialogue records from the immediate memory layer from the most recent rounds.

[0064] Simultaneously perform vector encoding on the current query and retrieve the most relevant summary record from the short summary vector library;

[0065] A one-hop graph query is performed in the long-term topic knowledge graph using the entity keywords in the query to obtain the neighbor relationships of related entities as structured context. The routing orchestrator assembles the above three layers of memory content into enhanced prompt words according to priority (the immediate memory layer has the highest priority, and the long-term topic graph layer has the lowest priority but serves as a structured fact anchor). After injecting system role instructions, the large language model inference engine is called to start the streaming generation task.

[0066] The reasoning and memory layer comprises three core sub-modules: a dynamic confidence-aware routing module, a hierarchical semantic memory graph module, and a large language model inference engine. The large language model inference engine is deployed based on a high-throughput inference framework that supports streaming word output and can expose the log probability of each word.

[0067] The dynamic confidence-aware routing module has real-time bidirectional signal interaction with the large language model inference engine. The dynamic confidence-aware routing module monitors the generation flow of the large language model inference engine and can send interception signals to the routing orchestrator.

[0068] A hot update write-back channel is marked between the routing orchestrator and the long-term topic knowledge graph, indicating the knowledge increment writing path.

[0069] The knowledge and storage layer includes three storage components: an immediate memory cache, a short-term summary vector library, and a long-term topic knowledge graph. These components correspond to the persistent backend of the three-layer memory architecture of the hierarchical semantic memory graph module. The long-term topic knowledge graph is stored in a graph database and supports precise entity matching and relation queries based on graph query language.

[0070] The short summary vector library is stored in a high-performance vector database and supports approximate nearest neighbor retrieval.

[0071] Implementation of Dynamic Confidence-Aware Routing Module

[0072] The complete workflow of the dynamic confidence-aware routing module is as follows.

[0073] The large language model inference engine pushes generated content word by word in a streaming manner, with each push containing the text representation of the current word. and its log probability ,Right now ,in For the first The sequence of all generated words before the step, This is the position index of the current word in the generated sequence, starting from 1.

[0074] The dynamic confidence-aware routing module for the first The local uncertainty score is defined by each lexical position. :

[0075] ;

[0076] in A higher value indicates that the model is less certain about that term. The maintenance length of the dynamic confidence-aware routing module is... A sliding window that is closest to the view within the window. The arithmetic mean of the local uncertainty scores of each word is taken to obtain the window average uncertainty. :

[0077] ;

[0078] in, This is the length of the sliding window, a system-preset hyperparameter with a default value of 8.

[0079] This is the index of the position of the word within the window.

[0080] The dynamic confidence-aware routing module runs a lightweight named entity recognition streaming model in parallel. It performs named entity boundary detection on the accumulated lexical sequences. The named entity recognition model uses sequence labeling to output the label category of the current lexical in the named entity span: B (entity start), I (entity interior), O (non-entity). When the end of the current named entity span is detected, the entity span is extracted. Calculate the uncertainty score of all terms within the entity node aggregation. :

[0081] ;

[0082] in, This is the starting lexical position index for the current named entity span;

[0083] This is the index of the end term position for the current named entity span;

[0084] This represents the number of lexical units within the entity's span.

[0085] The dynamic confidence-aware routing module calculates the dynamic security threshold for the current dialogue round. It is dynamically adjusted based on the following two factors: domain type coefficient. (A lower threshold is used for high-risk areas such as medical and legal fields, and a higher threshold is used for general Q&A.)

[0086] and the average uncertainty score of the confirmed fact nodes in the current context. (The threshold can be appropriately relaxed when the overall confidence level of the generated portion is high), dynamic security threshold. The calculation formula is:

[0087] ;

[0088] in, The base threshold is calibrated in experiments according to the model size, with a reference range of 2.0 to 4.0;

[0089] The domain risk coefficient has a range of values. The value is approximately 0.6 in the medical and legal fields, and approximately 1.0 in the general question-and-answer field.

[0090] This is the context adjustment coefficient, which controls the weight of the overall confidence of the generated content on the threshold. The default value is 0.3.

[0091] It is the mean of the average uncertainty scores of all entity nodes in the currently generated content (excluding the current entity node);

[0092] , , The units are all negative logarithmic probability units below the natural logarithm.

[0093] Will With dynamic security threshold Comparison: When No more than When the entity node is determined to be a low-uncertainty node, the dynamic confidence-aware routing module sends a pass signal to the routing orchestrator, the entity lexical sequence is appended to the output buffer, and the large language model inference engine continues to generate;

[0094] when Exceed When the entity node is determined to be a high-uncertainty node, the dynamic confidence-aware routing module sends an interception signal to the routing orchestrator, triggering the inference interception and fact alignment sub-process.

[0095] The specific process of the inference interception and fact alignment sub-process is as follows: The routing orchestrator pauses the generation call of the large language model inference engine, discards the generated lexical buffer content after the interception point, rolls back to the starting position of the intercepted entity node, and automatically generates a targeted fact verification subquery based on the semantic context of the generated content and the category of the currently intercepted entity node (the entity type label output by the named entity recognition module, covering types such as person name, organization name, date, and number). After the subquery is vectorized and encoded, a hybrid retrieval strategy is executed in the knowledge storage layer:

[0096] Prioritize performing exact entity matching and one-hop relation queries in the long-term topic knowledge graph using graph database query language. If a match is found, directly return the fact triples stored in the graph as fact text fragments. ;

[0097] If no match is found, the algorithm degenerates to performing an approximate nearest neighbor semantic search in the short-term summary vector library, returning the segment with the highest semantic relevance as the most relevant segment. The end-to-end time control target for the entire subquery process is 200 to 500 milliseconds, and the routing orchestrator will... This serves as a binding fact, injected as a contextual prefix to the current prompt word, and explicitly instructs the large language model inference engine to ensure that the entity location is consistent with... To maintain consistency, the generation call of the large language model inference engine is then resumed, and generation continues from the starting word position of the intercepted entity node. After generation is resumed, the dynamic confidence-aware routing module continues to monitor subsequent words in real time.

[0098] Once the large language model inference engine has completed generating all the content, the routing orchestrator gathers the complete word sequence in the output buffer, decodes it into the final answer text, and returns it to the user interaction layer.

[0099] Implementation of Hierarchical Semantic Memory Graph Module

[0100] The hierarchical semantic memory graph module organizes the dialogue history into three levels, with each level having the following responsibilities, triggering conditions, and storage backend.

[0101] The immediate memory layer maintains the original dialogue records from the most recent few rounds (10 rounds by default), stored in a memory cache as key-value pairs, preserving absolute semantic precision without any compression. Each record contains a round number. User query text System answer text and the set of entities obtained by named entity recognition in this round. ,in The number of entities in this round is specified. After each round of dialogue is completed, the new round record is written to the queue in an append-to-queue manner.

[0102] When the total number of records exceeds the preset capacity, the earliest record is removed from the immediate memory layer, triggering the incremental update process of the short summary layer.

[0103] The short-term summarization layer, based on a fine-tuned sequence-to-sequence intent summarization model, compresses dialogue records removed from the immediate memory layer into semantic summary text. The summary length is capped at 128 words, and then encoded using a text vectorization model (semantic encoder) to obtain the summary semantic vector. ,in The abstract semantic vector and metadata (round number) are vector-dimensional (usually 768 or 1024). Session identifier, summary text, and set of entities involved. The data is written into the short-term summary vector library to complete the incremental update of the short-term summary layer. During retrieval, the routing orchestrator performs vector encoding on the current query and then performs an approximate nearest neighbor search in the short-term summary vector library. It takes several of the most relevant summary records and concatenates the corresponding summary text to the context of the prompt words to supplement the historical semantic background not included in the immediate memory layer.

[0104] The long-term topic graph layer utilizes information extraction technology to dynamically write entities and their relationships extracted from multiple rounds of conversation into the graph database, forming a continuously evolving knowledge graph. After each round of dialogue, the hierarchical semantic memory graph module extracts the entity set from the immediate memory layer records of the current round. and the complete text pair The text pair input relation extraction model, for The entity pairs in the data are used to identify relationships, and a set of relationship triples is output. ,in For the head entity, For relation type labels, For tail entities, This represents the number of triplets drawn in this round.

[0105] right For each triple in the graph, an existence check is performed in the graph database: if the triple does not exist, the corresponding node and relation edge are created incrementally, and the source round is written into the relation edge attribute. Timestamps and confidence scores (output by the relation extraction model);

[0106] If a triple already exists but the new confidence score in this round is higher, then update the confidence value and last update timestamp in the relation edge attributes. The system periodically performs graph consistency maintenance: for confidence scores below the minimum retention threshold... (Default value is 0.4) and the last update time exceeds the aging window. (Default value is 72 hours) Relationship edges are soft-deleted (marked as pending archiving, not physically deleted) to control the graph size and remove outdated, low-quality relations. The range of values ​​is , The unit is hours.

[0107] Implementation of Knowledge Graph Hot Update and Write-back Mechanism

[0108] The knowledge graph hot update and write-back mechanism is triggered in two types of scenarios, and no parameter updates or retraining operations are required on the underlying large language model throughout the process.

[0109] Scenario 1 is triggered by user-initiated feedback, where the user submits feedback signals (positive confirmation or error correction text) to the system's answer in the dialogue interface. ), routing orchestrator receives Determine the feedback type:

[0110] If it is a positive confirmation, then the set of triplets in this round will be... The confidence score is below the median threshold The confidence level of the triplet (default value is 0.7) is increased to Then update the corresponding edge attributes in the graph database, and the process ends;

[0111] If it is error correction text, then the error correction text will be... The input relation extraction model extracts a set of modified triples contained in the user's actual intent. ,Will Compared with the original sampling results of this round Perform a difference comparison and mark the set of old triples that were replaced. With the newly added set of correct triples ;

[0112] right In the graph database, the confidence of the corresponding edge in the triplet is reduced to below the minimum retention threshold, triggering soft deletion logic.

[0113] right The triples in the array are used to perform incremental write operations with full confidence (1.0).

[0114] Scenario 2 is triggered by multi-hop logic inference. After each round of dialogue, the routing orchestrator performs lightweight multi-hop logic inference based on the existing nodes and relationships in the current long-term topic graph: starting from the newly added nodes in this round, it performs a path search of up to 3 hops along the relationship edges in a depth-first traversal manner to identify the implicit triple candidate set that satisfies the predefined inference rules. The inference rules are pre-configured in the form of a rule engine, and each rule is defined as follows:

[0115] If the graph contains and Then it can be inferred that ,in Defined by rules, it supports types such as transitive relations, inverse relations, and merging of equivalent entities.

[0116] right Each inference triplet in The large language model inference engine verifies and scores the reasonableness of the inference using zero-shot hints, and outputs a confidence estimate. The value ranges from 0 to 1, and only when... Not lower than the middle threshold At that time, the inferred triple is incrementally written into the graph database, and the source is marked as multi-hop inference and the inference confidence is marked in the relation edge attribute. ;

[0117] Otherwise, discard the candidate triplet.

[0118] After all write operations are completed, the hierarchical semantic memory graph module synchronously updates the summary vectors corresponding to the affected session segments in the short-term summary vector library to ensure the consistency of the three-layer memory content. At this point, a complete hot update write-back process is completed, and the system returns to the ready state, waiting for the next round of user input.

[0119] Implementation of the complete dialogue processing flow of the system

[0120] Complete system dialogue processing flow (Part 1) The end-to-end process of the wheel is as follows.

[0121] User submitted the Round query The routing orchestrator receives This triggers the memory retrieval process of the hierarchical semantic memory graph module: retrieving the original dialogue records from the immediate memory layer for the most recent rounds;

[0122] Synchronization Perform vector encoding and retrieve the most relevant short summary records from the short summary vector library (the default number of records retrieved is 3).

[0123] by The entity keywords in the graph are used to perform a one-hop graph query in the long-term topic knowledge graph to obtain the neighbor relationships of related entities as a structured context.

[0124] The routing orchestrator assembles the above three layers of memory content into enhanced prompt words according to priority, injects them into system role instructions, and then calls the large language model inference engine to start the next step. The task is generated in a streaming manner.

[0125] The large language model inference engine pushes words in a streaming manner. The dynamic confidence-aware routing module performs word-level uncertainty score calculation and named entity boundary detection in real time, and triggers inference interception and fact alignment sub-processes as needed.

[0126] After the large language model inference engine completes all generation, the routing orchestrator decodes the output buffer and outputs the final answer text. Return to the user interaction layer and display it to the user.

[0127] Simultaneously with the output of the answer, the routing orchestrator asynchronously triggers the hierarchical semantic memory graph module update process: immediate memory layer appending. Record;

[0128] If the instant memory layer queue is full, the short-term summary layer incremental update process is triggered.

[0129] Parallel execution of information extraction, resulting in new triples Incremental writing to the long-term theme graph layer.

[0130] The system displays a user feedback entry on the interface. If the user submits feedback within a preset time window (default 30 seconds), the hot update write-back process is triggered.

[0131] Otherwise, the system will enter the next ready state after the timeout.

[0132] Alternative implementation methods

[0133] In the uncertainty perception stage of the dynamic confidence-aware routing module, a sampling consistency test can be used instead of the lexical log probability method. Specifically, when the generated content reaches the boundary of the named entity node, the system does not rely on log probability, but instead independently performs 3 to 5 samplings on the entity node position with a high temperature parameter. The character-level edit distance between the multiple sampling results is compared. If the character-level edit distance between the multiple sampling results exceeds a preset difference threshold, it is regarded as a high uncertainty node, triggering the same inference interception process as the main solution. This alternative method does not rely on the large language model inference engine to expose the log probability interface, and is suitable for commercial interface scenarios that only provide text output, but it will introduce higher computational overhead and latency.

[0134] In the long-term topic graph layer of the hierarchical semantic memory graph module, a temporal knowledge graph can be used instead of a regular knowledge graph as the storage structure for long-term memory. The temporal knowledge graph is used for each relation triple. Add a timestamp dimension to form a quadruple. ,in Given the valid time interval of a relationship, the routing orchestrator prioritizes returning relationship facts with the most recent timestamp and within the validity period when retrieving long-term memory, and automatically reduces the weight of expired relationships. It naturally supports temporal logic queries and is suitable for professional domain question-answering systems with explicit needs for temporal reasoning.

[0135] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. An intelligent question-answering system based on natural language processing, characterized in that, The system includes a routing orchestrator, a dynamic confidence-aware routing module, a hierarchical semantic memory graph module, a large language model inference engine, and a knowledge storage layer; The large language model inference engine supports streaming word output and log probability exposure of each word, and accepts prompt word assembly and segmentation generation control instructions from the routing orchestrator. The dynamic confidence-aware routing module receives streaming lexical units and their log probabilities output by the large language model inference engine in real time. It calculates the local uncertainty score of each lexical unit based on a sliding window, runs the named entity recognition streaming model in parallel to perform named entity boundary detection on the accumulated lexical unit sequence, and aggregates the arithmetic mean of the uncertainty scores of all lexical units within the span when the named entity span ends to obtain the aggregated uncertainty score of the entity node. It compares the aggregated uncertainty score of the entity node with a dynamic security threshold. When the aggregated uncertainty score of the entity node exceeds the dynamic security threshold, it sends an interception signal to the routing orchestrator. The routing orchestrator is the central control module of the system, responsible for receiving user queries, reading the three-layer memory context of the hierarchical semantic memory graph module, driving the large language model inference engine to generate answers in a streaming manner, and listening to the interception signals of the dynamic confidence perception routing module. When an interception signal is received, the generation call of the large language model inference engine is paused and rolled back to the starting position of the intercepted entity node. Based on the semantic context of the generated content and the category of the intercepted entity node, a targeted fact verification subquery is generated. The subquery is routed to the knowledge storage layer for retrieval. The retrieved fact text fragments are injected with the context prefix of the prompt words and then the generation is resumed. After each round of answer generation is completed, the hierarchical semantic memory graph module is driven to perform memory update and knowledge graph write-back. The hierarchical semantic memory graph module maintains a three-layer memory structure consisting of an immediate memory layer, a short-term summary layer, and a long-term topic graph layer, and is responsible for memory reading and writing and cross-layer update scheduling after each round of dialogue. The knowledge storage layer consists of three parts: an immediate memory cache, a short-term summary vector library, and a long-term topic knowledge graph, which respectively correspond to the persistent backend of the three-layer memory architecture of the hierarchical semantic memory graph module.

2. The system according to claim 1, characterized in that, The dynamic confidence-aware routing module for the first The local uncertainty score is defined by each lexical position. Its calculation method is based on the model in context. Generating lexical elements under conditions The conditional probability is inverted after taking the natural logarithm: ； in, A higher value indicates that the model is less certain about that term; For the first Text representation of each word element; For the model in context Generating lexical elements under conditions The conditional probability; For the first The sequence of all generated words before the step; Take the natural logarithm; Length is The arithmetic mean of the local uncertainty scores of all tokens within the sliding window constitutes the window average uncertainty. ； in, The sliding window length is a system-preset hyperparameter. This is the position index of the word within the window; When Named Entity Span At the end, the entity nodes aggregate uncertainty scores. The calculation formula is: ； in, This is the starting lexical position index for the current named entity span; This is the index of the end term position for the current named entity span; This represents the number of lexical units within the entity's span.

3. The system according to claim 1, characterized in that, The dynamic security threshold The calculation formula is as follows: The domain type coefficient is dynamically adjusted based on the overall confidence of the generated context. ； in, Based on the threshold, The value range is from 2.0 to 4.0; The domain risk coefficient has a range of values. The values ​​for high-risk areas are lower than those for general question-and-answer areas. The value is 0.5-0.7 in high-risk areas and 0.9-1.0 in general question-and-answer areas; The context adjustment coefficient controls the weight of the overall confidence level of the generated content on the threshold. The default value is 0.3; This is the mean of the average uncertainty scores of all entity nodes in the currently generated content; , , The units are all negative logarithmic probability units below the natural logarithm.

4. The system according to claim 1, characterized in that, When the routing orchestrator routes the directed fact-verification subquery to the knowledge storage layer, it employs a hybrid retrieval strategy: Prioritize performing exact entity matching and one-hop relation queries in the long-term topic knowledge graph using graph database query language. If a match is found, directly return the fact triple stored in the graph as a fact text fragment. If no match is found, the process degenerates to performing an approximate nearest neighbor semantic search in the short-term summary vector library, returning the segment with the highest semantic relevance as the factual text segment; The routing orchestrator injects the factual text fragment into the context prefix of the cue word and constrains the generation result of the large language model inference engine to be consistent with the factual text fragment with explicit instructions, and then restores the generation from the starting word position of the intercepted entity node.

5. The system according to claim 1, characterized in that, The instant memory layer stores several rounds of original dialogue records in the memory cache in the form of key-value pairs. Each record contains the round number, user query text, system answer text, and the entity set obtained by named entity recognition in that round, without any compression. When the total number of records exceeds the preset capacity, the earliest round record is removed from the instant memory layer, triggering the incremental update process of the short-term summary layer. The short-term summary layer, based on a sequence-to-sequence intent summarization model, compresses dialogue records removed from the immediate memory layer into semantic summary text, encodes them into summary semantic vectors through a text vectorization model, and writes them together with the turn number, session identifier, summary text, and the set of entities involved into the short-term summary vector library. During retrieval, the routing orchestrator performs vector encoding on the current query and then performs an approximate nearest neighbor search in the short-term summary vector library, taking several of the most relevant summary records and concatenating them into the context of the prompt words.

6. The system according to claim 1, characterized in that, The long-term topic graph layer uses information extraction technology to dynamically write entities and their relationships extracted from multi-turn conversations into the graph database. The graph database uses nodes to represent entities and directed edges to represent semantic relationships between entities, forming a triplet structure. After each round of dialogue, the relation extraction model is used to identify the relation pairs of the current round, outputting a set of relation triples. After performing an existence check on each triple, it is written to the graph database in an incremental manner. At the same time, the source round, timestamp, and confidence score are recorded in the relation edge attributes. The system periodically performs soft deletion on relation edges whose confidence scores are below the minimum retention threshold and whose last update time exceeds the aging window, marking them as pending archiving rather than physically deleting them, in order to control the size of the graph.

7. The system according to claim 6, characterized in that, The system also includes a knowledge graph hot update and write-back mechanism. In scenarios triggered by user feedback, the routing orchestrator receives user feedback signals and determines the feedback type. When the feedback is positive confirmation, the confidence of triples whose confidence is below the middle threshold in this round is raised to the middle threshold and the corresponding relation edge attributes in the graph database are updated. When the feedback is error correction text, the error correction text is input into the relation extraction model to extract the set of corrected triples. The set of corrected triples is then compared with the original extraction results of this round. For the old triples that have been deleted, the confidence level is reduced to below the minimum retention threshold to trigger soft deletion. For the newly added correct triples, an incremental write operation is performed with full confidence.

8. The system according to claim 7, characterized in that, In the multi-hop logic inference triggering scenario, the knowledge graph hot update write-back mechanism uses a depth-first traversal method to start from the newly added node in this round, and performs a path search of up to 3 hops along the relation edges to identify the implicit triple candidate set that satisfies the predefined inference rules. The large language model inference engine performs a reasonableness verification score for each candidate triple using zero-shot hints. Only when the verification confidence is not lower than the intermediate threshold is the inferred triple written into the graph database in an incremental manner, and the source is marked as multi-hop inference and the corresponding inference confidence in the relation edge attribute. After all write operations are completed, the summary vectors corresponding to the affected session segments in the short-term summary vector library are updated synchronously, without needing to update the parameters of the underlying large language model.

9. The system according to claim 1, characterized in that, The dynamic confidence-aware routing module is also configured to: in scenarios where the logarithmic probability of a lexical unit is unavailable, use a sampling consistency test to assess the uncertainty of named entity nodes; When the generated content reaches the boundary of the named entity node, multiple sampling and generation are performed independently at the location of the entity node using high temperature parameters, and the character-level editing distance between the multiple sampling results is compared. When the character-level edit distance of multiple sampling results exceeds a preset difference threshold, the entity node is determined to be a high-uncertainty node, triggering the inference interception process.

10. An intelligent question-answering method based on natural language processing, characterized in that, This method is executed collaboratively by a route orchestrator, a dynamic confidence-aware routing module, a hierarchical semantic memory graph module, a large language model inference engine, and a knowledge storage layer, and includes the following processes: The routing orchestrator receives user queries, reads the original dialogue records of the immediate memory layer, the near nearest neighbor retrieval results of the short summary layer, and the one-hop graph query results of the long-term topic graph layer from the hierarchical semantic memory graph module, assembles the three layers of memory content into enhanced prompt words according to priority, and calls the large language model inference engine to start streaming generation. The large language model inference engine pushes word units and their log probabilities in a streaming manner. The dynamic confidence-aware routing module calculates word-level local uncertainty scores in real time. Combined with the named entity recognition streaming model, it performs named entity boundary detection. It aggregates the arithmetic mean of word uncertainty scores within the span of named entities to obtain the aggregated uncertainty score of entity nodes. This score is compared with a safety threshold dynamically calculated based on the domain risk coefficient and the overall confidence of the generated context. When the aggregated uncertainty score of entity nodes exceeds the safety threshold, the inference interception process is triggered. A targeted fact verification subquery is generated and routed to the knowledge storage layer to perform hybrid retrieval. The retrieved fact text fragments are injected with the context prefix of the prompt word and then restored from the interception position. After the large language model inference engine completes the generation, the routing orchestrator returns the final answer to the user and asynchronously triggers the hierarchical semantic memory graph module to update the immediate memory layer records, short-term summary layer vectors, and long-term topic graph layer triples. When the user submits feedback, the knowledge graph hot update write-back process is triggered. Based on the feedback type, the confidence of the corresponding triples in the graph database is adjusted or incremental writing is performed, and the summary vectors corresponding to the affected conversation paragraphs in the short-term summary vector library are updated synchronously. There is no need to update the parameters of the underlying large language model.

Images