Knowledge base engine construction method and system based on large language model

Abstract

The invention relates to a knowledge base engine construction method and system based on a large language model, and belongs to the technical field of artificial intelligence. According to the system, a triangular closed-loop iterative architecture is formed by a question and answer center module, a note module and a knowledge base module, and the following core functions are achieved: an automatic knowledge extraction mechanism based on a large language model converts unstructured interaction data into a standard knowledge unit; according to the traceable retrieval enhancement generation method integrated with the reflection token, full-link monitoring is realized through retrieval judgment, correlation evaluation and verification of the reflection token; and the introspection verification closed-loop system outputs deterministic scores through the knowledge evaluator and executes hierarchical correction measures. According to the method, the defects of illusion problem, low retrieval efficiency, lack of introspection mechanism and the like in the prior art are effectively solved, high efficiency, accuracy and self-evolution ability of a knowledge base engine are realized, and the method can be widely applied to intelligent question answering and auxiliary decision making systems in various professional fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and knowledge engineering, in particular to a construction, iteration and optimization method of a knowledge base engine based on a large language model (LLM), specifically including automatic knowledge extraction, traceable retrieval-augmented generation (RAG) and self-inspection verification closed-loop system. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, knowledge base systems have become the core tool of enterprise knowledge management and intelligent decision-making. Current retrieval-augmented generation (RAG) systems based on large language models can partially alleviate the model hallucination and knowledge lag problems, but still have the following limitations: first, the hallucination problem is not completely solved, the model may ignore or distort the retrieval evidence; second, the retrieval accuracy and efficiency are insufficient, lacking intelligent retrieval judgment mechanism; third, knowledge update relies on manual work, with high cost and low efficiency; fourth, there is a lack of traceability and self-inspection mechanism, which cannot realize the error detection and correction closed loop. These defects seriously restrict the application of the system in key task scenarios, and a new generation of knowledge base engine with high efficiency, accuracy and self-evolution ability is urgently needed. SUMMARY

[0003] The present application aims to overcome the shortcomings of the prior art and provide a knowledge base engine construction method and system based on a large language model. The system realizes automatic knowledge extraction, accurate retrieval and intelligent self-inspection through a triangular closed-loop iterative architecture composed of a question and answer hub module, a note module and a knowledge base module, significantly improving the accuracy, real-time performance and reliability of the knowledge base.

[0004] The core functional modules of the application include a question and answer hub module, a note module and a knowledge base module, which form a stable closed loop that cooperates and drives each other. The question and answer hub module is constructed based on a large language model and serves as the main interface for the system to interact with the user. It receives the user's natural language question and answer, note organization instructions and knowledge base construction instructions, processes and analyzes them, and outputs the question and answer results, structured notes and knowledge items. The note module serves as the core knowledge entry and buffer of the system. It receives direct records from the user, high-quality question and answer records generated by the question and answer hub and optimization inputs for iterative knowledge base, which can be preliminarily structured by the question and answer hub module and output standard knowledge units to be confirmed and stored in the knowledge base. The knowledge base module serves as the structured knowledge storage and application center of the system. It receives knowledge units from the note module, user's raw input (such as file upload) and queries from the question and answer hub, outputs knowledge sharing and precise retrieval results, and provides retrieval feedback to the question and answer hub to complete answer generation. The three modules form a triangular closed loop iterative relationship of "question and answer hub-note-knowledge base": the question and answer hub (model) generates notes, the notes are deposited into the knowledge base, and the knowledge base in turn supports and optimizes the accuracy and explainability of the question and answer hub, thereby realizing the self-evolution of the system.

[0005] The core innovation of the application mainly includes three parts: automatic knowledge extraction mechanism, traceable retrieval enhancement generation method and self-reflection verification closed loop system.

[0006] The automatic knowledge extraction mechanism analyzes the high-value user question and answer records generated by the question and answer hub, the notes actively submitted by the user and the data in the daily workflow in a regular or event-triggered manner. With the summarization and induction ability of the large language model under the question and answer hub module, these unstructured interaction data can be automatically extracted and summarized into structured and accurate notes. These notes are formatted into standard knowledge items by the question and answer hub module again, which are automatically or manually confirmed and updated to the knowledge base module, realizing the autonomous evolution and continuous optimization of the knowledge base.

[0007] The traceable retrieval enhancement generation method integrates the 'Reflection Tokens' mechanism based on the prior art to enhance the traceability and decision accuracy of the retrieval link. Specifically, the method realizes fine-grained monitoring through three optimization stages: in the retrieval judgment stage, a custom threshold algorithm is used to generate retrieval necessity reflection tokens to dynamically evaluate the dependence of the query on the knowledge base, thereby reducing unnecessary retrieval overhead; in the paragraph relevance evaluation stage, a multi-dimensional similarity calculation model is introduced to finely filter the retrieval results, improving the matching accuracy of the content and the query intent; in the generation and verification stage, combined with the evidence chain verification mechanism, it is ensured that the final answer is strictly based on the retrieval content and maintains the integrity. The whole process is realized through structured log recording to achieve full traceability, significantly improving the explainability and running efficiency of the system, and overcoming the problems of retrieval redundancy and insufficient verification in the prior art.

[0008] The self-reflection verification link performs macroscopic evaluation and intelligent correction on the retrieval-generation whole process output through a specially designed knowledge evaluator. The system outputs a three-level certainty score (sure, uncertain, and irrelevant), and takes graded correction measures according to the evaluation results: the sure result is directly output; the uncertain result triggers the knowledge optimization process, including supplementary retrieval, query rewriting, or marking knowledge gaps and flowing into the automatic knowledge refining process; the irrelevant result starts the reprocessing mechanism and is handed over to manual processing. This design forms a self-correction and continuous optimization mechanism for the system, significantly improving the reliability and stability of the output.

[0009] Through the above innovations, the present application solves the problems of hallucination, low retrieval efficiency and lack of self-reflection mechanism in the prior art, and realizes efficient, accurate and self-evolution of the knowledge base engine. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 Fig. 1 is a schematic diagram of the system triangular closed-loop iterative architecture, showing the interaction between the question and answer central module, the note module, and the knowledge base module; Figure 2 Fig. 2 is a schematic diagram of the retrieval enhancement generation process, detailing the retrieval judgment, relevance evaluation, and verification steps based on reflection tokens; Figure 3 Fig. 3 is a schematic diagram of the self-reflection verification process, showing the evaluation and correction measures process. The drawings are helpful for those skilled in the art to understand the embodiments of the present application. DETAILED DESCRIPTION

[0011] The specific steps of the knowledge base engine construction method and system based on large language models are described in detail below in conjunction with the best embodiment of the present application. This embodiment takes a software system as an example, but the present application is not limited to this and can be applied to hardware, cloud platforms, or hybrid deployment environments.

[0012] First, the system is initialized, and the question and answer hub module, note module, and knowledge base module are deployed. The question and answer hub module is built based on a pre-trained large language model (such as DeepSeek R1 or similar models), and is integrated with the user interface and other modules through an application programming interface (API) or a message queue (such as RocketMQ or Kafka). The note module is designed as a lightweight data processing unit that supports natural language processing functions and is used for temporary storage and structured processing of data. The knowledge base module uses a vector database (such as ElasticSearch) with high search capability, which can be combined with a relational database (such as PostgreSQL) to realize multi-modal knowledge storage, supporting the retrieval of text, images, and structured data.

[0013] Implement the automatic knowledge extraction step: the system sets a timing task (for example, execute every morning) or an event trigger (such as when more than 100 new records are accumulated in the note module), and starts the knowledge extraction process. The question and answer hub module analyzes user interaction data, including high-quality question and answer records (such as answers marked as "useful" by users) and user-submitted notes. Using the summarization capabilities of the large language model, structured knowledge entries are generated, such as Q-A pairs or factual statements extracted from a conversation. The note module formats these entries into standard knowledge units in JSON format, which are then automatically or confirmed by administrators and updated to the knowledge base module through API calls. For example, in the programming field, the system can automatically extract new standard code components from programmers' question and answer.

[0014] Implement the traceable retrieval enhancement generation step: when a user submits a query, the question and answer hub module first generates a retrieval judgment reflection token. Specifically, it calls the large language model to generate a binary output (yes / no) to determine whether the knowledge base needs to be searched. If not, the answer is generated directly based on the model's internal knowledge; if so, a search request is sent to the knowledge base module. The knowledge base module returns the preliminary search results, and for each knowledge paragraph, a paragraph relevance reflection token is generated, which is evaluated by the large language model to obtain a relevance score (0-1 range), and paragraphs with a score below 0.5 are filtered out. Then, the high-relevance paragraphs are input into the large language model along with the query to generate the final answer, and a verification reflection token is generated to ensure that the answer is based on evidence. The generation and evaluation of all reflection tokens are recorded in the log database, enabling full-link traceability.

[0015] Implement self-reflection verification step: knowledge evaluator (an independent large language model instance) evaluates the "question-retrieval evidence-generate answer" triple, outputs a certainty score (sure, uncertain, irrelevant). If it is sure, the answer is directly output; if it is uncertain, trigger knowledge optimization process, such as supplement retrieval more paragraphs, rewrite query (use large language model to rewrite user query to improve retrieval), or mark the case as knowledge gap, flow into automatic knowledge extraction process; if it is irrelevant, discard the result and transfer to manual processing. The knowledge evaluator is regularly trained and fine-tuned to improve evaluation accuracy.

[0016] Operation of the triangular closed-loop iterative system: in daily user interaction, the data generated by the question and answer hub module is deposited into the knowledge base module through the note module, and the update of the knowledge base module optimizes the performance of the question and answer hub through retrieval feedback. For example, in the financial field, the system automatically learns new market trends from the question and answer of investment consultants, updates the knowledge base, and thus improves the accuracy of subsequent question and answer.

[0017] The method of the present application supports multi-modal data processing, for example, the knowledge base module can store and retrieve text reports, image charts and structured tables. Modules communicate through REST API or message queue, ensuring loose coupling and scalability. The method is applied to intelligent question and answer and decision support systems in professional fields such as programming, law, medicine and finance, significantly improving the accuracy, real-time performance and reliability of knowledge management.

[0018] The above embodiments are only examples, and those skilled in the art can adjust the parameters and components according to actual needs without departing from the core idea of the present application. The scope of protection of the present application is defined by the claims.

Claims

1. A method for constructing a knowledge base engine based on a large language model, characterized in that, Includes the following steps: The system receives and processes user input in natural language questions and answers, note-taking instructions, and knowledge base construction instructions through a question-and-answer central module, outputting corresponding question-and-answer results, structured notes, and knowledge entries. The note-taking module receives output from the question-and-answer central module, direct user input, and iterative optimization input from the knowledge base, performs structured processing through the central module, and outputs standard knowledge units. The knowledge base module receives standard knowledge units from the note-taking module, raw user input data, and query requests from the central module, performs structured knowledge storage, multimodal retrieval, and application processing, and feeds back the retrieval results to the central module to support answer generation. The question-and-answer central module, note-taking module, and knowledge base module form a triangular closed-loop iterative system, enabling knowledge to be transferred from the question-and-answer central module to the knowledge base module via the note-taking module. Simultaneously, the knowledge base module enhances the accuracy and interpretability of the central module's output through retrieval and feedback.

2. The method according to claim 1, characterized in that, It also includes an automatic knowledge extraction step: periodically or based on an event-triggered mechanism, it analyzes high-value user Q&A records, user-submitted notes, and daily workflow data generated by the Q&A hub module, and uses the inductive and summarizing capabilities of the large language model to automatically extract unstructured data into clearly structured and factually accurate knowledge items. After being formatted into standard knowledge units by the note module, the data is automatically or manually updated to the knowledge base module.

3. The method according to claim 1, characterized in that, It also includes traceable retrieval enhancement generation steps: when a user query is received, a retrieval judgment reflection token is first generated, and the large language model is used to determine whether it is necessary to rely on knowledge base retrieval to generate a reliable answer; if not, the answer is generated directly based on the knowledge inside the large language model. If so, a retrieval request is sent to the knowledge base module; for the preliminary retrieval results returned by the knowledge base module, a paragraph relevance reflection token is generated; the relevance score of each paragraph to the query intent is evaluated through a large language model, and paragraphs below the set threshold are filtered out; the highly relevant paragraphs and the user query input are combined to generate the final answer through the large language model, and a verification reflection token is generated to ensure that the answer is strictly based on the retrieval evidence and the content is complete.

4. The method according to claim 3, characterized in that, The generation and evaluation processes of the retrieval judgment reflection token, paragraph relevance reflection token, and verification reflection token are all recorded in the log database, realizing full-link knowledge traceability and system interpretability.

5. The method according to claim 1, characterized in that, It also includes a self-verification step: through an independent knowledge evaluator, the triplet of question-retrieval evidence-generated answer output from the retrieval-generation process is macroscopically evaluated, and a certainty score is output, including three levels: certain, uncertain, and irrelevant; based on the certainty score, graded corrective measures are implemented: for certain results, the answer is directly output; for uncertain results, a knowledge optimization process is triggered, including supplementing the retrieval, query rewriting, or marking knowledge gaps and flowing into the automatic knowledge extraction process; for irrelevant results, the generated content is discarded and transferred to manual processing.

6. The method according to claim 5, characterized in that, The knowledge optimization process specifically includes: rewriting the user query semantically using a large language model and then re-retrieving it, or automatically extracting new knowledge from high-quality question and answer records and updating it to the knowledge base module through the notes module.

7. The method according to claim 1, characterized in that, The triangular closed-loop iterative system enables the continuous self-evolution of the knowledge base. Specifically, the data generated by the question-and-answer hub module based on user interaction is precipitated into structured knowledge in the knowledge base module through the note-taking module. The updates of the knowledge base module, in turn, enhance the accuracy and interpretability of the question-and-answer hub module through retrieval feedback.

8. The method according to claim 1, characterized in that, The knowledge base module supports the storage and retrieval of multimodal knowledge, including text, images, and structured data.

9. The method according to claim 1, characterized in that, The interaction between the question-and-answer central module, the note-taking module, and the knowledge base module is achieved through an application programming interface or a message queue, ensuring loose coupling and scalability of the system architecture.

10. A knowledge base engine construction system based on a large language model, used to implement the method of any one of claims 1 to 9, characterized in that, The system includes a question-and-answer hub module for receiving and processing user input, outputting question-and-answer results, structured notes, and knowledge entries; a note module for receiving and temporarily storing output from the question-and-answer hub module, user-generated notes, and knowledge base optimized input, and outputting standard knowledge units; and a knowledge base module for receiving standard knowledge units from the note module, original user input, and query requests from the question-and-answer hub module, performing structured storage, multimodal retrieval, and feedback processing. These three modules are interconnected via interfaces or message queues, forming a triangular closed-loop iterative architecture to achieve automatic knowledge extraction, traceable retrieval enhancement generation, and introspective verification.

Images