AI local knowledge base construction method and system based on hybrid retrieval and closed-loop optimization

Abstract

The embodiment of the invention discloses an AI local knowledge base construction method and system based on hybrid retrieval and closed-loop optimization, and the method and system are used in an enterprise and aim at guaranteeing data safety and improving retrieval accuracy and efficiency. The method comprises the steps that intelligent blocking and vectorization storage are conducted on an original document; screening most relevant knowledge fragments by adopting a hybrid retrieval mechanism in which semantic vector retrieval and keyword retrieval are parallel and combining a reordering model; generating answers by using a local large language model and marking reference sources to ensure traceability; and performing continuous closed-loop optimization on the retrieval model and the knowledge base based on user feedback. The method is operated in a completely privatized deployment environment, has high retrieval accuracy, strong data security and adaptive evolution capability, and effectively overcomes the defects of traditional keyword retrieval and a general RAG scheme in the aspects of semantic comprehension, complex document processing and continuous optimization.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for constructing an AI local knowledge base based on hybrid retrieval and closed-loop optimization. Background Technology

[0002] Traditional knowledge management methods primarily rely on keyword-matching search engines. These methods have significant limitations:

[0003] (1) Poor semantic understanding: The system is unable to understand the user's true intent and is insensitive to synonyms, near-synonyms, and contextual relationships, resulting in low recall and poor precision of search results. Users often need to spend a lot of time filtering irrelevant results.

[0004] (2) Weak ability to process unstructured data: Traditional systems have difficulty effectively processing unstructured data in various formats such as PDF, Word, and images, resulting in a large amount of knowledge assets not being effectively utilized.

[0005] (3) High maintenance costs: The classification, tagging and association of knowledge are highly dependent on manual labor, which is time-consuming, labor-intensive and difficult to keep updated.

[0006] With the development of Large Language Model (LLM) technology, some question-answering systems based on public cloud large models have emerged. However, uploading private enterprise data to public cloud platforms for processing poses serious risks of data security and privacy breaches. Enterprise core intellectual property and sensitive data may be used for model training or subjected to unauthorized access, and data sovereignty cannot be guaranteed.

[0007] To address these issues, the industry has proposed a private knowledge base solution based on Retrieval-Augmented Generation (RAG). This solution deploys a model in a local environment, retrieves private documents from a vector database, and then generates answers using a large model, which can, to some extent, ensure data security and improve question-answering performance.

[0008] However, existing general-purpose RAG frameworks still face challenges in practical applications:

[0009] (1) Bottleneck in retrieval accuracy: Simple vector similarity retrieval sometimes fails to accurately recall the most relevant information, especially when dealing with long documents, complex tables or specific terms, where incomplete or incorrect recall may still occur.

[0010] (2) Lagging knowledge updates and optimization: Existing solutions are mostly built in a one-time manner, lacking effective and automated feedback and optimization mechanisms. The system cannot adaptively learn and iterate based on user usage, resulting in insufficient "activity" of the knowledge base and a decline in accuracy over time.

[0011] (3) Coarse data preprocessing: The cleaning and block processing of the original documents are relatively simple, and the document structure and semantic coherence are not fully considered, which affects the quality of subsequent vectorization and retrieval.

[0012] Therefore, there is an urgent need for a new technical solution that can further improve the retrieval accuracy, efficiency, and adaptive optimization capabilities of private knowledge bases while ensuring absolute data security, in order to address the shortcomings of existing technologies. Summary of the Invention

[0013] To address these issues, this invention provides a method and system for constructing an AI local knowledge base based on hybrid retrieval and closed-loop optimization, thereby solving the technical problems of insufficient retrieval accuracy, poor performance in handling complex documents, and lack of continuous optimization capabilities in existing technologies when deployed on a private scale.

[0014] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0015] According to a first aspect of the present invention, a method for constructing an AI local knowledge base based on hybrid retrieval and closed-loop optimization is provided, the method comprising:

[0016] The input raw documents are parsed, cleaned, and semantically coherently segmented to form knowledge fragments, and key entities and metadata are extracted.

[0017] Using a corpus-fine-tuned sentence converter model, the knowledge fragments are converted into vectors and stored in a private vector database to establish an efficient index;

[0018] Receive user queries, perform multi-path retrieval in parallel, recall relevant knowledge fragments, reorder the recalled knowledge fragments, and select the most relevant Top-N knowledge fragments;

[0019] Based on the selected knowledge fragments, prompt words are constructed, which are then input into a locally deployed large language model to generate answers, and the source citations are labeled for key information in the answers.

[0020] Collect user feedback on answers, use the feedback data to optimize the re-ranking model and / or embedding model, and revise and update the knowledge base.

[0021] Furthermore, the input raw document is parsed, cleaned, and semantically coherently segmented to form knowledge fragments, and key entities and metadata are extracted, including:

[0022] The semantic coherence segmentation is based on the paragraph structure, heading level, and punctuation of the text to intelligently segment it, ensuring that each knowledge fragment is semantically complete and cohesive.

[0023] Furthermore, using a corpus-fine-tuned sentence converter model, the knowledge fragments are converted into vectors and stored in a private vector database to establish an efficient index, including:

[0024] The sentence converter model is the Sentence-BERT model, which has been fine-tuned using a professional corpus.

[0025] Furthermore, upon receiving user queries, the system performs parallel multi-path retrieval to recall relevant knowledge fragments. These fragments are then reordered, and the top-N most relevant fragments are selected, including:

[0026] The multi-channel recall includes:

[0027] Semantic vector recall: Vectorize queries and retrieve similar knowledge fragments from a vector database;

[0028] Keyword recall: Use a sparse retrieval model to perform full-text retrieval of keywords in the query.

[0029] Furthermore, the system receives user queries, performs parallel multi-path searches, retrieves relevant knowledge fragments, reorders the retrieved knowledge fragments, and selects the most relevant Top-N knowledge fragments. This also includes:

[0030] The multi-path recall also includes knowledge graph association recall: based on the entities in the query, related entities and relationship knowledge are recalled from the constructed knowledge graph.

[0031] Furthermore, the system receives user queries, performs parallel multi-path searches, retrieves relevant knowledge fragments, reorders the retrieved knowledge fragments, and selects the most relevant Top-N knowledge fragments. This also includes:

[0032] The re-ranking uses a lightweight re-ranking model that comprehensively evaluates the semantic similarity between knowledge fragments and queries, keyword matching degree, and metadata freshness, and then re-ranks them.

[0033] Furthermore, based on the selected knowledge fragments, prompt words are constructed, input into a locally deployed large language model to generate answers, and the key information in the answers is labeled with citation sources, including:

[0034] The sources cited in the annotations include the original text location, document name, or page number information of the corresponding knowledge fragment.

[0035] Furthermore, user feedback on the answers is collected, and the feedback data is used to optimize the re-ranking model and / or embedding model, and to revise and update the knowledge base, including:

[0036] Based on the feedback data analysis, identify problems in knowledge preprocessing or segmentation, and prompt the administrator to correct or supplement the knowledge source.

[0037] Furthermore, the method operates in a private deployment environment, with all data processing and model inference completed within the internal network.

[0038] According to a second aspect of the present invention, an AI local knowledge base construction system based on hybrid retrieval and closed-loop optimization is provided, the system comprising:

[0039] The data preprocessing module is used to parse, clean, and semantically coherently segment the input raw documents to form knowledge fragments and extract key entities and metadata.

[0040] The knowledge vectorization module is used to convert the knowledge fragments into vectors using a corpus-fine-tuned sentence converter model and store them in a private vector database to establish an efficient index.

[0041] The hybrid retrieval module is used to receive user queries, perform multi-path retrieval in parallel, recall relevant knowledge fragments, reorder the recalled knowledge fragments, and select the most relevant Top-N knowledge fragments.

[0042] The intelligent generation module is used to construct prompt words based on filtered knowledge fragments, input them into a locally deployed large language model to generate answers, and annotate the sources of key information in the answers.

[0043] The closed-loop optimization module is used to collect user feedback on answers, use the feedback data to optimize the re-ranking model and / or embedding model, and correct and update the knowledge base.

[0044] The embodiments of the present invention have the following advantages:

[0045] This invention intelligently segments and vectorizes the original document; employs a hybrid retrieval mechanism that combines semantic vector retrieval and keyword retrieval, and uses a re-ranking model to select the most relevant knowledge fragments; generates answers using a local large language model and annotates citation sources to ensure traceability; and continuously optimizes the retrieval model and knowledge base based on user feedback. This invention operates in a fully private deployment environment, possessing high retrieval accuracy, strong data security, and adaptive evolution capabilities, effectively addressing the shortcomings of traditional keyword retrieval and general RAG solutions in semantic understanding, complex document processing, and continuous optimization. Attached Figure Description

[0046] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0047] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0048] Figure 1 A schematic diagram of the logical structure of an AI local knowledge base construction system based on hybrid retrieval and closed-loop optimization provided in an embodiment of the present invention;

[0049] Figure 2 This is a flowchart illustrating an AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization, provided in an embodiment of the present invention. Detailed Implementation

[0050] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To address the issues mentioned above, such as the inability of traditional keyword retrieval to understand semantics, leading to inaccurate and inefficient retrieval; the data leakage and privacy security risks associated with public cloud big data model solutions; and the technical problems of insufficient retrieval accuracy, poor performance in handling complex documents, and lack of continuous optimization capabilities when deploying general-purpose RAG solutions in private environments.

[0052] refer to Figure 1 This invention discloses an AI local knowledge base construction system based on hybrid retrieval and closed-loop optimization. This invention operates in a completely private deployment environment. The system includes: a data preprocessing module 1, a knowledge vectorization module 2, a hybrid retrieval module 3, an intelligent generation module 4, and a closed-loop optimization module 5.

[0053] The system proposed in this embodiment of the invention is deployed on the user's private server or private cloud, ensuring that the entire data processing process does not leave the enterprise intranet and achieving complete control over the data.

[0054] Corresponding to the AI ​​local knowledge base construction system based on hybrid retrieval and closed-loop optimization disclosed above, this invention also discloses an AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization. The following details the AI ​​local knowledge base construction method based on hybrid retrieval and closed-loop optimization disclosed in this invention, in conjunction with the AI ​​local knowledge base construction system based on hybrid retrieval and closed-loop optimization described above.

[0055] refer to Figure 2 This invention discloses an AI-based local knowledge base construction method based on hybrid retrieval and closed-loop optimization, comprising: parsing, cleaning, and semantically coherently segmenting the input original document to form knowledge fragments, and extracting key entities and metadata; using a corpus-fine-tuned sentence converter model to convert the knowledge fragments into vectors and store them in a private vector database to establish an efficient index; receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge fragments, reordering the recalled knowledge fragments, and selecting the most relevant Top-N knowledge fragments; constructing prompt words based on the selected knowledge fragments, inputting them into a locally deployed large language model to generate answers, and annotating the source of references for key information in the answers; collecting user feedback on the answers, using the feedback data to optimize the reordering model and / or embedding model, and correcting and updating the knowledge base.

[0056] S1. Intelligent preprocessing of multidimensional data:

[0057] This step aims to clean, parse, and segment raw documents from diverse sources and in various formats to generate high-quality knowledge fragments that are suitable for machine understanding.

[0058] 1. Data Access and Format Parsing: The system supports batch access of documents in various formats, including but not limited to PDF, Word, Excel, PPT, Markdown, plain text, and images. By calling the corresponding parsing libraries, it extracts text, tables, and image content from the documents.

[0059] The documents include commonly used text, tables, and images. Images need to be labeled with their associated text information.

[0060] 2. Deep Content Cleaning: Multi-dimensional semantic denoising and normalization algorithms are used to clean the extracted content. This algorithm differs from simple deduplication and special character removal; it includes:

[0061] Page layout analysis and noise removal: Automatically identify and remove non-knowledge-related content such as headers, footers, page numbers, advertisements, and watermarks from documents.

[0062] Format standardization: redundant line breaks and spaces in the text are integrated, the table data is extracted in a structured manner, and OCR technology is used to recognize the text in the image; the original table data is mainly data, and the extracted data is in JSON format.

[0063] Semantic Chunking: Abandoning the coarse approach of traditional fixed-length chunking, this method intelligently chunks text based on its semantic integrity (such as paragraphs, heading levels, and punctuation marks), ensuring that each knowledge segment (chunk) is semantically complete and cohesive.

[0064] In the process of intelligent segmentation, the segment size is determined based on semantic analysis.

[0065] Entity and Metadata Extraction: While segmenting, Natural Language Processing (NLP) technology is used to automatically extract key entities (person names, place names, organization names, technical terms) and metadata (such as document source, creation date, chapter title) for each knowledge fragment, providing support for subsequent mixed retrieval.

[0066] S2, Domain-Adaptive Knowledge Vectorization:

[0067] This step transforms the preprocessed knowledge fragments into high-dimensional vector representations and stores them in a vector database.

[0068] 1. Embedding Model Selection and Fine-tuning: We selected an industry-leading sentence converter model (such as Sentence-BERT) and fine-tuned it using our internal professional corpus. Through fine-tuning, the model can more accurately understand the terminology and semantic nuances of specific domains, thereby generating more discriminative vector representations.

[0069] 2. Vector Generation and Storage: For each knowledge fragment, a high-dimensional vector is generated using a fine-tuned embedding model. Then, the original text of the knowledge fragment, its corresponding vector, and the metadata extracted in the first step are recorded as a whole and stored in a privately deployed vector database (such as Milvus or ChromaDB).

[0070] 3. Build efficient indexes: In vector databases, build efficient approximate nearest neighbor (ANN) indexes for stored vectors, such as HNSW (Hierarchical Navigable Small World), to ensure that millisecond-level fast similarity retrieval can be achieved in massive amounts of data.

[0071] S3. Hybrid retrieval combining multi-path recall and re-ranking:

[0072] This step is one of the core innovations of this invention, aiming to overcome the limitations of a single retrieval method and significantly improve the comprehensiveness and accuracy of knowledge retrieval. When a user enters a query, the system initiates multi-path retrieval in parallel:

[0073] 1. Semantic Vector Recall: The user's query is also converted into a query vector through an embedding model. A similarity search is performed in the vector database to recall the Top-K knowledge fragments that are most semantically relevant to the query. This is the foundation of RAG.

[0074] 2. Keyword Recall: Simultaneously, the system extracts core keywords from the user's query and uses a traditional sparse vector model (such as BM25) to perform a full-text search in the knowledge base, recalling knowledge fragments containing these keywords. This effectively compensates for the shortcomings of vector models in accurately matching specific terms or low-frequency words.

[0075] 3. Knowledge Graph Association Recall (Optional Enhancement): For highly structured data, the system can construct a knowledge graph. During retrieval, by analyzing the entities in the query, related entities and relationships are recalled from the knowledge graph as supplementary knowledge sources.

[0076] The recall process includes:

[0077] Entity recognition and linking: Mapping entities in the input text to corresponding nodes in the knowledge graph.

[0078] Path query: Starting from the initial entity, query related entities and relationship paths through the graph database.

[0079] Semantic ranking: Rank entities based on the strength of their association (such as frequency of occurrence and attribute matching).

[0080] For example, when a user searches for "apple", the system will retrieve entities related to "apple" from the knowledge graph, including information such as fruit attributes and technology company background, and sort them using a semantic model.

[0081] 4. Intelligent Re-ranking: The set of knowledge fragments retrieved through the multi-path recall is fed into a lightweight re-ranking model. This model (such as Cross-Encoder) comprehensively evaluates multiple dimensions such as the precise relevance of each knowledge fragment to the original query, semantic similarity, keyword matching, and metadata freshness. It scores and re-ranks all candidate fragments, and finally selects the top-N most relevant and authoritative knowledge fragments as the context provided to the larger model.

[0082] S4. Traceable intelligent question-answering generation:

[0083] 1. Dynamic Prompt Engineering: The system dynamically constructs a rich and well-structured prompt from the user's original query and the high-quality knowledge fragments selected in the third step, according to a preset template.

[0084] 2. Private Large Model Generation: The constructed prompts are input into a locally deployed Large Language Model (LLM). The LLM is strictly instructed to answer questions only based on the provided context (knowledge fragments), thus effectively preventing the model from generating "illusions" or fabricating facts;

[0085] During the instruction process, the Agent filters out different setting context priorities, and when outputting the results, it filters out answers that do not conform to the rules.

[0086] 3. Answer Generation and Source Tracing: After understanding the context, LLM generates a fluent, accurate, and logical natural language answer. Crucially, the system clearly labels the original knowledge fragments referenced by each key information point, along with links to their source documents or page numbers. This ensures that all answers are verifiable and traceable, greatly enhancing the system's credibility.

[0087] S5, Closed-loop optimization driven by user feedback:

[0088] This step constitutes another core innovation of the present invention, enabling the knowledge base system to have the ability to learn and iterate on its own.

[0089] 1. User Feedback Collection: The system provides a simple feedback interface below each answer (such as "like", "dislike", "irrelevant answer", etc.). Users can also submit suggestions for correction.

[0090] 2. Data Labeling and Analysis: The system automatically collects this feedback data in the background. Negative feedback (such as "dislike") and its corresponding query and recall context will be considered high-quality negative samples; positive feedback will be considered positive samples.

[0091] 3. Model Iteration and Fine-tuning: After accumulating a certain amount of feedback data, the system will automatically or semi-automatically use this data to optimize the key models in the system.

[0092] Optimize the re-ranking model: Using this labeled data, retrain the re-ranking model to make it more accurate in judging the relevance between knowledge fragments and queries;

[0093] During the re-ranking process, negative samples will be filtered according to rules and will not participate in the subsequent ranking. Correction suggestions will first be evaluated and scored based on semantic sub-items. Low scores will not be adopted, while high scores will be manually reviewed to determine whether they can be included as valid knowledge in the knowledge base.

[0094] Fine-tuning the embedding model: For queries in a specific domain, if the vector recall effect is found to be poor, the embedding model can be further fine-tuned using this feedback data to improve its domain adaptability.

[0095] The criteria for judging poor vector recall performance are:

[0096] 1) Confirm whether the currently used pre-trained model is suitable for the current data;

[0097] 2) Compare and analyze the results of vector recall and similarity recall to identify whether there are any omissions due to inappropriate labeling strategies;

[0098] 3) Conduct phased testing, adjusting model parameters and validating the recall strategy while ensuring data quality.

[0099] Improve the knowledge base: Analyze incorrect answers, trace back to problems in the data preprocessing or knowledge segmentation stages, and prompt the administrator to correct or supplement the knowledge sources.

[0100] After discovering an incorrect answer, the source of the incorrect answer is identified. The location of the data source is located by retrieval, and the results of the source data after preprocessing or after the source data is divided into blocks are queried to see if there are any defects. Then, the defects are repaired.

[0101] The embodiments of the present invention have the following advantages:

[0102] 1) Extremely high retrieval accuracy: By combining the advantages of semantics, keywords and knowledge graphs through a multi-path recall and re-ranking hybrid retrieval strategy, the recall rate and accuracy of complex queries are significantly improved.

[0103] 2) Excellent search efficiency: Users can ask questions directly in natural language and get accurate answers without having to manually sift through a large number of documents, which significantly shortens the information acquisition time.

[0104] 3) Absolute data security and privacy protection: The data, models and applications in this embodiment of the invention are all deployed privately, and the data does not leave the internal network, completely eliminating the risk of data leakage from public cloud solutions and protecting the sovereignty and security of the enterprise's core digital assets.

[0105] 4) Strong adaptive and evolutionary capabilities: Through a closed-loop optimization mechanism based on user feedback, the knowledge base can continuously learn and evolve during use, continuously improve the quality of question and answer, and maintain the activity and long-term value of the knowledge base.

[0106] 5) High credibility and traceability: All generated answers provide a clear index of the original source, allowing users to trace the source for verification with one click. This greatly avoids the illusion problem of large models and enhances users' trust in the AI ​​system.

[0107] 6) Wide applicability and flexibility: The embodiments of the present invention support the processing of a variety of complex document formats. The modular architecture design makes it easy to extend and integrate, and can flexibly adapt to the specific needs of different industries and enterprises.

[0108] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. An AI local knowledge base construction method based on hybrid retrieval and closed loop optimization, characterized in that, The method comprises: parsing, cleaning and semantic coherence blocking of input raw documents to form knowledge pieces and extract key entities and metadata; using a corpus fine-tuned sentence converter model to convert the knowledge pieces into vectors and store them in a privatized vector database to establish efficient indexing; receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge pieces, reordering the recalled knowledge pieces, and screening the most relevant Top-N knowledge pieces; constructing prompt words based on the screened knowledge pieces, inputting a locally deployed large language model to generate answers, and labeling the reference sources of key information in the answers; collecting user feedback on the answers, using feedback data to optimize the reordering model and / or embedding model, and correcting and updating the knowledge base.

2. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Parsing, cleaning and semantic coherence blocking of input raw documents to form knowledge pieces and extract key entities and metadata, including: The semantic coherence blocking is intelligent blocking according to the paragraph structure, title level and punctuation of the text, to ensure that each knowledge piece is complete and cohesive in semantics.

3. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Using a corpus fine-tuned sentence converter model to convert the knowledge pieces into vectors and store them in a privatized vector database to establish efficient indexing, including: The sentence converter model is a Sentence-BERT model fine-tuned with professional corpus.

4. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge pieces, reordering the recalled knowledge pieces, and screening the most relevant Top-N knowledge pieces, including: The multi-path recall includes: Semantic vector recall: vectorize the query and retrieve similar knowledge pieces in the vector database; keyword recall: use a sparse retrieval model to perform full-text retrieval on keywords in the query.

5. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 4, characterized in that, Receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge pieces, reordering the recalled knowledge pieces, and screening the most relevant Top-N knowledge pieces, also including: The multi-path recall also includes knowledge graph association recall: based on the entities in the query, recall associated entities and relationship knowledge from the constructed knowledge graph.

6. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge pieces, reordering the recalled knowledge pieces, and screening the most relevant Top-N knowledge pieces, also including: The reordering uses a lightweight reordering model to comprehensively evaluate the semantic similarity between the knowledge pieces and the query, the keyword matching degree and the metadata freshness, and reorders them.

7. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Based on the screened knowledge pieces, constructing prompt words, inputting a locally deployed large language model to generate answers, and labeling the reference sources of key information in the answers, including: The labeling of reference sources includes the original text location, document name or page number information of the corresponding knowledge piece.

8. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, Collecting user feedback on the answers, using feedback data to optimize the reordering model and / or embedding model, and correcting and updating the knowledge base, including: According to the feedback data, analyze the problems existing in knowledge preprocessing or blocking, and prompt the administrator to correct or supplement the knowledge source.

9. The AI local knowledge base construction method based on hybrid retrieval and closed-loop optimization according to claim 1, wherein, The method runs in a private deployment environment, and all data processing and model inference are completed in an internal network.

10. An AI local knowledge base construction system based on hybrid retrieval and closed-loop optimization, characterized in that, The system comprises: a data preprocessing module for parsing, cleaning and semantic coherence blocking of input raw documents, forming knowledge segments, and extracting key entities and metadata; a knowledge vectorization module for converting the knowledge segments into vectors using a corpus fine-tuned sentence converter model and storing them in a private vector database with efficient indexing; a hybrid retrieval module for receiving user queries, performing multi-path retrieval in parallel, recalling relevant knowledge segments, reordering the recalled knowledge segments, and selecting the top-N most relevant knowledge segments; an intelligent generation module for constructing prompt words based on the selected knowledge segments, inputting a locally deployed large language model to generate answers, and labeling the source of key information in the answers; a closed-loop optimization module for collecting user feedback on the answers, optimizing the reordering model and / or embedding model using the feedback data, and correcting and updating the knowledge base.

Images