Enterprise intelligent knowledge base management system and method based on vector model

Abstract

This invention discloses an enterprise intelligent knowledge base management system and method based on a vector model, relating to the fields of intelligent information retrieval and natural language processing. This invention parses unstructured documents in various enterprise formats, extracting semantic structure information such as text heading levels and paragraph boundaries, and segmenting them into a set of text fragments. The text fragments are then cleaned to obtain effective text fragments. These effective text fragments are integrated into semantic text blocks with the same topic. Sentence vectors are generated using a pre-trained semantic encoding model. Semantic segmentation points are located based on the semantic similarity of adjacent sentences to obtain standardized knowledge units. A dual-database collaborative query is constructed. After receiving user queries, structured pre-filtering is performed through a relational database, followed by semantic retrieval through a vector database. The intersection of the results from both databases is taken, and a comprehensive score is calculated based on multi-dimensional weights. After filtering, aggregation, and tracing, the results are output. If no matching results are found, a tiered fallback process is triggered, completing the entire retrieval loop.

Description

Technical Field

[0001] This invention relates to the field of intelligent information retrieval and natural language processing technology, specifically to an enterprise intelligent knowledge base management system and method based on a vector model. Background Technology

[0002] Enterprises accumulate massive amounts of unstructured documents during their long-term operations, such as office documents, technical reports, and rules and regulations. These documents contain rich knowledge assets. However, traditional enterprise knowledge retrieval methods mainly rely on keyword matching or simple full-text search, which cannot accurately understand the semantic information of user queries, resulting in poor relevance and low recall of search results.

[0003] Meanwhile, existing retrieval systems often neglect the structured information of documents, such as chapter levels and title attribution, resulting in fragmented search results that make it difficult for users to quickly locate information; enterprise knowledge management also often involves strict access control, departmental affiliation and other multi-dimensional attributes.

[0004] Therefore, there is an urgent need for an intelligent knowledge retrieval method that can integrate structured metadata filtering and semantic similarity retrieval while preserving the integrity of document structure, so as to achieve an efficient, accurate and traceable enterprise knowledge retrieval method. Summary of the Invention

[0005] The purpose of this invention is to provide an enterprise intelligent knowledge base management system and method based on a vector model to solve the problems raised in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for managing an enterprise intelligent knowledge base based on a vector model, the method comprising the following steps: Step S100: Parse the unstructured documents of the enterprise and extract structured semantic information, divide them into a set of text fragments, and then remove the garbled fragments in the text fragments; integrate the text fragments to generate sentence vectors, calculate the cosine similarity and divide the sentence vectors into the smallest knowledge fragments to form knowledge units; Step S101: Using Apache Tika as a unified scheduling framework, unstructured documents of the enterprise are parsed. The Apache POI office document parsing component and the PyMuPDF document parsing component are integrated to parse the enterprise's unstructured documents. The parsing process is as follows: Apache Tika identifies the unstructured document format, character encoding, and encryption status, intercepts encrypted documents without access permissions, and decrypts authorized documents before routing them to the corresponding parsing component. For office documents within the unstructured documents, Apache POI extracts the main text, headings, paragraphs, and list text content, removing non-text content. For PDF documents within the unstructured documents, PyMuPDF extracts the text. For plain text within the unstructured documents, the content is directly read. The office documents include .doc, .docx, .xlsx, and .txt format files. The document retains the heading hierarchy, paragraph boundaries, and chapter affiliation of the unstructured document, and outputs textual content that retains the complete semantic structure, dividing it into a set of text fragments based on paragraph boundaries. Step S102: Clean the obtained textual content and remove invalid and redundant data: remove garbled segments with a Unicode replacement character ratio exceeding α%, where α is set by professionals; quantify the text information density by calculating information entropy, and remove redundant data with low information content. For a single text segment X to be judged, calculate the information entropy H(X): ; In the formula, N is the total number of unique words in the current text segment X, and p(x) i ) is the word x i The probability of a word appearing in a segment is given by word x. i The frequency of occurrence in the text fragment X divided by the total number of words in the text fragment X, x i This represents the i-th word in the text segment X; When H(X) is lower than the preset low information entropy threshold δ, the current text segment X is determined to be redundant data without effective information and is directly removed; finally, the cleaned text segment set is output. Step S103: Based on the title hierarchy and chapter affiliation of the unstructured document, integrate the cleaned text fragments according to their respective chapters and titles: Continuous text fragments under the same title and chapter are integrated into independent semantic text blocks with the same theme, ensuring a unified thematic context within each text block and providing complete contextual support for subsequent semantic boundary recognition. For each semantic text block with the same theme, sentences are segmented according to punctuation marks to obtain a continuous sequence of sentences. This sequence of sentences is then input into a pre-trained Sentence-BERT semantic encoding model to generate a fixed-dimensional sentence vector for each sentence. For the adjacent i-th sentence vector V... i With the (i+1)th sentence vector V i+1 Calculate the cosine similarity between the two, cos_sim(V). i V i+1 ): ; In the formula, For vector dot product operation, It is an L2 norm operation of vectors; when the calculated similarity is lower than the preset semantic boundary threshold β, the punctuation interval of the current adjacent clause is determined as the semantic segmentation point; Based on the semantic segmentation points that have been located, the semantic text blocks of the same topic are divided into several minimum knowledge fragments. For each segmented minimum knowledge fragment, corresponding metadata is bound. The metadata includes the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the title level, the page number, the document creator, the creation time, the update time, the department to which it belongs, and the access permission level. This forms a knowledge unit consisting of "minimum knowledge fragment + metadata", providing a standardized input object for subsequent processing.

[0007] Step S200: Create metadata records for each knowledge unit using a relational database, and generate a unique identifier for the knowledge unit as the primary key; convert the smallest knowledge fragment of the knowledge unit into a semantic vector, store the vector using a vector database, and establish a vector index with the unique identifier of the knowledge unit as the associated field; Step S201: Using a relational database, create a metadata record for each knowledge unit. The metadata record includes the unique identifier of the knowledge unit, the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the title level, the page number, the document creator, the creation time, the update time, the department to which it belongs, and the access permission level fields. The unique identifier is automatically generated using a unique identifier (UUID) and serves as the primary key for each knowledge unit in the relational database. To support efficient combined retrieval, a composite index is established on the fields of the unstructured document's sequence number, department, access permission level, and chapter, improving the response speed of multi-condition combined queries. A sorting index is established on the creation time and update time fields to accelerate the efficiency of chronological or paginated queries. Through this relational database, metadata records centered on knowledge units are formed, providing structured filtering and precise query capabilities for subsequent retrieval. Step S202: Using a vector database, for the smallest knowledge fragment content of each knowledge unit, call the pre-trained Sentence-BERT semantic encoding model described in step S103 to convert the smallest knowledge fragment into a fixed-dimensional semantic vector. The dimension of the semantic vector is consistent with the dimension of the sentence vector in step S103. In the vector database, the semantic vector is stored using the unique identifier of the knowledge unit as the primary key association field. A vector index is established based on cosine similarity. The vector index adopts a hierarchical navigable small-world graph (HNSW) index type, which realizes efficient approximate nearest neighbor search through a multi-level graph structure, thereby improving the efficiency of large-scale vector retrieval. Each record in the vector database contains two core fields: the unique identifier of the knowledge unit and the semantic vector, ensuring that subsequent retrievals can quickly recall knowledge units that meet the user's query requirements based on vector similarity.

[0008] Step S300: For user query requests, pre-filtering is performed through a relational database to obtain a set of unique identifiers for knowledge units. The user's query text is converted into a query vector and semantic retrieval is performed through a vector database to recall the set of unique identifiers for knowledge units that match the semantics. The intersection of the two sets is taken to calculate the corresponding comprehensive ranking score. If there is no matching result, a three-gradient adjustment is triggered. Step S301: When a user query request is received, pre-filtering is performed on the metadata records of the relational database based on the access permission level, department, document name, and time range involved in the query to obtain a set I of unique identifiers for knowledge units that meet the conditions. meta The user's natural language query text is converted into a query vector using the pre-trained Sentence-BERT semantic encoding model described in step S103, and then retrieved using vector similarity in the vector database. Specifically: Calculate the cosine similarity cos_sim(Q,V) between the query vector and each semantic vector in the vector database, where Q is the query vector and V is the semantic vector. Sort all the cosine similarity values ​​in the vector database from high to low, and retrieve the unique identifiers of the knowledge units corresponding to the top K cosine similarities in the sorting results, where K is a preset retrieval recall number, and the set of retrieved unique identifiers of knowledge units is denoted as I. vector ; Step S302: Establish a dual-database collaborative query mechanism to achieve the integration of structured filtering and semantic retrieval; combine the unique identifier set I of knowledge units obtained from the pre-filtering... meta With the recall of the knowledge unit unique identifier set I vector Perform a fusion and filtering process to obtain the intersection I of the two sets. final :I final =I meta ∩I vector Regarding the final intersection I finalThe system uses unique identifiers for all knowledge units within a given knowledge unit. It retrieves complete metadata information for each knowledge unit from the metadata records of a relational database, extracts semantic vectors for each knowledge unit from a vector database, and uses the cosine similarity between the semantic vectors and the user query vector to construct a scoring model for each knowledge unit and calculate the comprehensive ranking score S. S=ω1×cos_sim(Q,V)+ω2×T+ω3×P; In the formula, ω1, ω2, and ω3 are the semantic similarity weight, time freshness weight, and permission matching weight, respectively, satisfying ω1+ω2+ω3=1; cos_sim(Q,V) is the cosine similarity between the current knowledge unit semantic vector V and the user query vector Q, normalized to the [0,1] interval; P is the permission matching coefficient, which takes a value of 1 when the access permission level of the knowledge unit matches the user's current access permission level, and a value of 0 when the access permission level is higher than the user's permission level; T is the time freshness coefficient, calculated based on the update time of the document corresponding to the knowledge unit, and the calculation formula is: ; In the formula, Δt is the time difference between the current system time and the document update time, and T0 is the preset time decay period constant, which takes the value of a positive integer; Step S303: If and only if the final intersection I final =I meta ∩I vector When the condition is equal to ∅, step S302 is terminated, triggering a three-gradient adjustment. A small-amplitude conditional adjustment is used to quickly find the intersection with the vector retrieval result, restoring the normal retrieval process. Specifically: First tier: Retain access permission level and department, adjust the time range to the preset time period, and re-pre-filter to obtain set I. meta1 ; Calculate the intersection I final1 =I meta1 ∩I vector , if I final1 =∅, enter the second gradient; Second tier: Retain access permission levels, adjust the department and time range to the entire department and maximum time range, and re-pre-filter to obtain set I. meta2 ; Calculate the intersection I final2 =I meta2 ∩I vector , if I final2 =∅, entering the third gradient; Third gradient: Only retain access permission levels, remove all other pre-filtering conditions, and pre-filter to obtain the full permission set I. meta3 ; Calculate the intersection I final3 =I meta3 ∩I vector , if Ifinal3 =∅, generates a no-result response, prompting the user to adjust the query conditions. The remaining pre-filtering conditions include the department, document name, and time range.

[0009] Step S400: Sort the knowledge units within the intersection in descending order according to the comprehensive ranking score, clean the knowledge units, group and aggregate the retrieval result set, and output the retrieval results that retain the original semantic structure; Step S401: Based on the calculated comprehensive ranking score S, perform the following steps on the final intersection I. final All knowledge units within the scope are sorted in descending order from high to low. Knowledge units with a comprehensive ranking score S lower than the preset effective score threshold β are removed to obtain the final retrieval result set I. Each retained knowledge unit is bound with corresponding metadata information to form traceable and standardized retrieval output content. Step S402: Group and aggregate the knowledge units in the final search result set I according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level. Output the search results that retain the semantic structure of the original document. At the same time, provide the original document name, page number, chapter position and other source information for each knowledge unit. This allows users to directly jump to the corresponding position in the original document to view the complete context content, thus completing the full-process search loop that integrates structured filtering and semantic search.

[0010] An enterprise intelligent knowledge base management system based on a vector model, the system includes a knowledge construction module, a dual-base construction module, a retrieval and fusion module, and a result output module; The knowledge construction module is responsible for the unified parsing and preprocessing of unstructured documents of enterprises. It uses Apache Tika to identify document format and encryption status, routes the data to the corresponding parsing component to extract text content, and retains structured semantic information such as heading levels and paragraph boundaries. The extracted text is cleaned to remove garbled characters and low-information-entropy redundant data, and integrated into semantic text blocks with the same topic based on heading level. Sentence-BERT is used to calculate sentence vector similarity to identify semantic boundaries, and the text blocks are divided into the smallest knowledge fragments. Complete metadata is bound to form standardized knowledge units, providing high-quality input for subsequent storage and retrieval. The knowledge construction module includes a document parsing unit, a text cleaning unit, and a semantic segmentation unit; The document parsing unit adopts the Apache Tika unified scheduling framework, integrating Apache POI and PyMuPDF parsing components. It identifies unstructured document formats, character encodings, and encryption status, blocks access to encrypted documents without authorization, and decrypts authorized documents before routing them to the corresponding components. Office documents are extracted using Apache POI, PDF documents are extracted using PyMuPDF, and plain text is read directly. It preserves semantic information such as heading levels, paragraph boundaries, and chapter affiliation, and divides text into sets of fragments based on paragraph boundaries. The text cleaning unit cleans the text content and removes invalid and redundant data, removing garbled segments with a Unicode replacement character ratio exceeding α%. It quantifies the text information density by calculating information entropy, calculates the information entropy H(X) for a single text segment X to be judged, and judges it as redundant data without valid information when H(X) is lower than the preset low information entropy threshold δ, and outputs a set of cleaned text segments. The semantic segmentation unit integrates the cleaned text fragments into semantic text blocks with the same theme according to the title level and chapter affiliation. It then segments the text into continuous sentence sequences based on punctuation marks, inputs them into Sentence-BERT to generate fixed-dimensional sentence vectors, calculates the cosine similarity between adjacent sentence vectors, and determines the semantic segmentation point when the similarity is lower than the preset semantic boundary threshold β. Based on the segmentation point, the text block is segmented into the smallest knowledge fragment, and metadata is bound to form a knowledge unit of "smallest knowledge fragment + metadata".

[0011] The dual-database construction module establishes a dual storage system for knowledge units, consisting of a relational database and a vector database. The relational database stores complete metadata records of the knowledge units, establishes joint indexes and sorted indexes, and provides structured filtering and precise query capabilities. The vector database stores semantic vectors of the smallest knowledge fragments of the knowledge units, establishes HNSW vector indexes based on cosine similarity, and achieves efficient approximate nearest neighbor retrieval, laying a data foundation for subsequent fusion retrieval. The dual-database construction module includes a metadata storage unit and a vector storage unit; The metadata storage unit uses a relational database to create metadata records for each knowledge unit, including a unique identifier (automatically generated UUID, used as the primary key), the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the heading level, the page number, the creator, the creation time, the update time, the department to which it belongs, and the access permission level. A composite index is created on the document serial number, department to which it belongs, access permission level, and chapter to which it belongs, and a sorted index is created on the creation time and update time fields, forming a metadata record centered on the knowledge unit. The vector storage unit uses a vector database to store the semantic vectors of knowledge units. It calls Sentence-BERT to convert the smallest knowledge fragment of each knowledge unit into a fixed-dimensional semantic vector. The semantic vector is stored using the unique identifier of the knowledge unit as the primary key association field. An HNSW vector index is built based on cosine similarity. Each record contains two core fields: the unique identifier of the knowledge unit and the semantic vector, ensuring the efficiency of subsequent vector similarity retrieval.

[0012] The retrieval fusion module implements a dual-database collaborative query mechanism of relational database and vector database. When receiving a user query request, it performs structured pre-filtering in the relational database to obtain a set of knowledge unit identifiers that meet the conditions, and performs semantic similarity retrieval and recall of the identifier set in the vector database based on the query vector. It takes the intersection of the two sets, constructs a scoring model for the knowledge units in the intersection, and calculates a comprehensive ranking score by combining semantic similarity, time freshness, and permission matching. When the intersection is empty, it triggers a three-gradient hierarchical processing, gradually relaxing the conditions until the result is returned or no result is indicated. The retrieval fusion module includes a dual-database pre-detection unit, a fusion scoring unit, and a fallback processing unit; When the dual-database pre-screening unit receives a user query request, it performs pre-filtering on the metadata records of the relational database based on the access permission level, department, document name, and time range involved in the query, to obtain a set of unique identifiers I for knowledge units that meet the conditions. meta The user's natural language query text is converted into a query vector using Sentence-BERT. This vector vector is then searched and ranked using a cosine similarity index in a vector database, retrieving the top K corresponding unique identifiers for each knowledge unit. vector ; The fusion scoring unit will use the pre-filtered set I meta With vector retrieval recall set I vector Perform fusion filtering and take the intersection I final For each unique identifier of knowledge unit within the intersection, metadata and semantic vectors are retrieved, and a scoring model is constructed to calculate the comprehensive ranking score S. The fallback processing unit is defined if and only if the final intersection I final When =∅, a three-gradient adjustment is triggered, specifically: First tier: Retain access permission level and department affiliation, adjust the time range to the preset time period, and re-pre-filter and calculate the intersection I. final1 If the intersection is still empty, proceed to the second gradient; Second gradient: retain access permission level, adjust the affiliated department, time range to the entire department range and the maximum time range, and calculate the intersection I. final2If the intersection is still empty, proceed to the third gradient; Third gradient: retain only access permission levels, clear other pre-filtering conditions, and calculate intersection I. final3 If the intersection is still empty, a no-result response is generated, prompting the user to adjust the query conditions.

[0013] The result output module is responsible for sorting, filtering, and structured output of the search results. It sorts the knowledge units within the intersection in descending order according to the comprehensive ranking score, removes knowledge units with scores below the effective threshold, and obtains the final search result set. The knowledge units in the results are grouped and aggregated according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level, while preserving the semantic structure of the original document. It provides source information such as document name, page number, and chapter position for each smallest knowledge fragment, allowing users to jump to the original document to view the complete context and complete the search loop.

[0014] The result output module includes a sorting and filtering unit and a result aggregation unit; The sorting and filtering unit sorts the intersection I according to the calculated comprehensive sorting score S. final All knowledge units within the scope are sorted in descending order from high to low. Knowledge units with a comprehensive ranking score S lower than the preset effective score threshold β are removed to obtain the final retrieval result set I. Each retained knowledge unit is bound with corresponding metadata information to form traceable and standardized retrieval output content. The result aggregation unit groups and aggregates the knowledge units in the final search result set I according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level. It outputs search results that retain the semantic structure of the original document. At the same time, it provides source information such as the document name, page number, and chapter position of the original document for each knowledge unit, allowing users to directly jump to the corresponding position in the original document to view the complete context content, thus completing the full-process search closed loop that integrates structured filtering and semantic retrieval.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. Achieve high-quality knowledge transformation of unstructured documents and enhance enterprise knowledge asset governance capabilities: This invention, through a multi-component collaborative document parsing process, redundant data removal based on information entropy, and knowledge segmentation process with precise semantic boundary identification, preserves the hierarchical structure and semantic information of documents, transforming multi-format and fragmented enterprise unstructured documents into standardized knowledge units with complete metadata, thereby realizing the standardized and reusable management of enterprise unstructured knowledge assets.

[0016] 2. Construct a dual-database collaborative retrieval mechanism to balance retrieval accuracy, compliance, and efficiency: This invention achieves deep semantic understanding and accurate retrieval of natural language queries through the deep integration of relational database structured pre-filtering and vector database semantic retrieval; ensures compliant control of enterprise document access through pre-filtering of metadata dimensions such as permissions, department of ownership, and time range; and improves retrieval response speed in large-scale knowledge base scenarios.

[0017] 3. Closed-loop design of the entire retrieval process to improve retrieval robustness and user experience: This invention uses a comprehensive ranking model that integrates semantic similarity, time freshness, and permission matching to accurately rank retrieval results; it is equipped with a three-tier hierarchical fallback mechanism to avoid no results under strict filtering conditions; the retrieval results provide complete source tracing and jump capabilities to ensure the contextual integrity of knowledge content and form a closed loop of the entire retrieval process. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the application of the present invention to an enterprise intelligent knowledge base management method based on a vector model; Figure 2 This is a schematic diagram of the knowledge construction module of an enterprise intelligent knowledge base management system based on a vector model, according to the present invention. Figure 3 This is a schematic diagram of the structure of the dual-database construction module in an enterprise intelligent knowledge base management system based on a vector model, according to the present invention. Figure 4 This is a schematic diagram of the structure of the retrieval and fusion module of an enterprise intelligent knowledge base management system based on a vector model, according to the present invention. Figure 5 This is a schematic diagram of the structure of the result output module in an enterprise intelligent knowledge base management system based on a vector model, according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example: Figures 1-5 As shown, this invention provides a technical solution. Let's assume there is a company G that has accumulated a large number of unstructured documents, including product technical manuals, project acceptance reports, market research documents, internal management systems, etc., scattered across multiple departments. The company has deployed an enterprise intelligent knowledge base management method based on a vector model. The method includes the following steps: Step S100: Parse the unstructured documents of the enterprise and extract structured semantic information, divide them into a set of text fragments, and then remove the garbled fragments in the text fragments; integrate the text fragments to generate sentence vectors, calculate the cosine similarity and divide the sentence vectors into the smallest knowledge fragments to form knowledge units; Step S101: Using Apache Tika as a unified scheduling framework, unstructured documents of the enterprise are parsed. The Apache POI office document parsing component and the PyMuPDF document parsing component are integrated to parse the enterprise's unstructured documents. The parsing process is as follows: Apache Tika identifies the unstructured document format, character encoding, and encryption status, intercepts encrypted documents without access permissions, and decrypts authorized documents before routing them to the corresponding parsing component. For office documents within the unstructured documents, Apache POI extracts the main text, headings, paragraphs, and list text content, removing non-text content. For PDF documents within the unstructured documents, PyMuPDF extracts the text. For plain text within the unstructured documents, the content is directly read. The office documents include .doc, .docx, .xlsx, and .txt format files. The document retains the heading hierarchy, paragraph boundaries, and chapter affiliation of the unstructured document, and outputs textual content that retains the complete semantic structure, dividing it into a set of text fragments based on paragraph boundaries. Step S102: Clean the obtained textual content and remove invalid and redundant data: remove garbled segments with a Unicode replacement character ratio exceeding α%, where α is set by professionals; quantify the text information density by calculating information entropy, and remove redundant data with low information content. For a single text segment X to be judged, calculate the information entropy H(X): ; In the formula, N is the total number of unique words in the current text segment X, and p(x) i ) is the word x i The probability of a word appearing in a segment is given by word x. i The frequency of occurrence in the text fragment X divided by the total number of words in the text fragment X, x i This represents the i-th word in the text segment X; When H(X) is lower than the preset low information entropy threshold δ, the current text segment X is determined to be redundant data without effective information and is directly removed; finally, the cleaned text segment set is output. Step S103: Based on the title hierarchy and chapter affiliation of the unstructured document, integrate the cleaned text fragments according to their respective chapters and titles: Continuous text fragments under the same title and chapter are integrated into independent semantic text blocks with the same theme, ensuring a unified thematic context within each text block and providing complete contextual support for subsequent semantic boundary recognition. For each semantic text block with the same theme, sentences are segmented according to punctuation marks to obtain a continuous sequence of sentences. This sequence of sentences is then input into a pre-trained Sentence-BERT semantic encoding model to generate a fixed-dimensional sentence vector for each sentence. For the adjacent i-th sentence vector V... i With the (i+1)th sentence vector V i+1 Calculate the cosine similarity between the two, cos_sim(V). i V i+1 ): ; In the formula, For vector dot product operation, It is an L2 norm operation of vectors; when the calculated similarity is lower than the preset semantic boundary threshold β, the punctuation interval of the current adjacent clause is determined as the semantic segmentation point; Based on the semantic segmentation points that have been located, the semantic text blocks of the same topic are divided into several minimum knowledge fragments. For each segmented minimum knowledge fragment, corresponding metadata is bound. The metadata includes the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the title level, the page number, the document creator, the creation time, the update time, the department to which it belongs, and the access permission level. This forms a knowledge unit consisting of "minimum knowledge fragment + metadata", providing a standardized input object for subsequent processing.

[0021] Step S200: Create metadata records for each knowledge unit using a relational database, and generate a unique identifier for the knowledge unit as the primary key; convert the smallest knowledge fragment of the knowledge unit into a semantic vector, store the vector using a vector database, and establish a vector index with the unique identifier of the knowledge unit as the associated field; Step S201: Using a relational database, create a metadata record for each knowledge unit. The metadata record includes the unique identifier of the knowledge unit, the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the title level, the page number, the document creator, the creation time, the update time, the department to which it belongs, and the access permission level fields. The unique identifier is automatically generated using a unique identifier (UUID) and serves as the primary key for each knowledge unit in the relational database. To support efficient combined retrieval, a composite index is established on the fields of the unstructured document's sequence number, department, access permission level, and chapter, improving the response speed of multi-condition combined queries. A sorting index is established on the creation time and update time fields to accelerate the efficiency of chronological or paginated queries. Through this relational database, metadata records centered on knowledge units are formed, providing structured filtering and precise query capabilities for subsequent retrieval. Step S202: Using a vector database, for the smallest knowledge fragment content of each knowledge unit, call the pre-trained Sentence-BERT semantic encoding model described in step S103 to convert the smallest knowledge fragment into a fixed-dimensional semantic vector. The dimension of the semantic vector is consistent with the dimension of the sentence vector in step S103. In the vector database, the semantic vector is stored using the unique identifier of the knowledge unit as the primary key association field. A vector index is established based on cosine similarity. The vector index adopts a hierarchical navigable small-world graph (HNSW) index type, which realizes efficient approximate nearest neighbor search through a multi-level graph structure, thereby improving the efficiency of large-scale vector retrieval. Each record in the vector database contains two core fields: the unique identifier of the knowledge unit and the semantic vector, ensuring that subsequent retrievals can quickly recall knowledge units that meet the user's query requirements based on vector similarity.

[0022] Step S300: For user query requests, pre-filtering is performed through a relational database to obtain a set of unique identifiers for knowledge units. The user's query text is converted into a query vector and semantic retrieval is performed through a vector database to recall the set of unique identifiers for knowledge units that match the semantics. The intersection of the two sets is taken to calculate the corresponding comprehensive ranking score. If there is no matching result, a three-gradient adjustment is triggered. Step S301: When a user query request is received, pre-filtering is performed on the metadata records of the relational database based on the access permission level, department, document name, and time range involved in the query to obtain a set I of unique identifiers for knowledge units that meet the conditions. meta The user's natural language query text is converted into a query vector using the pre-trained Sentence-BERT semantic encoding model described in step S103, and then retrieved using vector similarity in the vector database. Specifically: Calculate the cosine similarity cos_sim(Q,V) between the query vector and each semantic vector in the vector database, where Q is the query vector and V is the semantic vector. Sort all the cosine similarity values ​​in the vector database from high to low, and retrieve the unique identifiers of the knowledge units corresponding to the top K cosine similarities in the sorting results, where K is a preset retrieval recall number, and the set of retrieved unique identifiers of knowledge units is denoted as I. vector ; Step S302: Establish a dual-database collaborative query mechanism to achieve the integration of structured filtering and semantic retrieval; combine the unique identifier set I of knowledge units obtained from the pre-filtering... meta With the recall of the knowledge unit unique identifier set I vector Perform a fusion and filtering process to obtain the intersection I of the two sets. final :I final =I meta ∩I vector Regarding the final intersection I final The system uses unique identifiers for all knowledge units within a given knowledge unit. It retrieves complete metadata information for each knowledge unit from the metadata records of a relational database, extracts semantic vectors for each knowledge unit from a vector database, and uses the cosine similarity between the semantic vectors and the user query vector to construct a scoring model for each knowledge unit and calculate the comprehensive ranking score S. S=ω1×cos_sim(Q,V)+ω2×T+ω3×P; In the formula, ω1, ω2, and ω3 are the semantic similarity weight, time freshness weight, and permission matching weight, respectively, satisfying ω1+ω2+ω3=1; cos_sim(Q,V) is the cosine similarity between the current knowledge unit semantic vector V and the user query vector Q, normalized to the [0,1] interval; P is the permission matching coefficient, which takes a value of 1 when the access permission level of the knowledge unit matches the user's current access permission level, and a value of 0 when the access permission level is higher than the user's permission level; T is the time freshness coefficient, calculated based on the update time of the document corresponding to the knowledge unit, and the calculation formula is: ; In the formula, Δt is the time difference between the current system time and the document update time, and T0 is the preset time decay period constant, which takes the value of a positive integer; Step S303: If and only if the final intersection I final =I meta ∩I vector When the condition is equal to ∅, step S302 is terminated, triggering a three-gradient adjustment. A small-amplitude conditional adjustment is used to quickly find the intersection with the vector retrieval result, restoring the normal retrieval process. Specifically: First tier: Retain access permission level and department, adjust the time range to the preset time period, and re-pre-filter to obtain set I. meta1 ; Calculate the intersection I final1 =I meta1 ∩I vector , if I final1 =∅, enter the second gradient; Second tier: Retain access permission levels, adjust the department and time range to the entire department and maximum time range, and re-pre-filter to obtain set I. meta2 ; Calculate the intersection I final2 =I meta2 ∩I vector , if I final2 =∅, entering the third gradient; Third gradient: Only retain access permission levels, remove all other pre-filtering conditions, and pre-filter to obtain the full permission set I. meta3 ; Calculate the intersection I final3 =I meta3 ∩I vector , if I final3 =∅, generates a no-result response, prompting the user to adjust the query conditions. The remaining pre-filtering conditions include the department, document name, and time range.

[0023] Step S400: Sort the knowledge units within the intersection in descending order according to the comprehensive ranking score, clean the knowledge units, group and aggregate the retrieval result set, and output the retrieval results that retain the original semantic structure; Step S401: Based on the calculated comprehensive ranking score S, perform the following steps on the final intersection I. final All knowledge units within the scope are sorted in descending order from high to low. Knowledge units with a comprehensive ranking score S lower than the preset effective score threshold β are removed to obtain the final retrieval result set I. Each retained knowledge unit is bound with corresponding metadata information to form traceable and standardized retrieval output content. Step S402: Group and aggregate the knowledge units in the final search result set I according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level. Output the search results that retain the semantic structure of the original document. At the same time, provide the original document name, page number, chapter position and other source information for each knowledge unit. This allows users to directly jump to the corresponding position in the original document to view the complete context content, thus completing the full-process search loop that integrates structured filtering and semantic search.

[0024] An enterprise intelligent knowledge base management system based on a vector model, the system includes a knowledge construction module, a dual-base construction module, a retrieval and fusion module, and a result output module; The knowledge construction module is responsible for the unified parsing and preprocessing of unstructured documents of enterprises. It uses Apache Tika to identify document format and encryption status, routes the data to the corresponding parsing component to extract text content, and retains structured semantic information such as heading levels and paragraph boundaries. The extracted text is cleaned to remove garbled characters and low-information-entropy redundant data, and integrated into semantic text blocks with the same topic based on heading level. Sentence-BERT is used to calculate sentence vector similarity to identify semantic boundaries, and the text blocks are divided into the smallest knowledge fragments. Complete metadata is bound to form standardized knowledge units, providing high-quality input for subsequent storage and retrieval. The knowledge construction module includes a document parsing unit, a text cleaning unit, and a semantic segmentation unit; The document parsing unit adopts the Apache Tika unified scheduling framework, integrating Apache POI and PyMuPDF parsing components. It identifies unstructured document formats, character encodings, and encryption status, blocks access to encrypted documents without authorization, and decrypts authorized documents before routing them to the corresponding components. Office documents are extracted using Apache POI, PDF documents are extracted using PyMuPDF, and plain text is read directly. It preserves semantic information such as heading levels, paragraph boundaries, and chapter affiliation, and divides text into sets of fragments based on paragraph boundaries. The text cleaning unit cleans the text content and removes invalid and redundant data, removing garbled segments with a Unicode replacement character ratio exceeding α%. It quantifies the text information density by calculating information entropy, calculates the information entropy H(X) for a single text segment X to be judged, and judges it as redundant data without valid information when H(X) is lower than the preset low information entropy threshold δ, and outputs a set of cleaned text segments. The semantic segmentation unit integrates the cleaned text fragments into semantic text blocks with the same theme according to the title level and chapter affiliation. It then segments the text into continuous sentence sequences based on punctuation marks, inputs them into Sentence-BERT to generate fixed-dimensional sentence vectors, calculates the cosine similarity between adjacent sentence vectors, and determines the semantic segmentation point when the similarity is lower than the preset semantic boundary threshold β. Based on the segmentation point, the text block is segmented into the smallest knowledge fragment, and metadata is bound to form a knowledge unit of "smallest knowledge fragment + metadata".

[0025] The dual-database construction module establishes a dual storage system for knowledge units, consisting of a relational database and a vector database. The relational database stores complete metadata records of the knowledge units, establishes joint indexes and sorted indexes, and provides structured filtering and precise query capabilities. The vector database stores semantic vectors of the smallest knowledge fragments of the knowledge units, establishes HNSW vector indexes based on cosine similarity, and achieves efficient approximate nearest neighbor retrieval, laying a data foundation for subsequent fusion retrieval. The dual-database construction module includes a metadata storage unit and a vector storage unit; The metadata storage unit uses a relational database to create metadata records for each knowledge unit, including a unique identifier (automatically generated UUID, used as the primary key), the serial number of the unstructured document to which it belongs, the document name, the chapter to which it belongs, the heading level, the page number, the creator, the creation time, the update time, the department to which it belongs, and the access permission level. A composite index is created on the document serial number, department to which it belongs, access permission level, and chapter to which it belongs, and a sorted index is created on the creation time and update time fields, forming a metadata record centered on the knowledge unit. The vector storage unit uses a vector database to store the semantic vectors of knowledge units. It calls Sentence-BERT to convert the smallest knowledge fragment of each knowledge unit into a fixed-dimensional semantic vector. The semantic vector is stored using the unique identifier of the knowledge unit as the primary key association field. An HNSW vector index is built based on cosine similarity. Each record contains two core fields: the unique identifier of the knowledge unit and the semantic vector, ensuring the efficiency of subsequent vector similarity retrieval.

[0026] The retrieval fusion module implements a dual-database collaborative query mechanism of relational database and vector database. When receiving a user query request, it performs structured pre-filtering in the relational database to obtain a set of knowledge unit identifiers that meet the conditions, and performs semantic similarity retrieval and recall of the identifier set in the vector database based on the query vector. It takes the intersection of the two sets, constructs a scoring model for the knowledge units in the intersection, and calculates a comprehensive ranking score by combining semantic similarity, time freshness, and permission matching. When the intersection is empty, it triggers a three-gradient hierarchical processing, gradually relaxing the conditions until the result is returned or no result is indicated. The retrieval fusion module includes a dual-database pre-detection unit, a fusion scoring unit, and a fallback processing unit; When the dual-database pre-screening unit receives a user query request, it performs pre-filtering on the metadata records of the relational database based on the access permission level, department, document name, and time range involved in the query, to obtain a set of unique identifiers I for knowledge units that meet the conditions. meta The user's natural language query text is converted into a query vector using Sentence-BERT. This vector vector is then searched and ranked using a cosine similarity index in a vector database, retrieving the top K corresponding unique identifiers for each knowledge unit. vector ; The fusion scoring unit will use the pre-filtered set I meta With vector retrieval recall set I vector Perform fusion filtering and take the intersection I final For each unique identifier of knowledge unit within the intersection, metadata and semantic vectors are retrieved, and a scoring model is constructed to calculate the comprehensive ranking score S. The fallback processing unit is defined if and only if the final intersection I final When =∅, a three-gradient adjustment is triggered, specifically: First tier: Retain access permission level and department affiliation, adjust the time range to the preset time period, and re-pre-filter and calculate the intersection I. final1 If the intersection is still empty, proceed to the second gradient; Second gradient: retain access permission level, adjust the affiliated department, time range to the entire department range and the maximum time range, and calculate the intersection I. final2 If the intersection is still empty, proceed to the third gradient; Third gradient: retain only access permission levels, clear other pre-filtering conditions, and calculate intersection I. final3 If the intersection is still empty, a no-result response is generated, prompting the user to adjust the query conditions.

[0027] The result output module is responsible for sorting, filtering, and structured output of the search results. It sorts the knowledge units within the intersection in descending order according to the comprehensive ranking score, removes knowledge units with scores below the effective threshold, and obtains the final search result set. The knowledge units in the results are grouped and aggregated according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level, while preserving the semantic structure of the original document. It provides source information such as document name, page number, and chapter position for each smallest knowledge fragment, allowing users to jump to the original document to view the complete context and complete the search loop.

[0028] The result output module includes a sorting and filtering unit and a result aggregation unit; The sorting and filtering unit sorts the intersection I according to the calculated comprehensive sorting score S. final All knowledge units within the scope are sorted in descending order from high to low. Knowledge units with a comprehensive ranking score S lower than the preset effective score threshold β are removed to obtain the final retrieval result set I. Each retained knowledge unit is bound with corresponding metadata information to form traceable and standardized retrieval output content. The result aggregation unit groups and aggregates the knowledge units in the final search result set I according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level. It outputs search results that retain the semantic structure of the original document. At the same time, it provides source information such as the document name, page number, and chapter position of the original document for each knowledge unit, allowing users to directly jump to the corresponding position in the original document to view the complete context content, thus completing the full-process search closed loop that integrates structured filtering and semantic retrieval.

[0029] Example: Using ApacheTika as the core scheduling framework, a full scan of 1268 documents is performed, automatically identifying document format, character encoding, and encryption status; 12 encrypted documents without access rights are intercepted, and the remaining 1256 authorized documents are decrypted and routed to the corresponding parsing components according to their format. For.doc,.docx, and.xlsx office documents, extract the body text, headings, paragraphs, and list text content through Apache POI, and剔除 non-text content such as images, comments, and macro codes; for.pdf format documents, extract the full text content through PyMuPDF; for.txt plain text files, directly read the complete content; Semantic structure preservation and segmentation: Completely preserve the heading levels, paragraph boundaries, and chapter affiliations of all documents, and output text-based content with the complete semantic structure retained; divide all text-based content into a set of text fragments with paragraph boundaries as the delimiters, and set that a total of 20,156 initial text fragments are obtained in this processing; Traverse all text fragments, calculate the proportion of Unicode replacement characters, and剔除 text fragments with a proportion exceeding 15%. For the remaining 20,138 text fragments, calculate the information entropy of each one. The information entropy of a single text fragment X, example: the content of a meaningless text fragment is "好的好的收到收到", the total number of words is 8, the number of non-repeated words is 2, namely好的 and收到, calculated to get H(X)=1.0, which is lower than the threshold δ=2.8, and is判定 as redundant data and directly剔除; after cleaning, finally output the set of valid text fragments, and set that it contains a total of 18,642 high-quality text fragments; Based on the document heading levels and chapter affiliations as the core basis, integrate the text fragments after cleaning according to the chapters and headings they belong to. For each semantic text block with the same theme, complete sentence splitting according to punctuation marks such as full stops, semicolons, and question marks, and obtain a continuous sequence of split sentences. The sequence of split sentences is input into the pre-trained Sentence-BERT model to generate 384-dimensional fixed-dimensional sentence vectors for each split sentence, and calculate the cosine similarity. When the similarity calculation result is lower than the semantic boundary threshold β=0.65, it is判定 that the punctuation interval between the adjacent split sentences is the semantic segmentation point; finally, form a standardized knowledge unit composed of "minimum knowledge fragment + metadata"; Generate a UUID as the unique identifier for each knowledge unit, and create metadata records in the relational database. The system creates a combined index on the fields of "document serial number identifier", "attribution department", "access permission level", and "chapter affiliation", and creates a sorting index on "creation time" and "update time"; When receiving a user query request, start the dual-database collaborative query mechanism. Example: The user is an engineer in the technology R & D department, querying "What type of welding wire is used for the crane main girder", and限定 the attribution department to the technology R & D department and the time range to the past 2 years. The system first performs pre-filtering in the relational database to筛选 out the set of knowledge unit UUIDs that meet the conditions such as department and time, and set that 1,286 UUIDs are obtained this time; at the same time, convert the user's natural language query into a 384-dimensional query vector, calculate the cosine similarity with all semantic vectors in the vector database, and召回 the top 200 UUIDs with the highest similarity; Take the intersection of two UUID sets, and set the number of valid UUIDs obtained this time to 32. For the knowledge units corresponding to these UUIDs, calculate the comprehensive ranking score, which consists of semantic similarity (weight 0.7), time freshness (weight 0.2), and permission matching degree (weight 0.1). For example, the knowledge unit ranked first has a semantic similarity of 0.92, a document update time of half a year ago (time freshness coefficient 0.8), and permission matching, so its comprehensive score is 0.7×0.92+0.2×0.8+0.1×1=0.884. Sort the results by comprehensive score from highest to lowest, remove two knowledge units with scores below 0.4, and finally obtain 30 valid results. These results are then aggregated and displayed according to the chapter and title level of the document to which they belong. At the same time, each smallest knowledge fragment is labeled with the original document name, page number, chapter and other source information. Users can click to jump to the corresponding position in the original document to view the complete content and complete the entire search process.

[0030] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for managing an enterprise intelligent knowledge base based on a vector model, characterized in that: The method includes the following steps: Step S100: Parse the unstructured documents of the enterprise and extract structured semantic information, divide them into a set of text fragments, and then remove the garbled fragments in the text fragments; integrate the text fragments to generate sentence vectors, calculate the cosine similarity and divide the sentence vectors into the smallest knowledge fragments to form knowledge units; Step S200: Create metadata records for each knowledge unit using a relational database, and generate a unique identifier for the knowledge unit as the primary key; convert the smallest knowledge fragment of the knowledge unit into a semantic vector, store the vector using a vector database, and establish a vector index with the unique identifier of the knowledge unit as the associated field; Step S300: For user query requests, pre-filtering is performed through a relational database to obtain a set of unique identifiers for knowledge units. The user's query text is converted into a query vector and semantic retrieval is performed through a vector database to recall the set of unique identifiers for knowledge units that match the semantics. The intersection of the two sets is taken to calculate the corresponding comprehensive ranking score. If there is no matching result, a three-gradient adjustment is triggered. Step S400: Sort the knowledge units within the intersection in descending order according to the comprehensive ranking score, clean the knowledge units, group and aggregate the retrieval result set, and output the retrieval results that retain the original semantic structure.

2. The enterprise intelligent knowledge base management method based on vector model according to claim 1, characterized in that: Step S100 includes: Step S101: Using ApacheTika as a unified scheduling framework, the unstructured documents of the enterprise are parsed, the format, character encoding and encryption status of the unstructured documents are identified, the semantic information of the title level, paragraph boundaries and chapter affiliation of the unstructured documents are preserved, and the text content is output, which is divided into a set of text segments by paragraph boundaries. Step S102: Clean the obtained text segments and remove invalid and redundant data. Specifically, remove garbled segments with a Unicode replacement character ratio exceeding α%, where α is set by professionals; for a single text segment X to be judged, calculate the information entropy H(X). When H(X) is lower than the preset low information entropy threshold δ, the current text segment X is determined to be redundant data without valid information and is directly removed; output the cleaned text segment set. Step S103: According to the title hierarchy and chapter affiliation of the unstructured document, the cleaned text fragments are integrated according to their respective chapters and titles, specifically as follows: Continuous text fragments under the same title and chapter are integrated into independent semantic text blocks of the same topic. For each semantic text block of the same topic, sentences are segmented according to punctuation marks to obtain a continuous sentence sequence. A fixed-dimensional sentence vector is generated for each sentence. The semantic segmentation point is determined by using cosine similarity. Based on the semantic segmentation point that has been located, the semantic text block of the same topic is divided into several minimum knowledge fragments. For each segmented minimum knowledge fragment, the corresponding metadata is bound to form a knowledge unit composed of "minimum knowledge fragment + metadata".

3. The enterprise intelligent knowledge base management method based on vector model according to claim 1, characterized in that: Step S200 includes: Step S201: Using a relational database, create a metadata record for each knowledge unit. The metadata record includes a unique identifier for the knowledge unit, which serves as the primary key for each knowledge unit in the relational database. Establish a composite index on the sequence number, department, access permission level, and chapter fields of the unstructured document. Establish a sorted index on the creation time and update time fields to form a metadata record centered on the knowledge unit. Step S202: Using a vector database, the content of the smallest knowledge fragment of each knowledge unit is converted into a semantic vector of fixed dimension. The dimension of the semantic vector is consistent with the dimension of the sentence vector in step S103. In the vector database, the semantic vector is stored with the unique identifier of the knowledge unit as the primary key association field, and a vector index is established based on cosine similarity.

4. The enterprise intelligent knowledge base management method based on a vector model according to claim 1, characterized in that: Step S300 includes: Step S301: When a user query request is received, pre-filtering is performed on the metadata records of the relational database according to the query to obtain a set I of unique identifiers for knowledge units that meet the conditions. meta The user's natural language query text is converted into a query vector, and the vector similarity is used to retrieve the unique identifier set I of knowledge units from the vector database. vector ; Step S302: Establish a dual-database collaborative query mechanism, and use the pre-filtered set of unique identifiers for knowledge units (I) to perform a query. meta The set of unique identifiers for knowledge units retrieved by vector indexing I vector Perform a fusion and filtering process to obtain the intersection I of the two sets. final :I final =I meta ∩I vector For the intersection I final Each knowledge unit within the knowledge unit is uniquely identified, and a scoring model is constructed for each corresponding knowledge unit to calculate the comprehensive ranking score S. Step S303: If and only if the final intersection I final =I meta ∩I vector When the value is equal to ∅, step S302 is terminated, and the three-gradient adjustment is triggered, specifically as follows: First tier: Retain access permission level and department, adjust the time range to the preset time period, and re-pre-filter to obtain set I. meta1 ; Calculate the intersection I final1 =I meta1 ∩I vector , if I final1 =∅, enter the second gradient; Second tier: Retain access permission levels, adjust the department and time range to the entire department and maximum time range, and re-pre-filter to obtain set I. meta2 ; Calculate the intersection I final2 =I meta2 ∩I vector , if I final2 =∅, entering the third gradient; Third gradient: Only retain access permission levels, remove all other pre-filtering conditions, and pre-filter to obtain the full permission set I. meta3 ; Calculate the intersection I final3 =I meta3 ∩I vector , if I final3 =∅, generates a no-result response, prompting the user to adjust the query conditions. The remaining pre-filtering conditions include the department, document name, and time range.

5. The enterprise intelligent knowledge base management method based on a vector model according to claim 1, characterized in that: Step S400 includes: Step S401: Based on the calculated comprehensive ranking score S, perform the following steps on the final intersection I. final All knowledge units within the scope are sorted in descending order from high to low. Knowledge units with a comprehensive ranking score S lower than the preset effective score threshold β are removed to obtain the final retrieval result set I. The corresponding metadata information is then bound to each retained knowledge unit. Step S402: Group and aggregate the knowledge units in the final search result set I according to the serial number of the unstructured document to which they belong, the chapter to which they belong, and the title level, and output the search results that retain the semantic structure of the original document; provide the source information of the original document's document name, page number, and chapter position for each knowledge unit, and support users to directly jump to the corresponding position in the original document to view the complete context content.

6. An enterprise intelligent knowledge base management system based on a vector model, characterized in that: The system includes a knowledge construction module, a dual-database construction module, a retrieval and fusion module, and a result output module; The knowledge construction module uses a parsing component to extract corresponding text content from unstructured enterprise documents, cleans the extracted text, removes garbled characters and low-information-entropy redundant data, segments it to generate the smallest knowledge fragments, binds corresponding metadata, and forms standardized knowledge units. The dual-database construction module constructs a relational database and a vector database for knowledge units, and establishes joint indexes and sorted indexes; it also establishes joint indexes and sorted indexes for the fields corresponding to metadata, and establishes HNSW vector indexes for semantic vectors. When the retrieval fusion module receives a user query request, it establishes a dual-database collaborative query mechanism. It performs pre-filtering through a relational database to obtain a set of knowledge unit identifiers that meet the conditions, and performs semantic retrieval through a vector database to recall the set of knowledge unit identifiers that meet the conditions. It takes the intersection of the two sets to construct a scoring model and calculates a comprehensive ranking score. When the intersection is empty, it triggers a three-gradient hierarchical processing. The result output module sorts the knowledge units within the intersection in descending order according to the comprehensive ranking score, obtains the final search result set, binds complete metadata and traceability information, and outputs traceable standardized search content.

7. The enterprise intelligent knowledge base management system based on a vector model according to claim 6, characterized in that: The knowledge construction module includes a document parsing unit, a text cleaning unit, and a semantic segmentation unit; The document parsing unit uses ApacheTika as a unified scheduling framework, integrates document parsing components, identifies the format, character encoding and encryption status of unstructured documents, extracts the corresponding text content, retains structured semantic information, and divides the text into a set of text segments according to paragraph boundaries. The text cleaning unit removes garbled segments and redundant data with information entropy below a preset threshold from the parsed text segments, and outputs a set of cleaned text segments. Based on document title level and chapter affiliation, text fragments are integrated into semantic text blocks with the same theme, sentence vectors are generated, the cosine similarity of adjacent sentence vectors is calculated to locate semantic segmentation points, the smallest knowledge fragments are segmented and bound to metadata to form knowledge units.

8. The enterprise intelligent knowledge base management system based on a vector model according to claim 6, characterized in that: The dual-database construction module includes a metadata storage unit and a vector storage unit; The metadata storage unit uses a relational database to create metadata records for each knowledge unit and establish composite indexes and sorted indexes. The vector storage unit uses a vector database, calls a pre-trained semantic encoding model, converts the smallest knowledge fragment of the knowledge unit into a fixed-dimensional semantic vector, stores it with the unique identifier of the knowledge unit as the associated primary key, and establishes a vector index based on cosine similarity.

9. The enterprise intelligent knowledge base management system based on a vector model according to claim 6, characterized in that: The retrieval fusion module includes a dual-database pre-detection unit, a fusion scoring unit, and a fallback processing unit; When the dual-database pre-detection unit receives a user query request, it performs pre-filtering in the metadata records of the relational database, then converts the user's natural language query text into a query vector, and performs similarity retrieval and recall in the vector database. The fusion scoring unit takes the intersection of the unique identifiers of the pre-filtered and recalled knowledge units, constructs a comprehensive ranking score model, and calculates a comprehensive ranking score for each knowledge unit within the intersection. When the intersection of the pre-filter and the recall set of the vector retrieval is empty, the fallback processing unit triggers a three-gradient adjustment, relaxes the pre-filter conditions, retains the access permission level constraints, and generates a no-result response when there are no matching results, prompting the user to adjust the query conditions.

10. The enterprise intelligent knowledge base management system based on a vector model according to claim 6, characterized in that: The result output module includes a sorting and filtering unit and a result aggregation unit; The sorting and filtering unit sorts all knowledge units within the intersection in descending order from high to low according to the calculated comprehensive sorting score, removes knowledge units whose comprehensive sorting score is lower than the preset effective score threshold, binds corresponding metadata information to the retained knowledge units, and forms the final retrieval result set. The result aggregation unit groups and aggregates the knowledge units in the final retrieval result set, outputs retrieval results that retain the semantic structure of the original document, provides source information, and supports jumping to the corresponding position in the original document to view the complete context.

Images