Knowledge base document association search method and system based on knowledge graph and storage

Abstract

The invention relates to a knowledge base document association search method and system based on a knowledge graph and storage, and the method comprises the steps: building association relationships between documents and between chapters and paragraphs in the documents by combining a directory tree of a knowledge base, chapter structures of the documents, internal directory structures of the documents, document labels and the like; and constructing a knowledge base network based on a knowledge base, and carrying out association search. Compared with a traditional search mode, the method has the advantages that the query intention of the user can be better understood, and the knowledge retrieval efficiency can be better improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a knowledge base document association search method, system and storage based on a knowledge graph. Background Art

[0002] As a carrier for knowledge storage, organization and utilization, the knowledge base systematically and organizedly stores and manages knowledge. Knowledge bases are widely used in fields such as enterprise knowledge management, academic research, and R&D knowledge management. While organizing and storing a large amount of knowledge, the knowledge base also provides users with a convenient way to acquire and share knowledge.

[0003] Generally speaking, from the perspective of the types of data sources processed, a large number of current knowledge bases mainly tend to process unstructured documents, such as R&D materials during the R&D process, project process documents during project implementation, internal corporate rules and policies, etc. These documents have complex formats, and the types and numbers of documents are relatively large, and there is a strong demand for document retrieval.

[0004] At present, the unstructured document retrieval based on knowledge base is mainly based on keyword matching, which leads to low efficiency of users in obtaining information. In organizations and enterprises, there are a large number of document types such as systems, standards, specifications, manuals, etc. These documents are usually stored in various storage systems. As the number of documents increases, it is difficult for users to quickly and accurately find the required information. On the one hand, traditional document management methods often rely on the naming of folders and files. This method makes it easy for documents to be lost, duplicated, and confusingly named. When a specific document needs to be found, users need to spend a lot of time and energy on manual search; on the other hand, even if the required document is found, because the document generally contains more semantic units, traditional retrieval methods are difficult to quickly and efficiently provide accurate answers. In general, the current knowledge base document retrieval has the following problems: 1. Insufficient semantic understanding of search queries; 2. Search lacks connections between knowledge; 3. Lack of comprehensive consideration of search context. Summary of the invention

[0005] In order to solve the shortcomings of existing knowledge base document retrieval, the present invention provides a knowledge base document association search method, system and storage based on knowledge graph, establishes semantic associations between documents and documents, and between chapters and paragraphs within documents, and improves the accuracy and efficiency of queries.

[0006] The technical objectives of the present invention are achieved through the following technical solutions: A knowledge base document association search method based on knowledge graph, the method comprising the following steps: Step 1: Build a knowledge base network: Analyze the chapter structure of the document and extract the directory architecture of the document. The directory architecture is a hierarchical structure, and each level of the directory architecture contains a document title and a directory. Add corresponding entity tags to document titles and directories, extract entity tags based on the knowledge graph, and build associations between documents and between semantic units within documents based on whether they contain the same entity tags; Step 2: Perform an association search based on the constructed knowledge base network to search for matching entity tags to identify key entities; then extract other entities associated with the key entities from the knowledge base network and obtain the relationships between all entities; Step 3: Sort by matching degree, set matching degree threshold, and filter out search results with higher matching degree.

[0007] Furthermore, the knowledge base network includes several levels of knowledge nodes, each level of knowledge nodes has a label and a knowledge attachment, and each knowledge node has a document as a knowledge attachment; each knowledge attachment contains a title and a directory, and the title contains a label; the directory contains several levels of directory nodes, and each level of directory nodes has a label.

[0008] Furthermore, the associated search in step 2 includes the following steps: Step 2.1, the user enters a query statement; Step 2.2: Perform entity matching in the knowledge base network according to the query statement, match entity labels, and identify key entities in the knowledge base network; Step 2.3: When no entity tag is matched, fuzzy matching is performed based on the user query statement and the entity tag to form a candidate entity tag set; Step 2.4: Extract other entities associated with the key entity in the knowledge base network and obtain the relationship between the entities.

[0009] Furthermore, the relationships between entities obtained in step 2 include the relationships between documents and the association relationships between semantic units in the documents.

[0010] Furthermore, when executing step 3, the following steps are included: Step 3.1, the set of directory nodes of the document directly associated with the entity tag is recorded as V_1, and the weight of each directory node in V_1 is set to weight X, X>0; Step 3.2, for each directory node in V_1, continuously traverse the directory nodes of the sub-layer of the directory node to obtain the directory node set V_2 associated with the sub-layer directory nodes of the directory node in V_1, and increase the weight of each directory node in V_2 by Y, Y>0; Step 3.3: The set of documents to which the entity tag is indirectly associated is denoted as V_3; Step 3.4: For each document in V_3, traverse its directory nodes to obtain a set of directory nodes V_4 associated with the entity label. The weight of each directory node in V_4 is increased by Y. Step 3.5: Merge all directory nodes in V_1, V_2 and V_4 and merge their weights, and select the TOP weighted directory nodes according to the weight setting threshold.

[0011] Furthermore, the query statement includes at least one of a keyword, a phrase, and a question.

[0012] The present invention also provides a knowledge base document association search system based on knowledge graph, comprising: A knowledge base network construction module is used to construct a knowledge base network containing the association relationships within and between documents; Input module, used for users to input query statements; A matching module, used to perform an associative matching search from the knowledge base network according to the query statement input by the input module; The output module is used to output the results of the matching module's associated matching search.

[0013] The present invention also provides a storage medium storing computer software, which, when running, executes according to the steps of a knowledge base document association search method based on a knowledge graph.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The knowledge base network constructed in the present invention can capture the contextual relationship between entities with the help of knowledge graph, help the search engine better understand the background of user queries, and provide search results that are more in line with user intentions; knowledge graph, as a visible form of knowledge storage and association, can well establish semantic associations between documents and documents, and between chapters and paragraphs within documents, and can provide more efficient retrieval efficiency compared to traditional keyword retrieval.

[0015] 2. The knowledge base network constructed in the present invention contains the association relationship between documents and the fine-grained knowledge within the documents. With the help of associative search, the user's query intention can be better understood and the efficiency of knowledge retrieval can be better improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a knowledge base network design diagram based on knowledge graph in the present invention.

[0017] Figure 2 It is a schematic diagram of the knowledge base document association search process based on the knowledge graph in the present invention.

[0018] Figure 3 This is the network structure of the knowledge base related to DC battery swap stations. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is further described below in conjunction with specific embodiments: A knowledge base document association search method based on knowledge graph, the method comprising the following steps: Step 1. Build a knowledge base network: Analyze the chapter structure of the document, and extract the directory system structure of the document through set rules or directory extraction based on machine algorithms. The directory system structure is a hierarchical structure, and each level of the directory system structure contains document titles and directories; the extraction rules will specify the format templates of each directory system, such as the first-level title is 1xxx, the second-level title is 1.1xxx, and the third-level title is 1.1.1xxx.

[0020] The document titles and directories are marked with corresponding entity tags, entity tags are extracted based on the knowledge graph, and weak semantic associations are established between documents and between semantic units within a document based on whether they contain the same entity tags.

[0021] Knowledge base network Figure 1 As shown, the knowledge base network includes several levels of knowledge nodes, each level of knowledge nodes has a label and a knowledge attachment, each of the knowledge nodes has a document as a knowledge attachment; each knowledge attachment contains a title and a directory, and the title contains a label; the directory contains several levels of directory nodes, and each level of directory nodes has a label.

[0022] Step 2: Perform association search based on the constructed knowledge base network to search for matching entity tags to identify key entities; then extract other entities associated with the key entity from the knowledge base network, and obtain the relationships between all entities (including key entities and other entities associated with key entities), including the association relationships between key entities, the association relationships between other entities associated with key entities, and the association relationships between key entities and other associated entities; the relationships between entities include the relationships between documents and the association relationships between semantic units within the documents.

[0023] refer to Figure 2 , specifically including the following steps: Step 2.1, the user inputs a query statement; the query statement may be a keyword, a phrase, a question, etc., at least one of which is input as the query statement.

[0024] Step 2.2: Perform entity matching in the knowledge base network according to the query statement, match entity labels, and identify key entities in the knowledge base network; Step 2.3: When no entity label is matched, fuzzy matching is performed based on the user query statement and the entity label, candidate entity mentions are extracted from the query and retrieved from the knowledge base network, the semantic similarity between the entity and the entity mention in the knowledge base network is calculated, and then the output is sorted according to the semantic similarity to form a set of candidate entity labels; common machine learning models for calculating semantic similarity include: DSSM (Deep Structured Semantic Models), CNN-DSSM (Convolutional latent Semantic Model), LSTM-DSSM (Long-Short-Term Memory Deep Structured Semantic Models), and can also be implemented using the angle cosine (Cosine) algorithm.

[0025] Step 2.4: Extract other entities associated with the key entity in the knowledge base network and obtain the relationship between the entities.

[0026] Step 3: Sort by matching degree, set matching degree threshold, and filter out search results with higher matching degree. Figure 2 , specifically including the following steps: Step 3.1, the set of directory nodes of the document directly associated with the entity tag is recorded as V_1, and the weight of each directory node in V_1 is set to weight X, X>0; in this embodiment, X=10; Step 3.2: for each directory node in V_1, continuously traverse the directory nodes of the sub-layer of the directory node to obtain a directory node set V_2 associated with the sub-layer directory nodes of the directory node in V_1, and increase the weight of each directory node in V_2 by Y, where Y>0, and Y=1 in this embodiment; Step 3.3, the set of documents to which the entity tag is indirectly associated (such as tag-title-knowledge attachment or tag-knowledge node-knowledge attachment) is denoted as V_3; Step 3.4: For each document in V_3, traverse its directory nodes to obtain a set of directory nodes V_4 associated with the entity label. The weight of each directory node in V_4 is increased by Y, where Y=1. Step 3.5: merge all directory nodes in V_1, V_2 and V_4 by directory nodes and weights, and filter directory nodes with higher weights according to the weight setting threshold, such as filtering the top five directory nodes by weight.

[0027] In order to understand the content of the present invention more clearly, the following is described in conjunction with specific examples: Step 1: Figure 3 As shown, construct Figure 3 The knowledge base network shown, Figure 3The network structure of the knowledge base related to DC battery swap station: The DC power swap station is a first-level knowledge node, and detection management and maintenance management are second-level knowledge nodes; "Circuit breaker inspection details.docx" and "Converter transformer inspection details.docx" are knowledge attachments under the inspection management knowledge node, and "Converter transformer maintenance details.docx" is a knowledge attachment under the maintenance management knowledge node; "Circuit breaker inspection details.docx" includes the first-level directory node (3. Inspection method) and the second-level directory node (3.1 Casing inspection); "Converter transformer inspection details.docx" includes the first-level directory node (4. Inspection method), the second-level directory node (4.1 short circuit inspection, 4.2 bushing inspection); "Converter transformer maintenance details.docx" includes the first-level directory node (5. Maintenance key process quality space requirements), the second-level directory node (5.1 Bushing device maintenance, 5.2 Oil storage tank maintenance, 5.3, Breathing device maintenance); "Detection" is used as the label node between "3. Detection method", "3.1 Casing detection", "4. Detection method", "4.1 Short circuit detection" and "4.2 Casing detection"; "Converter transformer" is used as the label node between "Converter transformer inspection details.docx" and "Converter transformer maintenance details.docx"; "Casing" is used as the label node between "3.1 Casing Inspection" and "5.1 Casing Device Maintenance"; "5.2 Oil storage tank maintenance" uses "Oil storage tank" as the label node; “5.3 Respirator Maintenance” uses “Respirator” as the label node.

[0028] Step 2: Perform association search based on the constructed knowledge base network: The user inputs "converter bushing detection method" and extracts label entities from the knowledge base network: [converter transformer] (through converter fuzzy matching), [4.2 bushing detection] (through bushing detection exact matching), [bushing] (through bushing exact matching), [detection] (through detection exact matching); Step 3: Sort by matching degree, set matching degree threshold, and filter out search results with higher matching degree; the reachable paths are as follows: Path 1: [4.2 Casing Detection] Path 2: [Converter transformer] → [Converter transformer inspection details.docx] → [4. Inspection method] → [4.2 Bushing inspection] Path 3: [Detection] → [4. Detection method] → [4.2 Casing detection] Path 4: [Casing] → [4.2 Casing Detection] According to the optimal sorting of the paths, path 1 is output as the result with the highest score.

[0029] This embodiment also provides a knowledge base document association search system based on a knowledge graph, including: The knowledge base network construction module is used to construct a knowledge base network containing the relationships within and between documents, such as Figure 1 As shown; Input module, used for users to input query statements; The matching module is used to perform associated matching search from the knowledge base network according to the query statement input by the input module; the matching module performs entity matching in the knowledge base network according to the query statement, matches the entity label, and identifies the key entity in the knowledge base network; when the entity label cannot be matched, fuzzy matching is performed based on the user query statement and the entity label to form a set of candidate entity labels; then other entities associated with the key entity are extracted in the knowledge base network, and the relationship between the entities is obtained.

[0030] The output module is used to output the results of the matching module's associated matching search. The output module sorts them according to the matching degree, sets the matching degree threshold, and filters out the search results with a higher matching degree for output.

[0031] This embodiment also provides a storage medium, which stores computer software. The storage medium is such as a computer, a mobile phone, a tablet computer, a mobile hard disk, a mobile terminal, a vehicle-mounted computer device, a network storage space, etc. When the computer software is running, it is executed according to a method of a knowledge base document association search method based on a knowledge graph in the above embodiment.

[0032] This embodiment is only a further explanation of the present invention, not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but as long as it is within the scope of the claims of the present invention, it will be protected by the patent law.

Claims

1. A knowledge base document association search method based on knowledge graph, characterized in that: The method comprises the following steps: Step 1: Build a knowledge base network: Analyze the chapter structure of the document and extract the directory architecture of the document. The directory architecture is a hierarchical structure, and each level of the directory architecture contains a document title and a directory. Add corresponding entity tags to document titles and directories, extract entity tags based on the knowledge graph, and build associations between documents and between semantic units within documents based on whether they contain the same entity tags; Step 2: Perform an association search based on the constructed knowledge base network to search for matching entity tags to identify key entities; then extract other entities associated with the key entities from the knowledge base network and obtain the relationships between all entities; the relationships between entities include the relationships between documents and the association relationships between semantic units in the documents; Step 3: Sort by matching degree, set matching degree threshold, and filter out search results with higher matching degree; When executing step 3, the following steps are included: Step 3.1, the set of directory nodes of the document directly associated with the entity tag is recorded as V_1, and the weight of each directory node in V_1 is set to weight X, X>0; Step 3.2, for each directory node in V_1, continuously traverse the directory nodes of the sub-layer of the directory node to obtain the directory node set V_2 associated with the sub-layer directory nodes of the directory node in V_1, and increase the weight of each directory node in V_2 by Y, Y>0; Step 3.3: The set of documents to which the entity tag is indirectly associated is denoted as V_3; Step 3.4: For each document in V_3, traverse its directory nodes to obtain a set of directory nodes V_4 associated with the entity label. The weight of each directory node in V_4 is increased by Y. Step 3.5: Merge all directory nodes in V_1, V_2 and V_4 and merge their weights, and select the TOP weighted directory nodes according to the weight setting threshold.

2. According to claim 1, a knowledge base document association search method based on knowledge graph is characterized in that: The knowledge base network includes several levels of knowledge nodes, each level of knowledge nodes has a label and a knowledge attachment, and each of the knowledge nodes has a document as a knowledge attachment; Each knowledge attachment contains a title and a directory, and the title contains tags; The directory includes several levels of directory nodes, and the directory nodes at each level have labels respectively.

3. A knowledge base document association search method based on knowledge graph according to claim 2, characterized in that: The associated search in step 2 includes the following steps: Step 2.1, the user enters a query statement; Step 2.2: Perform entity matching in the knowledge base network according to the query statement, match entity labels, and identify key entities in the knowledge base network; Step 2.3: When no entity tag is matched, fuzzy matching is performed based on the user query statement and the entity tag to form a candidate entity tag set; Step 2.4: Extract other entities associated with the key entity in the knowledge base network and obtain the relationship between the entities.

4. A knowledge base document association search method based on knowledge graph according to claim 3, characterized in that: The query statement includes at least one of a keyword, a phrase, and a question.

5. A knowledge base document association search system based on knowledge graph, characterized in that: include: The knowledge base network construction module is used to construct a knowledge base network containing the association relationship within and between documents, extract a hierarchical directory system, and each level of the directory system contains document titles and directories; build the association relationship between documents and between semantic units within a document; Input module, used for users to input query statements; A matching module, used to perform an associative matching search from the knowledge base network according to the query statement input by the input module; The output module is used to output the results of the matching module's associated matching search. The set of directory nodes of the documents directly associated with the entity tag is V_1, and a weight X is set for each directory node in V_1; the set of directory nodes associated with the sub-layer directory nodes of each directory node in V_1 is V_2, and a weight Y is added to each directory node in V_2; the set of documents indirectly associated with the entity tag is V_3, the set of directory nodes associated with each document in V_3 and the entity tag is V_4, and the weight of each directory node in V_4 is increased by Y, all the directory nodes in V_1, V_2 and V_4 are merged for weight merging, and the TOP weight directory nodes in the merged result are filtered according to the set threshold.

6. A storage medium, characterized in that: The storage medium stores computer software, and when the computer software is running, it is executed according to the method according to any one of claims 1 to 4.

Images