A knowledge graph construction method based on term chain merging

Abstract

The application relates to the technical field of information extraction, and particularly discloses a knowledge graph construction method based on word item chain merging, which constructs a sequential relationship network, adopts a node merging mechanism, and step by step constructs a knowledge graph covering an original semantic structure from bottom to top on the basis of retaining the original sequence of word items, so that the accuracy and completeness of graph construction are improved, a solid foundation is provided for subsequent knowledge reasoning and structure conversion of a traditional knowledge graph, and a simplification method based on word item chain merging is adopted to automatically construct a structured relationship network from bottom to top without relying on external dictionaries or pre-training models, so that the word item sequence information in the text is effectively retained to form a knowledge network graph reflecting the original semantic structure.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of information extraction, and specifically discloses a knowledge graph construction method based on word chain merging. BACKGROUND

[0002] A knowledge graph can provide high-quality structured data and is widely used in many fields such as artificial intelligence. The knowledge graph can realize automatic question answering, can be used as a search engine, and can play a role in information extraction.

[0003] Traditional knowledge graph construction methods usually rely on external dictionaries, complex models and ontology annotation, and construct graphs through methods such as relation extraction and entity recognition. However, these methods have the following significant shortcomings when dealing with complex texts, open domains or low-resource scenarios:

[0004] Sequential information loss: existing knowledge graph construction methods mostly use a static relation extraction mode, which is difficult to preserve the order relationship of the words in the original text, resulting in loss of context semantic information. In addition, the degree of automation in the graph construction process is low, which limits the feasibility of its large-scale application.

[0005] Dependence on external dictionaries and pre-trained models: existing knowledge graph construction methods generally rely on a large amount of manually annotated data, external dictionaries or pre-trained models, have strict requirements for ontology annotation, and have a significant performance decline in low-resource environments, lacking the ability to adapt to different scenarios.

[0006] In summary, the existing knowledge graph construction method is difficult to effectively preserve the order information in the original text in the processing process, resulting in a lack of accurate expression of context and full use of original information in the knowledge graph.

[0007] The present application provides a knowledge graph construction method based on word chain merging to solve the above problems. SUMMARY

[0008] The present application provides a knowledge graph construction method based on word chain merging, which uses a simplified method based on word chain merging to automatically construct a structured relation network from bottom to top without relying on external dictionaries or pre-trained models, effectively preserving the word order information in the text to form a knowledge network graph that reflects the original semantic structure.

[0009] To achieve the above purpose, the basic scheme of the present application provides a knowledge graph construction method based on word chain merging, comprising the following steps:

[0010] Step A1: obtaining original text and inputting the original text into a Neo4j graph database, performing sentence-level segmentation on the original text to obtain a sentence set, and respectively performing Chinese word segmentation on the sentences to obtain corresponding word chains, the word chains including sequentially arranged words;

[0011] Step A2: combining the word chains of all sentences to form a global set;

[0012] Step A3: creating a node for each word in each word chain, establishing a sequential connection relationship between the nodes according to the word order information in the original text, and connecting all nodes in each word chain based on the sequential connection relationship to form an edge, and the initial graph structure formed by the nodes and edges is used as an initial knowledge graph;

[0013] Step A4: traversing the global set, counting nodes with the same name and respectively merging and updating them to the node in the front position as a new node, and updating all relationships pointing to the repeated nodes to point to the merged new node;

[0014] Step A5: updating the initial graph structure based on the updated nodes and their sequential connection relationship and using it as a new knowledge graph.

[0015] Further, the sentence-level segmentation step includes punctuation-based segmentation of the original text to obtain punctuation-separated sentences, and the sentence set is formed by all segmented sentences.

[0016] Further, the sequential connection relationship between the nodes is found through a Cypher query statement, and the nodes are connected through a RELATES_TO relationship.

[0017] Further, in step A4, the process of counting repeated nodes with the same name includes:

[0018] Traverse the global set and traverse all nodes in each word chain. During the traversal process, find all nodes with the same name as the currently traversed node. If a word appears twice or more in multiple sentences, it is considered to be repeated, and a list duplicate_nodes is established to store the word corresponding to the node that appears twice or more.

[0019] Further, the process of finding all nodes with the same name as the currently traversed node is specifically using the MATCH function to query all nodes in the knowledge graph that have the same name as the currently traversed node.

[0020] Further, for each repeated node corresponding to a word in the list duplicate_nodes, the following merging operation is performed:

[0021] The relationship between the updated nodes points to the new node, and for the repeated nodes, the repeated nodes are merged and the first repeated node is selected to be reserved as the new node, and all relationships pointing to the repeated nodes are updated to point to the merged new node.

[0022] The principle and effect of the scheme are that:

[0023] 1. Compared with the prior art, the application constructs a sequential relationship network and adopts a node merging mechanism to gradually construct a knowledge graph covering the original semantic structure from bottom to top on the basis of preserving the original order of the word items, thereby improving the accuracy and completeness of the graph construction and providing a solid foundation for subsequent knowledge reasoning and structural transformation of the traditional knowledge graph.

[0024] 2. Compared with the prior art, the application gradually constructs a knowledge graph covering the original semantic structure from bottom to top, directly extracts word item chains from the original text and gradually constructs a knowledge graph representing nodes and sequential relationships, does not rely on a pre-existing ontology or semantic model, does not rely on a pre-constructed ontology, takes a sentence as a basic unit, merges repeated word items one by one to form a knowledge graph with a sequential network, and maximizes the preservation of the word item order information in the original text, thereby constructing a knowledge graph network with a sequential structure. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0026] Figure 1 A flowchart of a knowledge graph construction method based on word item chain merging according to an embodiment of the present application is shown.

[0027] Figure 2 A schematic diagram of an initial graph structure according to an embodiment of the present application is shown, wherein figure (a) is an initial graph structure constructed by word item chains A schematic diagram of an initial graph structure constructed by word item chains A schematic diagram of an initial graph structure constructed by word item chains

[0028] Figure 3 A schematic diagram of an updated graph structure according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to make the technical means adopted by the present application and the effects achieved in realizing the predetermined object of the present application more comprehensible, specific embodiments, structures, features and effects according to the present application will be described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0030] A knowledge graph construction method based on term chain merging, as shown in Figure 1 , includes the following steps:

[0031] Step A1: Obtain the original text and input it into the Neo4j graph database, and sequentially perform sentence-level segmentation on the original text to obtain sentences and store them into a sentence set S, , represents the kth sentence obtained by performing sentence-level segmentation on the original text, and then the sentence is segmented into Chinese to obtain a term chain including terms arranged in order Each term chain is represented as: , represents the mth term of the kth term chain.

[0032] Specifically, the above-mentioned sentence-level segmentation step includes segmenting the original text according to punctuation marks such as periods, question marks, exclamation marks, etc. to obtain sentences separated by punctuation marks.

[0033] In this embodiment, the obtained original text is:

[0034] "The article is written by Xiaoming, and Xiaoming is the author of the article".

[0035] After sentence-level segmentation, the sentence set , and the corresponding term chain , are obtained:

[0036] = "The article is written by Xiaoming" -> = [Article, by, Xiaoming, write];

[0037] = "Xiaoming is the author of the article" -> = [Xiaoming, is, article, is, author].

[0038] Step A2: Combine the term chains of all sentences to form a global set W_all, , and the global set W_all is the input basis for subsequent term combination and knowledge graph construction.

[0039] Specifically, the global set W_all obtained through step A2 is represented as:

[0040] W_all=[ =[[Article, written by, Xiaoming],[Xiaoming is, the, writer of, the article]].

[0041] Step A3: Use the CREATE function in the Neo4j graph database to create a node for each word in each word necklace. For example, for a word necklace... Each term in the document creates a corresponding node: , Let m be the node corresponding to the mth term in the kth word necklace. Then, based on the word order information in the original text, establish sequential connections between the nodes in each word necklace. Based on the sequential connections, connect all the nodes in each word necklace to form edges. The initial graph structure composed of nodes and edges serves as the initial knowledge graph.

[0042] Regarding the obtained word necklace This section describes how nodes are created and their sequential connections are determined using the following Cypher query:

[0043] CREATE(a:Entity{name:'article'});

[0044] CREATE(b:Entity{name:'by'});

[0045] CREATE(c:Entity{name:'Xiaoming'});

[0046] CREATE(d:Entity{name:'Writing'});

[0047] The sequential connection relationship of the nodes is as follows:

[0048] CREATE(a)-[:RELATES_TO]->(b)-[:RELATES_TO]->(c)-[:RELATES_TO]->(d);

[0049] During the process of creating a node for each term, each term is also synchronously inserted as a node into the Neo4j graph database, and the nodes are connected using the RELATES_TO relationship. These steps preserve the term order information of the original text, ensuring the integrity of the text information.

[0050] Specifically, two word necklaces are obtained through step A3. , The sequential connection relationships between the nodes in the data are as follows:

[0051] (Article) -> (by) -> (Xiaoming) -> (write);

[0052] (Xiaoming) -> (is) -> (Article) -> (the) -> (writer).

[0053] By the word chain The resulting node and edge structure has an initial graph structure as shown in FIG. (a) of Figure 2 By the word chain The resulting node and edge structure has an initial graph structure as shown in FIG. (b) of Figure 2 .

[0054] Step A4: Traverse the global set W_all, count the repeated nodes with the same name, and respectively merge and update them to the node with the earliest position as the new node, and update all the relationships pointing to the repeated nodes to point to the new node.

[0055] Specifically, the process of counting the repeated nodes with the same name includes:

[0056] Traverse the global set W_all, traverse all nodes in each word chain, and in the traversal process, find all nodes in the word chain with the same name as the currently traversed node. In this embodiment, the name of the node is the word corresponding to the node. Specifically, use the MATCH function to query all nodes in the initial knowledge graph with the same name as the currently traversed node.

[0057] If a node represented by a word appears twice or more in multiple sentences, it is considered that the word and the corresponding node are repeated and need to be merged, and a list duplicate_nodes is established to store the words corresponding to the nodes that appear twice or more.

[0058] For example, when traversing to the first node of the word chain , the node corresponds to the word "Article", and the MATCH function is used to query all nodes in the initial knowledge graph with the name "Article":

[0059] MATCH(n:Entity{name:'Article'});

[0060] RETURN n;

[0061] Where n is a variable name used to temporarily represent the node matched in the query.

[0062] When nodes with the same name are queried, for example, when nodes and both represent "Article", then the node and node As two duplicate nodes, and store the term "article" corresponding to the two duplicate nodes into the list duplicate_nodes.

[0063] Then, for each duplicate node corresponding to the term in the list duplicate_nodes, perform the following merging operation:

[0064] Update the relationship between nodes to point to the new node. For duplicate nodes, merge the nodes and node and select the first node to be retained as the new node , and update all relationships pointing to the duplicate node to point to the new node.

[0065] This can be implemented by using the MATCH function and the MERGE function, which represent the nodes of "article" and For example, point all relationships pointing to the node or the node to the new node after merging :

[0066] MATCH(n:Entity{name:'article'});

[0067] MATCH(other)-[r]->(n);

[0068] MERGE(new:Entity{name:'article'});

[0069] MERGE(other)-[r]->(new);

[0070] Where r is a variable name used to temporarily represent the relationship of the other node other pointing to the matched node n in the query.

[0071] The new node is located at , and the new node represents the term "article". In the same step, the new node representing the term "Xiaoming" is located at . Based on the updated nodes and , and based on the updated node order relationship, the new graph structure shown in Figure 3 is finally obtained, in which all relationships point to the unique node representing "article" and the node representing "Xiaoming" , and the connection relationship preserves the order information of the two original sentences.

[0072] Step A5: update the graph structure by the updated nodes and their sequential connection relationship and take it as the new knowledge graph knowledge_graph, and the final graph data is stored in the Neo4j graph database in the form of graph structure. The graph structure is taken as the output parameter knowledge_graph of the method and can be directly referenced for knowledge reasoning, graph query and structure expansion and other application scenarios.

[0073] In the embodiment, the final knowledge graph structure retaining semantic order is represented as follows:

[0074] (Xiaoming)-[:RELATES_TO]->(is)-[:RELATES_TO]->(article);

[0075] (article)-[:RELATES_TO]->(of)-[:RELATES_TO]->(writer);

[0076] (article)<-[:RELATES_TO]-(by)<-[:RELATES_TO]-(Xiaoming);

[0077] (Xiaoming)-[:RELATES_TO]->(writes).

[0078] Through the above steps A1 to A5, two word chain or multiple word chains with repeated word items are merged into one word chain, and the position of the merged word chain is the same as the position of the first repeated word item, and the word chains without repeated word items do not affect each other.

[0079] The application proposes a knowledge graph construction method based on word chain merging, which focuses on merging nodes with the same name to construct a knowledge graph containing only nodes and their sequential connections. Compared with the traditional top-down knowledge graph construction method, the application constructs a knowledge graph covering the original semantic structure from bottom to top. The word chain is directly extracted from the original text and the knowledge graph representing the nodes and the sequential relationship is gradually constructed. The method does not depend on the pre-existing ontology or semantic model, and does not depend on the pre-constructed ontology or semantic model. Taking a sentence as a basic unit, the repeated word items are merged one by one to form a knowledge graph with a sequential network, which maximizes the retention of word order information in the original text, thereby constructing a knowledge graph network with a sequential structure.

[0080] Compared with the traditional knowledge graph construction method, the application effectively avoids the problem of loss of word order information, thereby improving the accuracy and integrity of the knowledge graph, and also provides a solid foundation framework for subsequent knowledge derivation and application, which has important significance for accurately expressing context and understanding the semantics of the text.

[0081] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been described above with reference to preferred embodiments, the present application is not intended to be limited to the above-described embodiments, and any person skilled in the art, without departing from the technical scope of the present application, can make some changes or modifications to the above-described technical content to obtain equivalent embodiments with equivalent changes. Any indirect modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical scope of the present application, still belongs to the scope of the technical solutions of the present application.

Claims

1. A knowledge graph construction method based on word necklace merging, characterized in that, Includes the following steps: Step A1: Obtain the original text and input it into the Neo4j graph database. Perform sentence-level segmentation on the original text to obtain a set of sentences. Perform Chinese word segmentation on each sentence to obtain the corresponding word necklace. The word necklace includes words arranged in order. Step A2: Combine the word chains of all sentences to form a global set; Step A3: Create a node for each word item in each word necklace, establish sequential connections between nodes based on the word order information in the original text, and connect all nodes in each word necklace to form edges based on the sequential connections. The initial graph structure composed of nodes and edges serves as the initial knowledge graph. Step A4: Traverse the global set, count nodes with the same name, merge them and update the first node as the new node, and update all relationships pointing to the duplicate nodes to point to the merged new node. Step A5: Update the initial graph structure with the updated nodes and their sequential connections, and use it as a new knowledge graph.

2. The knowledge graph construction method based on word necklace merging according to claim 1, characterized in that, The sentence-level segmentation step includes segmenting the original text according to punctuation marks to obtain sentences separated by punctuation marks, and then forming a sentence set from all the segmented sentences.

3. The knowledge graph construction method based on word necklace merging according to claim 1, characterized in that, The sequential connection relationships between nodes are found using Cypher queries and the nodes are connected using the RELATES_TO relation.

4. The knowledge graph construction method based on word necklace merging according to claim 1, characterized in that, In step A4, the process of counting nodes with the same name includes: Traverse the global collection, and then traverse all nodes within each word chain. During the traversal, find all nodes with the same name as the currently traversed node. If a word appears twice or more in multiple sentences, it is considered a duplicate word, and a list `duplicate_nodes` is created to store the word corresponding to nodes that appear twice or more.

5. The knowledge graph construction method based on word necklace merging according to claim 4, characterized in that, The process of finding all nodes with the same name as the currently traversed node involves using the MATCH function to query all nodes in the knowledge graph that have the same name as the currently traversed node.

6. The knowledge graph construction method based on word necklace merging according to claim 4, characterized in that, For each duplicate node corresponding to a term in the list `duplicate_nodes`, perform the following merge operation: Update the relationships between nodes to point to the new node. For duplicate nodes, merge the duplicate nodes and select the first duplicate node to keep as the new node. Update all relationships pointing to the duplicate nodes to point to the merged new node.

Images