A method and system for constructing domain knowledge graph based on rules and deep learning

By combining rules-based and deep learning methods, documents are classified and screened, and structured and unstructured text are extracted, the problems of low generalization ability and low accuracy in domain knowledge graph construction are solved, and high Quality domain knowledge graph construction and automation improvement.

CN114064915BActive Publication Date: 2024-11-08BEIJING SIMULATION CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111230908.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-22
Publication Date
2024-11-08
Estimated Expiration
2041-10-22

AI Technical Summary

Technical Problem

In the construction of domain knowledge graphs, the generalization ability of rule-based extraction methods is low, while the accuracy of deep learning methods is not high, making it difficult to achieve high-quality domain knowledge graph construction, especially in the case of untimely communication between different departments within the enterprise. Next, it is difficult to extract all entities and relationships.

Method used

A method based on the integration of rules and deep learning is adopted to construct a domain knowledge graph by classifying and screening documents, combining rules-based knowledge extraction and deep learning knowledge extraction. The specific steps include: performing rules-based extraction of structured text, serializing labeling and deep learning extraction of unstructured text, and finally building a domain knowledge graph.

Benefits of technology

It achieves a high accuracy rate and a certain degree of automation, reduces the participation of experts, effectively manages and visualizes the display of knowledge, and provides support for intelligent retrieval and rapid capture and understanding of knowledge.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114064915B_ABST
    Figure CN114064915B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a method and system for constructing a domain knowledge graph based on rules and deep learning, the method comprising: S1, inputting a document; S2, classifying and screening the document to obtain structured text and unstructured text; S3, extracting knowledge based on rules on the structured text to obtain a first triple; S4, serializing and annotating the unstructured text to obtain annotated text; S5, extracting knowledge based on deep learning on the annotated text to obtain a second triple; S6, constructing a domain knowledge graph based on the first triple and the second triple. The present application realizes the construction of a domain knowledge graph by integrating extraction methods based on rules and deep learning, solving the problem of low generalization ability of rule-based extraction methods and low accuracy of extraction methods based on deep learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and more specifically, to a method and system for constructing a domain knowledge graph based on rules and deep learning. Background Art

[0002] With the explosive growth of data, traditional manual retrieval methods can no longer meet the needs of humans for rapid capture and understanding of knowledge. Knowledge graphs have shown great advantages in solving the accuracy and scalability of knowledge queries. Knowledge graphs provide a way to better organize, manage and understand data information by expressing massive amounts of data information in a form that is closer to human cognition. Since Google proposed the concept of knowledge graphs in 2012, more and more large-scale general knowledge graphs have been built, such as Freebase, Wikidata, DBpedia, etc., which have been widely used in personalized recommendations, intelligent question and answer, information retrieval and other fields.

[0003] The knowledge graph is essentially a structured knowledge base and a huge structured knowledge network. The entities in the network represent things or concepts in the real world and are generally represented in the form of triples (head entity, relationship, tail entity). The process of building a knowledge graph refers to the process of extracting entities and relationships from structured or unstructured text, that is, the process of extracting knowledge from documents.

[0004] The construction method of general knowledge graph is already very mature, and it mostly uses deep learning to extract entities and relationships. However, due to the difference between domain knowledge graph and general knowledge graph, the general knowledge graph is oriented to general fields, mainly based on common sense knowledge, and the wider the knowledge coverage, the better, while the domain knowledge graph is a knowledge graph oriented to specific fields, and the user target is experts at different levels in the field. Therefore, compared with the general knowledge graph, the domain knowledge graph has more stringent quality requirements, higher expert participation, and lower degree of automation, making it difficult to directly use deep learning methods to achieve high-quality domain knowledge graph construction. In addition, within the industry, due to the untimely communication between departments, the internal structure of the documents generated within the enterprise is different, making it difficult to directly use rule-based extraction methods to completely extract all entities and relationships. Summary of the invention

[0005] Therefore, there is an urgent need for an extraction method based on the integration of rules and deep learning to build a domain knowledge graph.

[0006] The purpose of the present invention is to provide a method and system for constructing a domain knowledge graph based on rules and deep learning. In order to achieve at least one of the above purposes, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides a method for constructing a domain knowledge graph based on rules and deep learning, comprising:

[0008] S1. Input document;

[0009] S2, classifying and screening the documents to obtain structured text and unstructured text;

[0010] S3, performing rule-based knowledge extraction on the structured text to obtain a first triple;

[0011] S4, performing serialization annotation on the unstructured text to obtain annotated text;

[0012] S5, performing deep learning-based knowledge extraction on the annotated text to obtain a second triple;

[0013] S6. Construct a domain knowledge graph based on the first triple and the second triple.

[0014] In a specific embodiment, the S2 further includes:

[0015] S21, classifying the documents according to the document types to obtain classified documents;

[0016] S22, screening the classified documents by using a text screening method to screen out structured text and unstructured text;

[0017] in,

[0018] The document type is determined based on the document title.

[0019] In a specific embodiment, the S3 further includes:

[0020] Formulate extraction rules based on the experts’ definition and conduct extraction;

[0021] The rule-based knowledge extraction includes: knowledge extraction based on article structure characteristics and index knowledge extraction in professional fields;

[0022] The structure of the indicator is: indicator name: indicator value.

[0023] In a specific embodiment, the S5 includes:

[0024] S51, extracting entities from the annotated text;

[0025] S52: extracting relations from the annotated text.

[0026] In a specific embodiment, the S51 includes:

[0027] S511, preprocessing the annotated text through a BERT pre-trained language model to obtain a word vector;

[0028] S512, processing the word vector through a bidirectional GRU module to obtain context information of the entity;

[0029] S513, decoding the context information through a fully connected module to obtain a probability distribution of the entity;

[0030] S514, modifying the probability distribution by using a conditional random field model;

[0031] S515: Extract and classify the entities to complete entity extraction.

[0032] In a specific embodiment, the S52 includes:

[0033] S521, preprocessing the annotated text through a BERT pre-trained language model to obtain a word vector;

[0034] S522, processing the word vector through a bidirectional GRU module to obtain context information of the entity;

[0035] S523, obtaining the head entity vector and the tail entity vector of the relationship through a multi-layer perceptron module;

[0036] S524. Determine the relationship between the head entity vector and the tail entity vector through a bilinear model to complete relationship extraction.

[0037] The second aspect of the present application provides a domain knowledge graph construction system based on rules and deep learning, including:

[0038] Input module, used to input documents;

[0039] A classification and screening module, used to classify and screen the documents to obtain structured text and unstructured text;

[0040] A rule extraction module is used to extract knowledge based on rules from the structured text to obtain the first triple

[0041] A labeling module, used for serializing and labeling the unstructured text to obtain a labeled text; a deep learning extraction module, used for performing deep learning-based knowledge extraction on the labeled text to obtain a second triple;

[0042] The domain knowledge graph output module is used to construct a domain knowledge graph according to the first triple and the second triple.

[0043] In a specific embodiment, the deep learning module further includes:

[0044] An entity extraction module, used to extract entities from the annotated text;

[0045] The relation extraction module is used to extract relations from the annotated text.

[0046] A third aspect of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0047] A fourth aspect of the present application provides a computer-readable medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0048] The beneficial effects of this application are as follows:

[0049] In view of the problems existing in the current prior art, this application provides a method and system for constructing a domain knowledge graph based on rules and deep learning. This application realizes the construction of a domain knowledge graph by integrating the extraction methods based on rules and deep learning, solving the problem of low generalization ability of the extraction methods based on rules and the problem of low accuracy of the extraction methods based on deep learning, and can obtain a higher accuracy rate and a certain degree of automation, reducing the participation of experts. This application realizes the effective management and visual display of domain knowledge, and provides support for the realization of intelligent retrieval and the rapid capture and understanding of knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0051] Figure 1 A step diagram of a method for constructing a domain knowledge graph based on rules and deep learning in one embodiment of the present application is shown.

[0052] Figure 2 A module diagram of a domain knowledge graph construction system based on rules and deep learning in one embodiment of the present application is shown.

[0053] Figure 3 A flowchart of entity extraction based on deep learning in one embodiment of the present application is shown.

[0054] Figure 4 A flowchart of deep learning-based relationship extraction in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] In the following description, for the purpose of explanation, in order to provide a thorough understanding of one or more embodiments, numerous specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details.

[0056] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0057] It should also be noted that, in the description of the present application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "comprise one..." do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0058] At present, there are two main widely used knowledge extraction methods: rule-based knowledge extraction methods and deep learning-based knowledge extraction methods.

[0059] The traditional rule-based knowledge extraction method is to define the characters, grammar or semantic patterns that express the relationship in the text, and use the matching of patterns with text as the main means to achieve the acquisition of relationship instances. For known relationships, relevant patterns are constructed according to their expression in the text, so that relationship instances can be further extracted through pattern matching, thereby achieving knowledge extraction.

[0060] The traditional deep learning-based knowledge extraction method is to identify the entities in the sentence and the relationship between entity pairs by labeling each word in the sentence.

[0061] However, both extraction methods have their own limitations. Rules play an important role in early knowledge extraction. Rule-based extraction has a high accuracy rate, but it requires expert participation throughout the process and has low generalization ability. It can only match a few patterns identified manually. Patterns beyond the rules cannot be matched, that is, the rules cannot be applied to phrases with the same semantics but different expressions. Therefore, it is only suitable for extracting knowledge with strong structure. Deep learning can match two sentences with different expressions but similar semantics based on representation learning. It has a high degree of automation and less human participation, but deep learning relies heavily on natural language processing annotations such as part-of-speech tagging and syntactic parsing to provide classification features. Natural language processing annotation work often has a large number of errors. These errors will continue to propagate and amplify in the knowledge extraction system, and ultimately affect the effect of knowledge extraction. The comparison of the two extraction methods is shown in the following table:

[0062]

[0063]

[0064] Therefore, in order to obtain a higher accuracy and a certain degree of automation, reduce the participation of experts, and solve the problem of low generalization ability, an embodiment of the present application provides a method for constructing a domain knowledge graph based on rules and deep learning, such as Figure 1 As shown, including:

[0065] S1. Input document;

[0066] S2, classifying and screening the documents to obtain structured text and unstructured text;

[0067] In a specific embodiment, the S2 further includes:

[0068] S21, classifying the documents according to the document types to obtain classified documents;

[0069] Generally, different document types have different document structures, while documents of the same type have the same document structure, that is, the titles are the same. Therefore, preferably, the documents are classified according to their titles to obtain classified documents.

[0070] S22, screening the classified documents by using a text screening method to screen out structured text and unstructured text;

[0071] Preferably, some screening rules are manually formulated to screen the classified documents, which are usually rules formulated by experts. It can be found through observation that the same chapters in the same category of documents are either unstructured or structured, so the keyword matching method can be used to match the unstructured text;

[0072] The proportion of structured text is then used for further screening. Preferably, a counting method is used to calculate the proportion of structured text. It should be noted that the judgment standard of the proportion is evaluated by experts.

[0073] Preferably, if the proportion of structured text is high, rule-based knowledge extraction is directly performed on the structured text to construct the knowledge graph.

[0074] S3, performing rule-based knowledge extraction on the structured text to obtain a first triple;

[0075] In a specific embodiment, the rule-based knowledge extraction includes: knowledge extraction based on article structure and index knowledge extraction in professional fields; the description of the entity "index" in the document has the same rule characteristics, so the extraction rules for this type of structured knowledge are formulated according to the definition of experts, thereby realizing the extraction of structured knowledge;

[0076] The structure of the indicator is: indicator name: indicator value.

[0077] S4, performing serialization annotation on the unstructured text to obtain annotated text;

[0078] S5, performing deep learning-based knowledge extraction on the annotated text to obtain a second triple;

[0079] In a specific embodiment, the S5 further includes:

[0080] S51, extracting entities from the annotated text;

[0081] S52: extracting relations from the annotated text.

[0082] In a specific embodiment, taking "We love Beijing Tiananmen" as an example, the step of S51 is as follows: Figure 3 As shown, including:

[0083] S511, pre-processing "We love Beijing Tiananmen" through the BERT pre-trained language model to obtain word vectors;

[0084] S512, processing the word vector through a bidirectional GRU module to obtain context information of the entity;

[0085] S513, decoding the context information through a fully connected module to obtain a probability distribution of the entity;

[0086] S514, modifying the probability distribution by using a conditional random field model;

[0087] S515: Extract and classify the entities to complete entity extraction.

[0088] The steps of S52 are as follows: Figure 4 As shown, including:

[0089] S521, pre-processing “We love Beijing Tiananmen” through the BERT pre-trained language model to obtain a word vector;

[0090] S522, processing the word vector through a bidirectional GRU module to obtain context information of the entity;

[0091] S523, obtaining the head entity vector and the tail entity vector of the relationship through a multi-layer perceptron module;

[0092] S524, determining the relationship between the head entity vector and the tail entity vector through a bilinear model, and completing relationship extraction;

[0093] in, Figure 2 S ij arc is an energy function for evaluating the relationship between the head entity vector and the tail entity vector.

[0094] S6. Construct a domain knowledge graph based on the first triple and the second triple.

[0095] In the second aspect, an embodiment of the present application also provides a domain knowledge graph construction system based on rules and deep learning, such as Figure 2 As shown, including:

[0096] Input module, used to input documents;

[0097] A classification and screening module, used to classify and screen the documents to obtain structured text and unstructured text;

[0098] A rule extraction module, used for performing rule-based knowledge extraction on the structured text to obtain a first triple;

[0099] Annotation module, used for serializing and annotating the unstructured text to obtain annotated text;

[0100] A deep learning extraction module, used for performing deep learning-based knowledge extraction on the annotated text to obtain a second triple;

[0101] The domain knowledge graph output module is used to construct a domain knowledge graph according to the first triple and the second triple.

[0102] In a specific embodiment, the deep learning module further includes:

[0103] An entity extraction module, used to extract entities from the annotated text;

[0104] The relation extraction module is used to extract relations from the annotated text.

[0105] In a specific embodiment, the entity extraction module further includes:

[0106] A BERT preprocessing module, used to process the annotated text through a BERT pre-trained language model to obtain a word vector;

[0107] A GRU module is used to process the word vector to obtain context information of the entity;

[0108] A fully connected module, used for decoding the context information to obtain a probability distribution of the entity;

[0109] The conditional random field module is used to modify the probability distribution.

[0110] The relationship extraction module also includes:

[0111] A BERT preprocessing module, used to process the annotated text through a BERT pre-trained language model to obtain a word vector;

[0112] A GRU module is used to process the word vector to obtain context information of the entity;

[0113] Multi-layer perceptron module, used to obtain the head entity vector and tail entity vector of the relationship;

[0114] The bilinear module is used to determine the relationship between the head entity vector and the tail entity vector to complete the relationship extraction.

[0115] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0116] In a fourth aspect, an embodiment of the present application further provides a computer-readable medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0117] This application realizes the construction of a domain knowledge graph by integrating rule-based and deep learning extraction methods, solving the problem of low generalization ability of rule-based extraction methods and low accuracy of deep learning-based extraction methods, and can achieve a higher accuracy rate and a certain degree of automation, reducing the involvement of experts.

[0118] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A method for constructing a domain knowledge graph based on rules and deep learning, characterized in that: include: S1. Input document; S2, classifying and screening the documents to obtain structured text and unstructured text; S3, performing rule-based knowledge extraction on the structured text to obtain a first triple; S4, performing serialization annotation on the unstructured text to obtain annotated text; S5, performing deep learning-based knowledge extraction on the annotated text to obtain a second triple; S6. Construct a domain knowledge graph according to the first triple and the second triple; The S2 further includes: S21, classifying the documents according to the document types to obtain classified documents; S22, screening the classified documents by using a text screening method to screen out structured text and unstructured text; in, The document type is determined based on the document title; The S3 further includes: Formulate extraction rules based on the experts’ definition and conduct extraction; The rule-based knowledge extraction includes: knowledge extraction based on article structure characteristics and index knowledge extraction in professional fields; The structure of the indicator is: indicator name: indicator value; The S5 includes: S51, extracting entities from the annotated text; S52, extracting relations from the annotated text; The S51 includes: S511, preprocessing the annotated text through a BERT pre-trained language model to obtain a word vector; S512, processing the word vector through a bidirectional GRU module to obtain context information of the entity; S513, decoding the context information through a fully connected module to obtain a probability distribution of the entity; S514, modifying the probability distribution by using a conditional random field model; S515, extracting and classifying the entities to complete entity extraction; The S52 includes: S521, preprocessing the annotated text through a BERT pre-trained language model to obtain a word vector; S522, processing the word vector through a bidirectional GRU module to obtain context information of the entity; S523, obtaining the head entity vector and the tail entity vector of the relationship through a multi-layer perceptron module; S524. Determine the relationship between the head entity vector and the tail entity vector through a bilinear model to complete relationship extraction.

2. A domain knowledge graph construction system based on rules and deep learning using the method of claim 1, characterized in that: include: Input module, used to input documents; A classification and screening module, used to classify and screen the documents to obtain structured text and unstructured text; A rule extraction module, used for performing rule-based knowledge extraction on the structured text to obtain a first triple; Annotation module, used for serializing and annotating the unstructured text to obtain annotated text; A deep learning extraction module, used for performing deep learning-based knowledge extraction on the annotated text to obtain a second triple; The domain knowledge graph output module is used to construct a domain knowledge graph according to the first triple and the second triple.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method as claimed in claim 1 is implemented.

4. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method as claimed in claim 1 is implemented.