A knowledge graph automatic generation method and system based on document content analysis

Abstract

The application discloses a kind of knowledge graph automatic generation method and system based on document content analysis.Method includes: obtaining target document for constructing knowledge graph;Through domain fusion model, domain knowledge fusion is carried out to target document, and target document containing domain knowledge is generated;Through entity recognition model, the recognition of entity and the relationship between entities is carried out to target document containing domain knowledge, and several groups of head entity-relation-tail entity triplets are generated;According to all head entity-relation-tail entity triplets, the corresponding document knowledge graph is constructed.The application can improve the accuracy of entity and its relationship extraction by carrying out domain knowledge fusion to document, and effectively construct knowledge graph by generating head entity-relation-tail entity triplets, to improve the accuracy of knowledge graph construction.

Description

Technical Field

[0001] This invention relates to the fields of Internet and big data technology, specifically to a method and system for automatically generating knowledge graphs based on document content analysis. Background Technology

[0002] A knowledge graph is a relational network composed of structured representations of knowledge about a specific topic. It consists of nodes and edges connecting different nodes. Essentially, a knowledge graph is a semantic network, combining features of databases, graphs, and knowledge bases. In a knowledge graph, each node represents an entity or concept within that topic, and edges represent semantic relationships between entities or concepts. Knowledge graphs allow data retrieval through structured queries, can be analyzed like other network data structures, and the data within them possesses formal semantics, which can be used to interpret data and infer new facts.

[0003] Regarding the construction of knowledge graphs for documents, Chinese Patent Publication No. CN115956242A discloses a method for automatic knowledge graph construction, comprising: receiving text documents and training a first machine learning system to predict entities in the text documents; training a second machine learning system to predict relational data between entities; receiving a set of second text documents, determining a second embedding vector therefrom, and predicting entities and edges; thereby using the set of second text documents, the determined second embedding vector, and the predicted entities and their associated embedding vectors as inputs to the first and second trained machine learning models; and constructing triples representing entities and edges of the new knowledge graph.

[0004] The existing solutions described above achieve automatic knowledge graph construction by predicting entities and relationships within documents. In practical applications, these solutions can effectively identify and extract entities and their relationships from documents in general domains, thus enabling automatic knowledge graph construction. However, for highly specialized fields such as education or the military, which involve large amounts of technical terminology, existing machine reading comprehension models struggle to accurately segment and understand the semantic features of documents. This results in low accuracy in entity recognition and extraction, consequently leading to poor accuracy in knowledge graph construction. Therefore, designing a method to improve the accuracy of knowledge graph construction is a pressing technical problem that needs to be addressed. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem this invention aims to solve is: how to provide a method and system for automatically generating knowledge graphs based on document content analysis, which can improve the accuracy of entity and relation extraction by fusing domain knowledge into documents, and effectively construct knowledge graphs by generating head entity-relationship-tail entity triples, thereby improving the accuracy of knowledge graph construction.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A method for automatically generating knowledge graphs based on document content analysis includes:

[0008] S1: Obtain the target document for constructing the knowledge graph;

[0009] S2: Use a domain fusion model to fuse domain knowledge into the target document, generating a target document containing domain knowledge;

[0010] S3: Using an entity recognition model, identify entities and relationships between entities in target documents containing domain knowledge, and generate several sets of head entity-relationship-tail entity triples;

[0011] S4: Construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

[0012] Preferably, in step S2, the processing steps of the domain fusion model are as follows:

[0013] S201: Encode the target document to generate corresponding target document word vectors;

[0014] S202: Construct a corresponding document domain dictionary based on existing documents in the domain to which the target document belongs;

[0015] S203: Embed character vectors into each character in the document domain dictionary to generate document domain character vectors;

[0016] S204: Integrate the target document word vectors and the document domain character vectors, and use the generated document word vectors as the target document containing domain knowledge.

[0017] Preferably, in step S201, the target document is first segmented to obtain several words; then, a corresponding word ID is assigned to each word, and the word is replaced with the corresponding word ID; finally, the corresponding word vector is embedded for each word ID to generate the corresponding target document word vector.

[0018] Preferably, step S202 specifically includes the following steps:

[0019] S2021: Obtain existing documents in the domain of the target document as a corpus;

[0020] S2022: Preprocess existing documents, including noise removal, text cleaning, and text normalization;

[0021] S2023: Perform word segmentation on the preprocessed existing documents and count the frequency of each word in the corpus after segmentation;

[0022] S2024: Generate the corresponding document domain dictionary based on the word frequency statistics results.

[0023] Preferably, step S203 specifically includes the following steps:

[0024] S2031: Select a neural network with an embedding layer as the character vector training model;

[0025] S2032: Extract all unique characters from the document domain dictionary and generate the corresponding character set;

[0026] S2033: Initialize each character in the character set as a random vector in the embedding layer of the character vector training model;

[0027] S2034: Obtain existing documents in the domain of the target document as a corpus, convert the text in the corpus into character sequences, and label each character sequence with corresponding contextual information as training data;

[0028] S2035: Input the training data into the character vector training model and calculate the corresponding model output through forward propagation;

[0029] S2036: Use a loss function to calculate the difference between the model output and the true value of the character vector training model, and update the weights of the embedding layer and other layers through the backpropagation algorithm to minimize the loss;

[0030] S2037: Repeat steps S2035 to S2036 until the model converges or reaches the preset number of training rounds;

[0031] S2038: Extract character vectors from the embedding layer of the trained character vector training model to generate corresponding document-domain character vectors.

[0032] Preferably, in step S3, the processing steps of the entity recognition model are as follows:

[0033] S301: Use the Bi-LSTM model to capture the semantic features of the target document containing domain knowledge and generate the corresponding document context representation;

[0034] S302: Calculate the corresponding entity type label sequence based on the document context representation using the trained CRF model, and identify all entities based on the entity type label sequence;

[0035] S303: Sequentially identify the corresponding tail entity by treating each entity as a head entity, and identify the relationship between each head entity and its corresponding tail entity.

[0036] S304: Output all head entity-relationship-tail entity triples.

[0037] Preferably, in step S303, the tail entity corresponding to the head entity is determined through the following steps:

[0038] S3031: Replace each word in the target document with a word embedding to obtain the document embedding representation;

[0039] S3032: Convert the header entity into the corresponding header entity word embedding representation;

[0040] S3033: The document embedding representation and the head entity word embedding representation are used as input to the attention layer. The similarity score between the head entity word embedding and the embedding at each position in the target document is calculated through the attention layer.

[0041] S3034: Normalize the similarity scores using the softmax function to obtain the attention weights for each position;

[0042] S3035: The document embedding representation is weighted and summed using attention weights to obtain the weighted document embedding representation;

[0043] S3036: Use sequence labeling models to determine candidate tail entities based on weighted document embedding representations;

[0044] S3037: Based on the weighted document embedding representation and the representation of the tail entity candidates, calculate the score when each candidate is used as the tail entity of the corresponding head entity, and then select one or more candidates with the highest scores as the tail entity corresponding to the head entity.

[0045] Preferably, in step S3036, the sequence labeling model determines the candidates for the tail entity through the following steps:

[0046] 1) Construct a Bi-LSTM-CRF model and train it using a labeled dataset;

[0047] 2) Input the weighted document embedding representation into the Bi-LSTM layer of the trained Bi-LSTM-CRF model, and output a hidden state vector for each position of the target document through the Bi-LSTM layer;

[0048] 3) Use the output of the Bi-LSTM layer as the input of the CRF layer of the Bi-LSTM-CRF model, and output a label distribution for each position in the target document through the CRF layer;

[0049] 4) Based on the tag distribution output by the CRF layer, identify the entity boundaries in the target document, and then filter out the head entity itself, retaining other entities that have potential relationships with the head entity as candidates for the tail entity.

[0050] Preferably, step S4 specifically includes the following steps:

[0051] S401: Clean all head entity-relationship-tail entity triples;

[0052] S402: Use a graph database to store the cleaned triples;

[0053] S403: Create nodes for each unique head and tail entity in the knowledge graph and assign them corresponding attributes;

[0054] S404: Based on the relationship between the head entity and the tail entity in the triple, create an edge between the corresponding head entity and tail entity nodes and mark the type of the relationship.

[0055] This invention also discloses an automatic knowledge graph generation system based on document content analysis. The implementation of the automatic knowledge graph generation method based on this invention specifically includes:

[0056] The document import module is used to obtain target documents for building the knowledge graph.

[0057] The domain knowledge fusion module is used to fuse domain knowledge into the target document and generate a target document containing domain knowledge.

[0058] The entity recognition module identifies entities and relationships between entities in target documents containing domain knowledge, and generates corresponding head entity-relationship-tail entity triples.

[0059] The knowledge graph construction module is used to construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

[0060] Compared with existing technologies, the knowledge graph automatic generation method and system based on document content analysis in this invention have the following advantages:

[0061] Beneficial effects:

[0062] This invention utilizes a domain fusion model to generate target documents containing domain knowledge through domain knowledge fusion. First, domain knowledge fusion enriches the document with domain knowledge, providing more comprehensive and in-depth information for subsequent entity extraction. This allows for more accurate identification of domain-related entities during extraction, reducing the possibility of misjudgments and omissions. Furthermore, due to the fusion of domain knowledge, entity extraction can encompass more domain-specific terms and concepts, thereby improving the accuracy of automatic knowledge graph construction and enriching the knowledge graph's content. Second, domain knowledge fusion helps capture complex relationships between entities, leading to more accurate extraction of these relationships after entity extraction. Guided by domain knowledge, the relationship extraction algorithm can better understand the semantic connections between entities, thus extracting more accurate and meaningful entity relationships. These stronger entity relationships make the knowledge graph more complete and accurate, providing better support for subsequent reasoning and analysis tasks. Finally, domain knowledge integration makes the construction of knowledge graphs more systematic and standardized, ensuring that entities and relationships in the knowledge graph conform to the domain's norms and requirements. Furthermore, the constructed knowledge graph is closer to real-world application scenarios. Whether in search, question answering, recommendation, or domain-specific tasks such as decision analysis and knowledge discovery, knowledge graphs that integrate domain knowledge can provide more accurate and useful information support.

[0063] This invention uses an entity recognition model to identify entities and relationships between entities in target documents containing domain knowledge, generating several sets of head entity-relationship-tail entity triples, and then constructing a corresponding document knowledge graph. First, by generating head entity-relationship-tail entity triples, the knowledge in the document is transformed into a structured form, making the knowledge clearer, easier to understand and query. Furthermore, the structured knowledge representation helps improve the readability and interpretability of the knowledge, facilitating a better and more accurate construction of the document's knowledge graph. Second, by extracting entities and relationships between entities from the document, the content of the knowledge graph can be continuously expanded and enriched, enabling the knowledge graph to cover a wider range of domains and topics. Finally, the knowledge graph constructed based on the head entity-relationship-tail entity triples can efficiently support complex query and reasoning tasks. By traversing the entities and relationships in the graph, knowledge related to the query can be quickly found, helping to improve the accuracy and efficiency of information retrieval, thereby further enhancing the practicality of automatic knowledge graph construction. Attached Figure Description

[0064] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0065] Figure 1 This is a logical flowchart of a method for automatically generating knowledge graphs based on document content analysis. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0067] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not mean that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0068] The following detailed explanation illustrates the specific implementation methods:

[0069] Example 1:

[0070] This embodiment discloses a method for automatically generating knowledge graphs based on document content analysis.

[0071] like Figure 1 As shown, an automatic knowledge graph generation method based on document content analysis includes:

[0072] S1: Obtain the target document for constructing the knowledge graph;

[0073] S2: Use a domain fusion model to fuse domain knowledge into the target document, generating a target document containing domain knowledge;

[0074] S3: Using an entity recognition model, identify entities and relationships between entities in target documents containing domain knowledge, and generate several sets of head entity-relationship-tail entity triples;

[0075] S4: Construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

[0076] This invention utilizes a domain fusion model to generate target documents containing domain knowledge through domain knowledge fusion. First, domain knowledge fusion enriches the document with domain knowledge, providing more comprehensive and in-depth information for subsequent entity extraction. This allows for more accurate identification of domain-related entities during extraction, reducing the possibility of misjudgments and omissions. Furthermore, due to the fusion of domain knowledge, entity extraction can encompass more domain-specific terms and concepts, thereby improving the accuracy of automatic knowledge graph construction and enriching the knowledge graph's content. Second, domain knowledge fusion helps capture complex relationships between entities, leading to more accurate extraction of these relationships after entity extraction. Guided by domain knowledge, the relationship extraction algorithm can better understand the semantic connections between entities, thus extracting more accurate and meaningful entity relationships. These stronger entity relationships make the knowledge graph more complete and accurate, providing better support for subsequent reasoning and analysis tasks. Finally, domain knowledge integration makes the construction of knowledge graphs more systematic and standardized, ensuring that entities and relationships in the knowledge graph conform to the domain's norms and requirements. Furthermore, the constructed knowledge graph is closer to real-world application scenarios. Whether in search, question answering, recommendation, or domain-specific tasks such as decision analysis and knowledge discovery, knowledge graphs that integrate domain knowledge can provide more accurate and useful information support.

[0077] This invention uses an entity recognition model to identify entities and relationships between entities in target documents containing domain knowledge, generating several sets of head entity-relationship-tail entity triples, and then constructing a corresponding document knowledge graph. First, by generating head entity-relationship-tail entity triples, the knowledge in the document is transformed into a structured form, making the knowledge clearer, easier to understand and query. Furthermore, the structured knowledge representation helps improve the readability and interpretability of the knowledge, facilitating a better and more accurate construction of the document's knowledge graph. Second, by extracting entities and relationships between entities from the document, the content of the knowledge graph can be continuously expanded and enriched, enabling the knowledge graph to cover a wider range of domains and topics. Finally, the knowledge graph constructed based on the head entity-relationship-tail entity triples can efficiently support complex query and reasoning tasks. By traversing the entities and relationships in the graph, knowledge related to the query can be quickly found, helping to improve the accuracy and efficiency of information retrieval, thereby further enhancing the practicality of automatic knowledge graph construction.

[0078] In specific implementation, the processing steps of the domain fusion model are as follows:

[0079] S201: Encode the target document to generate corresponding target document word vectors;

[0080] In this embodiment, the target document is first segmented into several words; then, a corresponding word ID is assigned to each word, and the word is replaced with the corresponding word ID; finally, the corresponding word vector is embedded for each word ID to generate the corresponding target document word vector.

[0081] S202: Construct a corresponding document domain dictionary based on existing documents in the domain to which the target document belongs;

[0082] S203: Embed character vectors into each character in the document domain dictionary to generate document domain character vectors;

[0083] S204: Integrate the target document word vectors and the document domain character vectors, and use the generated document word vectors as the target document containing domain knowledge.

[0084] In this embodiment, the target document word vector and the document domain character vector can be directly concatenated: find the word vector corresponding to each word in the target document, and concatenate it with the average value of the document domain character vector of all characters in the word, so that each word is represented by a vector that combines word-level and character-level information.

[0085] In other preferred embodiments, the fusion of word vectors and character vectors can also be achieved through more complex methods:

[0086] 1) Weighted vector summation: A weight is pre-assigned to each word vector and character vector, and then they are summed according to these weights. The choice of weights can be determined based on the specific task and dataset; for example, the optimal weight combination can be found through experimentation.

[0087] 2) Using Neural Networks for Fusion: Design a neural network structure that takes word vectors and character vectors as input and allows information to interact and fuse through network layers. For example, a Convolutional Neural Network (CNN) or a Recurrent Neural Network (RNN) can be used to process the character vectors and concatenate or add their output to the word vectors. Then, further neural network layers can be used to capture higher-level feature representations.

[0088] Step S202 specifically includes the following steps:

[0089] S2021: Obtain existing documents in the domain of the target document as a corpus;

[0090] In this embodiment, the existing documents acquired should cover all aspects of the field to ensure the richness and accuracy of the dictionary. Furthermore, the quality and reliability of the existing documents should be ensured, avoiding the inclusion of errors or irrelevant information.

[0091] S2022: Preprocess existing documents, including noise removal (such as HTML tags, special characters, etc.), text cleaning (such as removing stop words, punctuation marks, etc.), and text normalization (such as converting to lowercase, unified encoding, etc.);

[0092] S2023: Perform word segmentation on the preprocessed existing documents and count the frequency of each word in the corpus after segmentation;

[0093] In this embodiment, word segmentation is a crucial step for languages ​​like Chinese that lack explicit word boundaries. Existing word segmentation tools or algorithms can be selected to segment existing documents. Word segmentation tools include HanLP and jieba: HanLP is a Chinese natural language processing toolkit developed over many years by the Social Computing and Information Retrieval Research Center of Harbin Institute of Technology, providing a one-stop Chinese natural language processing solution. It supports various word segmentation methods, such as standard word segmentation, NLP word segmentation, indexed word segmentation, N-shortest path word segmentation, CRF word segmentation, and ultra-fast dictionary word segmentation. Jieba is a Python-based Chinese word segmentation tool that supports three segmentation modes: precise mode, full mode, and search engine mode. It uses a Trie tree structure to achieve efficient word graph scanning, generating a directed acyclic graph (DAG) composed of all possible word combinations of Chinese characters in a sentence, and employs dynamic programming to find the maximum probability path, identifying the maximum segmentation combination based on word frequency. Word segmentation algorithms include dictionary-based and statistical-based algorithms. Dictionary-based algorithms, also known as string matching algorithms, include forward maximum matching, backward maximum matching, and bidirectional matching. These algorithms match the string to be matched against words in a pre-built, sufficiently large dictionary. If a word is found, the match is successful, and the word is identified. Statistical-based algorithms include Hidden Markov Models (HMMs), Conditional Random Fields (CRFs), and deep learning. They are based on large corpora and train models using statistical learning methods for word segmentation. This approach considers not only word frequency but also context, effectively eliminating ambiguity and identifying out-of-vocabulary words.

[0094] S2024: Generate the corresponding document domain dictionary based on the word frequency statistics results.

[0095] In this embodiment, the document domain dictionary may include keywords, commonly used words and their frequency information within the domain.

[0096] Step S203 specifically includes the following steps:

[0097] S2031: Select a neural network with an embedding layer as the character vector training model;

[0098] In this embodiment, recurrent neural networks (RNN), long short-term memory networks (LSTM), or Transformers can be selected as character vector training models.

[0099] S2032: Extract all unique characters from the document domain dictionary and generate the corresponding character set;

[0100] S2033: Initialize each character in the character set as a random vector in the embedding layer of the character vector training model;

[0101] In this embodiment, the dimension of the embedding layer matches the target dimension of the character vector. The dimension of the character vector can be determined based on the specific task and the size of the dataset. The higher the dimension, the richer the information it can express, but the computational complexity will also increase accordingly.

[0102] S2034: Obtain existing documents in the domain of the target document as a corpus, convert the text in the corpus into character sequences, and label each character sequence with corresponding contextual information as training data;

[0103] In this embodiment, the generation of context information can be achieved through methods such as sliding windows and n-grams (e.g., the previous and next characters of a character) in order to capture local dependencies between characters.

[0104] S2035: Input the training data into the character vector training model and calculate the corresponding model output through forward propagation;

[0105] S2036: Use a loss function to calculate the difference between the model output and the true value of the character vector training model, and update the weights of the embedding layer and other layers through the backpropagation algorithm to minimize the loss;

[0106] In this embodiment, the loss function is used to measure the difference between the model prediction and the true value, and the training objective can be to maximize the probability of the character sequence or minimize the prediction error.

[0107] S2037: Repeat steps S2035 to S2036 until the model converges or reaches the preset number of training rounds;

[0108] S2038: Extract character vectors from the embedding layer of the trained character vector training model to generate corresponding document-domain character vectors.

[0109] In this embodiment, the output document domain character vector contains the semantic information of that document domain.

[0110] In the specific implementation process, the entity recognition model processes the following steps:

[0111] S301: Use the Bi-LSTM model to capture the semantic features of the target document containing domain knowledge and generate the corresponding document context representation;

[0112] In this embodiment, using a Bi-LSTM (Bidirectional Long Short-Term Memory) model to capture semantic features and generate document context representation is a common and effective method. Bi-LSTM can simultaneously consider the preceding and following information of the input sequence, thus capturing semantic features more accurately. Specifically: the Bi-LSTM model is initialized, consisting of two LSTM layers, one processing the forward (left-to-right) direction of the input sequence and the other processing the backward (right-to-left) direction; word embedding vectors are used as input to the Bi-LSTM model, with one word embedding vector input at each time step; at each time step of the Bi-LSTM, the model updates its internal state and generates an output based on the current input and the state of the previous time step; the outputs of the forward and backward LSTM layers are concatenated at each time step to generate a representation containing preceding and following information; finally, the output of the last time step of the Bi-LSTM is selected as the context representation of the entire document.

[0113] The target document containing domain knowledge is actually the document word vector. The semantic features of the document word vector can be captured by the Bi-LSTM model, and the document context representation can be generated based on the hidden state of the last layer of the Bi-LSTM model.

[0114] The formula is expressed as:

[0115]

[0116]

[0117] In the formula: This represents the last hidden state h of the Bi-LSTM model. i and the i-th feature representation g of the global representation i The context representation (obtained by max pooling over all the last hidden states); e i It is the word representation of the i-th word embedding; d h It is the dimension of the last hidden state in the Bi-LSTM model.

[0118] S302: Calculate the corresponding entity type label sequence based on the document context representation using the trained CRF model, and identify all entities based on the entity type label sequence;

[0119] In this embodiment, the CRF model refers to a Conditional Random Field (CRF), a special type of Markov random field used for labeling or analyzing sequence data, such as natural language text or biological sequences. To train a CRF model, pre-labeled entity data needs to be constructed. This data should include the document context representation and its corresponding entity type. The document context representation is used as input to train the CRF model. The CRF model learns how to label the corresponding entity for each word based on the context information in the input sequence. During training, the CRF model learns how to calculate the probability of different labeled sequences and finds the parameters that maximize the probability of labeled sequences in the training data. Finally, the trained CRF model is used to perform entity recognition on new, unlabeled target documents. The CRF model labels each word with entity tags based on the document context representation of the target document and the learned parameters, forming an entity type label sequence.

[0120] This invention utilizes the global optimization capability of the CRF model to consider the contextual information of the entire sequence, thereby improving the accuracy of entity recognition.

[0121] S303: Sequentially identify the corresponding tail entity by treating each entity as a head entity, and identify the relationship between each head entity and its corresponding tail entity.

[0122] In this embodiment, relation extraction techniques from Natural Language Processing (NLP) can be used to identify relationships between entities. These relation extraction techniques include rule-based methods, template-based methods, and deep learning-based methods.

[0123] S304: Output all head entity-relationship-tail entity triples.

[0124] In step S303, the tail entity corresponding to the head entity is determined through the following steps:

[0125] S3031: Replace each word in the target document with a word embedding to obtain the document embedding representation;

[0126] S3032: Convert the header entity into the corresponding header entity word embedding representation;

[0127] S3033: Design an attention layer, taking the document embedding representation and the head entity word embedding representation as input to the attention layer, and calculate the similarity score between the head entity word embedding and the embedding at each position in the target document through the attention layer;

[0128] In this embodiment, the similarity score can be calculated using dot product, cosine similarity, or other similarity metrics.

[0129] S3034: Normalize the similarity scores using the softmax function to obtain the attention weights for each position;

[0130] In this embodiment, the attention weight reflects the degree of relevance between different positions in the document and the head entity.

[0131] S3035: The document embedding representation is weighted and summed using attention weights to obtain the weighted document embedding representation;

[0132] In this embodiment, the weighted document embedding representation emphasizes the document portion most relevant to the head entity.

[0133] S3036: Use sequence labeling models to determine candidate tail entities based on weighted document embedding representations;

[0134] S3037: Based on the weighted document embedding representation and the representation of the tail entity candidates, calculate the score when each candidate is used as the tail entity of the corresponding head entity, and then select one or more candidates with the highest scores as the tail entity corresponding to the head entity.

[0135] In this embodiment, the candidate score can be calculated by calculating the similarity between the representations.

[0136] In step S3036, the sequence labeling model determines the candidates for the tail entity through the following steps:

[0137] 1) Construct a Bi-LSTM-CRF model and train it using a labeled dataset;

[0138] In this embodiment, the labeled data includes head entities, tail entities, and their relationship types. During training, the model parameters, including those of the Bi-LSTM and CRF layers, are optimized using backpropagation and gradient descent. During training, an appropriate loss function (such as the negative log-likelihood loss of the CRF layer) is used to measure the difference between the model's predictions and the true labels, and model performance is optimized by minimizing the loss.

[0139] 2) Input the weighted document embedding representation into the Bi-LSTM layer of the trained Bi-LSTM-CRF model, and output a hidden state vector (which contains the context information of that position) for each position of the target document through the Bi-LSTM layer;

[0140] In this embodiment, the Bi-LSTM layer can simultaneously consider the forward and backward information of the input sequence, thereby capturing richer contextual features.

[0141] 3) Use the output of the Bi-LSTM layer as the input of the CRF layer of the Bi-LSTM-CRF model, and output a label distribution for each location in the target document through the CRF layer (these labels correspond to possible entity boundaries or types);

[0142] In this embodiment, the CRF layer considers the transition probability between tags to ensure that the output tag sequence is legal and reasonable.

[0143] 4) Based on the tag distribution output by the CRF layer, identify the entity boundaries in the target document (by finding consecutive sequences with the same entity type tags), and then filter out the head entity itself, retaining other entities that have potential relationships with the head entity as candidates for the tail entity.

[0144] In the specific implementation process, step S4 includes the following steps:

[0145] S401: Clean all head entity-relationship-tail entity triples, removing duplicate, incorrect, or irrelevant triples;

[0146] S402: Use a graph database (such as Neo4j, OrientDB, etc.) to store the cleaned triples;

[0147] In this embodiment, it is necessary to ensure that each node and edge has a unique identifier.

[0148] S403: Create nodes for each unique head and tail entity in the knowledge graph and assign corresponding attributes (such as name, type, etc.);

[0149] S404: Based on the relationship between the head entity and the tail entity in the triple, create an edge between the corresponding head entity and tail entity nodes and mark the type of the relationship.

[0150] In this embodiment, if the triple contains additional attribute information (such as the weight of the relation, confidence level, etc.), this information can be added to the edge or node.

[0151] Example 2:

[0152] This embodiment discloses an automatic knowledge graph generation system based on document content analysis, which is implemented based on the automatic knowledge graph generation method of Embodiment 1.

[0153] A knowledge graph automatic generation system based on document content analysis includes:

[0154] The document import module is used to obtain target documents for building the knowledge graph.

[0155] The domain knowledge fusion module is used to fuse domain knowledge into the target document and generate a target document containing domain knowledge.

[0156] The entity recognition module identifies entities and relationships between entities in target documents containing domain knowledge, and generates corresponding head entity-relationship-tail entity triples.

[0157] The knowledge graph construction module is used to construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

[0158] This invention utilizes a domain fusion model to generate target documents containing domain knowledge through domain knowledge fusion. First, domain knowledge fusion enriches the document with domain knowledge, providing more comprehensive and in-depth information for subsequent entity extraction. This allows for more accurate identification of domain-related entities during extraction, reducing the possibility of misjudgments and omissions. Furthermore, due to the fusion of domain knowledge, entity extraction can encompass more domain-specific terms and concepts, thereby improving the accuracy of automatic knowledge graph construction and enriching the knowledge graph's content. Second, domain knowledge fusion helps capture complex relationships between entities, leading to more accurate extraction of these relationships after entity extraction. Guided by domain knowledge, the relationship extraction algorithm can better understand the semantic connections between entities, thus extracting more accurate and meaningful entity relationships. These stronger entity relationships make the knowledge graph more complete and accurate, providing better support for subsequent reasoning and analysis tasks. Finally, domain knowledge integration makes the construction of knowledge graphs more systematic and standardized, ensuring that entities and relationships in the knowledge graph conform to the domain's norms and requirements. Furthermore, the constructed knowledge graph is closer to real-world application scenarios. Whether in search, question answering, recommendation, or domain-specific tasks such as decision analysis and knowledge discovery, knowledge graphs that integrate domain knowledge can provide more accurate and useful information support.

[0159] This invention uses an entity recognition model to identify entities and relationships between entities in target documents containing domain knowledge, generating several sets of head entity-relationship-tail entity triples, and then constructing a corresponding document knowledge graph. First, by generating head entity-relationship-tail entity triples, the knowledge in the document is transformed into a structured form, making the knowledge clearer, easier to understand and query. Furthermore, the structured knowledge representation helps improve the readability and interpretability of the knowledge, facilitating a better and more accurate construction of the document's knowledge graph. Second, by extracting entities and relationships between entities from the document, the content of the knowledge graph can be continuously expanded and enriched, enabling the knowledge graph to cover a wider range of domains and topics. Finally, the knowledge graph constructed based on the head entity-relationship-tail entity triples can efficiently support complex query and reasoning tasks. By traversing the entities and relationships in the graph, knowledge related to the query can be quickly found, helping to improve the accuracy and efficiency of information retrieval, thereby further enhancing the practicality of automatic knowledge graph construction.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for automatically generating knowledge graphs based on document content analysis, characterized in that, include: S1: Obtain the target document for constructing the knowledge graph; S2: The target document is fused with domain knowledge using a domain fusion model to generate a target document containing domain knowledge. The processing steps of the domain fusion model in step S2 are as follows: S201: Encode the target document to generate corresponding target document word vectors; S202: Construct a corresponding document domain dictionary based on existing documents in the domain to which the target document belongs; S203: Embed character vectors into each character in the document domain dictionary to generate document domain character vectors; S204: Integrate the target document word vector and the document domain character vector, and use the generated document word vector as the target document containing domain knowledge; S3: Using an entity recognition model, identify entities and relationships between entities in target documents containing domain knowledge, and generate several sets of head entity-relationship-tail entity triples; In step S3, the processing steps of the entity recognition model are as follows: S301: Use the Bi-LSTM model to capture the semantic features of the target document containing domain knowledge and generate the corresponding document context representation; S302: Calculate the corresponding entity type label sequence based on the document context representation using the trained CRF model, and identify all entities based on the entity type label sequence; S303: Sequentially identify the corresponding tail entity by treating each entity as a head entity, and identify the relationship between each head entity and its corresponding tail entity. In step S303, the tail entity corresponding to the head entity is determined through the following steps: S3031: Replace each word in the target document with a word embedding to obtain the document embedding representation; S3032: Convert the header entity into the corresponding header entity word embedding representation; S3033: The document embedding representation and the head entity word embedding representation are used as input to the attention layer. The similarity score between the head entity word embedding and the embedding at each position in the target document is calculated through the attention layer. S3034: Normalize the similarity scores using the softmax function to obtain the attention weights for each position; S3035: The document embedding representation is weighted and summed using attention weights to obtain the weighted document embedding representation; S3036: Use sequence labeling models to determine candidate tail entities based on weighted document embedding representations; S3037: Based on the weighted document embedding representation and the representation of the tail entity candidates, calculate the score when each candidate is used as the tail entity of the corresponding head entity, and then select one or more candidates with the highest scores as the tail entity corresponding to the head entity. S304: Output all head entity-relationship-tail entity triples; S4: Construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

2. The method for automatically generating knowledge graphs based on document content analysis as described in claim 1, characterized in that: In step S201, the target document is first segmented into several words; then, each word is assigned a corresponding word ID and the word is replaced with the corresponding word ID; finally, the corresponding word vector is embedded for each word ID to generate the corresponding target document word vector.

3. The method for automatically generating knowledge graphs based on document content analysis as described in claim 1, characterized in that: Step S202 specifically includes the following steps: S2021: Obtain existing documents in the domain of the target document as a corpus; S2022: Preprocess existing documents, including noise removal, text cleaning, and text normalization; S2023: Perform word segmentation on the preprocessed existing documents and count the frequency of each word in the corpus after segmentation; S2024: Generate the corresponding document domain dictionary based on the word frequency statistics results.

4. The method for automatically generating knowledge graphs based on document content analysis as described in claim 1, characterized in that: Step S203 specifically includes the following steps: S2031: Select a neural network with an embedding layer as the character vector training model; S2032: Extract all unique characters from the document domain dictionary and generate the corresponding character set; S2033: Initialize each character in the character set as a random vector in the embedding layer of the character vector training model; S2034: Obtain existing documents in the domain of the target document as a corpus, convert the text in the corpus into character sequences, and label each character sequence with corresponding contextual information as training data; S2035: Input the training data into the character vector training model and calculate the corresponding model output through forward propagation; S2036: Use a loss function to calculate the difference between the model output and the true value of the character vector training model, and update the weights of the embedding layer and other layers through the backpropagation algorithm to minimize the loss; S2037: Repeat steps S2035 to S2036 until the model converges or reaches the preset number of training rounds; S2038: Extract character vectors from the embedding layer of the trained character vector training model to generate corresponding document-domain character vectors.

5. The method for automatically generating knowledge graphs based on document content analysis as described in claim 1, characterized in that: In step S3036, the sequence labeling model determines the candidates for the tail entity through the following steps: 1) Construct a Bi-LSTM-CRF model and train it using a labeled dataset; 2) Input the weighted document embedding representation into the Bi-LSTM layer of the trained Bi-LSTM-CRF model, and output a hidden state vector for each position of the target document through the Bi-LSTM layer; 3) Use the output of the Bi-LSTM layer as the input of the CRF layer of the Bi-LSTM-CRF model, and output a label distribution for each position in the target document through the CRF layer; 4) Based on the tag distribution output by the CRF layer, identify the entity boundaries in the target document, and then filter out the head entity itself, retaining other entities that have potential relationships with the head entity as candidates for the tail entity.

6. The method for automatically generating knowledge graphs based on document content analysis as described in claim 1, characterized in that: Step S4 specifically includes the following steps: S401: Clean all head entity-relationship-tail entity triples; S402: Use a graph database to store the cleaned triples; S403: Create nodes for each unique head and tail entity in the knowledge graph and assign them corresponding attributes; S404: Based on the relationship between the head entity and the tail entity in the triple, create an edge between the corresponding head entity and tail entity nodes and mark the type of the relationship.

7. A knowledge graph automatic generation system based on document content analysis, characterized in that: The implementation based on the knowledge graph automatic generation method described in claim 1 specifically includes: The document import module is used to obtain target documents for building the knowledge graph. The domain knowledge fusion module is used to fuse domain knowledge into the target document and generate a target document containing domain knowledge. The processing steps of the domain knowledge fusion module are as follows: S201: Encode the target document to generate corresponding target document word vectors; S202: Construct a corresponding document domain dictionary based on existing documents in the domain to which the target document belongs; S203: Embed character vectors into each character in the document domain dictionary to generate document domain character vectors; S204: Integrate the target document word vector and the document domain character vector, and use the generated document word vector as the target document containing domain knowledge; The entity recognition module identifies entities and relationships between entities in target documents containing domain knowledge, and generates corresponding head entity-relationship-tail entity triples. The processing steps of the entity recognition module are as follows: S301: Use the Bi-LSTM model to capture the semantic features of the target document containing domain knowledge and generate the corresponding document context representation; S302: Calculate the corresponding entity type label sequence based on the document context representation using the trained CRF model, and identify all entities based on the entity type label sequence; S303: Sequentially identify the corresponding tail entity by treating each entity as a head entity, and identify the relationship between each head entity and its corresponding tail entity. In step S303, the tail entity corresponding to the head entity is determined through the following steps: S3031: Replace each word in the target document with a word embedding to obtain the document embedding representation; S3032: Convert the header entity into the corresponding header entity word embedding representation; S3033: The document embedding representation and the head entity word embedding representation are used as input to the attention layer. The similarity score between the head entity word embedding and the embedding at each position in the target document is calculated through the attention layer. S3034: Normalize the similarity scores using the softmax function to obtain the attention weights for each position; S3035: The document embedding representation is weighted and summed using attention weights to obtain the weighted document embedding representation; S3036: Use sequence labeling models to determine candidate tail entities based on weighted document embedding representations; S3037: Based on the weighted document embedding representation and the representation of the tail entity candidates, calculate the score when each candidate is used as the tail entity of the corresponding head entity, and then select one or more candidates with the highest scores as the tail entity corresponding to the head entity. S304: Output all head entity-relationship-tail entity triples; The knowledge graph construction module is used to construct the corresponding document knowledge graph based on all head entity-relationship-tail entity triples.

Images