A data integration method based on knowledge graph

Abstract

The present application discloses a data integration method based on knowledge graph, which relates to the field of knowledge graph technology, including: inputting text data into a large language model based on Prompt; the Qwen‑14B pre-training model uses a self-attention mechanism to encode the input text data and generate semantic vectors at the word and sentence levels; according to the generated semantic vectors, entities and relationships are obtained; structured knowledge in the form of subject-predicate-object triples SPO is extracted; SPO is mapped to a knowledge graph of a graph database Neo4j, and Neo4jCypher statements are used to map the subject and object entities in SPO to nodes; the predicate relationship in SPO is mapped to the relationship between nodes; a graph storage engine based on Neo4j is used to store the mapped knowledge graph in a graph database, and the graph database uses an adjacency list structure; and data integration is performed using the knowledge graph. In view of the problem of low data integration efficiency in the prior art, the present application improves the integration efficiency of unstructured text data.

Description

Technical Field

[0001] The present application relates to the field of knowledge graph technology, and in particular to a data integration method based on knowledge graph. Background Art

[0002] As a structured knowledge representation and storage method, knowledge graphs can manage and retrieve knowledge more efficiently and flexibly by organizing entities, relationships, and attributes in the form of graphs. Integrating unstructured text data into knowledge graphs can achieve unified representation, association analysis, and intelligent application of knowledge, and has broad application prospects, such as intelligent search, question-answering systems, and recommendation systems.

[0003] However, integrating unstructured text data into knowledge graphs faces many challenges. First, unstructured text data lacks clear structural and semantic information, and it is difficult to accurately identify entities and relationships in text. Second, the amount of text data is huge and covers a wide range of fields. The cost and cycle of manually constructing knowledge graphs are high, and it is difficult to meet the needs of practical applications. In addition, the storage and query of knowledge graphs also face challenges in efficiency and scalability.

[0004] In the related technology, for example, Chinese patent document CN117150050B provides a knowledge graph construction method and system based on a large language model on December 1, 2023, which belongs to the field of text processing technology. The method includes: performing text clustering on knowledge text data to obtain a knowledge text data set T of several different text types; submitting the knowledge text data set T to the first HDFS for distributed file storage; extracting the knowledge text type from the first HDFS in order according to the length of the knowledge text type, and using the preset large language model CoT to perform knowledge entity recognition on the extracted knowledge text type to obtain the association information of each knowledge entity; submitting the association information of each knowledge entity to the second HDFS for distributed file storage; according to the association information of each knowledge entity stored in the second HDFS, constructing the graph node links between each knowledge entity to obtain the knowledge graph. However, this solution only uses HDFS for distributed file storage, and faces efficiency bottlenecks when querying complex graph data. Summary of the invention

[0005] 1. Technical problems to be solved

[0006] In response to the problem of low efficiency in integrating unstructured text data in knowledge graphs in the prior art, the present application provides a data integration method based on knowledge graphs, which improves the integration efficiency of unstructured text data by performing entity recognition and relationship extraction through a large language model.

[0007] 2. Technical solution

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] The present specification provides a data integration method based on a knowledge graph, including: obtaining unstructured text data; inputting the obtained text data into a large language model based on Prompt for feature extraction; wherein the large language model adopts a Qwen-14B pre-trained model with a general meaning of hundreds of billions of parameters, and adopts a self-attention mechanism to encode the input text data to generate semantic vectors at the word and sentence levels; according to the generated semantic vectors, a named entity recognition algorithm is used to identify entities in the text, and the steps of the named entity recognition algorithm include: extracting features from the text data, extracting character features and part-of-speech features, the character features include vector representations of characters, and the part-of-speech features include vector representations of parts of speech; extracting the extracted The character features and part-of-speech features are combined into a feature vector sequence A1 according to the order in the text as the input of the Bi-LSTM network; the feature vector sequence A1 is input into the Bi-LSTM network, and the context representation of characters and parts of speech is learned from both the forward and backward directions through the bidirectional LSTM layer. The bidirectional LSTM layer is composed of multiple LSTM units, each of which receives the input of the current time step and the hidden state of the previous time step, and generates the hidden state and output of the current time step. The outputs of the forward LSTM and the backward LSTM are concatenated into the output vector sequence of the Bi-LSTM network; the output vector sequence of the Bi-LSTM network is used as the emission matrix of the conditional random field CRF layer, and the C The RF layer learns the transfer matrix and state sequence of BIO annotations, and the CRF layer uses the transfer feature function and the state feature function to calculate the dependency between labels and between labels and observations respectively; among them, the BIO annotation is used to mark the boundary and type of the entity in the text, B indicates the beginning of the entity, I indicates the inside of the entity, and O indicates the outside of the entity; in the conditional random field CRF layer, the state sequence output by the Bi-LSTM network is decoded, and the Viterbi algorithm is used to obtain the state sequence with the highest probability; according to the state sequence and BIO labels obtained by the Viterbi algorithm, the entities identified in the text data and the corresponding types are obtained; the dependency syntax and semantic role labeling method are used to obtain the relationship between entities, and the steps of the method include: decoding the text This data is used for feature extraction, and part-of-speech features and word features are extracted. The part-of-speech features include the vector representation of the part-of-speech, and the word features include the vector representation of the word. The extracted part-of-speech features and word features are combined into a feature vector sequence A2 in the order in the text as the input of the Stack-Pointer network. The feature vector sequence A2 is input into the Stack-Pointer network, and the output of the Stack-Pointer network is decoded using a transfer decoding algorithm to obtain the head word index and dependency label of each word, and a syntactic dependency tree is constructed. The part-of-speech feature vector sequence in the extracted feature vector sequence A2 and the constructed syntactic dependency tree are used as the input of the Bi-LSTM network.The syntactic dependency tree and part-of-speech feature vector sequence are input into the Bi-LSTM network, and the semantic role representation of words is learned through the bidirectional LSTM layer; the output vector of the bidirectional LSTM layer is input into the multi-layer perceptron, and the semantic role of each word is classified through the multi-layer perceptron; the output probability of the multi-layer perceptron is converted into the emission matrix of semantic role labeling, and the transfer matrix of semantic role labeling is defined; the Viterbi algorithm is used to decode the emission matrix and the transfer matrix to obtain the semantic role argument and type of each word; according to the syntactic dependency tree, argument boundary and semantic role type dictionary, the semantic role argument is filtered to obtain the semantic role labeling result of each word in the sentence; according to the acquired entities and relations, the structured knowledge in the form of subject-predicate-object triples SPO is extracted. ;

[0010] The extracted triples SPO are mapped to the knowledge graph of the graph database Neo4j. The Neo4jCypher statement is used to map the subject and object entities in the triples SPO to nodes. The attributes of the nodes include the metadata of the entity noun and entity type; the predicate relationship in the triples SPO is mapped to the relationship between nodes. The attributes of the relationship include the metadata of the relationship type and the relationship strength; the Neo4j-based graph storage engine is used to store the mapped knowledge graph in the graph database. The graph database uses an adjacency list structure to store nodes and relationships separately; the graph database storing the knowledge graph is used for data integration. The steps of data integration include: using Neo4jCypher statements to retrieve and analyze the knowledge graph stored in the graph database; using the MATCH command to match the query graph pattern, the WHERE command to filter the matched graph pattern, and the RETURN command to return the query result; using the CREATE command to create new nodes and relationships, and the MERGE command to update existing nodes and relationships; using the query graph pattern as input, calling Page through the Neo4jCypher statement The Rank algorithm calculates the importance of the node, and the Louvain algorithm is called to obtain the community label of the node; the Neo4jETL tool is used to import the external data to be integrated into the graph database, and the mapping logic between the external data and the graph database is defined through the Neo4jCypher statement, and the external data and the graph database are associated using the mapping logic. This application obtains unstructured text, uses a large language model to extract semantic features, identifies entities and relationships, builds a knowledge graph, stores it in a graph database, and uses the graph database for data integration, thereby achieving automated extraction and integration from unstructured data to structured knowledge, expanding the scale and quality of the knowledge graph, and improving the efficiency and accuracy of data integration.

[0011] Furthermore, a graph database storing knowledge graphs is used for data integration, including: using Neo4jCypher statements to retrieve and analyze the knowledge graphs stored in the graph database, the Neo4jCypher statements include MATCH commands, WHERE commands, RETURN commands, CREATE commands and MERGE commands; using the MATCH command to match the queried graph pattern, the graph pattern consists of nodes, relationships, node attribute conditions and relationship attribute conditions, the node attribute conditions are used to match nodes of attribute values, and the relationship attribute conditions are used to match relationships of attribute values; wherein the query conditions of the graph pattern are derived from user input or query requests of applications, and the MATCH command searches for matching subgraphs in the graph database according to the query conditions; using the WHERE command to filter the matching graph pattern, and screening the matching nodes and relationships according to the node attribute conditions, relationship attribute conditions and topological structure conditions, the topological structure conditions including the degree of the node and the path length; wherein the WHERE command further filters the nodes and relationships in the subgraph according to the attribute conditions and the structure conditions based on the subgraph matched by the MATCH command to obtain the target subgraph that meets the conditions.

[0012] The query results are returned with the RETURN command, which includes the matched nodes, relationships, and properties of nodes and relationships. The RETURN command returns the nodes, relationships, and properties in the subgraph according to the target subgraph filtered by the WHERE command, and outputs them to the user or application as the query results. The CREATE command is used to create new nodes and relationships. According to the entities and relationships in the external data, the corresponding nodes and relationships are created in the graph database, and the properties of the nodes and relationships are set. The external data comes from heterogeneous data sources such as relational databases, NoSQL databases, RDF data, and CSV files. The TE command converts external data into nodes and relationships of a graph database and stores them in the graph database to expand the scale of the knowledge graph; the MERGE command is used to update existing nodes and relationships, and to search for matching nodes and relationships in the graph database based on the entities and relationships in the external data. If found, the properties of the corresponding nodes or relationships are updated, and if not found, new nodes and relationships are created; wherein, the MERGE command merges external data with the existing knowledge graph in the graph database, updates the property information of existing entities and relationships, and creates new nodes and relationships for new entities and relationships, thereby updating and completing the knowledge graph. This application uses the graph database query language Neo4jCypher, uses the MATCH, WHERE, and RETURN commands to retrieve and analyze the knowledge graph, and uses the CREATE and MERGE commands to construct and update the knowledge graph, thereby realizing flexible query, integrated fusion, and continuous evolution of knowledge graph data, and improving the query performance and storage efficiency of data integration.

[0013] Furthermore, data integration is performed using a graph database storing knowledge graphs, and the method also includes: using the graph pattern obtained by querying the RETURN command as input, and calling the Page Rank algorithm and the louvain algorithm through the Neo4jCypher statement; wherein the graph pattern contains matching nodes, relationships and their attributes, which serve as input data for the Page Rank algorithm and the louvain algorithm; calling the Page Rank algorithm to calculate the importance measure of the node according to the topological structure of the node in the graph pattern; wherein the Page Rank algorithm iteratively calculates the Page Rank value of each node by calculating the in-degree, out-degree and importance of neighboring nodes of the node, and the Page Rank value reflects the importance of the node in the entire graph structure, and the higher the Page Rank value, the more important the node; calling the louvain algorithm to obtain the community label of the node according to the connection relationship of the node in the graph pattern; wherein the louvain algorithm divides the nodes in the graph into multiple communities by maximizing the modularity, so that the nodes within the same community are closely connected, and the nodes between different communities are sparsely connected, and each node is marked with the label of the community to which it belongs.

[0014] The node importance measurement Page Rank value calculated by the Page Rank algorithm and the node community label obtained by the louvain algorithm are added as new attributes to the corresponding nodes, and the node Page Rank attribute and community attribute are stored in the graph database through the CREATE command or the MERGE command; in the subsequent knowledge graph query and analysis, the user or application can sort, filter and cluster the query results according to the Page Rank attribute and community attribute of the node, for example, select important nodes according to the Page Rank value, and select nodes in the same community according to the community label, so as to improve the pertinence and effectiveness of the query and analysis. This application expands the semantic information of the knowledge graph by calling the graph algorithms Page Rank and louvain in the process of knowledge graph data integration, calculating the importance measurement and community label of the node, and storing the calculation results as node attributes in the graph database, providing a new attribute dimension for subsequent knowledge graph query and analysis, and improving the intelligence level and application value of data integration.

[0015] Furthermore, using a graph database storing knowledge graphs for data integration also includes: using the Neo4jETL tool to import external data to be integrated into the graph database; wherein the external data comes from heterogeneous data sources such as relational databases, NoSQL databases, RDF data, CSV files, etc., and the Neo4jETL tool supports the connection and data import of multiple data sources, and converts the external data into the node and relationship format of the graph database; defining the mapping logic between the external data and the graph database through the Neo4jCypher statement, and the mapping logic includes the conversion rules from the external data to the nodes and relationships of the graph database; wherein the mapping logic maps the external data to the node types and relationship types in the graph database according to the pattern and semantics of the external data, determines that the entities in the external data correspond to the nodes in the graph database, and the relationships in the external data correspond to the relationships in the graph database, and specifies the attribute mapping rules of the nodes and relationships.

[0016] Use mapping logic to associate external data with the graph database, specifically including: according to the mapping logic, map entities in the external data to nodes in the graph database, map entity attributes to node attributes, and create new nodes or update existing nodes through the CREATE command or the MERGE command; according to the mapping logic, map relationships in the external data to relationships in the graph database, map relationship attributes to relationship attributes, and create new relationships or update existing relationships through the CREATE command or the MERGE command; when creating nodes and relationships, use the MATCH command to retrieve existing nodes and relationships, and match them through the WHERE command and attribute conditions to ensure that the newly created nodes and relationships are associated and integrated with the existing knowledge graph; through the Neo4jETL tool and Cypher mapping logic, continuously import and integrate external data into the graph database to achieve dynamic update and expansion of the knowledge graph; at the same time, the Page Rank algorithm and the louvain algorithm measure the importance and calculate community tags for the newly integrated nodes and relationships to maintain the semantic richness and analytical value of the knowledge graph. This application uses the Neo4jETL tool and Cypher mapping logic to seamlessly connect and semantically map external heterogeneous data sources with the graph database, realize automatic and standardized data integration of the knowledge graph, improve the efficiency and quality of data integration, expand the scale and coverage of the knowledge graph, and provide more extensive and in-depth knowledge support for intelligent applications.

[0017] Furthermore, obtaining entities includes: performing feature extraction on text data, extracting character features and part-of-speech features, wherein the character features include vector representations of characters, and the part-of-speech features include vector representations of parts of speech; wherein the text data comes from structured or unstructured text files, such as web pages, news, papers, etc., and the feature extraction process converts the text data into numerical vector representations that can be processed by the machine learning model; the extracted character features and part-of-speech features are combined into a feature vector sequence A1 according to the order in the text as the input of the Bi-LSTM network; wherein each element in the feature vector sequence A1 corresponds to a character or part of speech in the text, and the order of the elements reflects the time series characteristics of the text, and the Bi-LSTM network can Effectively model the context information of the text; input the feature vector sequence A1 into the Bi-LSTM network, and learn the context representation of characters and parts of speech from both forward and backward directions through the bidirectional LSTM layer. The bidirectional LSTM layer is composed of multiple LSTM units, each LSTM unit receives the input of the current time step and the hidden state of the previous time step, and generates the hidden state and output of the current time step, and concatenates the outputs of the forward LSTM and the backward LSTM into the output vector sequence of the Bi-LSTM network; among them, the forward LSTM learns the context information from the beginning to the end of the text, and the backward LSTM learns the context information from the end to the beginning of the text. The bidirectional LSTM can capture the semantic features of the text more comprehensively.

[0018] The output vector sequence of the Bi-LSTM network is used as the emission matrix of the conditional random field CRF layer. The CRF layer learns the transfer matrix and state sequence of the BIO annotation. The CRF layer uses the transfer feature function and the state feature function to calculate the dependency between labels and between labels and observations respectively. Among them, the emission matrix represents the dependency between the observation sequence and the label sequence, and the transfer matrix represents the dependency between adjacent labels. The CRF layer solves the optimal label sequence through the dynamic programming algorithm to realize the boundary and type recognition of the entity. The BIO annotation sequence output by the CRF layer is decoded to extract the entity and the corresponding type label. Among them, the BIO annotation is used to mark the edge of the entity in the text. Boundary and type, B indicates the beginning of the entity, I indicates the inside of the entity, O indicates the outside of the entity, continuous BI annotations indicate a complete entity, and the entity type is indicated by the BI annotation suffix; the extracted entities are mapped to the normalized entities in the knowledge graph, and the corresponding relationship between the entity mention and the normalized entity is created through the MERGE command of the graph database, and the text-based entity recognition results are fused and associated with the knowledge graph to expand the entity information of the knowledge graph; wherein, entity mention refers to the entity name in the text, and the normalized entity refers to the standard entity in the knowledge graph. The entity mapping process needs to solve the problems of homonymous entity disambiguation and reference resolution to ensure the consistency of text entities and knowledge graph entities. This application proposes an entity recognition method based on feature learning and sequence labeling, which learns the contextual representation of the text through the Bi-LSTM network, and learns the boundary and type annotation of the entity through the CRF layer, so as to achieve efficient and accurate entity acquisition, and map and associate the entity recognition results with the knowledge graph, enriching the entity information of the knowledge graph, and providing data support for subsequent relationship extraction and attribute completion.

[0019] Furthermore, obtaining the entity also includes: in the conditional random field CRF layer, decoding the state sequence output by the Bi-LSTM network, and using the Viterbi algorithm to obtain the state sequence with the highest probability; the Viterbi algorithm finds the state path with the highest probability through dynamic programming, the state of the dynamic programming is the label of each position, the state transition is the label of the previous position to the label of the next position, the transition probability is the transition feature function learned by the CRF layer, the emission probability is the state feature function learned by the CRF layer, the score of each state sequence is calculated by the product of the transition probability and the emission probability, and the backtracking pointer of the state sequence is recorded; wherein, the state The sequence corresponds to the character sequence in the text, and each state represents the BIO label of the corresponding character. The Viterbi algorithm achieves the global optimal sequence labeling by maximizing the conditional probability of the label sequence. The state sequence with the highest probability is obtained according to the backtracking pointer, and the BIO labels of the characters corresponding to each state in the state sequence are labeled. The BIO labels are used to mark the boundaries and types of entities in the text. Among them, the backtracking pointer records the transfer path of the optimal state sequence. Starting from the terminal state, traversing in reverse along the backtracking pointer can obtain the complete optimal state sequence, correspond the state sequence to the character sequence, and complete the BIO label labeling of the characters.

[0020] According to the boundary information of the BIO tag, the B tag and the adjacent I tag after the B tag in the state sequence are merged into one entity until an O tag or a different type of B tag is encountered; wherein the B tag indicates the beginning of the entity, the I tag indicates the inside of the entity, and the O tag indicates the outside of the entity. The adjacent BI tags constitute a complete entity fragment. By identifying the boundary of the BI tag, the character sequence can be divided into entity and non-entity parts; according to the type information of the BIO tag, the type of each entity is determined. The BIO tag contains the letter code of the entity type. According to the mapping relationship between the letter code and the predefined entity type, the letter code is converted into the entity type; wherein the entity type is represented by the suffix of the BI tag, For example, B-PER represents the beginning of a name entity, and I-PER represents the inside of a name entity. By constructing a mapping dictionary between letter codes and entity types, the BIO tag can be converted into an actual entity type tag. The boundaries and types of the obtained entities are combined to obtain the entities and corresponding types identified in the text data. Among them, the boundaries of the entities are determined by the position of the BI tags, and the types of the entities are determined by the suffix of the BI tags. The character sequence within the entity boundaries is used as the entity name, and the entity type is used as the semantic label of the entity to form a complete entity recognition result. The identified entities and types are subjected to entity mapping and knowledge graph fusion to complete the entity acquisition and integration from text data to the knowledge graph.

[0021] Furthermore, the relationship between entities is obtained, including: extracting features from text data, extracting part-of-speech features and word features, the part-of-speech features including the vector representation of the part-of-speech, and the word features including the vector representation of the word; wherein the part-of-speech features and word features respectively represent the part-of-speech and semantic information of each word in the sentence, providing grammatical and semantic clues for relationship extraction; the extracted part-of-speech features and word features are combined into a feature vector sequence A2 according to the order in the text, as the input of the Stack-Pointer network; wherein each element in the feature vector sequence A2 corresponds to a word in the sentence, the order of the elements reflects the word order characteristics of the sentence, and the Stack-Pointer network can model the grammatical and semantic structure information of the sentence; The feature vector sequence A2 is input into the Stack-Pointer network, which consists of a stacked LSTM layer and a pointer-type attention layer. The stacked LSTM layer pushes the input feature vector sequence A2 into the stack and encodes it into a hidden state. The pointer-type attention layer calculates the head word index of each word and the probability distribution of the dependency label according to the hidden state of the stack; the head word index represents the position index of the parent node of each word in the syntactic dependency tree in the input sequence; among them, the stacked LSTM layer simulates the shift-reduce process in syntactic analysis and constructs the syntactic structure of the sentence from the bottom up, and the pointer-type attention layer learns the long-distance dependency relationship between words through the Attention mechanism.

[0022] A transfer decoding algorithm is used to decode the output of the Stack-Pointer network. The transfer decoding algorithm moves and pops on the input feature vector sequence A2 through transfer operations and stack operations, and selects the head word index and dependency label according to the probability distribution of the attention layer to obtain the head word index and dependency label of each word; wherein, the transfer operation includes left arc, right arc and shift, which respectively represent the establishment of left-pointing and right-pointing dependency arcs in the syntax tree, or the movement of the current word without establishing a dependency arc, and the stack operation includes push and pop, which respectively represent the pushing or popping of words into or out of the stack. The transfer decoding algorithm obtains the global optimal dependency analysis result by maximizing the probability of the correct transfer sequence; a syntactic dependency tree is constructed according to the head word index and dependency label of each word, and the nodes in the syntactic dependency tree represent words, the directed edges represent the dependency relationship between words, the direction of the edge represents the direction of the dependency relationship, and the edge The label indicates the type of dependency relationship; the head word index determines the direction of the edge in the syntax tree, and the dependency label determines the type of the edge in the syntax tree. By connecting the word nodes according to the head word index and assigning dependency labels, the syntactic dependency tree representation of the sentence can be obtained; according to the predefined dependency relationship template, the relationship between entities is extracted from the syntactic dependency tree. The dependency relationship template defines the relationship type corresponding to different dependency paths. By matching the dependency relationship template in the syntactic dependency tree, the semantic relationship between entities can be identified; the extracted entity relationship is formed into triples, each triple includes a head entity, a tail entity and a relationship type, and the entities in the triples are mapped to the normalized entities in the knowledge graph. Through the MERGE command of the graph database, the edges between the head entity, the tail entity and the relationship are created, and the text-based relationship extraction results are fused and associated with the knowledge graph to expand the relationship information of the knowledge graph. This application proposes a relationship extraction method based on syntactic analysis and dependency parsing. The dependency structure of sentences is learned through the Stack-Pointer network, the globally optimal syntactic dependency tree is obtained through the transfer decoding algorithm, and the semantic relationship between entities is identified from the syntactic dependency tree using the predefined dependency relationship template, thus achieving efficient and accurate relationship extraction. At the same time, the relationship extraction results are fused and associated with the knowledge graph, enriching the relationship information of the knowledge graph and providing structured knowledge support for intelligent applications.

[0023] Furthermore, obtaining the relationship between entities also includes: using the part-of-speech feature vector sequence in the extracted feature vector sequence A2 and the constructed syntactic dependency tree as the input of the Bi-LSTM network, the syntactic dependency tree provides the syntactic structure information between words, and the part-of-speech feature vector sequence provides the syntactic attribute information of the words; wherein, the syntactic dependency tree and the part-of-speech feature vector sequence are combined as the input of the Bi-LSTM network, which can simultaneously model the syntactic structure and syntactic attributes of the sentence and improve the accuracy of semantic role labeling; the syntactic dependency tree and the part-of-speech feature vector sequence are input into the Bi-LSTM network, and the semantic role representation of the word is learned from the syntactic structure and syntactic attributes through the bidirectional LSTM layer, and the output vector sequence of the bidirectional LSTM layer integrates the contextual information of the syntactic structure and syntactic attributes; wherein, the syntactic dependency tree is encoded in a manner similar to TreeLSTM, and the dependency relationship of the words is used as the topological structure of the LSTM unit, and the part-of-speech feature vector sequence is encoded through the time-based LSTM, and the bidirectional LSTM layer is used to store the word semantic role information. The LSTM layer learns the context representation of words from the forward and reverse directions, and concatenates the two to obtain the semantic role features of the words; the output vector of the bidirectional LSTM layer is input into the multi-layer perceptron, and the semantic role of each word is classified by the multi-layer perceptron. The output of the multi-layer perceptron is the probability distribution of each word belonging to different semantic role types; among them, the semantic role types include agent, patient, time, place, etc. Each word may have multiple semantic roles, so multi-label classification is required. The multi-layer perceptron maps the feature vector to a probability distribution through the Softmax function, and the type with the largest probability is used as the semantic role label of the word; according to the semantic role probability distribution output by the multi-layer perceptron, each word is semantically labeled to obtain the semantic role labeling result of the sentence; among them, semantic role labeling is to assign a semantic role label to each word on the basis of the syntactic dependency tree. The label represents the semantic component of the word in the sentence. Through semantic role labeling, the role positioning of the word in the relationship can be clarified.

[0024] Preferably, the relationship between entities is extracted from the semantic role labeling results according to a predefined semantic role combination template, and the semantic role combination template defines the relationship type corresponding to the combination of different semantic roles. By matching the semantic role combination template in the semantic role labeling results, the semantic relationship between entities can be identified, and the relationship type is determined by the semantic role combination template. The extracted entity relationships are formed into triples, each triple includes a head entity, a tail entity and a relationship type, and the semantic roles of the head entity and the tail entity determine the direction of the relationship. The entities in the triples are mapped to the normalized entities in the knowledge graph, and the edges between the head entity, the tail entity and the relationship are created through the MERGE command of the graph database. The relationship extraction results based on semantic role labeling are fused and associated with the knowledge graph to expand the relationship information of the knowledge graph. A relationship extraction method based on semantic role labeling is introduced, and the semantic role representation of sentences is learned through a Bi-LSTM network, the semantic roles of words are classified through a multi-layer perceptron, and the semantic relationship between entities is identified from the semantic role labeling results using a predefined semantic role combination template, as a supplement to syntactic dependency analysis, to further improve the recall rate of relationship extraction.

[0025] Furthermore, obtaining the relationship between entities also includes: converting the output probability of the multi-layer perceptron into an emission matrix of semantic role labeling, the rows of the emission matrix represent words, the columns represent BIO tags, and the elements represent the probability of words being labeled with corresponding BIO tags; wherein the BIO tags are used to mark the boundaries and types of semantic role arguments, B represents the beginning of the argument, I represents the inside of the argument, and O represents the outside of the argument. The emission matrix represents the emission probability of each word corresponding to different BIO tags, and the emission probability is obtained by the Softmax probability output by the multi-layer perceptron; defining a transfer matrix of semantic role labeling, the rows and columns of the transfer matrix represent BIO tags, the elements represent the probability of the previous BIO tag transferring to the next BIO tag, and the transfer probability is estimated based on the prior knowledge of semantic role labeling in the corpus; wherein the transfer matrix represents the transfer constraint relationship between different BIO tags, for example, the I label can only follow the B label or The O label cannot follow the I label of the same type. The transition frequency of BIO labels in the corpus can be used to estimate the value of the transition probability. The Viterbi algorithm is used to decode on the emission matrix and the transition matrix. The Viterbi algorithm finds the BIO label sequence with the highest probability through dynamic programming. The state of dynamic programming is the BIO label at each position. The state transition is from the BIO label at the previous position to the BIO label at the next position. The state transition probability is the element of the transition matrix, and the emission probability is the element of the emission matrix. The score of each state sequence is calculated by the product of the state transition probability and the emission probability, and the backtracking pointer of the state sequence is recorded. Among them, the Viterbi algorithm comprehensively considers the emission probability and the transition probability, and finds the global optimal BIO label sequence under the premise of satisfying the transition constraints. The backtracking pointer records the transition path of the optimal state sequence, which is used to generate a complete label sequence in reverse from the terminal state.

[0026] According to the backtracking pointer, the BIO tag sequence with the highest probability is obtained, and the consecutive BIO tags of the same type in the BIO tag sequence are merged into the boundary of the argument. The boundary of the argument is determined by the B tag and the I tag, and the words within the argument boundary are taken as candidate arguments; among them, the B tag indicates the starting position of the semantic role argument, and the I tag indicates the middle and end position of the semantic role argument. The consecutive BI tags define a complete semantic role argument. The words inside the argument constitute the candidate argument, and the candidate argument is the basic unit of relation extraction; the semantic role type of the candidate argument is determined according to the type of the BIO tag, and the type of the BIO tag contains the type information of the semantic role. The semantic role type of the candidate argument is matched with the semantic role type of the corresponding central word, and the unmatched candidate arguments are filtered out to obtain the semantic role argument and type to which each word belongs; among them, the central word refers to the predicate or core verb in the sentence, which is usually the attachment center of the semantic role, and the language of the candidate argument is the semantic role type of the candidate argument. The semantic role type needs to be consistent with the semantic role type of the central word, otherwise invalid semantic role arguments may be generated and need to be filtered; according to the predefined semantic role combination template, the relationship between entities is extracted from the obtained semantic role arguments and types. The semantic role combination template defines the relationship type corresponding to the combination of different semantic role arguments. By matching the semantic role combination template in the semantic role argument, the semantic relationship between entities can be identified. The relationship type is determined by the semantic role combination template; the extracted entity relationship is formed into triples, each triple includes a head entity, a tail entity and a relationship type. The head entity and the tail entity are respectively represented by the core arguments in the semantic role argument. The entities in the triples are mapped to the normalized entities in the knowledge graph. Through the MERGE command of the graph database, the edges between the head entity, the tail entity and the relationship are created. The relationship extraction results based on semantic role annotation are fused and associated with the knowledge graph to expand the relationship information of the knowledge graph. This application optimizes the relationship extraction method based on semantic role labeling. By introducing the emission matrix and the transfer matrix, and using the Viterbi algorithm to decode the BIO label sequence, it overcomes the defect that the semantic role labeling method based on word classification cannot characterize the constraint relationship between labels, and improves the accuracy and coherence of semantic role argument recognition. At the same time, by matching the semantic role combination template, the relationship between entities is extracted from the semantic role argument, and the extraction results are fused and associated with the knowledge graph, continuously expanding and updating the relationship information of the knowledge graph.

[0027] Furthermore, the semantic role types of the candidate arguments are matched with the semantic role types of the corresponding central words, and the unmatched candidate arguments are filtered out to obtain the semantic role arguments and types to which each word belongs, including: determining the central word of each candidate argument according to the syntactic dependency tree and argument boundaries, where the central word represents the word on which the candidate argument depends; wherein, by locating the position of the candidate argument in the syntactic dependency tree, finding the parent node on which the candidate argument depends in the syntactic tree, and taking the word corresponding to the parent node as the central word of the candidate argument, where the central word is usually the core component of the semantic role assignment of the candidate argument; obtaining the semantic role type of each central word according to a semantic role-annotated dictionary and corpus, where the dictionary defines the semantic role type of each word, and the corpus obtains the frequency and probability of each word assuming different semantic role types through statistical methods; wherein, the semantic role-annotated dictionary is a pre-constructed The corpus is a large-scale annotated data set. By counting the frequency of each word playing different semantic role types in different corpora and dividing it by the total frequency of the word, the probability distribution of the word playing each semantic role type can be obtained; the cosine similarity between the semantic role type of the candidate argument and the semantic role type of the corresponding central word is calculated; among them, the semantic role type vector of the candidate argument and the semantic role type vector of the central word are both multidimensional one-hot vectors, and the dimension is equal to the number of semantic role types. Each dimension represents a semantic role type. If the word belongs to the semantic role type, the corresponding dimension takes the value of 1, otherwise it takes the value of 0. By calculating the cosine value of the angle between the two vectors, their similarity in the distribution of semantic role types can be measured. The larger the cosine value, the more matching the candidate argument and the central word in semantic role.

[0028] Set a similarity threshold, filter out candidate arguments below the threshold, and retain candidate arguments above the threshold as the final semantic role arguments; the similarity threshold is pre-set and is used to control the tightness of semantic role matching. The higher the threshold, the stricter the matching conditions, and the more candidate arguments are filtered out; the lower the threshold, the looser the matching conditions, and the more candidate arguments are retained. By setting an appropriate threshold, a balance can be achieved between matching accuracy and matching recall; for each word, combine the semantic role argument and type to form a word-argument-type triple as the semantic role labeling result of the corresponding word in the sentence; a word may belong to multiple semantic role arguments and correspond to multiple semantic role types. By combining them into a triple, the semantic role information of the word in the sentence can be fully represented. The first element of the triple is the word itself, and the second element is the word type. The second element is the semantic role argument to which the word belongs, and the third element is the type of the semantic role argument; according to the predefined semantic role combination template, the relationship between entities is extracted from the semantic role labeling results obtained. The semantic role combination template defines the relationship type corresponding to the combination of different semantic role triples. By matching the semantic role combination template in the semantic role labeling results, the semantic relationship between entities can be identified, and the extracted entity relationship is formed into a triple. Each triple includes a head entity, a tail entity and a relationship type. The head entity and the tail entity are respectively represented by the word elements in the semantic role triple. The entities in the triple are mapped to the normalized entities in the knowledge graph. Through the MERGE command of the graph database, the edges between the head entity, the tail entity and the relationship are created, and the relationship extraction results based on the semantic role labeling are fused and associated with the knowledge graph to expand the relationship information of the knowledge graph. This application refines the filtering method of semantic role arguments, and filters out unmatched candidate arguments by calculating the similarity between the candidate arguments and the central word in the semantic role type, and setting a similarity threshold, thereby improving the accuracy of semantic role argument recognition. At the same time, by binding semantic role arguments and types to words, a complete semantic role labeling result is formed, and the relationship between entities is extracted from it using the semantic role combination template, further expanding the coverage of relationship extraction. Finally, the relationship extraction results are fused and associated with the knowledge graph to continuously update the relationship information of the knowledge graph.

[0029] 3. Beneficial effects

[0030] Compared with the prior art, the advantages of this application are:

[0031] By using the Qwen-14B pre-trained model with hundreds of billions of parameters to encode unstructured text data, and using its powerful semantic understanding and feature extraction capabilities, high-quality semantic vectors at the word and sentence levels are obtained, providing a good semantic foundation for subsequent entity recognition and relationship extraction, and improving integration efficiency and accuracy;

[0032] Using deep learning models such as Bi-LSTM network and CRF layer, named entity recognition is performed on text data. By extracting character features and part-of-speech features, as well as context encoding of the bidirectional LSTM layer and sequence labeling of the CRF layer, entities and their types in the text are accurately identified, providing high-quality entity nodes for building knowledge graphs.

[0033] The method combines Stack-Pointer network and Bi-LSTM network, uses syntactic dependency analysis and semantic role labeling technology, extracts the relationship between entities from the syntactic and semantic dimensions of words, makes full use of the structured information contained in the text data, and obtains accurate and rich entity relationship knowledge;

[0034] The extracted entities and relationships are mapped to the knowledge graph in the graph database Neo4j. The storage and query methods of the graph database are adopted. The efficiency and flexibility of the graph are utilized to achieve rapid retrieval, reasoning and fusion of knowledge, greatly improving the efficiency of data integration and application.

[0035] Through Neo4jCypher statements, the knowledge graph can be queried, analyzed and updated. By using graph pattern matching, attribute filtering, path retrieval and other functions, the hidden associations and patterns in the knowledge graph can be flexibly mined to discover new knowledge and insights, thus expanding the depth and breadth of data integration.

[0036] Introducing graph-based data mining algorithms, such as Page Rank and Louvain community discovery algorithms, which use the network structure of knowledge graphs to calculate the importance of entity nodes and community affiliation, provide a novel way to organize and interpret knowledge, and enhance the intelligence and interpretability of data integration;

[0037] Neo4jETL tools and Cypher statements are used to achieve seamless integration and mapping of external data sources with knowledge graphs. Predefined mapping rules are used to automatically convert structured and semi-structured data into graph form, expanding the scale and diversity of knowledge graphs and improving the level of automation of data integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0039] Figure 1 is an exemplary flow chart of a data integration method based on a knowledge graph according to some embodiments of this specification;

[0040] Figure 2is an exemplary flow chart of obtaining entities and corresponding types according to some embodiments of this specification;

[0041] Figure 3 is an exemplary flow chart of constructing a syntactic dependency tree according to some embodiments of this specification;

[0042] Figure 4 This is an exemplary flowchart of obtaining semantic role labeling results according to some embodiments of this specification. DETAILED DESCRIPTION

[0043] The method and system provided in the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 This is an exemplary flow chart of a method for data integration based on a knowledge graph according to some embodiments of the present specification, a method for data integration based on a knowledge graph, comprising: obtaining unstructured text data; inputting the text data into a large language model based on Prompt, the large language model using a Qwen-14B pre-trained model with a general meaning of hundreds of billions of parameters; the Qwen-14B pre-trained model uses a self-attention mechanism to encode the input text data to generate semantic vectors at the word and sentence levels; based on the generated semantic vectors, a named entity recognition algorithm is used to identify entities in the text, and the relationship between entities is obtained through dependency syntax and semantic role labeling methods; based on the obtained entities and relationships , extract structured knowledge in the form of subject-predicate-object triples SPO; map the extracted triples SPO to the knowledge graph of the graph database Neo4j, use Neo4jCypher statements to map the subject and object entities in the triples SPO to nodes, and the attributes of the nodes include metadata of entity nouns and entity types; map the predicate relationship in the triples SPO to the relationship between nodes, and the attributes of the relationship include metadata of the relationship type and relationship strength; use a graph storage engine based on Neo4j to store the mapped knowledge graph in the graph database, which uses an adjacency list structure to store nodes and relationships separately; use the graph database that stores the knowledge graph to perform data integration.

[0045] Get unstructured text data. Text data can come from a variety of sources such as web pages, documents, and databases. For web page data on the Internet, you can use web crawler technology to automatically collect data. Commonly used Python crawler libraries include Scrapy, Beautiful Soup, and Selenium. For document files in local disks or network disks (such as Word, PDF, TXT, etc.), you can use the corresponding document parsing library to extract content. If the text data is stored in a structured database (such as MySQL, MongoDB, etc.), you can use a database query language (such as SQL) to extract the content of the text field.

[0046] Figure 2 According to the exemplary flowchart of obtaining entities and corresponding types shown in some embodiments of this specification, the input text data is preprocessed, including operations such as word segmentation, removal of stop words, and character normalization, and the text is converted into a standardized word sequence. The processed word sequence is encapsulated in the form of a Prompt, that is, a task-descriptive template text is added before the sequence. For example: "Please extract entities and relationships in the text." The Prompt form can guide the language model to perform a specific task. The encapsulated Prompt is input into the pre-trained Qwen-14B model. Qwen-14B is a general large-scale language model that adopts a Transformer structure, has 1.4 billion parameters, is pre-trained on a large-scale corpus, and has strong semantic understanding and generation capabilities. The Qwen-14B model uses a self-attention mechanism to encode the input Prompt. The self-attention mechanism can learn the dependencies between words and capture the contextual information of words. Specifically, each layer of Transformer blocks in the model contains two sub-layers: multi-head self-attention and feedforward neural network. Multi-head self-attention linearly maps the word sequence into three vectors: Q (query), K (key), and V (value). The attention weight is obtained by calculating the similarity between Q and K, and then the weight is applied to V to obtain the weighted sum as the context representation of the current word. Multiple heads can capture different subspace information.

[0047] The feedforward neural network further performs nonlinear transformation on the context representation to enhance the expressive power of the features.

[0048] Transformer blocks can be stacked in multiple layers to model long-distance semantic dependencies. Through the multi-layer self-attention encoding of the Qwen-14B model, the semantic vector representation of each word in the input text sequence can be obtained. This vector integrates the features of the word itself and the context information. By concatenating the word vectors in the order in the sequence, the semantic vector representation of the entire sentence can be obtained. Due to the position encoding mechanism of Transformer, the order relationship between word vectors is also preserved. Therefore, the Qwen-14B model can generate sentence-level semantic vectors based on word positions, reflecting the semantic information of the entire sentence. Finally, the word-level and sentence-level semantic vectors generated by the Qwen-14B model are output as input features for downstream tasks such as entity recognition and relation extraction. The semantic vectors convert unstructured text data into structured numerical representations, providing rich semantic information for subsequent knowledge extraction.

[0049] Character-level feature extraction is performed on the text data. The text is converted into a character sequence, and each character is represented by a unique index. For example: "The weather is nice today" -> [1, 2, 3, 4, 5, 6]. Each character is encoded using one-hot encoding or the embedding method to generate a fixed-length character vector. One-hot encoding: Each character is represented by an N-dimensional vector, where only the position corresponding to the character is 1, and the rest are 0. N is the size of the character set. For example, assuming the character set is {"今", "天", "气", "真", "好"}, the one-hot vector of "今" is [1, 0, 0, 0, 0], and the vector of "天" is [0, 1, 0, 0, 0], and so on. Embedding encoding: Each character is represented by an M-dimensional dense vector, and each element of the vector is a real value. M is the pre-set dimension of the character vector. For example, assuming M = 4, the embedding vector of "今" can be [0.1, -0.2, 0.3, 0.5], and the vector of "天" can be [-0.2, 0.4, 0.1, -0.3], and so on. The embedding vectors can be randomly initialized and optimized end-to-end during the model training process. The encoded vectors of each character are arranged in the order in the text to form a character vector sequence. For example: "The weather is nice today" -> [[0.1, -0.2, 0.3, 0.5], [-0.2, 0.4, 0.1, -0.3]].

[0050] Extract part-of-speech features from text data, use part-of-speech tagging tools to segment and tag text, and get the part-of-speech tag for each word. For example: "Today / t weather / n real / d good / a" -> [("Today", "t"), ("weather", "n"), ("real", "d"), ("good", "a")]. Convert part-of-speech tags to one-hot vectors or embedding vectors. One-hot encoding: Each part-of-speech tag is represented by an N-dimensional vector, where N is the size of the part-of-speech tag set. For example, assuming the part-of-speech tag set is {t, n, d, a}, the one-hot vector of "t" is [1, 0, 0, 0], the vector of "n" is [0, 1, 0, 0], and so on. Embedding encoding: Each part-of-speech tag is represented by an M-dimensional dense vector, where M is the pre-set part-of-speech vector dimension. For example, if M=3, the embedding vector of "t" can be [0.2, -0.1, 0.4], the embedding vector of "n" can be [-0.3, 0.2, 0.5], and so on. Arrange the part-of-speech vectors of each word in the order in the text to form a part-of-speech vector sequence. For example: "Today / tThe weather / nIt's really / dGood / a"->[[0.2, -0.1, 0.4], [-0.3, 0.2, 0.5]], concatenate the character vector sequence and the part-of-speech vector sequence in the way of word alignment to form the final word feature vector sequence. Let the character vector dimension be D1 and the part-of-speech vector dimension be D2, then the feature vector dimension of each word is D1+D2. For example, for the word "today", its character vector is [[0.1, -0.2, 0.3, 0.5], [-0.2, 0.4, 0.1, -0.3]], and its part-of-speech vector is [0.2, -0.1, 0.4], then the concatenated feature vector is [[0.1, -0.2, 0.3, 0.5, 0.2, -0.1, 0.4], [-0.2, 0.4, 0.1, -0.3, 0.2, -0.1, 0.4]]. The final word feature vector sequence is in the form of [[word 1 character vector, word 1 part-of-speech vector], [word 2 character vector, word 2 part-of-speech vector]]. The word feature vector sequence is input into the bidirectional LSTM network to learn the contextual representation of the word.

[0051] The forward LSTM traverses the sequence of word feature vectors from left to right, processes a word vector at each time step, and updates the hidden state. Suppose the sequence of word feature vectors is X = (x_1, x_2, ....., x_T), and the hidden state of the forward LSTM is over right arrow{h_t} = LSTM(x_t, over right arrow{h_{t-1}}). The sequence of word feature vectors is [[0.1, -0.2, 0.3], [-0.2, 0.4, 0.1]], then the forward LSTM processes vectors such as [0.1, -0.2, 0.3], [-0.2, 0.4, 0.1] in sequence, and updates the hidden state over right arrow{h_1}, over right arrow{h_2}. The reverse LSTM traverses the sequence of word feature vectors from right to left, processes a word vector at each time step, and updates the hidden state. Assume that the hidden state of the reverse LSTM is over left arrow{h_t}=LSTM(x_t, over left arrow{h_{t+1}}).

[0052] The word feature vector sequence is [[0.1, -0.2, 0.3], [-0.2, 0.4, 0.1]], then the reverse LSTM processes [-0.2, 0.4, 0.1], [0.1, -0.2, 0.3] in sequence, and updates the hidden state, over left arrow{h_{T-1}}, overleft arrow{h_T}. At each time step, the hidden states of the forward LSTM and the reverse LSTM are concatenated to obtain a word representation that incorporates context information. Let the context representation of the word w_t be h_t = [over right arrow{h_t}; overleft arrow{h_t}], where [;] represents the vector concatenation operation.

[0053] The output vector sequence of Bi-LSTM is used as the emission matrix of conditional random field (CRF). The output vector sequence of Bi-LSTM is H = (h_1, h_2, ..., h_T), where h_tinmath bb{R}^d is the contextual representation of the tth word and d is the dimension of the hidden state. H is used as the emission matrix Pinmathbb{R}^{Ttimesn} of the CRF layer, where n is the number of labels. P_{i, j} represents the emission probability that the i-th word is labeled with the j-th label. The CRF layer performs sequence labeling on the BIO labels of words by learning the transfer feature function and the state feature function. The transfer feature function characterizes the dependency between labels in the form of t(y_{i-1}, y_i, X), where y_i is the label of the i-th word.

[0054] Commonly used transfer features include: label transfer probability, such as p(y_i=B|y_{i-1}=O), label co-occurrence probability, such as p(y_i=I, y_{i-1}=B). The state feature function describes the correlation between words and labels, in the form of s(y_i, X, i), where X is the input sequence. Commonly used state features include: word emission probability, such as p(x_i|y_i=B), word part of speech, affix and other attributes and label correlation, such as p(y_i=I|x_itext{part of speech is NN}). The CRF layer learns the weight lambda_k of the transfer feature function and the weight mu_l of the state feature function.

[0055] Linearly combine the feature functions to obtain the conditional probability of the label sequence Y: p(Y|X)=frac{1}{Z(X)}exp(sum_{k}lambda_ksum_{i=1}^Tt_k(y_{i-1},y_i,X)+sum_{l}mu_lsum_{i=1}^Ts_l(y_i,X,i)). Where Z(X) is the normalization factor, t_k and s_l are the kth transition feature function and the lth state feature function, respectively.

[0056] The Viterbi algorithm is used to decode at the CRF layer to obtain the BIO label sequence with the highest probability. The Viterbi algorithm recursively calculates the maximum probability of the label sequence through the idea of ​​dynamic programming and records the optimal path. Let delta_t(j) represent the maximum probability of the partial observation sequence x_1, x_2, ..., x_t at position t and the label of the tth word is j, then: delta_{t+1}(k) = max_{j}delta_t(j)cdotpsi(j, k, x_{t+1}), where psi(j, k, x_{t+1}) is the non-normalized probability of the word x_{t+1} transferring from label j to label k, which is calculated by the transfer feature and state feature. At each step, the Viterbi algorithm selects the previous label j that maximizes delta_{t+1}(k) and records it in the backtracking pointer phi_{t+1}(k). At the termination position T, select the termination label y_T^*=argmax_{j}delta_T(j) with the highest probability, and then obtain the optimal label sequence Y^*=(y_1^, y_2^,......., y_T^*) through the backtracking pointer phi.

[0057] According to the boundary information of the BIO tag, the starting position and the ending position of the entity are extracted. The BIO tag adopts the following convention: the B (Begin) tag indicates the beginning of the entity, the I (Inside) tag indicates the middle part of the entity, and the O (Outside) tag indicates a non-entity. Continuous B and I tags constitute a complete entity. Specifically: the B tag is used as the starting position of the entity, and the continuous I tags are used as the middle part of the entity until an O tag or another B tag is encountered, marking the end of the entity. For example, the tag sequence [B, I, I, O, B, I, O] indicates that there are two entities, the first entity includes the first three words, and the second entity includes the last two words. According to the type information of the BIO tag, the type of each entity is identified. The BIO tag can use different letter suffixes to represent different entity types, for example: B-PER, I-PER represent person name entities, B-ORG, I-ORG represent organization name entities, and B-LOC, I-LOC represent place name entities. According to the pre-defined mapping relationship between tags and types, the specific type of each entity can be determined. For example, the label sequence [B-PER, I-PER, O, B-ORG, I-ORG] indicates that there are two entities, the first entity is a person's name, and the second entity is an organization's name.

[0058] Figure 3 It is an exemplary flow chart for constructing a syntactic dependency tree according to some embodiments of this specification; the syntactic dependency structure is learned using a Stack-Pointer network, and the Stack-Pointer network models the dependency relationship between words through a stacked LSTM and a pointer-type attention mechanism. The word feature vector sequence X = (x_1, x_2, ..., x_n) is input into the stacked LSTM to learn the contextual representation of the word. The stacked LSTM saves and updates the hidden state of the word in the stack through push and pop operations to capture long-distance dependencies. At each time step, the dependency weights of the current word and other words are calculated through the attention mechanism. The attention weight a_{ij} represents the dependency probability of word x_i on word x_j, which is calculated by the similarity of the word representation. The word with the highest attention weight is selected as the head node of the current word, and the corresponding dependency label (such as subject, object, etc.) is predicted. The selection of the head node is realized by a pointer network, and it is located in the word sequence according to the attention weight. The dependency label is predicted by multiple classifiers and classified according to the word representation and the head node representation. d. Through the iterative process of stacked LSTM, a complete syntactic dependency tree is obtained. The dependency tree represents the modification relationship and syntactic structure between words in the form of directed edges.

[0059] Input the part-of-speech feature sequence and the syntactic dependency tree into the Bi-LSTM network to learn the semantic role representation of words. The Bi-LSTM network can encode part-of-speech information and dependency structure at the same time and learn the syntactic semantic representation of words. Input the part-of-speech feature vector sequence P = (p_1, p_2, ...., p_n) and the adjacency matrix Ainmath bb{R}^{ntimesn} of the dependency tree into the Bi-LSTM. The part-of-speech feature characterizes the syntactic attributes of words, such as nouns, verbs, etc. The adjacency matrix represents the topological structure of the dependency tree, and A_{ij} = 1 means that word x_i is the head node of word x_j. The forward LSTM and reverse LSTM calculate the hidden state along the forward and reverse directions of the word sequence respectively. Forward hidden state over right arrow{h_i} = LSTM(x_i, p_i, sum_{j = 1}^nA_{ij}cdotover right arrow{h_j}). Reverse hidden state over left arrow{h_i}=LSTM(x_i,p_i,sum_{j=1}^nA_{ji}cdotover left arrow{h_j}). At each position, concatenate the forward and reverse hidden states to obtain the semantic role representation of the word h_i=[over right arrow{h_i};over leftarrow{h_i}]. The semantic role representation integrates the contextual information, syntactic attributes and dependencies of the word.

[0060] The output vector h_i of Bi-LSTM is passed to the multi-layer perceptron (MLP) to predict the semantic role label corresponding to each word. MLP models the complex mapping relationship between words and semantic roles through nonlinear transformation and stacked structure. The input of MLP is the output vector h_i of Bi-LSTM, which is extracted through multiple fully connected layers and nonlinear activation functions. h_i^{(1)}=ReLU(W_1cdoth_i+b_1), h_i^{(2)}=ReLU(W_2cdoth_i^{(1)}+b_2), where W_k and b_k are the weight matrix and bias vector of the kth layer. The output layer of MLP performs multi-classification through the softmax function to predict the probability that the word belongs to each semantic role label. y_i=softmax(W_ocdoth_i^{(L)}+b_o), where L is the number of layers of MLP and y_i is the probability distribution of the semantic role label of word x_i. Semantic role labels use BIO annotation, such as B-ARG0 (agent-start), I-ARG0 (agent-intermediate), O (non-argument), etc.

[0061] Decode the semantic role label sequence (y_1, y_2, ......., y_n) output by MLP to obtain the boundary and semantic role of the argument. Use the Viterbi algorithm to decode the BIO label and find the most likely boundary of the argument. The Viterbi algorithm recursively calculates the maximum probability of the label sequence through dynamic programming. Let delta_i(j) represent the maximum probability of the partial observation sequence at position i, and the label of the i-th word is j. delta_{i+1}(k)=maxlimits_{j}delta_i(j)cdotp(y_{i+1}=k|y_i=j)cdotp(x_{i+1}|y_{i+1}=k), where p(y_{i+1}=k|y_i=j) is the label transition probability and p(x_{i+1}|y_{i+1}=k) is the emission probability. According to the optimal label sequence obtained by the Viterbi algorithm, extract continuous labels of the same type as arguments. For example, the label sequence [B-ARG0, I-ARG0, O, B-ARG1, I-ARG1] represents two arguments ARG0 and ARG1. ARG0 consists of words x_1 and x_2, and ARG1 consists of words x_4 and x_5. SVO knowledge triples are extracted based on the boundaries and semantic roles of the arguments. For each predicate (central word), find the arguments of all its semantic role labels. The predicate is identified by syntactic dependencies and part-of-speech information, usually the main verb of the sentence. The arguments are classified according to semantic roles, and the agent (ARG0), patient (ARG1), time (ARGM-TMP), etc. are extracted. Different semantic roles correspond to different components in the knowledge triple.

[0062] Figure 4It is an exemplary flowchart for obtaining semantic role labeling results shown in some embodiments of this specification, and defines the Cypher creation statement of the node. For the subject and object entities in the triple, the CREATE statement is used to create the corresponding node. Among them, the entity type is the identified entity type label, and the entity name is the original mention of the entity in the text. The attributes of the node include the entity type and the entity name. Define the Cypher creation statement of the edge. For the predicate relationship in the triple, the CREATE statement is used to create the corresponding directed edge to connect the subject node and the object node. Among them, entity type 1 and entity type 2 are the entity types of the subject and the object, respectively, and the subject name and the object name are their entity mentions. The semantic relationship type determined by the relationship type. The relationship strength can be calculated based on indicators such as the confidence and frequency of the relationship extraction, reflecting the reliability of the relationship. The attributes of the edge include the relationship type and strength. Batch execute Cypher statements. All SPO triples are converted into Cypher statements in sequence and executed in batches through the Java API or Cypher Shell of Neo4j. Each triple generates two node creation statements and one edge creation statement. Use the MERGE statement to handle duplicate entities. For the same entity shared by multiple triples, the MERGE statement can ensure that the node is only created once to avoid duplication. MERGE will first match the existing node, and create a new node if it does not exist.

[0063] Using Neo4j's graph storage engine to store knowledge graphs: Neo4j uses a localized storage engine to persist knowledge graphs to disk. Graph data is stored in the form of an adjacency list, which supports efficient graph traversal and query. Node storage. Each node includes a unique node ID, node label, and attribute key-value pair. The node ID is used to identify the node, the node label indicates the type of the node, and the attribute stores the specific information of the node. Neo4j allocates a fixed-size storage record to each node, which contains the basic information and attribute values ​​of the node. Multiple node records are stored sequentially in the Node Store file. Edge storage. Each edge includes a unique edge ID, a starting node ID, a target node ID, an edge type, and an attribute key-value pair. The edge ID identifies the edge, the starting and target node IDs locate the two endpoints of the edge, and the edge type indicates the type of relationship. Neo4j allocates a fixed-size storage record to each edge, which contains the basic information and attribute values ​​of the edge. Multiple edge records are stored sequentially in the RelationshipStore file. Adjacency table storage. In order to speed up graph traversal and query, Neo4j also maintains an adjacency table structure. For each node, the adjacency list records all the incoming and outgoing edges of the node.

[0064] The adjacency list is stored in a linked list format. Each node corresponds to a set of linked lists of edges, which can quickly access the adjacent edges of the node. The adjacency list information is stored in files such as Relationship Group Store and Relationship Type Store. Index storage. In order to support efficient node and edge property queries, Neo4j also builds an index structure. The property index maps the property value to the corresponding node or edge ID, accelerating the equal value and range query of the property. Index information is stored in separate index files, such as schema index, full-text index, etc.

[0065] Use the MATCH statement to match the query graph pattern. The MATCH statement defines a graphical query template, using node and edge variable names to represent the nodes and relationships to be matched. The statement matches all nodes p of type Person and edges of type WORKS_AT from p to node c of type Company. Use the WHERE statement to set the filter condition. The WHERE statement specifies the constraints that the attributes of the nodes or edges must satisfy to filter out the results that meet the conditions. It requires that the age attribute of the Person node is greater than 30 and the name attribute of the Company node is "Google". Use the RETURN statement to specify the return result. The RETURN statement defines the nodes, edges, and attributes included in the query result. Returns the name attribute of the matching Person node and the name attribute of the Company node. Combine multiple query statements to implement complex queries. Cypher supports the combination of multiple MATCH, WHERE, and RETURN statements, which can implement complex queries such as multi-hop path matching, conditional filtering, and result set merging.

[0066] Use the CALL statement to call the Page Rank algorithm to calculate the importance of the node. The Page Rank algorithm calculates the score of each node based on the node's in-degree and the importance of its neighbor nodes, reflecting the influence of the node in the graph. Call the Page Rank algorithm for Page type nodes and LINKS type edges, set the number of iterations to 20, and the damping coefficient to 0.85. The algorithm returns the Page Rank value of each node. Use the CALL statement to call the Louvain algorithm to detect the community attributes of the node. The Louvain algorithm divides the graph into multiple communities by optimizing the modularity index. The nodes in the same community are closely connected, and the connections between different communities are sparse. Call the Louvain algorithm for Page type nodes and LINKS type edges, and set it not to perform hierarchical aggregation. The algorithm returns the community ID to which each node belongs. Use the SET statement to store the calculation results as node attributes. The SET statement can update or add node attributes to associate the importance and community attributes returned by the algorithm with the node. Store the Page Rank value as the Page Rank attribute of the Page node, and store the Louvain community ID as the community attribute.

[0067] Use Neo4jETL tool to import external data sources. ETL tool supports batch import of data from structured data sources such as CSV, JSON, XML to graph database, and automatically creates nodes and edges. Import Person node data from persons.csv file, use MERGE statement to match or create nodes, and set node properties. Use Cypher statement to map and merge external data with existing knowledge graph. Through MATCH and MERGE statements, entities in external data can be mapped to nodes in knowledge graph, and relationships can be mapped to edges to achieve knowledge fusion. Match Person node John, merge Company node Google (create if it does not exist), and create WORKS_AT relationship edge from John to Google. Execute ETL scripts regularly to achieve incremental updates. By scheduling ETL scripts to run regularly, new entities and relationships can be imported from external data sources, merged with existing knowledge graphs, and automatic expansion and update of knowledge base can be achieved. Neo4j supports transaction mechanism to ensure atomicity and consistency of updates.

Claims

1. A data integration method based on knowledge graph, comprising: Obtain unstructured text data; The acquired text data is input into a large language model based on Prompt for feature extraction. The large language model uses the Qwen-14B pre-trained model with hundreds of billions of parameters, and uses a self-attention mechanism to encode the input text data to generate semantic vectors at the word and sentence levels. Based on the generated semantic vectors, named entity recognition algorithm is used to identify entities in the text; the relationship between entities is obtained through dependency syntax and semantic role labeling methods; based on the obtained entities and relationships, structured knowledge in the form of subject-predicate-object triples (SPO) is extracted; The extracted triple SPO is mapped to the knowledge graph of the graph database Neo4j. The subject and object entities in the triple SPO are mapped to nodes using Neo4jCypher statements. The attributes of the nodes include the metadata of the entity noun and entity type. The predicate relationship in the triple SPO is mapped to the relationship between nodes. The attributes of the relationship include the metadata of the relationship type and relationship strength. A Neo4j-based graph storage engine is used to store the mapped knowledge graph in a graph database, which uses an adjacency list structure to store nodes and relationships separately. Use graph databases that store knowledge graphs for data integration; Get the relationships between entities, including: The semantic vector is used as A2 as the input of the Stack-Pointer network; The Stack-Pointer network consists of a stacked LSTM layer and a pointer-type attention layer. The stacked LSTM layer pushes the input feature vector sequence A2 into the stack and encodes it into a hidden state. The pointer-type attention layer calculates the head word index of each word and the probability distribution of the dependency label according to the hidden state of the stack; the head word index represents the position index of the parent node of each word in the syntactic dependency tree in the input sequence; The transfer decoding algorithm is used to decode the output of the Stack-Pointer network. The transfer decoding algorithm moves and pops up on the input feature vector sequence A2 through transfer operations and stack operations, and selects the head word index and dependency label according to the probability distribution of the attention layer to obtain the head word index and dependency label of each word; A syntactic dependency tree is constructed based on the head word index and dependency label of each word. The nodes in the syntactic dependency tree represent words, the directed edges represent the dependency relationship between words, the direction of the edge represents the direction of the dependency relationship, and the label of the edge represents the type of the dependency relationship; Get the relationship between entities, including: The part-of-speech feature vector sequence in the extracted feature vector sequence A2 and the constructed syntactic dependency tree are used as the input of the Bi-LSTM network. The syntactic dependency tree provides syntactic structure information between words, and the part-of-speech feature vector sequence provides syntactic attribute information of words. The syntactic dependency tree and part-of-speech feature vector sequence are input into the Bi-LSTM network. The bidirectional LSTM layer learns the semantic role representation of words from the syntactic structure and syntactic attributes. The output vector sequence of the bidirectional LSTM layer integrates the contextual information of the syntactic structure and syntactic attributes. The output vector of the bidirectional LSTM layer is input into a multi-layer perceptron, which is used to classify the semantic role of each word. The output of the multi-layer perceptron is the probability distribution of each word belonging to different semantic role types. Get the relationship between entities, including: The output probability of the multilayer perceptron is converted into an emission matrix of semantic role labeling. The rows of the emission matrix represent words, the columns represent BIO labels, and the elements represent the probability of a word being labeled with the corresponding BIO label. Define the transition matrix of semantic role labeling. The rows and columns of the transition matrix represent BIO labels, and the elements represent the probability of the previous BIO label transferring to the next BIO label. The transition probability is estimated based on the prior knowledge of semantic role labeling in the corpus. The Viterbi algorithm is used to decode on the emission matrix and the transfer matrix. The Viterbi algorithm finds the BIO tag sequence with the highest probability through dynamic programming. The state of dynamic programming is the BIO tag at each position, and the state transition is the BIO tag from the previous position to the BIO tag at the next position. The state transition probability is the element of the transfer matrix, and the emission probability is the element of the emission matrix. The score of each state sequence is calculated by the product of the state transition probability and the emission probability, and the backtracking pointer of the state sequence is recorded. Obtain the BIO tag sequence with the highest probability according to the backtracking pointer, merge the consecutive BIO tags of the same type in the BIO tag sequence into the boundary of the argument. The boundary of the argument is determined by the B tag and the I tag, and the words within the argument boundary are taken as candidate arguments. Determine the semantic role type of the candidate argument according to the type of the BIO tag. The type of the BIO tag contains the type information of the semantic role. Match the semantic role type of the candidate argument with the semantic role type of the corresponding central word, filter out the unmatched candidate arguments, and obtain the semantic role argument and type to which each word belongs. Match the semantic role types of candidate arguments with the semantic role types of the corresponding central words, filter out unmatched candidate arguments, and obtain the semantic role arguments and types of each word, including: According to the syntactic dependency tree and argument boundaries, determine the central word of each candidate argument, where the central word represents the word on which the candidate argument depends; According to the semantic role annotated dictionary and corpus, the semantic role type of each central word is obtained. The dictionary defines the semantic role type of each word, and the corpus obtains the frequency and probability of each word playing different semantic role types through statistical methods. Calculate the cosine similarity between the semantic role type of the candidate argument and the semantic role type of the corresponding central word; Set a similarity threshold, filter candidate arguments below the threshold, and retain candidate arguments above the threshold as the final semantic role arguments; For each word, the semantic role argument and type are combined to form a triple of word-argument-type, which serves as the semantic role labeling result of the corresponding word in the sentence.

2. The data integration method based on knowledge graph according to claim 1 is characterized in that: Use graph databases that store knowledge graphs for data integration, including: Neo4jCypher statements are used to retrieve and analyze the knowledge graph stored in the graph database. Neo4jCypher statements include MATCH commands, WHERE commands, RETURN commands, CREATE commands, and MERGE commands. in: Use the MATCH command to match the graph pattern of the query. The graph pattern consists of nodes, relationships, node attribute conditions, and relationship attribute conditions. Node attribute conditions are used to match nodes with attribute values, and relationship attribute conditions are used to match relationships with attribute values. Use the WHERE command to filter the matching graph patterns, and filter the matching nodes and relationships according to the node attribute conditions, relationship attribute conditions, and topological structure conditions. The topological structure conditions include the node degree and path length. The query results are returned with the RETURN command. The query results include the matched nodes, relationships, and properties of the nodes and relationships. Use the CREATE command to create new nodes and relationships. According to the entities and relationships in the external data, create corresponding nodes and relationships in the graph database, and set the properties of the nodes and relationships. Use the MERGE command to update existing nodes and relationships. According to the entities and relationships in the external data, search for matching nodes and relationships in the graph database. If found, update the properties of the corresponding nodes. If not found, create new nodes and relationships.

3. The data integration method based on knowledge graph according to claim 2 is characterized in that: Data integration using graph databases that store knowledge graphs also includes: The graph pattern obtained by the RETURN command query is used as input to call the Page Rank algorithm and the Louvain algorithm through the Neo4jCypher statement; wherein the graph pattern contains matching nodes, relationships, and attributes, which serve as input data for the Page Rank algorithm and the Louvain algorithm; Call the Page Rank algorithm to calculate the importance measurement Page Rank value of the node according to the topological structure of the nodes in the graph mode; Call the Louvain algorithm to obtain the community labels of nodes based on the connection relationship of nodes in the graph mode; The Page Rank value, a measure of the importance of the node, calculated by the Page Rank algorithm, and the community label of the node obtained by the Louvain algorithm are added to the corresponding node as new attributes, and the Page Rank value and community attribute of the node are stored in the graph database through the CREATE command or the MERGE command.

4. The data integration method based on knowledge graph according to claim 3 is characterized in that: Data integration using graph databases that store knowledge graphs also includes: Use Neo4jETL tool to import external data to be integrated into the graph database; The mapping logic between external data and the graph database is defined through Neo4jCypher statements. The mapping logic includes the conversion rules from external data to nodes and relationships in the graph database. According to the mapping logic, the entities in the external data are mapped to the nodes in the graph database, the attributes of the entities are mapped to the attributes of the nodes, and new nodes are created or existing nodes are updated through the CREATE command or the MERGE command; According to the mapping logic, the relationships in the external data are mapped to the relationships in the graph database, and the attributes of the relationships are mapped to the attributes of the relationships. New relationships are created or existing relationships are updated through the CREATE command or the MERGE command. When creating nodes and relationships, use the MATCH command to retrieve existing nodes and relationships, and match them through the WHERE command and attribute conditions to associate and integrate the newly created nodes and relationships with the existing knowledge graph.

5. The data integration method based on knowledge graph according to any one of claims 1 to 4, characterized in that: Get entities, including: Take the semantic vector as A1 and use it as the input of the Bi-LSTM network; The context representation of characters and parts of speech is learned from both the forward and backward directions through the bidirectional LSTM layer. The bidirectional LSTM layer consists of multiple LSTM units. Each LSTM unit receives the input of the current time step and the hidden state of the previous time step, and generates the hidden state and output of the current time step. The outputs of the forward LSTM and the backward LSTM are concatenated into the output vector sequence of the Bi-LSTM network. The output vector sequence of the Bi-LSTM network is used as the emission matrix of the conditional random field CRF layer. The transfer matrix and state sequence of the BIO annotation are learned through the CRF layer. The CRF layer uses the transfer feature function and the state feature function to calculate the dependency between labels and between labels and observations respectively. Among them, the emission matrix represents the dependency between the observation sequence and the label sequence, and the transfer matrix represents the dependency between adjacent labels. Decode the BIO annotation sequence output by the CRF layer to extract the entity and the corresponding type label; the BIO annotation is used to mark the boundary and type of the entity in the text, B indicates the beginning of the entity, I indicates the inside of the entity, and O indicates the outside of the entity; Map the extracted entities to entities in the knowledge graph.

6. The data integration method based on knowledge graph according to claim 5 is characterized in that: Get the entity, also includes: In the conditional random field (CRF) layer, the state sequence output by the Bi-LSTM network is decoded, and the Viterbi algorithm is used to obtain the state sequence with the highest probability. The Viterbi algorithm finds the state path with the highest probability through dynamic programming. The state of dynamic programming is the label of each position, and the state transition is the label from the previous position to the next position. The transition probability is the transition feature function learned by the CRF layer, and the emission probability is the state feature function learned by the CRF layer. The score of each state sequence is calculated by the product of the transition probability and the emission probability, and the backtracking pointer of the state sequence is recorded. Obtain the state sequence with the highest probability according to the backtracking pointer, and mark the BIO label of the character corresponding to each state in the state sequence; According to the boundary information of the BIO tag, the B tag and the adjacent I tag after the B tag in the state sequence are merged into one entity until the O tag or a different type of B tag is reached; Determine the type of each entity according to the type information of the BIO tag, the BIO tag contains the letter code of the entity type, and convert the letter code into the entity type according to the mapping relationship between the letter code and the predefined entity type; The boundaries and types of the obtained entities are combined to obtain the entities identified in the text data and the corresponding types; According to the identified entities and types, entity mapping and knowledge graph fusion are performed.

Images