A knowledge graph path mining method using conditional random fields and relation extraction
Through the conditional random field and relationship extraction model, path mining in the knowledge graph is solved, and the problem of lack of semantic information and diversity of paths in the existing technology is solved, and more accurate and comprehensive path discovery is achieved, meeting user needs and improving the knowledge graph value of application.
Patent Information
- Application Number
- CN202411096340.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-12
AI Technical Summary
The existing path mining method is based on topological structure and ignores semantic information, resulting in paths lacking semantic accuracy and richness, being single, and unable to meet user needs. The limitations of algorithms limit the complexity and diversity of data relationship processing.
The conditional random field model and end-to-end relationship extraction model are used to find multiple indirect correlation paths in the knowledge graph through deep-first search, extract the value characteristics and diverse characteristics of the path, comprehensively consider the overall value index and information enrichment index, and conduct the path. Arrange and filter.
It improves the efficiency and accuracy of data integration and information extraction, provides more comprehensive and accurate path mining results, meets user needs, and improves the application value and practicality of the knowledge graph.
Smart Images

Figure CN118627614B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and more specifically, to a knowledge graph path mining method using conditional random fields and relationship extraction. Background Art
[0002] Path mining refers to the process of finding the set of paths connecting two specific target entities (usually nodes or entities) in a given knowledge graph or network by mining and analyzing the indirect association paths between them. This mining method is widely used in investigating cybercrime.
[0003] However, existing path mining between two targets is mostly based on the topological information of the graph structure. This method has some problems, including:
[0004] Ignoring semantic information: Path mining based solely on topological structure tends to ignore the semantic information between entities, resulting in the lack of semantic accuracy and richness in the mined paths.
[0005] Path singleness: Path mining based on topological information tends to mine a single type of path, lacking diversity and comprehensiveness. This may lead to the missing of some potential associated paths, limiting the comprehensiveness of the mining results.
[0006] Ignoring user needs: This method often ignores the specific needs and application scenarios of users, and is unable to perform customized path mining based on user needs, resulting in mining results that do not meet user expectations.
[0007] Algorithm limitations: Some mining algorithms based on topological information may have limitations and cannot handle complex data relationships and diverse requirements, resulting in limited accuracy and completeness of mining results.
[0008] In order to solve the above problems, a technical solution is now provided. Summary of the invention
[0009] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a knowledge graph path mining method using conditional random fields and relationship extraction. First, through the conditional random field model and the end-to-end relationship extraction model, accurate identification of entities and precise extraction of relationships are achieved, thereby improving the efficiency and accuracy of data integration and information extraction. Secondly, a depth-first search is performed based on the constructed knowledge graph to effectively find multiple indirect association paths connecting the target entity, meeting the requirements of user needs and application scenarios. Then, by extracting the value features and diverse features in the path list, and comprehensively considering the overall value index and information richness index, the paths are arranged and screened, so that important paths are easier to be discovered and understood. It can then provide comprehensive and accurate support for the path mining of both target parties, helping users discover potential associations and paths, thereby providing a favorable reference basis for decision-making and application, so as to solve the problems raised in the above-mentioned background technology.
[0010] To achieve the above object, the present invention provides the following technical solution: Step S1, integrating and preprocessing the original data, using the conditional random field model to perform entity recognition and linking on the original data, and finding the most likely entity label sequence by learning the characteristics and relationships of the entities in the data to achieve the unification and linking of the entities;
[0011] Step S2: Based on the end-to-end relation extraction model and in combination with the attention mechanism, the semantic information and context information between entities are used to extract relations, predict the relation type and confidence between entity pairs, and retain the credible relation type.
[0012] Step S3, constructing the identified and linked entities and the extracted relationships into a knowledge graph structure, and assigning weights to the knowledge graph edges by screening similar entity pairs and calculating the co-occurrence frequency under time decay;
[0013] Step S4, based on the constructed knowledge graph, input the target parties and condition settings to obtain a path list;
[0014] Step S5, extracting the value features and diversity features of each path in the path list and arranging the paths.
[0015] In a preferred embodiment, step S1 includes the following contents:
[0016] Obtain raw data from relevant departments, pre-process the raw data, and convert text data into sequence form;
[0017] Extract features from the preprocessed data and use them as input for the conditional random field model to build and train the conditional random field model;
[0018] The trained conditional random field model is used to label the data, and the Viterbi algorithm is used to decode the unlabeled data.
[0019] In a preferred embodiment, the unlabeled data is decoded using the Viterbi algorithm, including the following:
[0020] For each position and each possible state in the input sequence, initialize the path probability matrix and path pointer matrix;
[0021] Through recursive calculation, the path probability and backtracking pointer of each position are calculated. The specific steps are as follows:
[0022] For the first position, calculate the path probability and backtracking pointer of the initial state;
[0023] For subsequent positions, calculate the maximum path probability to each state and the corresponding backtracking pointer;
[0024] At the last position, find the state corresponding to the maximum path probability as the final end state of the entity tag sequence;
[0025] According to the backtracking pointer matrix, starting from the last position, backtrack along the path with the highest path probability to the starting position to obtain a determined entity label sequence;
[0026] Preprocess the identified entities and extract the feature vector representation of the entities;
[0027] The similarity between entities is calculated using the similarity method. For each identified entity, the similarity is compared with the similarity threshold, and entities with similarity higher than the similarity threshold are linked to the same entity.
[0028] In a preferred embodiment, step S2 includes the following contents:
[0029] S2-1, use relationship extraction technology to extract the relationship between entities from the data;
[0030] S2-2, use the pre-trained word vector model to extract features from the text around the entity and obtain the context information of the entity;
[0031] S2-3, applies the attention mechanism to each node in the entity graph and dynamically adjusts the connection weights between entities according to the semantic relevance between nodes;
[0032] S2-4, taking the entity graph as input, the context information of the entity is learned through a multi-layer graph convolutional neural network to capture the semantic relationship between entities;
[0033] S2-5, a fully connected layer is applied to the output layer of the multi-layer graph convolutional neural network to predict the relationship between entity pairs, output the relationship type and its confidence, and retain the relationship type with a confidence greater than the confidence threshold.
[0034] In a preferred embodiment, step S3 includes the following contents:
[0035] Combine the vector representations of entities and relationships into a graph structure; for each entity, represent it as a node of the graph; for each relationship, represent it as an edge connecting two entity nodes, forming a knowledge graph structure with entities as nodes and relationships as edges;
[0036] First, through the similarity calculation based on the relationship vector, the entity pairs with semantic similarity scores greater than the first screening threshold are screened out. Then, for these entity pairs, the co-occurrence frequency between the entity pairs is calculated using the statistical calculation method based on the co-occurrence frequency. When calculating the co-occurrence frequency, the time decay factor is introduced to decay the long-term co-occurrence frequency. Finally, the result is used as the weight of the knowledge graph edge.
[0037] The formula for calculating collinear frequency is as follows: ;
[0038] in, and represents an entity pair, represents the co-occurrence frequency of entity pairs, represents the time point at which the entity pair first co-occurs, is the time decay factor, Indicates at a point in time The number of co-occurrences of the following entity pairs;
[0039] The co-occurrence frequency is used as the final weight of the corresponding edge.
[0040] In a preferred embodiment, step S4 includes the following contents:
[0041] Find the path connecting the target entities A and B through depth-first search. At the same time, set the maximum path length, relationship type filtering, specific attribute restrictions on entities in the path, and path deduplication conditions to ensure that the search results meet the needs and application scenarios.
[0042] Starting from target entity A, set the current path to empty and add target entity A to the current path;
[0043] Initialize the path list to be empty;
[0044] For each entity on the current path, find the entity directly connected to it and add it to the candidate node list;
[0045] For each entity in the candidate nodes list:
[0046] If the entity is directly connected to the target entity B, a path that meets the conditions is found and the current path is added to the path list;
[0047] If the path length does not reach the set maximum length limit, and the entity meets the relationship type filtering and entity attribute restriction conditions, then the entity is added to the current path, the path length is updated, and a recursive search is performed;
[0048] Recursively perform a depth-first search on the newly added entities to the path until an entity directly connected to the target entity B is found or the maximum path length is reached;
[0049] Whenever a path that meets the conditions is found, it is recorded in the path list;
[0050] When the search is completed or the maximum path length is reached, a list of all paths that meet the search criteria is returned.
[0051] In a preferred embodiment, step S5 includes the following contents:
[0052] The value characteristics include the overall value index, and the diversity characteristics include the information richness index.
[0053] In a preferred embodiment, the process of obtaining the overall value index is as follows:
[0054] d1, first, initialize each node in each path in the path list, and set its initial authority and hubness score to 1;
[0055] d2, then, update the authority and hubness scores of the nodes through iterative calculation until convergence;
[0056] For each node, update its authority score to the sum of the hubness scores of its neighboring nodes;
[0057] For each node, update its hubness score to the sum of the authority scores of its neighboring nodes;
[0058] d3, after each iteration, the authority and hubness scores of the nodes are normalized to ensure that the values are between 0 and 1;
[0059] d4, repeating steps d2 and d3 until the authority and hubness scores converge to a predetermined number of iterations;
[0060] d5, based on the calculated authority and hubness scores, comprehensively consider the weight of the nodes, add up the authority and hubness scores of each node, and then take the average of the scores of all nodes as the overall value index of the path.
[0061] In a preferred embodiment, the process of obtaining the information richness index is:
[0062] For each path, the information amount of each entity in the path is calculated by information entropy;
[0063] Then, the information amount of each relationship type in the path is calculated;
[0064] Finally, the amount of entity information and the amount of relationship information are comprehensively considered, and the weighted average of the amount of entity information and the amount of relationship information is taken as the information richness index of the path.
[0065] In a preferred embodiment, the overall value index and the information richness index are dimensionless and their numerical values are taken and normalized to obtain the correlation priority coefficient; the correlation priority coefficient is compared with the display threshold, and if the correlation priority coefficient is greater than or equal to the display threshold, the corresponding path is displayed, and the paths are sorted from large to small according to the numerical value of the correlation priority coefficient; otherwise, if the correlation priority coefficient is less than the display threshold, the path is not displayed.
[0066] The technical effects and advantages of the knowledge graph path mining method using conditional random fields and relationship extraction of the present invention are as follows:
[0067] 1. The present invention uses the conditional random field model to perform entity recognition and linking, and can accurately extract entities such as people, places, and time from multiple data sources and unify them. This processing method not only improves the efficiency of data integration, but also improves the accuracy of information extraction. By decoding unlabeled data through the Viterbi algorithm, the entity label sequence can be quickly and accurately identified, providing a reliable foundation for subsequent data processing and analysis. Finally, through the entity similarity calculation and linking method, entities with different expressions are unified, providing richer information resources for the construction and application of knowledge graphs, and providing users with more accurate and comprehensive query and analysis services.
[0068] 2. The present invention realizes an integrated process of extracting entities and relationships from data by integrating raw data, using conditional random field models for entity recognition and linking, and extracting relationships based on an end-to-end relationship extraction model combined with an attention mechanism. This integrated processing method not only improves the efficiency of entity linking and relationship extraction, but also provides a more comprehensive and coherent knowledge graph construction path while maintaining accuracy. By constructing the identified and linked entities and the extracted relationships into a knowledge graph structure, and assigning weights to the graph edges according to similar entity pairs and time-decayed co-occurrence frequencies, the connections between entities in the real world can be better reflected, thereby improving the accuracy and coherence of the knowledge graph. This not only improves the quality and accuracy of the knowledge graph, but also provides more possibilities for mining and analyzing relationships between entities, laying a solid foundation for the application and promotion of the knowledge graph.
[0069] 3. The present invention uses the constructed knowledge graph to find all possible indirect association paths through depth-first search given a target entity. In this process, multiple restrictions are set, including the maximum length of the path, filtering of relationship types, specific attribute restrictions on entities in the path, and deduplication of the path. These restrictions help to improve search efficiency and accuracy, and ensure that the obtained path meets user needs and application scenarios. Through depth-first search, multiple indirect association paths connecting target entities A and B can be quickly and accurately obtained, thereby providing users with more comprehensive and accurate association information. This path acquisition method that comprehensively considers multiple conditions can effectively meet user needs and enhance the application value and practicality of the knowledge graph.
[0070] 4. The present invention arranges the associated path list by extracting the value characteristics and diversity characteristics of each path in the path list. Among them, the value characteristics are mainly reflected by the overall value index. The authority and hubness scores of each node are calculated by the HITS algorithm, and the overall value index is obtained by integrating the node weights, which reflects the ability of the path to transmit knowledge or information; while the diversity characteristics are represented by the information richness index, which integrates the amount of information of entities and relationships, reflecting the information richness and diversity of the path. Such a processing method can make the arrangement of paths more reasonable and orderly, and help users to obtain path information that is meaningful to the target two-party relationship more quickly. By calculating the association priority coefficient, combining the overall value index and the information richness index, and filtering and sorting according to the display threshold, the visualization effect and user experience of the path list can be improved, making important paths easier for users to discover and understand. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 A schematic diagram of a process of a knowledge graph path mining method using conditional random fields and relationship extraction according to the present invention;
[0072] Figure 2 This is a flow chart of step S2 in a knowledge graph path mining method using conditional random fields and relationship extraction according to the present invention. DETAILED DESCRIPTION
[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0074] Example 1
[0075] Figure 1 The present invention provides a knowledge graph path mining method using conditional random fields and relationship extraction, comprising:
[0076] Step S1, integrate the original data and perform preprocessing, use the conditional random field model to perform entity recognition and linking on the original data, find the most likely entity label sequence by learning the characteristics and relationships of the entities in the data, and achieve entity unification and linking;
[0077] Step S2: Based on the end-to-end relation extraction model and in combination with the attention mechanism, the semantic information and context information between entities are used to extract relations, predict the relation type and confidence between entity pairs, and retain the credible relation type.
[0078] Step S3, constructing the identified and linked entities and the extracted relationships into a knowledge graph structure, and assigning weights to the knowledge graph edges by screening similar entity pairs and calculating the co-occurrence frequency under time decay;
[0079] Step S4, based on the constructed knowledge graph, input the target parties and condition settings to obtain a path list;
[0080] Step S5, extracting the value features and diversity features of each path in the path list and arranging the paths.
[0081] Step S1 includes the following contents:
[0082] Use entity recognition technology to identify target entities from raw data, such as people, places, time, etc., and perform entity linking to unify different representations of the same entity. Specifically, it includes the following:
[0083] Obtain raw data from relevant departments, including police systems, social systems, address books, and integration transactions. Raw data includes personnel information (such as name, ID number, phone number), location information (such as address, longitude and latitude), time information (such as communication time, transaction time), transaction information (such as communication records, financial transaction records), etc.
[0084] Preprocess the original data, including word segmentation, part-of-speech tagging and other operations, to convert the text data into sequence form.
[0085] Features are extracted from the preprocessed data, including information such as part of speech, word boundaries, and word forms, which are used as input for the conditional random field model.
[0086] Define the structure of the conditional random field model, including state sets, feature functions, and corresponding weights. The state set includes entity types and labels marked as non-entities, and the feature function is used to map the input sequence and state sequence to probabilities.
[0087] Entity types include entity types such as person, place, time, etc.;
[0088] Non-entity labels are used to represent the parts of the input sequence that do not belong to any entity.
[0089] Observation feature function: used to model the relationship between the features of the input sequence and the state (entity type or non-entity label). For example, you can define an observation feature function , indicating that at position The status is When the input sequence are associated with certain characteristics.
[0090] Transition feature function: used to model the relationship between adjacent states. For example, a transition feature function can be defined , indicating that at position Status by Transfer to relationship.
[0091] Among them, the weights of the conditional random field model are learned through maximum likelihood estimation or other optimization algorithms. The specific allocation process is as follows:
[0092] First, define a set of characteristic functions ,in represents the index of the characteristic function, and Respectively represent state and status, represents the input sequence, Indicates the current location.
[0093] For each sample data, calculate the value of the feature function under the current state sequence. Specifically, for each feature function, calculate the value of the feature function according to the current state sequence and the input sequence, for example, whether the condition defined by the feature function is met. If the condition is met, the value of the feature function is 1, otherwise it is 0.
[0094] Define a weight vector λ = [ λ 1 , λ 2 , …… , λ k ] , whose dimension is the same as the number of feature functions. Each feature function corresponds to a weight.
[0095] For each sample data, the weighted sum of the feature function value and the weight is calculated. Specifically, for each sample data, the weighted sum is calculated as follows: ;
[0096] in, represents the weighted sum, is the number of characteristic functions, It is The weight corresponding to the feature function is It is The value of a characteristic function in the current state sequence.
[0097] Use the softmax function to convert the weighted sum into conditional probability and calculate the conditional probability based on the weighted sum ;
[0098] in, is the conditional probability, is another possible weighted sum.
[0099] The maximum likelihood estimation is adopted and the training data set is used to learn the model parameters, and the final weight vector is the parameter of the model.
[0100] The log-likelihood loss function is used to define the loss function of the model. For each sample in the labeled dataset, the difference between the predicted sequence and the true sequence is calculated and used as part of the loss function.
[0101] Use optimization algorithms to minimize the loss function and adjust model parameters. Common optimization algorithms include stochastic gradient descent (SGD), quasi-Newton method, etc.
[0102] The model is trained iteratively using labeled data. In each iteration, the loss function is calculated and the model parameters are updated until the loss function converges or the specified number of iterations is reached.
[0103] Use the validation set or test set to evaluate the trained model. Evaluation indicators can include accuracy, recall, F1 value, etc.
[0104] Assume the evaluation results are as follows:
[0105] Accuracy: 90%
[0106] Recall rate: 85%
[0107] F1 value: 87%.
[0108] Based on the evaluation results, the model parameters are optimized, such as adjusting the feature set and regularization parameters, to improve model performance.
[0109] Use the trained conditional random field model to label the data.
[0110] Decode the unlabeled data and use the Viterbi algorithm to find the most likely entity tag sequence:
[0111] For each position and each possible state in the input sequence, initialize the path probability matrix and path pointer matrix;
[0112] Initialize the path probability matrix and the backtracking pointer matrix.
[0113] The path probability matrix records the maximum probability of reaching each state, and the backtracking pointer matrix records the previous state when reaching each state.
[0114] Through recursive calculation, the path probability and backtracking pointer of each position are calculated. The specific steps are as follows:
[0115] For the first position, calculate the path probability and backtracking pointer of the initial state;
[0116] For subsequent positions, the maximum path probability to each state and the corresponding backtracking pointer are calculated. The specific calculation formula is: ;
[0117] ;
[0118] in:
[0119] Indicates at location The previous status is The maximum probability of the path from the starting position to the position When the status is The maximum probability of
[0120] Indicates at location The previous status is When the state of the corresponding previous position is reached, that is, on the most likely path, When the status is The previous state of
[0121] Represents the characteristic function The weight parameter is used to adjust the importance of the feature in the model. In the conditional random field model, the feature function is used to capture the relationship between the input sequence and the state sequence, and the weight parameter determines the influence of the feature on the model;
[0122] Represents the characteristic function, which is used to define the relationship between the input sequence and the state sequence;
[0123] and Respectively indicate the current location Previous state and next position The status on.
[0124] At the last position, find the state corresponding to the maximum path probability as the end state of the final entity tag sequence.
[0125] According to the backtracking pointer matrix, starting from the last position, backtrack along the most likely path to the starting position to obtain the most likely entity label sequence.
[0126] The identified entities are preprocessed, including removing stop words, punctuation marks, etc., and extracting the feature vector representation of the entity.
[0127] The similarity between entities is calculated using methods such as string matching and word vector similarity. Commonly used similarity calculation methods include edit distance, cosine similarity, Jaccard similarity, etc.
[0128] For each identified entity, the similarity is compared with the similarity threshold, and entities with similarity higher than the similarity threshold are linked to the same entity.
[0129] The present invention uses the conditional random field model to perform entity recognition and linking, and can accurately extract entities such as people, places, and time from multiple data sources and unify them. This processing method not only improves the efficiency of data integration, but also improves the accuracy of information extraction. By decoding unlabeled data through the Viterbi algorithm, the entity label sequence can be quickly and accurately identified, providing a reliable foundation for subsequent data processing and analysis. Finally, through the entity similarity calculation and linking method, entities with different expressions are unified, providing richer information resources for the construction and application of knowledge graphs, and providing users with more accurate and comprehensive query and analysis services.
[0130] like Figure 2 As shown, step S2 includes the following contents:
[0131] S2-1, use relationship extraction technology to extract the relationship between entities from the original data, such as communication relationship between people, economic relationship, etc.
[0132] An end-to-end relation extraction model based on deep learning is adopted, combined with the attention mechanism and graph convolutional neural network (GCN), and the semantic information and contextual information between entity pairs are used to extract relations. The specific steps are as follows:
[0133] The entities in the text are taken as nodes, and an entity graph is constructed based on the semantic relevance between entities. Each node represents an entity, and the edges represent the semantic associations between entities.
[0134] S2-2, use pre-trained word vector models (such as Word2Vec, BERT, etc.) to extract features from the text around the entity and obtain the context information of the entity.
[0135] Among them, the training process of the word vector model is as follows:
[0136] Preprocess the original text data, including word segmentation, stop word removal, conversion to lowercase, etc., to convert the text data into a word sequence;
[0137] For the Continuous Bag of Words (CBOW) model, each training sample consists of a target word and its context words; for the Skip-gram model, each training sample consists of an input word and its surrounding context words;
[0138] Count the number of occurrences of all words and select the retained words as the vocabulary based on word frequency, assigning each word a unique index number;
[0139] Use training samples and vocabulary to train the Word2Vec model, and optimize the model parameters by maximizing the likelihood function of the sample or minimizing the loss function. In the CBOW model, the input is the average of the word vectors of the context words, and the output is the word vector of the target word; in the Skip-gram model, the input is the word vector of the target word, and the output is the word vector of the context word;
[0140] After training, each word will get a corresponding word vector, which can capture the semantic relationship between words, such as similarity, semantic analogy, etc.
[0141] S2-3, applies an attention mechanism to each node in the entity graph, dynamically adjusts the connection weights between entities based on the semantic relevance between nodes, and improves the attention between related entity pairs. In the attention mechanism, each node assigns a weight to its neighbor nodes, which indicates the degree of attention the node pays to the neighbor nodes. Typically, these weights are calculated through a learned attention distribution, which can be dynamically adjusted based on the similarity between nodes.
[0142] Use the pre-trained word vector model to obtain the semantic information of the entity, and perform feature representation on each node (i.e. entity) in the entity graph. Each node is represented as a vector that contains the semantic features of the entity.
[0143] The semantic relevance between nodes is measured by calculating the similarity between them. The similarity between nodes can be calculated using methods such as cosine similarity and inner product.
[0144] The attention mechanism is used to dynamically adjust the connection weights between entities to reflect the semantic relevance between them. The calculation steps are as follows:
[0145] a1, for each pair of entities and , calculate the similarity score between the two ;
[0146] a2, use the softmax function to convert the similarity score into attention weight : ;
[0147] in, Is with entity The number of all entities involved, Is Entity and entities The similarity score between .
[0148] a3, according to the obtained attention weight, calculate the weighted representation of the entity, and use the attention weight to weight the feature vector of the entity, that is: ;
[0149] in, and Is Entity and The feature vector of .
[0150] Through steps a1, a2, and a3, the attention mechanism is applied to calculate the attention weights between entities based on the similarity between entities. This can more accurately capture the semantic relevance between entities, thereby optimizing the representation and application effects of the knowledge graph.
[0151] Finally, according to the calculated attention weights, the connection weights between entities are dynamically adjusted. The adjusted connection weights reflect the semantic relevance between entities, making the key information in the graph more prominent for subsequent relationship extraction and knowledge graph construction.
[0152] S2-4, taking the entity graph as input, the multi-layer graph convolutional neural network (GCN) is used to learn the features of the entity context information to capture the semantic relationship between entities;
[0153] The knowledge graph is represented as a graph structure, where nodes represent entities and edges represent the relationships between entities. Such a graph is called an entity graph and can be represented by an adjacency matrix. To express;
[0154] Initialize a feature vector for each node. The feature vector can be the word vector, attribute vector, etc. of the node. The initial feature vectors form a node feature matrix.
[0155] Use a multi-layer GCN model to learn the feature representation of nodes. Assume that Layer GCN, each layer The feature update process is as follows:
[0156] b1, the weighted sum feature of the neighboring nodes of the calculation node, is expressed as: ;
[0157] in, Is a node The set of neighbor nodes of Is a node The number of neighbor nodes, Is a node In the The feature vector of the layer, Indicates the level at which neighbor information needs to be retained.
[0158] b2, using activation function Perform nonlinear transformation on the weighted and features of the nodes: ;
[0159] in, It is The weight matrix of the layer, is the first bias vector.
[0160] b3, repeat each layer in the multi-layer GCN model until the final feature representation is reached.
[0161] Through steps b1, b2, and b3, a multi-layer GCN model is used to learn features of the contextual information in the entity graph to capture the semantic relationships between entities.
[0162] S2-5, apply a fully connected layer to the output layer of the multi-layer graph convolutional neural network (GCN) to predict the relationship between entity pairs, output the relationship type and its confidence, and retain the relationship type with a confidence greater than the confidence threshold;
[0163] In the output layer of the GCN model, the feature vector of each node Represents the entity The final feature representation of is the number of layers of GCN;
[0164] Use a fully connected layer to predict the type of relationship between entity pairs and their confidence. Connect the feature vectors of the two entities, then pass through a fully connected layer and a softmax activation function to output the probability distribution of the relationship type. The calculation formula of the fully connected layer is as follows: z = σ ( W fc [ h o U , h v U ] + b fc ) ;
[0165] in, is the weight matrix of the fully connected layer, is the second bias vector, [ h o U , h v U ] It means to connect the feature vectors of two entities. is the output of the fully connected layer.
[0166] In order to distinguish and , which is now explained as follows:
[0167] Represents an entity The final output feature representation in the GCN model is is the number of layers of GCN, indicating the Entity features obtained after the GCN layer.
[0168] Represents an entity In the GCN model The feature representation of the layer, Represents the current level of GCN.
[0169] Perform a softmax operation on the output of the fully connected layer to obtain the probability distribution of each relationship type. The probability distribution of the management type is expressed as , the calculation formula is: ;
[0170] in, is the total number of relationship types. is the first in the output vector of the fully connected layer elements.
[0171] The cross entropy loss function is used to measure the difference between the predicted value and the true label. Assume that the true relationship type label is , then the cross entropy loss Calculated as: ;
[0172] in, is the first elements, is the probability distribution vector of the relationship type elements.
[0173] The back propagation algorithm is used to update the model parameters to optimize the prediction results. By calculating the gradient of the cross entropy loss function to the model parameters, the weight matrix and bias vector of the fully connected layer can be updated so that the model can better predict the relationship type between entity pairs.
[0174] Relationship types with confidence greater than or equal to the confidence threshold are retained.
[0175] By predicting the relationship type and confidence between entity pairs, the system can more accurately understand the relevance and semantic connection between entities. The prediction of relationship type can enable the system to understand the specific connection between entities, so as to better understand the semantic information in the text or data. The confidence can provide a credibility assessment of the relationship prediction results, helping the system determine the accuracy and reliability of the prediction results. Such predictions can enhance the natural language processing system's ability to understand text or data, and provide a more reliable foundation for subsequent tasks such as information retrieval and knowledge graph construction.
[0176] Step S3 includes the following contents:
[0177] The identified and linked entities and the extracted relationships are constructed into a knowledge graph structure, where entities are nodes and relationships are edges.
[0178] Combine the vector representations of entities and relationships into a graph structure. For each entity, represent it as a node of the graph; for each relationship, represent it as an edge connecting two entity nodes. Construct a knowledge graph structure with entities as nodes and relationships as edges.
[0179] First, through the similarity calculation based on the relationship vector, the entity pairs with semantic similarity scores greater than the first screening threshold are screened out. Then, for these entity pairs, the co-occurrence frequency between them is calculated using a statistical calculation method based on co-occurrence frequency. When calculating the co-occurrence frequency, consider introducing a time decay factor to appropriately decay the long-term co-occurrence frequency to better reflect the dynamic relationship between entities. Finally, the results are used as the weight of the knowledge graph edge to help build a more accurate knowledge graph structure.
[0180] Among them, the formula for calculating the collinear frequency is as follows: ;
[0181] in, and represents an entity pair, represents the co-occurrence frequency of entity pairs, represents the time point at which the entity pair first co-occurs, is the time decay factor, Indicates at a point in time The number of co-occurrences of the following entity pairs.
[0182] The co-occurrence frequency is used as the final weight of the corresponding edge.
[0183] The present invention realizes an integrated process of extracting entities and relationships from data by integrating raw data, using a conditional random field model for entity recognition and linking, and extracting relationships based on an end-to-end relationship extraction model combined with an attention mechanism. This integrated processing method not only improves the efficiency of entity linking and relationship extraction, but also provides a more comprehensive and coherent knowledge graph construction path while maintaining accuracy. By constructing the identified and linked entities and the extracted relationships into a knowledge graph structure, and assigning weights to the graph edges based on similar entity pairs and time-decayed co-occurrence frequencies, the connections between entities in the real world can be better reflected, thereby improving the accuracy and coherence of the knowledge graph. This not only improves the quality and accuracy of the knowledge graph, but also provides more possibilities for mining and analyzing relationships between entities, laying a solid foundation for the application and promotion of the knowledge graph.
[0184] Step S4 includes the following contents:
[0185] The combination of path maximum length limit, relationship type filtering, specific attribute limit of path entities, and path deduplication aims to optimize the search process and ensure that the obtained indirect association path list meets user needs and application scenarios. The path maximum length limit helps avoid searching for paths that are too long, improves search efficiency, and reduces invalid path exploration. Relationship type filtering can make search results more precise, considering only specific types of relationships that users are interested in, reducing the search space, and improving the relevance of search results. Limiting the specific attributes of entities in the path can ensure that the searched path meets user requirements. For example, in social networks, limiting the city attributes of entities in the path makes the path more meaningful. Path deduplication can avoid searching for duplicate paths and ensure the uniqueness and accuracy of search results. Taking these conditions into consideration, the search process can be effectively optimized, search efficiency can be improved, and a more indirect association path list that meets users' needs can be provided.
[0186] The path connecting the target entities A and B is found through depth-first search. At the same time, the maximum path length, relationship type filtering, specific attribute restrictions on entities in the path, and path deduplication conditions are set to ensure that the search results meet the requirements and application scenarios.
[0187] Starting from target entity A, set the current path to empty and add target entity A to the current path.
[0188] Initialize the path list to be empty.
[0189] For each entity on the current path, find the entity directly connected to it and add it to the candidate node list.
[0190] For each entity in the candidate nodes list:
[0191] If the entity is directly connected to the target entity B, a path that meets the conditions is found and the current path is added to the path list.
[0192] If the path length does not reach the set maximum length limit, and the entity meets the relationship type filtering and entity attribute restriction conditions, the entity is added to the current path, the path length is updated, and a recursive search is performed.
[0193] Recursively perform a depth-first search on the newly added entities to the path until an entity directly connected to the target entity B is found or the maximum path length is reached.
[0194] Whenever a path that meets the criteria is found, it is recorded in the path list.
[0195] When the search is completed or the maximum path length is reached, a list of all paths that meet the search criteria is returned.
[0196] For example, there is the following character relationship map:
[0197] Now we need to find all indirect connection paths between person A (Alice) and person B (Bob).
[0198] initialization:
[0199] Starting from target entity A (Alice), set the current path to empty and add Alice to the current path.
[0200] Initialize the path list to be empty.
[0201] Depth-First Search:
[0202] Search for entities directly connected to Alice and find that Alice is directly connected to Tom, Carol, and David.
[0203] Path expansion:
[0204] For Tom, check if he is directly connected to the target entity B (Bob).
[0205] Tom is directly connected to Bob, so we find a path: Alice→Tom→Bob.
[0206] For Carol, check if there is a direct connection to the target entity B.
[0207] Carol is directly connected to Bob, so find a path: Alice→Carol→Bob.
[0208] For David, check whether he is directly connected to the target entity B.
[0209] David is directly connected to Mary, and Mary is directly connected to Bob, so we find a path: Alice→David→Mary→Bob.
[0210] Recursive search:
[0211] For Tom, check whether the path can be further expanded.
[0212] Tom is directly connected to Mary, but the path length has reached the maximum limit, so the search for Tom stops.
[0213] For Carol, check whether the path can be further expanded.
[0214] Carol is directly connected to David, but the path length has reached the maximum limit, so the search for Carol stops.
[0215] For David, check whether the path can be further expanded.
[0216] David is directly connected to Carol, but the path length has reached the maximum limit, so the search for David stops.
[0217] Path record:
[0218] Record the three paths that meet the conditions found: Alice→Tom→Bob, Alice→Carol→Bob, Alice→David→Mary→Bob.
[0219] Path returns:
[0220] Returns a list of all paths that meet the search criteria: [Alice→Tom→Bob,Alice→Carol→Bob,Alice→David→Mary→Bob].
[0221] The present invention uses the constructed knowledge graph to find all possible indirect association paths through depth-first search given a target entity. In this process, multiple restrictions are set, including the maximum length of the path, filtering of relationship types, specific attribute restrictions on entities in the path, and deduplication of the path. These restrictions help to improve search efficiency and accuracy, and ensure that the obtained path meets user needs and application scenarios. Through depth-first search, multiple indirect association paths connecting target entities A and B can be quickly and accurately obtained, thereby providing users with more comprehensive and accurate association information. This path acquisition method that comprehensively considers multiple conditions can effectively meet user needs and enhance the application value and practicality of the knowledge graph.
[0222] Step S5 includes the following contents:
[0223] The value characteristics include the overall value index, and the diversity characteristics include the information richness index;
[0224] The process of obtaining the overall value index is as follows:
[0225] d1, first, initialize each node in each path in the path list, and set its initial authority and hubness score to 1;
[0226] d2, then, update the authority and hubness scores of the node through iterative calculation until convergence. In each round of iteration, the authority and hubness scores of the node are updated according to the weights of its neighbor nodes. The update formula is as follows:
[0227] For each node Authoritativeness score: ;
[0228] For each node, update its authority score to the sum of the hubness scores of its neighboring nodes;
[0229] For nodes Pivot score:
[0230] For each node, update its hubness score to the sum of the authority scores of its neighboring nodes;
[0231] in, Representation Node The set of neighbor nodes of Indicates the number of iterations.
[0232] d3, after each iteration, the authority and hubness scores of nodes are normalized to ensure that their values are between 0 and 1.
[0233] d4, repeat steps d2 and d3 until the authority and hubness scores converge to stable values or reach a predetermined number of iterations.
[0234] d5, based on the calculated authority and hubness scores, comprehensively consider the weight of the nodes, add up the authority and hubness scores of each node, and then take the average of the scores of all nodes as the overall value index of the path.
[0235] The overall value information reflects the ability of the path to transfer knowledge or information between the two connected targets. The greater the value information, the higher the information transfer ability of the path when connecting the two targets, and the higher the importance and credibility of the path. In the path list, the size of the value information is critical to the path ranking. Generally, paths with higher value information tend to be ranked higher because they are more likely to provide useful information or bring greater influence. On the contrary, paths with less value information may be ranked lower because they transfer relatively less information or are not credible enough, and have less impact on the relationship between the two targets.
[0236] When calculating the information diversity of a path, the first thing to consider is the amount of information of each entity. The amount of information of an entity refers to the amount of information carried by each entity that appears in the path. Entropy is used to measure the amount of information of an entity. Specifically, for each entity in the path, the probability distribution of the relationship type between it and the successor entity is calculated, and the amount of information of the entity is calculated based on this. The formula for calculating the amount of information is the weighted sum of the negative logarithm of the probability of each relationship type at the location of the entity.
[0237] Secondly, the amount of information of each relationship type in the path needs to be considered. The amount of information of a relationship type refers to the amount of information carried by each relationship type in the path. Entropy is also used to measure the amount of information of a relationship type. Specifically, for each relationship type in the path, the transition probability distribution from one entity to its successor entity is calculated, and the amount of information of the relationship type is calculated based on this. The calculation formula for the amount of information is the weighted sum of the negative logarithm of the transition probability between each entity at the location of the relationship type.
[0238] Finally, the information diversity of the path is calculated by comprehensively considering the amount of entity information and the amount of relationship information. The information diversity of a path refers to the amount of information contained in the path as a whole. The weighted average of the amount of entity information and the amount of relationship information is taken as the information diversity of the path. Specifically, the formula for calculating the information diversity of a path is half of the weighted average of the sum of the amount of information of all entities in the path and the sum of the amount of information of all relationship types in the path.
[0239] The process of obtaining the information richness index is as follows:
[0240] For each path, the amount of information of each entity in the path is calculated by information entropy. The amount of information of an entity can be calculated by the following formula: ;
[0241] in, Indicates the first entities, represents the total number of entities in the path, Representing Entities and its successor entities The type of relationship between Indicates the relationship type The probability of appearing in a path.
[0242] Then, the amount of information of each relationship type in the path is calculated. The amount of information of the relationship type can be calculated by the following formula:
[0243] ;
[0244] in, Indicates the first relationship type, represents the total number of relationships in the path, Represents the entity to its successor entity The transition probability.
[0245] Finally, the amount of entity information and the amount of relationship information are comprehensively considered, and the weighted average of the amount of entity information and the amount of relationship information is taken as the information richness index of the path.
[0246] The information richness index is used to represent and reflect the richness of the information content of the paths in the path list, as well as the diversity of different entities and relationship types contained in the paths. The larger the information richness index, the more diverse the entity and relationship types contained in the path, and the richer the information provided by the path. On the contrary, the smaller the information richness index, the relatively simple information content in the path, and the relatively simple information provided by the path. In the path list, the size of the information richness index is of great significance to the path ranking. Generally speaking, paths with larger information richness indexes tend to be ranked higher, because these paths contain richer information and may be more helpful for understanding the relationship between the target parties. On the contrary, paths with smaller information richness indexes may be ranked lower, because they provide relatively less information and may be less helpful for understanding the relationship between the target parties.
[0247] The overall value index and the information richness index are dimensionless and their values are normalized to obtain the correlation priority coefficient. For example, the following calculation formula can be used to obtain it: ;
[0248] in, represents the association priority coefficient, and are the normalized overall value index and information richness index, and They are the preset proportional coefficients of the normalized overall value index and information richness index, respectively, and both are greater than zero.
[0249] The relevance priority coefficient is used to represent and reflect the comprehensive priority of each path in the path list. It takes into account the overall value and information richness of the path, and is calculated by dedimensionalization and normalization, reflecting the importance and uniqueness of the path in the overall path set. A higher relevance priority coefficient means that the path has a higher priority and importance in the entire path list, while a lower relevance priority coefficient means that the path is relatively low in the ranking.
[0250] The association priority coefficient is compared with the display threshold. If the association priority coefficient is greater than or equal to the display threshold, the corresponding path is displayed and the paths are sorted from large to small according to the value of the association priority coefficient; otherwise, if the association priority coefficient is less than the display threshold, the path is not displayed.
[0251] The present invention arranges the associated path list by extracting the value characteristics and diversity characteristics of each path in the path list. Among them, the value characteristics are mainly reflected by the overall value index. The authority and hubness scores of each node are calculated by the HITS algorithm, and the overall value index is obtained by integrating the node weights, which reflects the ability of the path to transmit knowledge or information; while the diversity characteristics are represented by the information richness index, which integrates the amount of information of entities and relationships, reflecting the information richness and diversity of the path. Such a processing method can make the arrangement of the paths more reasonable and orderly, and help users to obtain path information that is meaningful to the target two-party relationship more quickly. By calculating the association priority coefficient, combining the overall value index and the information richness index, and filtering and sorting according to the display threshold, the visualization effect and user experience of the path list can be improved, making important paths easier for users to discover and understand.
[0252] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.
[0253] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or may be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.
[0254] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0255] In the several embodiments provided in the present application, it should be understood that the disclosed systems and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0256] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0257] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0258] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.
[0259] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0260] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A knowledge graph path mining method using conditional random fields and relationship extraction, characterized by: Step S1, integrate the original data and perform preprocessing, use the conditional random field model to perform entity recognition and linking on the original data, find the most likely entity label sequence by learning the characteristics and relationships of the entities in the data, and achieve entity unification and linking; Step S2: Based on the end-to-end relation extraction model and in combination with the attention mechanism, the semantic information and context information between entities are used to extract relations, predict the relation type and confidence between entity pairs, and retain the credible relation type. Step S3, constructing the identified and linked entities and the extracted relationships into a knowledge graph structure, and assigning weights to the knowledge graph edges by screening similar entity pairs and calculating the co-occurrence frequency under time decay; Step S4, based on the constructed knowledge graph, input the target parties and condition settings to obtain a path list; Step S5, extracting the value feature and diversity feature of each path in the path list and arranging the paths; Step S2 includes the following contents: S2-1, use relation extraction technology to extract the relationship between entities from the original data; S2-2, use the pre-trained word vector model to extract features from the text around the entity and obtain the context information of the entity; S2-3, applies the attention mechanism to each node in the entity graph and dynamically adjusts the connection weights between entities according to the semantic relevance between nodes; S2-4, taking the entity graph as input, the context information of the entity is learned through a multi-layer graph convolutional neural network to capture the semantic relationship between entities; S2-5, apply a fully connected layer to the output layer of the multi-layer graph convolutional neural network to predict the relationship between entity pairs, output the relationship type and its confidence, and retain the relationship type with a confidence greater than the confidence threshold; Step S3 includes the following contents: Combine the vector representations of entities and relationships into a graph structure; for each entity, represent it as a node of the graph; for each relationship, represent it as an edge connecting two entity nodes, forming a knowledge graph structure with entities as nodes and relationships as edges; First, through the similarity calculation based on the relationship vector, the entity pairs with semantic similarity scores greater than the first screening threshold are screened out. Then, for these entity pairs, the co-occurrence frequency between the entity pairs is calculated using the statistical calculation method based on the co-occurrence frequency. When calculating the co-occurrence frequency, the time decay factor is introduced to decay the long-term co-occurrence frequency. Finally, the result is used as the weight of the knowledge graph edge. The formula for calculating co-occurrence frequency is as follows: ; in, and represents an entity pair, represents the co-occurrence frequency of entity pairs, represents the time point at which the entity pair first co-occurs, is the time decay factor, Indicates at a point in time The number of co-occurrences of the following entity pairs; The co-occurrence frequency is used as the final weight of the corresponding edge; Step S4 includes the following contents: Find the path connecting the target entities A and B through depth-first search. At the same time, set the maximum path length, relationship type filtering, specific attribute restrictions on entities in the path, and path deduplication conditions to ensure that the search results meet the needs and application scenarios. Starting from target entity A, set the current path to empty and add target entity A to the current path; Initialize the path list to be empty; For each entity on the current path, find the entity directly connected to it and add it to the candidate node list; For each entity in the candidate nodes list: If the entity is directly connected to the target entity B, a path that meets the conditions is found and the current path is added to the path list; If the path length does not reach the set maximum length limit, and the entity meets the relationship type filtering and entity attribute restriction conditions, the entity is added to the current path, the path length is updated, and a recursive search is performed; Recursively perform a depth-first search on the newly added entities to the path until an entity directly connected to the target entity B is found or the maximum path length is reached; Whenever a path that meets the conditions is found, it is recorded in the path list; When the search is completed or the maximum path length is reached, a list of all paths that meet the search criteria is returned.
2. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 1, characterized in that: Step S1 includes the following contents: Obtain raw data from relevant departments, pre-process the raw data, and convert text data into sequence form; Extract features from the preprocessed data and use them as input for the conditional random field model to build and train the conditional random field model; The trained conditional random field model is used to label the data, and the Viterbi algorithm is used to decode the unlabeled data.
3. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 2, characterized in that: The unlabeled data is decoded using the Viterbi algorithm, which includes the following: For each position and each possible state in the input sequence, initialize the path probability matrix and path pointer matrix; Through recursive calculation, the path probability and backtracking pointer of each position are calculated. The specific steps are as follows: For the first position, calculate the path probability and backtracking pointer of the initial state; For subsequent positions, calculate the maximum path probability to each state and the corresponding backtracking pointer; At the last position, find the state corresponding to the maximum path probability as the final end state of the entity tag sequence; According to the backtracking pointer matrix, starting from the last position, backtrack along the path with the highest path probability to the starting position to obtain a determined entity label sequence; Preprocess the identified entities and extract the feature vector representation of the entities; The similarity between entities is calculated using the similarity method. For each identified entity, the similarity is compared with the similarity threshold, and entities with similarity higher than the similarity threshold are linked to the same entity.
4. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 3, characterized in that: Step S5 includes the following contents: The value characteristics include the overall value index, and the diversity characteristics include the information richness index.
5. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 4, characterized in that: The process of obtaining the overall value index is as follows: d1, first, initialize each node in each path in the path list, and set its initial authority and hubness score to 1; d2, then, update the authority and hubness scores of the nodes through iterative calculation until convergence; For each node, update its authority score to the sum of the hubness scores of its neighboring nodes; For each node, update its hubness score to the sum of the authority scores of its neighboring nodes; d3, after each iteration, the authority and hubness scores of the nodes are normalized to ensure that the values are between 0 and 1; d4, repeating steps d2 and d3 until the authority and hubness scores converge to a predetermined number of iterations; d5, based on the calculated authority and hubness scores, comprehensively consider the weight of the nodes, add up the authority and hubness scores of each node, and then take the average of the scores of all nodes as the overall value index of the path.
6. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 5, characterized in that: The process of obtaining the information richness index is as follows: For each path, the information amount of each entity in the path is calculated by information entropy; Then, the information amount of each relationship type in the path is calculated; Finally, the amount of entity information and the amount of relationship information are comprehensively considered, and the weighted average of the amount of entity information and the amount of relationship information is taken as the information richness index of the path.
7. A knowledge graph path mining method using conditional random fields and relationship extraction according to claim 6, characterized in that: The overall value index and information richness index are dimensionless and their values are taken and normalized to obtain the correlation priority coefficient; the correlation priority coefficient is compared with the display threshold. If the correlation priority coefficient is greater than or equal to the display threshold, the corresponding path is displayed and the paths are sorted from large to small according to the value of the correlation priority coefficient; otherwise, if the correlation priority coefficient is less than the display threshold, the path is not displayed.
