Knowledge graph completion algorithm combined with neighborhood path coding

Abstract

The invention relates to a knowledge graph completion algorithm combined with neighborhood path coding, which comprises the following steps of: constructing an attention network structure comprising three layers, respectively calculating weights of input entities, relations and neighborhood characteristics of the entities and the relations, finally calculating to obtain scores of the entities under all the input relations, and further completing a knowledge graph. The method is suitable for few-sample knowledge graph completion, can improve the efficiency of few-sample knowledge graph completion, and has excellent performance.

Description

Technical Field

[0001] The present invention relates to the field of completion algorithms, and in particular to a knowledge graph completion algorithm combined with neighborhood path coding. Background Art

[0002] In the 21st century, with the rapid development of science and technology, the amount of data has shown an exponential growth, resulting in complex information and difficulty in efficient processing. Traditional web applications and database technologies can no longer meet the needs of the current explosive data growth. In this context, knowledge graph technology has emerged as a key tool for structured organization and extraction of useful knowledge information. In 2012, Google first proposed the concept of knowledge graph, defining it as a large-scale semantic network that contains entities, concepts and the semantic relationships between them. Knowledge graph aims to organize and manage massive amounts of information in a structured way, thereby improving the query, recommendation and reasoning capabilities of data.

[0003] The core goal of the knowledge graph completion algorithm is to fill in the missing parts through the existing knowledge graph and other data sources to improve the accuracy and completeness of the graph and ensure that it can timely reflect changes in the real world and new knowledge. However, many relations in the knowledge graph often have only a few corresponding entities, which makes it difficult to effectively apply traditional large-scale training methods. With the continuous updating and expansion of the knowledge graph, some new relations may have only a few triple instances, so how to deal with few-sample relations has become an important and challenging topic.

[0004] Traditional knowledge graph completion methods assume that each relation has a large number of entity pairs for training, but in real scenarios, many relations only have a small number of samples.

[0005] Few-shot Knowledge Graph Completion (FKGC) is an important direction in knowledge graph research, and has broad application prospects, especially in the fields of medical care, personalized recommendations, and environmental monitoring.

[0006] For example, in a new product recommendation system, when user feedback data is very limited, FKGC can help the system predict user preferences more accurately. Few-sample knowledge graph completion technology does not rely on large-scale samples and can cope with relational scenarios with only a small number of samples. The focus of FKGC is to use a small number of triples as support sets to infer new relations and predict entities that may exist in the query set.

[0007] In summary, few-sample knowledge graph completion technology is crucial to ensure the integrity and accuracy of knowledge graphs. Combined with advanced neural network technology, this completion method can not only improve the semantic representation and reasoning capabilities of knowledge graphs, but also help solve practical problems in various fields through efficient data completion and noise processing, showing broad application prospects.

[0008] At present, the main challenges of FKGC include: how to mine as much useful information as possible from limited data, how to prevent model overfitting, and how to effectively deal with noise in the data.

[0009] It should be understood that the above statements merely provide background technology related to the present invention and do not necessarily constitute prior art. Summary of the invention

[0010] The purpose of the present invention is to provide a knowledge graph completion algorithm combined with neighborhood path coding, which has the advantage of significantly improving the efficiency of knowledge graph completion.

[0011] In order to achieve the above object, the present invention provides a knowledge graph completion algorithm combined with neighborhood path coding, comprising the following steps: S1, constructing a three-layer attention network structure, respectively calculating the weights of the neighborhood features of the entity, the weights of the neighborhood features of the relationship, and the weights of the neighborhood features of the entity-relationship; S2, using the attention mechanism to weight the weights of the neighborhood features of the entity, and obtaining the weighted weight X after the first layer is weighted 1 ; S3, generate entity neighbor nodes corresponding to all entities; S4, calculate the attention weights of all relations one by one; S5, use the attention mechanism to aggregate the hidden states in all relations, and obtain the weighted hidden states r′ of all relations under the bidirectional long short-term memory network model Bi-LSTM ; S6, obtain the gate value g; S7, use the gate value g as the adjustment hidden state r′ Bi-LSTM The weight vector is obtained by processing the gated multilayer perceptron network to obtain the output r′ gmlp ; Residual connection is used to obtain the final relationship representation r′, and the random manifold decoder is used to calculate the entity score Score under all input relationships.

[0012] Preferably, in S1, each layer calculates the weights of the neighborhood features corresponding to the entities, relationships, and entity-relationships one by one. The calculation formula is:

[0013]

[0014] Among them, i represents the count value of the i-th input entity, relationship, or entity-relationship; W is the weight matrix; d is the input value; b is the bias term; x represents the number of layers.

[0015] Preferably, in said S2, the calculation formula is:

[0016] X 1 =sigmoid(W attn u 1 );

[0017] Among them, u 1 W is the vector concatenated from the outputs of the attention heads representing the weights of the neighborhood features corresponding to all entities in the first layer; attn is a randomly initialized attention weight matrix; sigmoid is the activation function.

[0018] Preferably, an inter-layer attention mechanism is used to enhance the ability to distinguish various entities.

[0019] Preferably, in S3, a gating mechanism is used to generate the final entity neighbor node, and the calculation formula is:

[0020] g tmp =W g v (3) +b g ;

[0021] Among them, W g is the original weight of the gating mechanism; v (3) is the output vector representation of the weights of the neighborhood features corresponding to all entity-relationships in the third layer; b g is the bias of the gating mechanism; g tmp is the output mapping, ranging from [0,1];

[0022] O neighbor =o neigh +(1-g tmp )⊙e selfEmbed ;

[0024] Among them, e selfEmbed Represents the semantics of a single entity; ⊙ represents a mapping operation; o neigh The weight X after weighting the first layer 1 The area where the next neighbor node is located; O neighbor A single entity neighbor node corresponding to a single entity.

[0025] Preferably, in S4, the calculation formula is:

[0026]

[0027] Among them, W attn is a randomly initialized attention weight matrix; m riis the feature information of the entity in the first layer corresponding to the i-th entity neighbor node; N(v) is the set of all entity neighbor nodes; r i is the relational representation of the i-th relation; r′ is the set of relations of actual neighbor nodes; m r′i is the relationship of the i-th actual neighbor node; represents the attention weight of the i-th relation.

[0028] Preferably, in S5, aggregating hidden states refers to obtaining deeper information of the relationship in the model operation, and the calculation formula is:

[0029] r i =Tanh(W i s′ i +b i );

[0031]

[0032]

[0033] Among them, W i represents the weight matrix of the i-th relationship; b i represents the bias of the i-th relationship; r i is the relational representation of the i-th relation; c i represents the new attention weight of the i-th relation; s′ i represents the hidden output state of the i-th relationship under the bidirectional long short-term memory network model; r′ Bi-LSTM Represents the weighted hidden state of all relations in the bidirectional long short-term memory network; K represents the total number of currently input relations; Tanh is the hyperbolic tangent function.

[0034] Preferably, in S6, the calculation formula is:

[0035] g = sigmoid(W g r′ Bi-LSTM +b g );

[0037] Among them, W g is the original weight of the gating mechanism; b g is the bias of the gating mechanism; W g r′ Bi-LSTM is r′ Bi-LSTM The weighted representation of .

[0038] Preferably, in S7, the calculation formula is:

[0039] r′ gmlp =Gmlp(r′Bi-LSTM );

[0040] Among them, r′ gmlp It is the output after being processed by the gated multilayer perceptron network.

[0041] Preferably, in S8, the calculation formula is:

[0042] r′=r′ gmlp +r′ Bi-LSTM ;

[0044] Score=|S(h′,r′,t′)-MLP(z′) 2 |;

[0046] Where S(h′,r′,t′) represents the score function calculated by substituting the entity-relationship into the gated recurrent unit and using the transfer distance model; MLP is a multi-layer perceptron; z′ is r′ gmlp The initial default score.

[0047] In summary, compared with the prior art, the knowledge graph completion algorithm combined with neighborhood path coding provided by the present invention has at least the following beneficial effects:

[0048] (1) The present invention constructs a three-layer attention network structure to process the feature representation of entities, relations, and entities and relations respectively. Through the hierarchical structure and gating mechanism, key relationship information is effectively selected, the accuracy of feature representation is improved, and the complex relationship between entities is captured in more detail.

[0049] (2) The present invention dynamically adjusts the importance of relationship paths to further improve the accuracy of relationship prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of the knowledge graph completion algorithm combined with neighborhood path coding of the present invention. DETAILED DESCRIPTION

[0051] The following is combined with Figure 1 , the present invention is further explained by describing a preferred specific embodiment in detail.

[0052] It should be noted that the drawings are in a very simplified form and use non-precise proportions. They are only used to conveniently and clearly assist in explaining the embodiments of the present invention, and are not used to limit the conditions for the implementation of the present invention. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical content disclosed by the present invention.

[0053] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only the elements explicitly listed, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0054] like Figure 1 As shown, the present invention provides a knowledge graph completion algorithm combined with neighborhood path coding, which specifically includes the following steps:

[0055] S1. Construct a three-layer attention network structure to calculate the weights of the neighborhood features of the entity, the weights of the neighborhood features of the relationship, and the weights of the neighborhood features of the entity-relationship respectively;

[0056] S2. Use the attention mechanism to weight the neighborhood features of the entity and obtain the weight X after the first layer of weighting 1 ;

[0057] S3, generate entity neighbor nodes corresponding to all entities;

[0058] S4, calculate the attention weights of all relations one by one;

[0059] S5. Use the attention mechanism to aggregate the hidden states in all relations and obtain the weighted hidden states r′ of all relations under the Bi-LSTM (Bidirectional Long Short-Term Memory Network) model Bi-LSTM ; Aggregating hidden states refers to obtaining deeper information about the relationship in model operations;

[0060] S6, obtaining a gate value g;

[0061] S7, through the GMLP (gated Multi-Layer Perceptron, gated multi-layer perceptron) network, using the gated value g as the hidden state r′ Bi-LSTM The weight vector is obtained by processing the GMLP network to obtain the output r′ gmlp ;

[0062] S8. Use residual connection to obtain the final relationship representation r′, and use the stochastic manifold decoder (StochasticManifoldE Decoder, SMD) to calculate the entity score Score under all input relationships.

[0063] It should be noted that in S1, the attention network structure is a network architecture widely used in deep learning, and its core lies in the introduction of the attention mechanism. The attention mechanism allows the model to focus on the most relevant parts when processing information, thereby improving the efficiency and accuracy of the model.

[0064] Furthermore, in the S1, a three-layer attention network structure is designed, where the first layer processes entities, the second layer processes relationships, and the third layer processes entity-relationships. Entities and relationships are two core concepts in the knowledge graph, which together constitute the basic framework of the knowledge graph. Entities are the basic units in the knowledge graph, representing specific objects in the real world or the conceptual world, which can be specific physical objects. Relationships are the ties that connect different entities, representing various interactions and connections between entities.

[0065] Specifically, in S1, each layer calculates the weights of the neighborhood features corresponding to the entity, relationship, and entity-relationship respectively. The formula is as follows:

[0066]

[0067] Among them, i represents the count value of the i-th input entity, or relationship, or entity-relationship, which is only a quantitative reference and has no computational meaning; W is the weight matrix, d is the input value, that is, it represents the entity, or relationship, or entity-relationship, b is the bias term, and x represents the number of layers; through the above steps, the interactive information between entities and relationships can be more accurately represented. That is, represents the weight of the neighborhood feature corresponding to the i-th entity (because it is the first layer, it corresponds to the entity), represents the weight of the neighborhood feature corresponding to the i-th relationship (because it is the second layer, so the corresponding relationship), Represents the weight of the neighborhood feature corresponding to the i-th entity-relationship (because it is the third layer, it corresponds to the entity-relationship).

[0068] Furthermore, in S2, the weight X of the first layer after weighting 1 The calculation formula is as follows:

[0069] X 1 =sigmoid(W attn u 1 );

[0070] Among them, u 1 The vector W represents the concatenation of the outputs of the attention heads representing the weights of the neighborhood features corresponding to all entities in the first layer. attn is a randomly initialized attention weight matrix, sigmoid is an activation function that can map any real value of the input to between 0 and 1, and the output value can be interpreted as a probability or a decision boundary for a binary classification problem.

[0071] In a preferred embodiment of the present invention, an inter-layer attention mechanism is used to enhance the ability to distinguish various entities and improve the ability to represent information in the knowledge graph.

[0072] Furthermore, in S3, a gating mechanism is used to generate the final entity neighbor node. The gating mechanism can dynamically adjust the correspondence between the entity neighbor node and the entity, and generate a more accurate correspondence between the entity neighbor node and the entity by selecting or suppressing specific features according to context information;

[0073] The specific calculation formula is as follows:

[0074] g tmp =W g v (3) +b g ;

[0075] Among them, W g is the original weight of the gating mechanism, v (3) is the output vector representation of the weights of the neighborhood features corresponding to all entity-relationships in the third layer, b g is the bias of the gating mechanism, used to adjust the activity of the gate, g tmp is the output map, whose range is limited to [0,1].

[0076] O neighbor =o neigh +(1-g tmp )⊙e selfEmbed ;

[0077] Among them, e selfEmbed represents the semantics of a single entity, ⊙ represents the mapping operation, o neigh The weight X after weighting the first layer 1 The area where the next neighbor node is located, O neighbor A single entity neighbor node corresponding to a single entity.

[0078] In a preferred embodiment of the present invention, in S3, during the data transmission input and output process, a residual connection is used to enhance stability.

[0079] Furthermore, in the S4, the calculation formula is as follows:

[0080]

[0081] Among them, W attn is the randomly initialized attention weight matrix, m ri is the feature information of the entity in the first layer corresponding to the i-th entity neighbor node, N(v) is the set of all entity neighbor nodes, r i is the relational representation of the i-th relation, r′ is the set of relations of the actual neighbor nodes, m r′i is the relationship of the i-th actual neighbor node, which is naturally generated based on the weights of the neighborhood features corresponding to the entity neighbor nodes and the relationship; represents the attention weight of the i-th relation.

[0082] Furthermore, in S5, the calculation formula is as follows:

[0083] r i =Tanh(W i s′ i +b i );

[0084]

[0085]

[0086] Among them, W i represents the weight matrix of the i-th relationship, b i represents the bias of the i-th relationship, r i is the relational representation of the ith relation, c i Represents the new attention weight of the i-th relation, s′ i represents the hidden output state of the i-th relation under the Bi-LSTM model, r′ Bi-LSTM represents the weighted hidden state of all relations under Bi-LSTM, K represents the total number of relations currently input, and Tanh is the hyperbolic tangent function.

[0087] It should be noted that, in the above-mentioned S6 and S7, the gate value g is usually used to control the flow of information, that is, to determine which information should be transmitted and which information should be suppressed.

[0088] Furthermore, in S6, the calculation formula is:

[0089] g = sigmoid(W g r′ Bi-LSTM +b g );

[0090] Among them, W g is the original weight of the gating mechanism, b g is the bias of the gating mechanism, W g r′ Bi-LSTM is r′ Bi-LSTM The weighted representation of .

[0091] Furthermore, in the S7, the calculation formula is:

[0092] r′ gmlp =Gmlp(r′ Bi-LSTM );

[0093] Among them, r′ gmlp is the output after GMLP network processing. This step provides the Bi-LSTM Further characterization of r′ gmlp .

[0094] In a preferred embodiment of the present invention, residual connection is used to enhance the learning ability of the model. The specific formula is:

[0095] r′=r′ gmlp +r′ Bi-LSTM ;

[0096] Among them, r′ gmlp Represents r′ Bi-LSTM is a further feature representation of , and r′ is the final relationship representation.

[0097] Furthermore, in S8, using Stochastic ManifoldE Decoder, the calculation formula is:

[0098] Score=|s(h′,r′,t′)-MLP(z′) 2 |;

[0099] Among them, S(h′,r′,t′) represents the score function calculated by substituting the entity-relationship into the gated recurrent unit GAPE and using the TransE (Translational Distance Model) model; MLP (Multi-Layer Perceptron) is a multi-layer perceptron, and z′ is r′ gmlp The initial default score.

[0100] Therefore, the input value is analyzed according to the score to complete the knowledge graph. For example, a threshold can be set. When a certain threshold is reached, it means that the input entities, relationships, and entity-relationships in step S1 meet the requirements and enter the output or subsequent processing; if not, adjustments are made and the calculation is repeated; the threshold here can be set as needed.

[0101] In summary, the knowledge graph completion algorithm combined with neighborhood path coding provided by the present invention is suitable for few-sample knowledge graph completion, can improve the efficiency of few-sample knowledge graph completion, and has excellent performance.

[0102] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A knowledge graph completion algorithm combined with neighborhood path coding, characterized in that: The following steps are involved: S1. Construct a three-layer attention network structure to calculate the weights of the neighborhood features of the entity, the weights of the neighborhood features of the relationship, and the weights of the neighborhood features of the entity-relationship respectively; S2, use the attention mechanism to weight the neighborhood features of the entity to obtain the first-layer weighted weight X1; S3, generate entity neighbor nodes corresponding to all entities; S4, calculate the attention weights of all relations one by one; S5. Use the attention mechanism to aggregate the hidden states in all relations and obtain the weighted hidden states r′ of all relations under the bidirectional long short-term memory network model. Bi-LSTM ; S6, obtaining a gate value g; S7, using the gate value g as the adjustment hidden state r' Bi-LSTM The weight vector is obtained by processing the gated multilayer perceptron network to obtain the output r′ gmlp ; S8. Use residual connection to obtain the final relationship representation r′, and use the random manifold decoder to calculate the entity score Score under all input relationships.

2. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In S1, each layer calculates the weights of the neighborhood features corresponding to entities, relationships, and entity-relationships one by one. The calculation formula is: Among them, i represents the count value of the i-th input entity, relationship, or entity-relationship; W is the weight matrix; d is the input value; b is the bias term; x represents the number of layers.

3. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the S2, the calculation formula is: X1=sigmoid(W attn u1); Among them, u1 represents the vector of the concatenated output of the attention heads of the weights of the neighborhood features corresponding to all entities in the first layer; W attn is a randomly initialized attention weight matrix; sigmoid is the activation function.

4. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 3, characterized in that: The inter-layer attention mechanism is used to enhance the ability to distinguish various entities.

5. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the above S3, a gating mechanism is used to generate the final entity neighbor node, and the calculation formula is: g tmp =W g v (3) +b g ; Among them, W g is the original weight of the gating mechanism; v (3) is the output vector representation of the weights of the neighborhood features corresponding to all entity-relationships in the third layer; b g is the bias of the gating mechanism; g tmp is the output mapping, ranging from [0,1]; O neighbor =o neigh +(1-g tmp )⊙e selfEmbed ; Among them, e selfEmbed Represents the semantics of a single entity; ⊙ represents a mapping operation; o neigh is the area where the neighbor nodes are located under the weight X1 after the first layer weighting; neighbor A single entity neighbor node corresponding to a single entity.

6. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the S4, the calculation formula is: Among them, W attn is a randomly initialized attention weight matrix; m ri is the feature information of the entity in the first layer corresponding to the i-th entity neighbor node; N(v) is the set of all entity neighbor nodes; r i is the relational representation of the i-th relation; r′ is the set of relations of actual neighbor nodes; m r ' i is the relationship of the ith actual neighbor node; b ri represents the attention weight of the i-th relation.

7. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In S5, the aggregated hidden state refers to obtaining deeper information about the relationship in the model operation, and the calculation formula is: r i =Tanh(W i s i ′+b i ); Among them, W i represents the weight matrix of the i-th relationship; b i represents the bias of the i-th relationship; r i is the relational representation of the i-th relation; c i represents the new attention weight of the i-th relation; s i ′ represents the hidden output state of the i-th relationship under the bidirectional long short-term memory network model; r′ Bi-LSTM Represents the weighted hidden state of all relations in the bidirectional long short-term memory network; K represents the total number of currently input relations; Tanh is the hyperbolic tangent function.

8. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the above-mentioned S6, the calculation formula is: g=sigmoid(W g r′ Bi-LSTM +b g ); Among them, W g is the original weight of the gating mechanism; b g is the bias of the gating mechanism; W g r′ Bi-LSTM is r′ Bi-LSTM The weighted representation of .

9. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the above-mentioned S7, the calculation formula is: r′ gmlp =Gmlp(r′ Bi-LSTM ); Among them, r′ gmlp It is the output after being processed by the gated multilayer perceptron network.

10. The knowledge graph completion algorithm combined with neighborhood path coding as claimed in claim 1, characterized in that: In the above-mentioned S8, the calculation formula is: r′＝r′ gmlp +r′ Bi-LSTM ； Score=|S(h′,r′,t′)-MLP(z′) 2 |; Where S(h′,r′,t′) represents the score function calculated by substituting the entity-relationship into the gated recurrent unit and using the transfer distance model; MLP is a multi-layer perceptron; z′ is r′ gmlp The initial default score.

Images