Small-sample knowledge graph reasoning method based on graph neural network

Through the graph neural network-based method, the head and tail entity embedding representation is generated using entity relationship paths and attention mechanisms, and combined with Transformer and LSTM networks to learn entity semantic information, the problem of insufficient learning ability of relationship representation in the existing technology is solved and improved The reasoning performance and accuracy of the knowledge graph of few samples.

CN118916494APending Publication Date: 2024-11-08ZHENGZHOU UNIV

Patent Information

Application Number
CN202410966275.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing method of inference of less sample knowledge graph ignores the relationship path information and entity context semantic information between entities, resulting in insufficient learning ability of relational representation and inability to effectively complete the inference of less sample knowledge graph.

Method used

Using a graph neural network method, the relationship information is obtained through the relationship paths between entities, and the attention mechanism is used to distinguish the importance of different relationships in the relationship path to entities. The neighbor information is aggregated to generate a head and tail entity embedding representation of a small sample relationship, and Combining the context semantic information of Transformer encoder and LSTM neural network learning entities, the quality of the representation of the few-sample relationship is enhanced.

Benefits of technology

It improves the performance of knowledge graph inference in less sample, can more accurately predict the missing facts in the knowledge graph, and enhances the quality of the representation of the relationship between the few samples and the accuracy of the reasoning results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916494A_ABST
    Figure CN118916494A_ABST
Patent Text Reader

Abstract

The invention relates to the field of knowledge graph reasoning, and discloses a few-sample knowledge graph reasoning method based on a graph neural network. The method comprises the following steps: firstly, carrying out initialization embedding pre-training on knowledge graph data; secondly, an entity neighbor encoder is used for capturing relation path information between entities, embedding representation of the entities is enhanced, and more accurate relation representation is obtained by supporting neighbor relations of set entity pairs; then, a Transform encoder and an LSTM network are fused, and context semantic information of the entity is learned; and finally, the matching processor calculates the semantic similarity of the query triad and the support set, and sorts the query triad according to the similarity score of the query triad. According to the method, the information in the few-sample knowledge graph can be fully utilized, and the inference performance of the few-sample knowledge graph is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph reasoning, and specifically to a few-sample knowledge graph reasoning method based on graph neural network. Background Art

[0002] Knowledge graph refers to a large-scale semantic network composed of various entities, concepts and relationships in the real world. Knowledge graphs often represent and store real-world knowledge in the form of triples, which are generally composed of (head entity, relationship, tail entity). Entities are composed of concrete objects and abstract concepts in the real world, and relationships describe the connections between different entities or concepts. These facts are interconnected to form a huge knowledge semantic network.

[0003] Existing knowledge graph reasoning methods can achieve good reasoning performance on knowledge graphs with a large amount of entity and relationship information. However, the frequency distribution of relations in most knowledge graphs usually has a long tail problem, and many types of relations have only a small number of entity pairs, which are called few-shot relations. Few-shot knowledge graph reasoning aims to predict the missing unseen facts in the knowledge graph by giving a few-shot relation and a small number of entity pairs, and to complete the few-shot knowledge graph.

[0004] In recent years, the study of knowledge graph reasoning based on few-shot relations has attracted extensive exploration by researchers, and a variety of few-shot knowledge reasoning models have been proposed, such as metric learning-based models and meta-learner-based models. These models have achieved good results in few-shot knowledge graph reasoning, however, these methods still ignore the relationship path information and entity context semantic information between entities, and the problem of insufficient relationship representation learning ability. Summary of the invention

[0005] The present invention provides a few-sample knowledge graph reasoning method based on graph neural network. The method is a knowledge reasoning method designed to address the deficiencies of existing few-sample knowledge graph reasoning methods, and is used to make full use of the information in the few-sample knowledge graph and improve the performance of few-sample knowledge graph reasoning. The method of the present invention is based on a graph neural network. First, relationship information is obtained through the relationship path between entities, and the importance of different relationships in the relationship path to the entity is distinguished by the attention mechanism. The neighbor information is aggregated to generate the head and tail entity embedding representation of the few-sample relationship, and then the accurate relationship representation is learned by using the neighbor relationship information of the entity pairs associated with the few-sample relationship. Next, the Transformer encoder and the LSTM neural network are combined to learn the contextual semantic information of the entity, further enhance the quality of the few-sample relationship representation, and finally obtain the result of knowledge reasoning.

[0006] The present invention provides a few-sample knowledge graph reasoning method based on graph neural network, which mainly includes the following steps:

[0007] Step 100, knowledge graph data preprocessing module;

[0008] Step 110: The knowledge graph is represented as G = {E, R, F}. Where E represents the entity set {e1, e2, e3..., e E}, entity e is the most basic element in the knowledge graph. R represents the relationship set {r1,r2,r3...,r R}, a relation r represents a specific connection between different entities. F represents a set of facts {f1,f2,f3...,f F}, is the set of triples in the knowledge graph. For the fact set F, each f can be defined as a triple (h, r, t), that is, (head entity, relationship, tail entity), where h is the head entity, r is the relationship, and t is the tail entity. The background knowledge graph G′ is a subgraph of the knowledge graph G obtained by removing all the few-sample relationships from G;

[0009] Step 120: preprocess the knowledge graph using the TransE knowledge graph embedding method to obtain initial embedding vector representations of entities and relationships in the knowledge graph;

[0010] Step 130: Use R to represent the set of task relationships in the knowledge graph G. task Indicates. task Divided into meta-training set R train , meta-validation set R valid and the meta-test set R test The task relations in the test set are not visible in the training set, i.e. In the training phase, for each training task r∈R train , the related triplets are randomly divided into support sets S r and the query set Q r .

[0011] Step 200, entity neighbor encoder: for the triples (h i ,r,t i ), obtain relational information through the relational paths between entities, use the attention mechanism to distinguish the importance of different relations in the relational paths for entities, and aggregate neighbor information to generate the head and tail entity embedding representations of the few-sample relation r.

[0012] Step 300, relation representation encoder: using the neighbor relationship information of entities in the support set to reduce the dependence of the few-sample relation pairs on the associated entity pairs, and obtain a more accurate few-sample relation representation.

[0013] Step 400, Transformer learner: The LSTM neural network and the Transformer encoder are integrated. Before the Transformer encoder, the entities and relations are first processed by the LSTM network and then input into the Transformer encoder to obtain a further enhanced few-shot relation embedding representation.

[0014] Step 500, matching processor: use the attention mechanism to obtain the dynamic relationship representation of the support set, then calculate the semantic similarity between the support set and the query triple, sort the query triples according to the semantic similarity score, and obtain the inference prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the present invention;

[0016] Figure 2 is a schematic diagram of a physical neighbor encoder of the present invention;

[0017] Figure 3 is a schematic diagram of a relational representation encoder of the present invention;

[0018] Figure 4 is a schematic diagram of the Transformer learner of the present invention;

[0019] Figure 5 Schematic diagram of the matching processor of the present invention. DETAILED DESCRIPTION

[0020] In order to make the technical solutions of the embodiments of the present invention clearer and more complete, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods in the examples of the present invention.

[0021] Considering the existing few-sample knowledge graph reasoning methods have the problem of ignoring the relationship path information and entity context semantic information between entities, as well as the problem of insufficient relationship representation learning ability, the present invention example, see the attached Figure 1 , a few-sample knowledge graph reasoning method based on graph neural network is provided, which mainly includes the following five steps.

[0022] Step 100: perform pre-training for initial embedding on the knowledge graph, divide the set of task relations in the knowledge graph into a meta-training set, a meta-validation set, and a meta-test set, and randomly divide the relevant triples of each training task in the training set into a support set and a query set.

[0023] Step 200: Utilize an entity neighbor encoder to encode entity embedding representations associated with a few-sample relationship through relationship paths between entities.

[0024] Step 300: Use the relation representation encoder to learn the neighbor relationship information of the entities in the support set, reduce the dependence of the few-sample relation on the associated entity pairs, and learn a more accurate few-sample relation embedding representation.

[0025] Step 400: learn the contextual semantic information of the entity based on the Transformer learner that integrates the LSTM neural network and the Transformer encoder, so as to further enhance the quality of the few-sample relationship representation.

[0026] Step 500: Use the attention mechanism to obtain the dynamic relationship representation of the support set, then calculate the semantic similarity between the support set and the query triples, sort them according to the matching similarity scores of the query triples, and obtain the inference result.

[0027] Specifically, in some implementations, step 100 includes the following sub-steps:

[0028] Step 110: The knowledge graph is represented as G = {E, R, F}. Where E represents the entity set {e1, e2, e3..., e E}, entity e is the most basic element in the knowledge graph. R represents the relationship set {r1,r2,r3...,r R}, a relation r represents a specific connection between different entities. F represents a set of facts {f1,f2,f3...,f F}, is the set of triples in the knowledge graph. For the fact set F, each f can be defined as a triple (h, r, t), that is, (head entity, relationship, tail entity), where h is the head entity, r is the relationship, and t is the tail entity. The background knowledge graph G′ is a subgraph of the knowledge graph G obtained by removing all the few-sample relationships from G;

[0029] Step 120: Preprocess the knowledge graph using the TransE knowledge graph embedding method to obtain the initial embedding vector representation of the entity in the knowledge graph. The initial embedding vector representation of the relationship

[0030] Step 130: Use R to represent the set of task relationships in the knowledge graph G. task Indicates. task Divided into meta-training set R train , meta-validation set R valid and the meta-test set R test The task relations in the test set are not visible in the training set, i.e. In the training phase, for each training task r∈R train , the related triplets are randomly divided into support sets S r and the query set Q r .

[0031] See attached Figure 2 , given a triplet of a few-shot relation r (h i ,r,t i ), for the 2-hop relation path of the head entity or the tail entity, the relation message passing is performed through the graph neural network to aggregate the information in the relation path, and then the updated head and tail entity embedding representations are obtained.

[0032] Specifically, in some implementations, step 200 includes the following sub-steps:

[0033] Step 210: Given a triplet (h i ,r,t i ), obtain the 2-hop relationship path of the head entity or the tail entity through the graph neural network, and obtain the initial embedding representation of the entity e = {e1, e2, ..., e N} and the initial embedding representation of the relation r = {r1,r2,...,r N}.

[0034] Step 220, the initial embedding representations of the entities and relationships in step 210 are recursively processed by a graph neural network of L layers;

[0035] Step 221, at layer l, by setting the neighbor relationship r i And the entities connected to it are connected in series to get the attention value:

[0036]

[0037] in, and Respectively represent the entity v and neighbor relationship r of the l-1th layer i and neighbor entity u i Embedded representation of ;

[0038] Step 222: The attention value of step 221 is activated by LeakyReLU (·) function to obtain entity v and neighbor entity u. i Attention score:

[0039]

[0040] in, is the weight matrix;

[0041] Step 223: normalize the attention score of step 222 through the softmax function to obtain the attention weight value, which is used to distinguish the importance of different relationships to the entity:

[0042]

[0043] Among them, N(v) represents the set of neighbor relationships connected to v;

[0044] Step 224: For the attention weight value obtained in step 223, the features of the neighboring entities and the features of the entity itself are aggregated by message passing to obtain an updated entity embedding representation:

[0045]

[0046] in, and are the weight matrix and bias term of the lth layer respectively; ReLU(·) represents the activation function.

[0047] See attached Figure 3 , the neighbor relationship information of entity pairs connected by few-shot relations is used to enrich the semantic representation of few-shot relations and reduce the dependence of few-shot relation embeddings on the embeddings of associated entity pairs.

[0048] Specifically, in some implementations, step 300 includes the following sub-steps:

[0049] Step 310: adopt the idea of ​​modeling entity embedding and relation embedding of TransE model, and model the task relation embedding as the transformation of entity embedding h and t, that is, when the triple is established, h+r≈t, and the embedding representation r≈th of the few-sample task relation r is obtained;

[0050] Step 320: The i-th entity pair (h i ,h t )’s neighbor relationship, using the neighbor relationship set and To express, and They represent the neighbor relationship set of the i-th head entity and the neighbor relationship set of the tail entity respectively. The neighbor relationship vector represents:

[0051]

[0052] Where N represents the number of neighbor relationships, G R Represents a set of relations in a knowledge graph;

[0053] Step 330: To enhance the semantic expression of the few-sample relation r and reduce the interference of noise on the few-sample relation, the dot product similarity is used to calculate the similarity between the neighbor relation and r, that is,

[0054] Step 340: In an implementation method, for the result of step 330, a noise filtering strategy is used to filter out the neighbor relationship that is most similar to the task relationship. Specifically, only the first T neighbor relationship vectors with the highest similarity scores are retained, and the neighbor relationship vectors with lower scores are filtered out:

[0055]

[0056] Among them, the Top(·) function is used to select E sim (i) The top T neighbor relationship vectors with similarity scores;

[0057] Step 350: Neighbor relationship vector E′ filtered in step 340 sim (i) Encode and obtain the integrated neighbor relationship vector representation:

[0058]

[0059] Among them, r i ′ represents the integrated representation of the T neighbor relationship vectors of the support set entity pair; Represents the weight matrix during encoding;

[0060] Step 360: Set the task relations r and r of the support set i ′ is merged to obtain the task relationship embedding representation with neighbor relationship semantics:

[0061] r′=σ(w1r+w2r′ i ) (8)

[0062] Among them, σ(·) represents the activation function, represents a learnable parameter.

[0063] See attached Figure 4 In order to obtain a more accurate relational embedding representation of semantic information, a layer of LSTM network is added before the Transformer encoder. The LSTM network can process sequence data and capture sequence context information, which enhances the ability of the Transformer model to extract key information.

[0064] Specifically, in some implementations, step 400 includes the following sub-steps:

[0065] Step 410: represent the task relationship r and the corresponding entity pair (h, t) as a sequence X = (x h ,x r ,x t ), for each x in the sequence X, we need to add position embedding As the input of the learner,

[0066] Step 420: Get in step 410 After that, the first hidden state of the current LSTM unit is the last hidden state of the previous LSTM unit. Will The hidden state is input into the LSTM network together to obtain the hidden state output of the current LSTM unit.

[0067] Step 430: In the Transformer encoder, firstly, the multi-head attention layer and the residual connection & normalization layer are used for learning; then, the feedforward neural network layer composed of the fully connected layer and the activation function, and the subsequent residual connection & normalization layer introduce nonlinearity into the module, and finally the output of the Transformer learner is obtained:

[0068]

[0069] Among them, Transformer(·) represents the Transformer encoding operation, It is the final representation of the task relationship triple (h, r, t), and its representation form is

[0070] See attached Figure 5 ,The matching processor calculates the semantic similarity between the support set and the query triples, and sorts them according to the similarity scores, and uses the triples with the highest similarity score as the model inference prediction result.

[0071] Specifically, in some implementations, step 500 includes the following sub-steps:

[0072] Step 510: Generate a support set dynamic relation representation r for each query triple m ,Right now

[0073]

[0074] Step 520: According to the r obtained in step 510 m , using dot product similarity to calculate dynamic relationship representation r m and query relation representation r q The similarity score between:

[0075] score1(r m ,r q )=r m ·r q (10)

[0076] Among them, the larger the score value of score1, the higher the semantic similarity between the query triple and the support set, indicating that the possibility of the query triple is greater; the smaller the score value, the smaller the possibility of the query triple.

[0077] Step 530: query entity pair (h q , r q , t q ) Output sequence obtained by Transformer learner Then perform vector operations to get the score value

[0078] Step 540: When evaluating the matching degree of the query entity pair, the values ​​of score1 and score2 are comprehensively considered. For the query entity pair (h q , t q )'s final matching score score

[0079] score(h q , t q )=score1(r m , r q )-δscore2(h q , t q ) (11)

[0080] Among them, δ is the adjustment factor. For each entity in the candidate entity set, it is substituted into the query triple one by one to obtain the query triple set. The model will calculate the score of the query triple according to the above steps and sort it according to the score value. The one with the highest score is the inference prediction result of the triple under the few-sample task.

[0081] Finally, it should be noted that the specific embodiments described above are only preferred specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any technician familiar with the technical field, within the technical scope disclosed in the present application, who makes equivalent replacements or improvements based on the technical solutions and concepts of the present invention, should be covered within the scope of protection of the present application.

Claims

1. A few-sample knowledge graph reasoning method based on graph neural network, characterized in that: The method comprises: Initialize the knowledge graph for embedding pre-training and divide the data set; Utilize the entity neighbor encoder to encode the few-shot relational entity embedding representations through relation paths; Utilize the relation representation encoder to learn the neighbor relation information of entities in the support set; Integrate LSTM neural network and Transformer encoder to learn contextual semantic information of entities; The attention mechanism is used to obtain the dynamic relationship representation of the support set, calculate the semantic similarity, and sort the query triples according to the matching similarity scores.

2. The few-sample knowledge graph reasoning method based on graph neural network according to claim 1 is characterized in that: The steps to initialize the knowledge graph for embedding pre-training and divide the data set include: The knowledge graph is represented as G = {E, R, F}, where E represents the set of entities e, R represents the set of relations r, and F represents the set of facts f. For the fact set F, each f can be defined as a triple (h, r, t), that is, (head entity, relation, tail entity); Preprocess the knowledge graph using the TransE knowledge graph embedding method to obtain the initial embedding vector representation of entities and relationships in the knowledge graph; The set of task relations in the knowledge graph G is represented by R task Indicates. task Divided into meta-training set R train , meta-validation set R valid and the meta-test set R test The task relations in the test set are not visible in the training set, i.e. In the training phase, for each training task r∈R train , the related triplets are randomly divided into support sets S r and the query set Q r .

3. The few-sample knowledge graph reasoning method based on graph neural network according to claim 1 or 2, characterized in that: The steps of encoding the few-shot relationally associated entity embedding representations through relation paths using an entity neighbor encoder include at least: Given a triplet (h i ,r,t i ), obtain the 2-hop relationship path of the head entity or the tail entity through the graph neural network, and obtain the initial embedding representation of the entity and the relationship; The initial embedding representations of entities and relationships are recursively processed by the L-layer graph neural network to update the embedding representations of entities and relationships.

4. The few-sample knowledge graph reasoning method based on graph neural network according to any one of claims 1 to 3, characterized in that: The steps of using the relation representation encoder to learn the neighbor relation information of the entities in the support set include at least: The idea of ​​modeling entity embedding and relation embedding by TransE model is adopted, and task relation embedding is modeled as the transformation of entity embedding h and t. That is, when the triple is established, h+r≈t, and the embedding representation r≈th of the few-sample task relation r is obtained. The i-th entity pair (h i ,h t )’s neighbor relationship, using the neighbor relationship set and To express, and Represent the neighbor relationship set of the i-th head entity and the neighbor relationship set of the tail entity respectively; Use dot product similarity to calculate the similarity between neighbor relations and few-shot relations, and retain the top T neighbor relation vectors with the highest similarity scores; Encode the neighbor relationship vector to obtain the integrated neighbor relationship vector representation, and support the task relations r and r' i Merge them to obtain task relationship embedding representation with neighbor relationship semantics.

5. The few-sample knowledge graph reasoning method based on graph neural network according to any one of claims 1 to 4, characterized in that: The steps of fusing LSTM neural network and Transformer encoder to learn contextual semantic information of entities include at least: The task relationship r and the corresponding entity pair (h, t) are represented as a sequence X = (x h ,x r ,x t ), for each x in the sequence X, we need to add position embedding As the input of the learner, Will The hidden state is input into the LSTM network together to obtain the hidden state output of the current LSTM unit. Through the learning of the Transformer encoder, the output of the Transformer learner is obtained, that is, 6. The few-sample knowledge graph reasoning method based on graph neural network according to any one of claims 1 to 5, characterized in that: The steps of using the attention mechanism to obtain the dynamic relation representation of the support set, calculating the semantic similarity and sorting the query triples according to the matching similarity scores include at least: Generate a dynamic relation representation r of the support set for each query triple m ; Using dot product similarity to calculate dynamic relationship representation r m and query relation representation r q The similarity score between them, score1(r m ,r q )=r m ·r q ; Through vector operation, the translation vector score value is obtained. When evaluating the matching degree of the query entity pair, the values ​​of score1 and score2 are comprehensively considered. q ,t q )’s final matching score: score(h q ,t q )=score1(r m ,r q )-δscore2(h q ,t q ) Where δ is the adjustment factor.

Citation Information

Patent Citations

  • Knowledge graph reasoning method based on data enhancement and adaptive negative sampling technology

    CN114254117A

  • Local-global time sequence knowledge graph reasoning method based on relational graph

    CN117787411A

  • Modeling relation dynamic time dynamic knowledge graph reasoning method

    CN118013048A

Cited By

  • Medical knowledge graph construction method and system based on metric learning

    CN119884385A

  • User query behavior prediction method and device based on SQL query

    CN119938724A