Small sample knowledge graph completion method based on graph structure information

Abstract

The invention relates to the technical field of knowledge maps, discloses a small sample knowledge map complementing method based on graph structure information, and solves the problems of inaccurate entity embedding and insufficient path information utilization caused by incomplete utilization of a graph structure in the whole small sample knowledge map complementing process in the prior art. According to the method, in the training process, firstly, sub-graphs of head and tail entities are constructed, and the sub-graphs comprise neighbor sub-graphs and path sub-graphs; thirdly, head and tail entity embedding is constructed through a graph attention network, and relation embedding is constructed through Transform path interaction; and finally, converting the path-based similarity embedding phi into a scoring result by using a multi-layer perceptron layer. In the prediction process, a support set of a to-be-predicted sample is input into the model as fine tuning, and then the sample in the query set is input into the model to obtain a prediction result score.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and specifically to a small sample knowledge graph completion method based on graph structure information. Background Art

[0002] Knowledge graph is a data set stored in the form of triples, which has been widely used in intelligent fields such as text question answering and social recommendation. Existing knowledge graphs generally have problems such as insufficient information and sparse connections, resulting in limited support for downstream tasks. Based on the existing triple set, the knowledge graph completion method fully explores the potential relationship between entities and adds new fact triples to the knowledge graph, thereby achieving the effect of improving the completeness of the knowledge graph.

[0003] However, the effectiveness of traditional knowledge graph methods requires a large number of identical triple entity pairs for training, and knowledge graphs generally face problems of knowledge missing and long-tail effects, where a large amount of information is concentrated on a few popular nodes, resulting in scarcity of most entity and relationship information. To address these challenges, research has turned to the field of small sample knowledge graph completion, aiming to use meta-learning or metric learning algorithms to complete missing information and improve the completeness and coverage of the graph. However, current methods have failed to fully utilize the potential information of knowledge graphs. In response to this, this application has conducted research on small sample knowledge graph completion technology based on graph structure information, aiming to effectively utilize the neighborhood information and path information of entities and enhance the comprehensiveness of knowledge graphs.

[0004] The existing small sample knowledge graph completion technology is mainly divided into metric learning-based methods and meta-learning-based methods. Among them, the metric learning-based method aims to capture the complex interaction patterns between entities and relationships in the knowledge graph by learning low-dimensional representations. The core of this method is to construct an embedding space, which allows to determine whether there is a specific relationship between them by calculating the distance between entity and relationship vectors. Under this framework, the model training goal is to reduce the distance between positive examples and expand the distance between negative examples, so that effective knowledge reasoning and completion can be achieved even with a small amount of data. The meta-learning-based method relies on the meta-learning strategy, aiming to train a meta-model that can quickly adjust its parameters when faced with new and limited samples to achieve accurate predictions. With this, the model can learn a new relationship with only a very small number of examples, thereby adapting to new tasks.

[0005] However, the existing methods only consider part of the structural information in the knowledge graph, such as only considering neighbor node information or path information, and do not integrate the complete information of the graph structure well, so the completion effect needs to be improved. Summary of the invention

[0006] The technical problem to be solved by the present invention is: to propose a small sample knowledge graph completion method based on graph structure information, to make up for the problem of inaccurate entity and relationship embedding caused by the failure of existing methods to fully utilize subgraph structure information, so that it can better improve the completion effect in the small sample knowledge graph completion problem.

[0007] The technical solution adopted by the present invention to solve the above technical problems is:

[0008] A small sample knowledge graph completion method based on graph structure information includes the following steps:

[0009] S1: Pre-train the triples in the knowledge graph to obtain the entity relationship embedding information of all knowledge graphs; at the same time, build a small sample task set, divide different relationships into different task sets according to the knowledge graph relationship; and divide the support set and query set according to the number of training samples;

[0010] S2: Use the small sample task set as the positive sample set, perform negative sampling, add the negative sampling results to the positive sample set to form an expanded sample set, extract a knowledge graph subgraph for each sample in the sample set, and construct the corresponding neighbor subgraph and path subgraph;

[0011] S3: Dynamically assign weights to the first-layer neighbor nodes and the second-layer neighbor nodes through the graph attention network to aggregate the central node information and finally obtain the central entity embedding;

[0012] S4: Obtain relation embedding through the constructed path subgraph, construct dual relation embedding, model the relation representation from both aggregation and semantic interaction aspects, and thus realize triple relation representation embedding;

[0013] S5: Match and score the relationship representation embeddings of the triples in the training set and the triples in the test set to obtain the loss value, and train the small sample knowledge graph completion model based on the final loss value through the stochastic gradient descent method;

[0014] S6. Divide the small sample task to be tested into a support set and a query set. Generate scores for the candidate triples in the query set through the model, and finally select the one with the highest score as the completion result.

[0015] As an implementation method, step S1 is specifically as follows:

[0016] S11. Pre-train the triples in the knowledge graph by transferring the embedding model to obtain the pre-embedded information of all triples;

[0017] S12, divide the knowledge graph into small sample tasks according to the relationship, select a set number of triples of relationships as small sample tasks to construct a data set, and divide the training, verification and test sets according to the set ratio; and divide the other triples in the knowledge graph into background knowledge graphs;

[0018] As an implementation manner, step S2 specifically includes the following sub-steps:

[0019] S21: Negative sampling construction, using Bernoulli negative sampling, constructs n for each sample in the positive sample set S samp negative samples and compiled into a group of positive and negative samples, where n samp is the hyperparameter of the number of negative samples;

[0020] S22: Neighborhood subgraph construction; construct a two-layer neighbor subgraph based on the central node. For a pair of triples, construct a subgraph with the head entity and the tail entity as the central node. First, select the central node v0∈V in the entity set V, and obtain all the first-layer neighbor node sets N1(v0) of the central node:

[0021] N1(v0)={v∈V|(v0,v)∈E}

[0022] Among them, v is a layer of neighbor nodes, E is the edge set between nodes; then get the second layer neighbor node set N2(v0):

[0023]

[0024] Among them, u is the second-layer neighbor node; for the first-layer neighbor node set, all nodes are taken, and for the second-layer neighbor node set, the first n non-repeated nodes are taken;

[0025] Step 23, path subgraph construction: using a bidirectional breadth-first search algorithm to extract the relationship path subgraph between entity pairs;

[0026] For the set S of supporting entity pairs r Each supporting entity pair h s , t s ∈S r Specifically, given the maximum path length T, the starting entity h s Start Execution Breadth-first search of hops, collecting neighbors with different hop counts, i.e. and the de novo entity h s the corresponding paths of different lengths to these neighbors, i.e., the left paths;

[0027] Similarly, from the tail entity t s Start and execute Breadth-first search of hops to get neighbors And from ts to their corresponding paths, i.e. the right paths;

[0028] Then, calculate the intersection The neighbors in the intersection are used to connect the left and right paths, generating paths from h to h of different lengths. s to s The path is represented by Finally, a multi-hop path between the head and tail entities is obtained.

[0029] As an implementation method, the central entity embedding is obtained by the following steps:

[0030] First, define an attention mechanism; for any neighbor node Attention coefficient α vu is calculated as:

[0031]

[0032] Where a is the learnable weight vector of the attention mechanism, the superscript T represents the transpose; W is the weight matrix of the feature transformation, and h v and h u are the feature vectors of nodes v and u respectively, | represents the concatenation operation; LeakyReLU is the activation function; h k is the feature vector of node k. At this time, the attention coefficients of the first and second layer neighbor nodes to the central node are calculated;

[0033] Then calculate the attention coefficient of the second-layer neighbor nodes to the first-layer neighbor nodes; for any first-layer neighbor nodes And any second-layer neighbor node corresponding to the first-layer neighbor node u Attention coefficient α uk is calculated as:

[0034]

[0035] Aggregate feature h′ of the first layer neighbor nodes u The formula is as follows:

[0036]

[0037] Among them, σ() is the activation function, α vk is the attention coefficient of the central node to the second layer neighbor nodes, α uk is the attention coefficient of the first-layer neighbor nodes to the second-layer neighbor nodes;

[0038] Further, the embedding representation h′ of the central node v is v It is obtained by linear combination of weighted neighbor features, and the formula is as follows:

[0039]

[0040] Among them, r vu is the information of a layer of neighbor relations. For connection operation;

[0041] Finally, the node feature matrix X is constructed to represent the initial transformation of each node, with a shape of (N, F), where N is the number of nodes and F is the feature dimension of each node. Then, the adjacency matrix A is constructed to represent the structure of the graph, with a shape of (N, N). Then, the node embedding representation is optimized by minimizing the reconstruction error of the inner product decoder, and finally the embedding representation of the central entity node is obtained.

[0042] As an implementation method, the implementation process of embedding triple relationship representation is as follows:

[0043] S41: First, define the constructed path subgraph and traverse a path starting from the head node h; this path can be represented as a series of ordered pairs of nodes and edges (r1, r2, ..., r l ), formalized as:

[0044] h→u1→u2…→u l-1 →t

[0045] Each node u i and edge i Therefore, for any path π=(r1→r2…→r l ), we can define a sequence R π To accurately represent the traversal order of nodes and edges on the path π;

[0046] The attention mechanism is used to calculate the attention weight of each relationship. Given the combination of the head node and the timestamp (h, t), the attention weight n i The calculation formula is:

[0047]

[0048] Among them, c r is the target relation vector, The vector representation of the system in the path, U and θ are training parameters, U T and θ T represents its matrix transpose, and [·;·] represents the concatenation operation of vectors; the intermediate representation n through nonlinear transformation i To calculate the normalized importance weight of each relationship, we use a softmax function:

[0049]

[0050] Among them, R h→trepresents a set of path relationships from the head node h to the tail node t, β i is the normalized importance weight of each relationship.

[0051] Finally, a bidirectional gated recurrent unit is used to take each path as input and output the relationship level of the path.

[0052] S42: Model the relation representation by constructing triple semantic aspects, interactively process the N paths using the Transformer encoder, process the path embeddings through the self-attention and feedforward network in the Transformer encoder, and output h final It contains rich semantic representation that takes into account all path information, and obtains the embedding vectors corresponding to N paths.

[0053] Given a source node h in the support set s With the target node t s The path embedding collection And given the source node h in the query set q With the target node t q The path embedding collection Using the Transformer encoder, we construct a set of support set path embedding vectors and the query set path embedding vector collection Construct the similarity matrix S h , where the elements of the matrix express and similarities between;

[0054] Then based on the similarity matrix S h Use the radial basis function RBF kernel aggregation function to extract the accumulated similar features and obtain the final feature vector

[0055] As an implementation method, step 5 is specifically as follows:

[0056] A multi-layer perceptron (MLP) layer is used to convert the path-based similarity embedding φ into a score, and max pooling is performed on the scores of all samples to output a single score g(h q ,t q ,S r ), specifically expressed as:

[0057]

[0058] Where P(h i ,t i ), i = s, t is h i to iPath embedding set; φ(,) represents the feature vector obtained by step S42 using the similarity matrix, and is trained by the loss function, the loss function L r The expression is as follows:

[0059]

[0060] Among them, m represents the interval boundary between positive and negative samples, and the set The query entity pair containing the error They are the correct query entities Q r ={(h q ,t q )}Relative error entity pair.

[0061] The beneficial effects of the present invention are:

[0062] 1) The role of neighbor subgraphs in entity embedding is considered. The first-layer and second-layer neighbor nodes of the head and tail entities are represented, and then the graph attention network is used to dynamically assign weights to the first-layer and second-layer neighbor nodes respectively to aggregate the central node information, thereby improving the accuracy of the small sample knowledge graph completion problem.

[0063] 2) Considering the role of path subgraphs in relation embedding, the dual relation embedding is constructed to model relation representation from both aggregation and semantic interaction aspects. In terms of aggregation, the encoder model in Transformer is used to extract the structural hierarchical information of multiple paths between the head and tail entities, and the different hierarchical features of the relation paths and the head and tail entities are extracted in combination with the head and tail entity embeddings, and the paths between the head and tail entities are classified and aggregated. In terms of interaction, the multi-hop paths of the support set and the multi-hop paths of the query set are used to construct a similarity matrix for interactive matching, thereby realizing the embedding of triple relation representations and further improving the accuracy of the small sample knowledge graph completion problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Constructing a graph for the neighbor subgraph in the present invention;

[0065] Figure 2 Constructing a graph for the path subgraph in the present invention;

[0066] Figure 3 It is a small sample knowledge graph entity embedding model diagram based on neighborhood information aggregation in the present invention;

[0067] Figure 4 This is a small sample knowledge graph relation embedding model diagram based on relation path aggregation in the present invention. DETAILED DESCRIPTION

[0070] The present invention aims to propose a small sample knowledge graph completion method based on graph structure information to make up for the problem that the existing methods fail to effectively use the structure graph in the knowledge graph to embed entity and relationship representations. The method mainly includes the following steps:

[0071] S1. Pre-train the knowledge graph to obtain the entity relationship embedding information of all knowledge graphs. At the same time, build a small sample task set, divide different relationships into different task sets according to the knowledge graph relationship, and divide the support set and query set according to the number of training samples K;

[0072] S2. Use the positive sample set of the small sample knowledge graph to perform negative sampling operations, add the results of negative sampling to the positive sample set to form an expanded sample set, extract a knowledge graph subgraph for each sample in the sample set, and construct the corresponding neighbor subgraph and path subgraph;

[0073] S3, dynamically assign weights to the first-layer neighbor nodes and the second-layer neighbor nodes through the graph attention network to aggregate the central node information, and finally obtain the central entity embedding;

[0074] S4. Obtain relation embedding through constructed path subgraphs, construct dual relation embedding, and model relation representation from both aggregation and semantic interaction aspects. This ultimately achieves triple relation representation embedding;

[0075] S5. Match and score the triples in the support set and the triples in the query set to obtain a loss value, and train a small sample knowledge graph completion model based on the final loss value using the stochastic gradient descent method;

[0076] S6. Divide the small sample task to be tested into a support set and a query set. Generate scores for the candidate triples in the query set through the model, and finally select the one with the highest score as the completion result.

[0077] Example:

[0078] This example uses the NELL-One dataset to specifically illustrate the overall process:

[0079] Step 1: First, preprocess the sample data. The data preprocessing operation specifically includes two sub-steps: pre-training the knowledge graph and building a small sample task set:

[0080] Step 11: Pre-train the triples in the knowledge graph through the TransE (transfer embedding) model to obtain the pre-embedded information of all triples;

[0081] Step 12: Divide the knowledge graph into small sample tasks according to the relationship, select the relationship of triples less than 500 but greater than 50 as the small sample task to build the NELL-One dataset. For the training, verification and test task relationships, the division ratios of the NELL-One dataset are 51, 5, and 11 respectively. And divide the other triples in the knowledge graph into the background knowledge graph;

[0082] Step 2: Perform negative sampling on the NELL-One dataset and construct subgraphs in the training set and test set, including neighbor subgraphs and path subgraphs.

[0083] Step 21: Negative sampling construction, using Bernoulli negative sampling, construct n for each sample in the positive sample set S samp negative samples and compiled into a group of positive and negative samples, where n samp is a hyperparameter of the number of negative samples. The formula for Bernoulli negative sampling is as follows:

[0084]

[0085] Among them, ht is the average number of tail entities corresponding to each head entity in the knowledge graph G; th is the average number of head entities corresponding to each tail entity in the knowledge graph G; p is the Bernoulli probability of the replacement method, and a value is randomly generated in each negative sampling. When this value is less than p, the head entity of the positive sample is randomly replaced to obtain a negative sample, otherwise, the tail entity is randomly replaced to obtain a negative sample.

[0086] Step 22: Neighborhood subgraph construction, such as Figure 1 As shown, it is necessary to construct a two-layer neighbor subgraph based on the central node. For a pair of triples, a subgraph is constructed with the head entity and the tail entity as the central node. First, select the central node v0∈V in the entity set V. Get all the first-layer neighbor node sets N1(v0) of the central node:

[0087] N1(v0)={v∈V∣(v0,v)∈E},

[0088] Among them, v is a neighbor node in the first layer, and E is the edge set between nodes. Then get the second layer neighbor node set N2(v0):

[0089]

[0090] Among them, u is the second-layer neighbor node. For the first-layer neighbor node set, all nodes are taken, and for the second-layer neighbor node set, the first n non-repeated nodes are taken, and the repeated nodes in N2(v0) are considered to be removed. This method is used to construct a neighbor subgraph with the head and tail entities as the central nodes.

[0091] Step 23: Path subgraph construction, such as Figure 2As shown, considering the multi-hop path between the head and tail entities, a bidirectional breadth-first search algorithm (Bidirectional Breadth-First Search, bidirectional BFS) is used to extract the relationship path subgraph between entity pairs.

[0092] Then, for the supporting entity pair set S r Each supporting entity pair h s , t s ∈S r Specifically, given the maximum path length T, the starting entity h s Start Execution BFS of different hops collects neighbors with different hop counts, that is, And from h s The corresponding paths of different lengths to these neighbors are the left paths.

[0093] Similarly, from the tail entity t s Start and execute BFS of hops to get neighbors And from t s To their corresponding paths, that is, the right paths.

[0094] Then, calculate the intersection The neighbors in the intersection are used to connect the left and right paths, generating paths from h to h of different lengths. s to s The path is represented by Finally, the multi-hop path between the head and tail entities can be obtained.

[0095] Step 3: Entity embedding of small sample knowledge graph is performed by aggregating neighborhood information, such as Figure 3 As shown in , there is a secondary neighbor subgraph with the central node v0 as the center. This subgraph contains the direct neighbors (primary neighbors) of the central node v and the neighbors of these neighbors (secondary neighbors), but does not include the central node v itself. Represents the set of neighbors of v, using Represents a set of secondary neighbors.

[0096] In order to aggregate the information of these neighbor nodes through the graph attention network to obtain the embedded representation of entity v, we first define an attention mechanism that learns the relative importance of different neighbors to the central node. Attention coefficient α vu is calculated as:

[0097]

[0098] Among them, a is the learnable weight vector of the attention mechanism, W is the weight matrix of feature transformation, and h v and h u are the feature vectors of nodes v and u respectively, | represents the concatenation operation. LeakyReLU is the activation function. At this time, the attention coefficients of the first and second layer neighbor nodes to the central node are calculated.

[0099] Next, we need to calculate the attention coefficient of the second-layer neighbor nodes to the first-layer neighbor nodes. And any second-layer neighbor node corresponding to the first-layer neighbor node u Attention coefficient α uk is calculated as:

[0100]

[0101] Among them, h k is the feature vector of node k; after obtaining the attention coefficients of the double-layer neighbor nodes to the central node and the attention coefficients of the second-layer neighbor nodes to the first-layer neighbor nodes, it is necessary to add the importance of the central node to the second layer to the attention coefficients of the second-layer neighbor nodes to the first-layer neighbor nodes. In this way, the problem of too much redundant information of the second-layer neighbor nodes and weak correlation with the central node can be eliminated. Therefore, the aggregate feature h′ of the first-layer neighbor nodes is u The formula is as follows:

[0102]

[0103] Among them, σ() is the activation function, α vk is the attention coefficient of the central node to the second layer neighbor nodes, α uk is the attention coefficient of the first-layer neighbor nodes to the second-layer neighbor nodes. In this way, the embedded representation of the first-layer neighbor nodes is obtained. This embedded representation not only combines the neighborhood information of the second-layer neighbor nodes but also strengthens the relevance to the central node through the graph attention network.

[0104] Finally, the embedding representation h′ of the central node v is further transformed into v It is obtained by linear combination of weighted neighbor features, which not only combines the aggregation feature h′ of one layer of neighbor nodes u The embedding also contains information about the neighbor relationship r vu The connection operation is performed to embed, and the central node embedding representation is more perfected by aggregating the information of relationships and neighbors. The formula is as follows:

[0105]

[0106] After obtaining the embedded representation of the central node, it is equivalent to completing the encoding function of the image autoencoder. Then, the node feature matrix X is constructed to represent the initial normalization of each node. The shape is (N, F), where N is the number of nodes and F is the feature dimension of each node. Then, the adjacency matrix A is constructed to represent the structure of the graph. The shape is (N, N), where A ij Indicates whether there is an edge between node i and node j, Indicates A ij The transpose of the matrix. The node embedding representation is then optimized by minimizing the reconstruction error of the inner product decoder. The goal of the graph autoencoder is to minimize the reconstruction error, and a binary cross entropy loss function can be used.

[0107]

[0108] This results in a more accurate embedding representation of the central entity node.

[0109] Step 4: Relation embedding of small sample knowledge graph is performed through relationship path aggregation, such as Figure 4 As shown in the figure, the relation embedding is obtained by constructing the path subgraph, and the dual relation embedding is constructed to model the relation representation from the aggregation aspect, as follows:

[0110] Step 41, model the relationship representation by aggregating the relationship path aspects, such as Figure 4 As shown in the upper part, we first define the constructed path subgraph and traverse a path starting from the head entity h. This path can be represented as a series of ordered pairs of nodes and edges (r1, r2, …, r l ), formalized as:

[0111] h→u1→u2…→u l-1 →t

[0112] Each node u i and edge i Therefore, for any path π=(r1→r2…→r l ), we can define a sequence R π To accurately represent the traversal order of nodes and edges on the path π.

[0113] In the process of building a relation-level attention model based on neighborhood information, in order to correctly evaluate the relevance between each relation in the path and the target relation, an attention mechanism is adopted, which is based on the state vector h of the head node. t And the relationship vector c r To calculate the attention weight of each relationship. Given a combination of head node and timestamp (h, t), the calculation formula of attention weight is:

[0114]

[0115] Among them, c r is the target relation vector, Represents the vector representation of the system in the path, U and θ are trainable parameters, U T and θ T represents its matrix transpose, and [·;·] represents the concatenation operation of vectors. The intermediate representation n through nonlinear transformation i is used to calculate the normalized importance weight of each relationship, which is achieved through a softmax function:

[0116]

[0117] Here, R h→t represents a set of path relationships from the head node h to the tail node t, β i is the normalized importance weight of each relationship.

[0118] Finally, a Bi-GRU (Bidirectional Gated Recurrent Unit) model is used to take each path as input and output the relationship level of the path.

[0119] Step 42: Model the relation representation by constructing triple semantic aspects, such as Figure 4 As shown in the lower part, the Transformer encoder is introduced to interactively process N paths. The Transformer model relies on its self-attention mechanism to handle the interaction between elements in the sequence data; it processes the path embedding through self-attention and feedforward network. Output h final It will contain a rich semantic representation that takes into account all path information. In this way, the embedding vectors corresponding to N paths are obtained.

[0120] Given a source node h in the support set s With the target node t s The path embedding collection And given the source node h in the query set q With the target node t q The path embedding collection Using the previous method, we construct a support set path embedding vector set and the query set path embedding vector collection Construct the similarity matrix S h , where the elements of the matrix express and The similarities between.

[0121] Then, an aggregation function is applied to extract similarity features. Since the paths extracted from the graph are unordered and independent of each other, the RBF (Radial Basis Function) kernel aggregation function is used to extract the accumulated similarity features. Specifically, the matrix S is first h Each line of Perform the maximum pooling operation to obtain the maximum similarity It represents the set of embedding vectors h(h q ,t q ) is the most likely matching item h i Next, is converted into a feature vector of length Γ Generate through Γ RBF kernels, each feature The mean μ is used by the γth kernel γ and variance σ γ The formula is as follows. All kernels represent the distribution of similarity, so express Close to μ γ The conversion from one-dimensional similarity to Γ-dimensional feature vector can improve the distinguishing ability of similarity features.

[0122]

[0123] Step 5: Construct a loss function to score the triples, match and score the triples in the support set and the triples in the query set, and train the small sample knowledge graph completion model based on the final loss value through the stochastic gradient descent method. The specific contents are as follows:

[0124] A multi-layer perceptron (MLP) layer is used to convert the path-based similarity embedding φ into a score, and max pooling is performed on the scores of all K-samples to output a single score g(h q ,t q ,S r ), specifically expressed as:

[0125]

[0126] where h q ,t q ∈S r , P(h i ,t i ) is h i to i Path embedding set. φ(,) represents the feature vector obtained by the similarity matrix through step S42, which is then trained through the loss function and the parameters are updated through the Adam optimizer; the loss function expression is as follows:

[0127]

[0128] Among them, m represents the interval boundary between positive and negative samples, L r is the loss function. The query entity pair containing the error They are the correct query entities Q r ={(h q ,t q )}Relative error entity pair.

[0129] Step 6. After completing the model training in step 5, the model can be used for actual knowledge graph completion tasks.

[0130] For example, for the incomplete triple (h, r, ?) to be completed, the possible tail entity in the head entity h and the relation r needs to be predicted. First, each entity in the candidate entity set is taken out to construct a pending triple (h, r, t i ), and then in step 5, the triples in the support set and the pending triples in the query set are taken out and put into the loss function to calculate the scores. The score array [score0, score1, …, score m ], sort the score array, and take the relationship in the candidate triple with the highest score as the completion result.

[0131] Although the present invention is described herein with reference to the embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and implementations, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A small sample knowledge graph completion method based on graph structure information, characterized in that: The following steps are involved: S1: Pre-train the triples in the knowledge graph to obtain the entity relationship embedding information of all knowledge graphs; at the same time, build a small sample task set and divide different relationships into different task sets according to the knowledge graph relationship; And divide the support set and query set according to the number of training samples; S2: Use the small sample task set as the positive sample set, perform negative sampling, add the negative sampling results to the positive sample set to form an expanded sample set, extract a knowledge graph subgraph for each sample in the sample set, and construct the corresponding neighbor subgraph and path subgraph; S3: Dynamically assign weights to the first-layer neighbor nodes and the second-layer neighbor nodes through the graph attention network to aggregate the central node information and finally obtain the central entity embedding; S4: Obtain relation embedding through the constructed path subgraph, construct dual relation embedding, model the relation representation from both aggregation and semantic interaction aspects, and thus realize triple relation representation embedding; S5: Match and score the relationship representation embeddings of the triples in the training set and the triples in the test set to obtain the loss value, and train the small sample knowledge graph completion model based on the final loss value through the stochastic gradient descent method; S6. Divide the small sample task to be tested into a support set and a query set. Generate scores for the candidate triples in the query set through the model, and finally select the one with the highest score as the completion result.

2. According to claim 1, a small sample knowledge graph completion method based on graph structure information is characterized in that: The step S1 is specifically as follows: S11. Pre-train the triples in the knowledge graph by transferring the embedding model to obtain the pre-embedded information of all triples; S12, divide the knowledge graph into small sample tasks according to the relationship, select a set number of triples of relationships as small sample tasks to construct a data set, and divide the training, validation and test sets according to the set ratio; And divide the other triplets in the knowledge graph into the background knowledge graph.

3. According to claim 1, a small sample knowledge graph completion method based on graph structure information is characterized in that: The step S2 specifically comprises the following sub-steps: S21: Negative sampling construction, using Bernoulli negative sampling, constructs n for each sample in the positive sample set S samp negative samples and compiled into a group of positive and negative samples, where n samp is the hyperparameter of the number of negative samples; S22: Neighborhood subgraph construction; construct a two-layer neighbor subgraph based on the central node. For a pair of triples, construct a subgraph with the head entity and the tail entity as the central node. First, select the central node v0∈V in the entity set V, and obtain all the first-layer neighbor node sets N1(v0) of the central node: N1(v0)={v∈V|(v0,v)∈E} Among them, v is a layer of neighbor nodes, E is the edge set between nodes; then get the second layer neighbor node set N2(v0): Among them, u is the second-layer neighbor node; for the first-layer neighbor node set, all nodes are taken, and for the second-layer neighbor node set, the first n non-repeated nodes are taken; Step 23, path subgraph construction: using a bidirectional breadth-first search algorithm to extract the relationship path subgraph between entity pairs; For the set S of supporting entity pairs r Each supporting entity pair h s , t s ∈S r Specifically, given the maximum path length T, the starting entity h s Start Execution Breadth-first search of hops, collecting neighbors with different hop counts, i.e. and the de novo entity h s the corresponding paths of different lengths to these neighbors, i.e., the left paths; Similarly, from the tail entity t s Start and execute Breadth-first search of hops to get neighbors And from t s to their corresponding paths, i.e. the right paths; Then, calculate the intersection The neighbors in the intersection are used to connect the left and right paths, generating paths from h to h of different lengths. s to s The path is represented by Finally, a multi-hop path between the head and tail entities is obtained.

4. A small sample knowledge graph completion method based on graph structure information according to claim 3, characterized in that: The central entity embedding is specifically obtained by the following steps: First, define an attention mechanism; for any neighbor node Attention coefficient α vu is calculated as: Where a is the learnable weight vector of the attention mechanism, the superscript T represents the transpose; W is the weight matrix of the feature transformation, and h v and h u are the feature vectors of nodes v and u respectively, | represents the concatenation operation; LeakyReLU is the activation function; h k is the feature vector of node k. At this time, the attention coefficients of the first and second layer neighbor nodes to the central node are calculated; Then calculate the attention coefficient of the second-layer neighbor nodes to the first-layer neighbor nodes; for any first-layer neighbor nodes And any second-layer neighbor node corresponding to the first-layer neighbor node u Attention coefficient α uk is calculated as: Aggregate feature h′ of the first layer neighbor nodes u The formula is as follows: Among them, σ() is the activation function, α vk is the attention coefficient of the central node to the second layer neighbor nodes, α uk is the attention coefficient of the first-layer neighbor nodes to the second-layer neighbor nodes; Further, the embedding representation h′ of the central node v is v It is obtained by linear combination of weighted neighbor features, and the formula is as follows: Among them, r vu is the information of one layer of neighbor relationship, and ⊕ is the connection operation; Finally, the node feature matrix X is constructed to represent the initial transformation of each node, with a shape of (N, F), where N is the number of nodes and F is the feature dimension of each node. Then, the adjacency matrix A is constructed to represent the structure of the graph, with a shape of (N, N). Then, the node embedding representation is optimized by minimizing the reconstruction error of the inner product decoder, and finally the embedding representation of the central entity node is obtained.

5. According to claim 4, a small sample knowledge graph completion method based on graph structure information is characterized in that: The implementation process of the triple relationship representation embedding is as follows: S41: First, define the constructed path subgraph and traverse a path starting from the head node h; this path can be represented as a series of ordered pairs of nodes and edges (r1, r2, ..., r l ), formalized as: h→u1→u2…→u l-1 →t Each node u i and edge i Therefore, for any path π=(r1→r2…→r l ), we can define a sequence R π To accurately represent the traversal order of nodes and edges on the path π; The attention mechanism is used to calculate the attention weight of each relationship. Given the combination of the head node and the timestamp (h, t), the attention weight n i The calculation formula is: Among them, c r is the target relation vector, represents the vector representation of the system in the path, U and θ are training parameters, and represents its matrix transpose, and [·;·] represents the concatenation operation of vectors; the intermediate representation n through nonlinear transformation i To calculate the normalized importance weight of each relationship, we use a softmax function: Among them, R h→t represents a set of path relationships from the head node h to the tail node t, β i is the normalized importance weight of each relationship. Finally, a bidirectional gated recurrent unit is used to take each path as input and output the relationship level of the path. S42: Model the relation representation by constructing triple semantic aspects, interactively process the N paths using the Transformer encoder, process the path embeddings through the self-attention and feedforward network in the Transformer encoder, and output h final It contains rich semantic representation that takes into account all path information, and obtains the embedding vectors corresponding to N paths. Given a source node h in the support set s With the target node t s The path embedding collection And given the source node h in the query set q With the target node t q The path embedding collection Using the Transformer encoder, we construct a set of support set path embedding vectors and the query set path embedding vector collection Construct the similarity matrix S h , where the elements of the matrix express and similarities between; Then based on the similarity matrix S h Use the radial basis function RBF kernel aggregation function to extract the accumulated similar features and obtain the final feature vector 6. A small sample knowledge graph completion method based on graph structure information according to claim 5, characterized in that: The step 5 is specifically as follows: A multi-layer perceptron (MLP) layer is used to convert the path-based similarity embedding φ into a score, and max pooling is performed on the scores of all samples to output a single score g(h q ,t q ,S r ), specifically expressed as: Where P(h i ,t i ), i = s, t is h i to i Path embedding set; φ(,) represents the feature vector obtained by step S42 using the similarity matrix, and is trained by the loss function, the loss function L r The expression is as follows: Among them, m represents the interval boundary between positive and negative samples, and the set The query entity pair containing the error They are the correct query entities Q r ={(h q ,t q )}Relative error entity pair.

7. The method for completing a small sample knowledge graph based on graph structure information according to claim 5, characterized in that: The formula for the Bernoulli negative sampling is as follows: Among them, ht is the average number of tail entities corresponding to each head entity in the knowledge graph G; th is the average number of head entities corresponding to each tail entity in the knowledge graph G; p is the Bernoulli probability of the replacement method, and a value is randomly generated in each negative sampling. When this value is less than p, the head entity of the positive sample is randomly replaced to obtain a negative sample, otherwise, the tail entity is randomly replaced to obtain a negative sample.

8. The method for completing a small sample knowledge graph based on graph structure information according to claim 5, characterized in that: The minimization of the reconstruction error is achieved through the binary cross entropy loss function, the formula is as follows: Among them, A ij Indicates whether there is an edge between node i and node j in the adjacency matrix A, Indicates A ij Transpose of a matrix.

9. The method for completing a small sample knowledge graph based on graph structure information according to claim 5, characterized in that: Based on the similarity matrix S h Use the radial basis function RBF kernel aggregation function to extract the accumulated similar features and obtain the final feature vector The specific steps are as follows: First, the matrix S h Each line of Perform the maximum pooling operation to obtain the maximum similarity represents the embedding vector set h(h q ,t q ) is the most likely matching item h i ,Then is converted into a feature vector of length Γ Generate through Γ RBF kernels, each feature The mean μ is used by the γth kernel γ and variance σ γ Generate, the formula is as follows: All kernels represent a distribution of similarities, so express Close to μ γ , that is, the γth similarity feature.

Images