A knowledge graph link prediction method and system based on dual encoding and decoding model

Abstract

The present application relates to a knowledge graph link prediction method and system based on a dual encoding and decoding model. The method embeds entities and relationships in a target knowledge graph as entity sequences and relationship sequences of the same dimension, respectively. Since the dimensions of the entity sequence and the relationship sequence are the same, the ability of a prediction model composed of a first encoder and a first decoder to capture long-distance interactions can be improved, so that the model can make full use of the interactive feature information of the entity sequence and the relationship sequence, while ensuring effective interaction, reducing the size of the embedding dimension and model parameters, and improving prediction efficiency and accuracy; then a second encoder and a second decoder based on a multi-layer perceptron are nested in the first decoder, and the fully connected layer in the second encoder can be used to map and restore the encoded feature representation in a unified dimension, thereby further capturing the long-distance interactions between entities and relationships in a lower dimension, preventing dimensionality explosion, and improving prediction efficiency and accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of knowledge graph technology, and in particular, to a knowledge graph link prediction method and system based on a dual encoding and decoding model. Background Art

[0002] Due to the advancement of natural language processing technology, the powerful knowledge representation capabilities of knowledge graphs (KGs) have been widely used in question answering systems, recommendation systems, semantic search, and expert systems. Various artificial intelligence tasks have utilized knowledge graphs to improve their performance. However, the problem currently faced by knowledge graphs is that there is a widespread phenomenon of missing entities or relationships in a large number of triples, which seriously hinders the effectiveness of knowledge graphs in practical applications.

[0003] Therefore, in order to solve the problem of missing entities and relations, link prediction representation reasoning for knowledge graph completion has emerged. This method maps entities and relations in the knowledge graph into a low-dimensional vector space, namely knowledge graph embedding (KGE). KGE simplifies the link prediction task by converting the supplementary reasoning process into a computational process in this low-dimensional space.

[0004] At present, mainstream link prediction research mainly focuses on language pre-training and transformer-based graph neural network models. However, these models have problems such as a large number of parameters and dimensionality explosion, which seriously affect the prediction efficiency and accuracy of the prediction model. Summary of the invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The main purpose of the embodiments of the present disclosure is to propose a knowledge graph link prediction method and system based on a dual encoding and decoding model, which can improve the efficiency and accuracy of link prediction.

[0007] The first aspect of the embodiment of the present application proposes a knowledge graph link prediction method based on a dual codec model, and the knowledge graph link prediction method based on a dual codec model includes:

[0008] Embed entities and relations in the target knowledge graph into entity sequences and relationship sequences respectively, where the dimensions of the entity sequences and relationship sequences are equal;

[0009] extracting a first interaction feature vector from the entity sequence and the relationship sequence according to a first encoder;

[0010] extracting a link prediction result from the first interaction feature vector according to a first decoder corresponding to the first encoder;

[0011] Wherein, the first decoder includes a second encoder and a second decoder based on a multi-layer perceptron;

[0012] The extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes:

[0013] encoding the first interaction feature vector according to the second encoder and projecting it to a second interaction feature vector of the same dimension as the first interaction feature vector;

[0014] A link prediction result is extracted from the second interaction feature vector according to the second decoder.

[0015] The embodiments of the present application have at least the following beneficial effects:

[0016] The prediction method provided in the embodiment of the present application embeds the entities and relationships in the target knowledge graph into entity sequences and relationship sequences of the same dimension respectively. Since the dimensions of the entity sequence and the relationship sequence are the same, the ability of the prediction model composed of the first encoder and the first decoder to capture long-distance interactions can be improved, so that the prediction model composed of the first encoder and the first decoder can make full use of the interactive feature information of the entity sequence and the relationship sequence, while ensuring effective interaction, reducing the size of the embedding dimension and model parameters, and improving prediction efficiency and accuracy; then, the second encoder and the second decoder based on the multi-layer perceptron are nested in the first decoder, and the fully connected layer in the second encoder can be used to map and restore the encoded feature representation in a unified dimension, so as to further capture the long-distance interactions between entities and relationships in a lower dimension, prevent dimensionality explosion, and improve prediction efficiency and accuracy.

[0017] In some embodiments of the present application, embedding entities and relationships in the target knowledge graph into entity sequences and relationship sequences respectively includes:

[0018] Determine the triples of entities in the target knowledge graph;

[0019] Embed the head entity in the triple as an entity sequence of a target dimension, and embed the relationship between the head entity and the tail entity in the triple as a relationship sequence of the target dimension;

[0020] Before extracting the first interaction feature vector from the entity sequence and the relationship sequence according to the first encoder, the method further includes:

[0021] Merging the entity sequence and the relationship sequence into a target sequence;

[0022] The extracting a first interaction feature vector from the entity sequence and the relationship sequence comprises:

[0023] A first interactive feature vector is extracted from the target sequence.

[0024] In some implementations of the present application, the expression for extracting the first interaction feature vector from the entity sequence and the relationship sequence includes:

[0025] ;

[0026] ;

[0027] ;

[0028] in, are entity sequence and relationship sequence respectively, is the normalization function, is the multi-head attention function, , , , is the weight matrix of the query projection, is the weight matrix of key projection, is the weight matrix for value projection, is the Dropout function, is a feed-forward neural network, is the first interaction feature vector.

[0029] In some embodiments of the present application, the expression for encoding the first interaction feature vector and projecting it to a second interaction feature vector having the same dimension as the first interaction feature vector includes:

[0030] ;

[0031] is the first interaction eigenvector, and Generate the weight matrix and bias vector of the second interaction feature vector for the first fully connected layer projection of the second encoder, is the second interaction eigenvector, is the ReLU function;

[0032] In some implementations of the present application, the expression for extracting the link prediction result from the second interaction feature vector includes:

[0033] ;

[0034] and Generate the weight matrix and bias vector of the link prediction result for the second fully connected layer of the second decoder, Link prediction result.

[0035] In some embodiments of the present application, the loss functions of the first encoder and the first decoder include:

[0036] ;

[0037] in, is the predicted probability, is a smooth label, Indicates The predicted probability of a head, Indicates Smooth labels of heads, The total number of heads.

[0038] In some embodiments of the present application, the target dimension is 100.

[0039] The second aspect of the embodiment of the present application proposes a knowledge graph link prediction system based on a dual encoding and decoding model, the system comprising:

[0040] A sequence embedding unit, used to embed entities and relations in the target knowledge graph into entity sequences and relationship sequences respectively, where the dimensions of the entity sequences and relationship sequences are equal;

[0041] a feature encoding unit, configured to extract a first interaction feature vector from the entity sequence and the relationship sequence according to a first encoder;

[0042] a link prediction unit, configured to extract a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder;

[0043] The first decoder includes a second encoder and a second decoder based on a multi-layer perceptron.

[0044] The extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes:

[0045] encoding the first interaction feature vector according to the second encoder and projecting it to a second interaction feature vector of the same dimension as the first interaction feature vector;

[0046] A link prediction result is extracted from the second interaction feature vector according to the second decoder.

[0047] A third aspect of an embodiment of the present application proposes an electronic device, comprising at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by at least one of the controllers to enable at least one of the controllers to execute the knowledge graph link prediction method based on the dual encoding and decoding model as described above.

[0048] The fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, it implements the knowledge graph link prediction method based on the dual encoding and decoding model as described above.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 It is a flowchart of a knowledge graph link prediction method based on a dual encoding and decoding model provided in an embodiment of the present application;

[0052] Figure 2 It is a schematic diagram of the MRR change trend of MlpE on WN18RR provided in the embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a prediction model provided in an embodiment of the present application;

[0054] Figure 4 It is a structural diagram of a knowledge graph link prediction system based on a dual encoding and decoding model provided in an embodiment of the present application;

[0055] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] like Figure 1In one embodiment of the present application, a knowledge graph link prediction method based on a dual codec model is provided. The knowledge graph link prediction method based on a dual codec model includes the following steps S110 to S130:

[0058] Step S110: embed the entities and relations in the target knowledge graph into entity sequences and relationship sequences respectively, and the dimensions of the entity sequences and relationship sequences are equal.

[0059] Step S120: extracting a first interaction feature vector from the entity sequence and the relationship sequence according to the first encoder.

[0060] Step S130: extract a link prediction result from the first interaction feature vector according to the first decoder corresponding to the first encoder.

[0061] The first decoder includes a second encoder and a second decoder based on a multi-layer perceptron;

[0062] Extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes:

[0063] Step S1310: Encode the first interactive feature vector according to the second encoder and project it to a second interactive feature vector with the same dimension as the first interactive feature vector;

[0064] Step S1320: extract a link prediction result from the second interaction feature vector according to the second decoder.

[0065] In step S110, the target knowledge graph refers to the knowledge graph that needs to be processed currently (i.e., the knowledge graph for link prediction). The dimension here refers to the length of the sequence.

[0066] In some embodiments, step S110 includes the following steps:

[0067] Step S1110: determine the triples of entities in the target knowledge graph.

[0068] Step S1120: embed the head entity in the triple as an entity sequence of the target dimension, and embed the relationship between the head entity and the tail entity in the triple as a relationship sequence of the target dimension.

[0069] Before extracting the first interaction feature vector from the entity sequence and the relationship sequence according to the first encoder in step S120, the method further includes:

[0070] Merge entity sequence and relation sequence into target sequence;

[0071] Extracting the first interactive feature vector from the entity sequence and the relationship sequence in step S120 includes:

[0072] Extract the first interaction feature vector from the target sequence.

[0073] For example, the triple is , is the head entity, is the tail entity, For the relationship between two entities, the head entity Embedded as a sequence In the Embed in sequence In which, the sequence and sequence The target dimension refers to the sequence length of the entity sequence, for example, the length is 100 or 200.

[0074] Then the sequence and sequence Merge into a target sequence, where the sequence is: . It should be noted that the subsequent sequence subscripts are expressed in uppercase.

[0075] The following set of experiments demonstrates the advantages of having the same dimension for entity sequences and relationship sequences:

[0076] Experiments are conducted based on the Multi-Layer Perceptron Encoder (MlpE). MlpE is not described in detail here.

[0077] First, the score function of the MlpE model includes:

[0078] (1)

[0079] The scoring function shows that and The size of will affect the interaction effect, so the embedding dimension of entities and relations will affect the performance of the model. From the score function, it can be seen that the embedding dimension of entities and relations will affect the capture effect of the fully connected layer on long-distance interactions, and the number of extracted feature vectors is also related to the embedding dimension of entities and relations.

[0080] In order to understand the impact of entity and relationship embedding dimension combinations on the model link prediction performance, the experiment modified the embedding dimension size of entities and relationships to simulate the interaction of different entity and relationship embedding dimensions. The evaluation indicator used is the mean mutually exclusive rank (MRR), and the experimental dataset is WN18RR. According to the commonly used dimension size, seven entity and relationship embedding dimension combinations are set. The training time is 500 epochs, and other parameters are set the same.

[0081] As the embedding dimensions of entities and relations increase, the MRR change trend of MlpE on WN18RR is as follows: Figure 2 As shown in Figure 2, in general, as the dimensions of entity and relation embeddings increase, MRR shows an upward trend.

[0082] Specifically, when the entity and relationship embedding dimension combination changes from (100, 100) to (200, 200), the model performance indicator MRR shows an upward trend, indicating that the link prediction accuracy has improved. When the embedding dimension reaches (200, 200), MRR reaches a peak of 0.471, but as the embedding dimension changes from (200, 200) to (256, 200), MRR decreases again. This may be due to overfitting; however, when the embedding dimension reaches (256, 256), MRR stabilizes without further fluctuations.

[0083] Therefore, the experiments show that within a certain range of embedding dimensions, the larger the embedding dimensions of entities and relationships, the better the interaction effect and the higher the model performance. In addition, when the embedding dimensions of entities and relationships are the same, the interaction effect is the largest and the model performance is the best.

[0084] In step S120, a first interaction feature vector is extracted from the entity sequence and the relationship sequence according to the first encoder.

[0085] In this application, the prediction model is composed of a first encoder (i.e., encoding part) and a first decoder (i.e., decoding part) (the above steps are not shown). Here, the first encoder is used to capture the interaction information in the entity sequence and the relationship sequence to obtain the first interaction feature vector. It should be noted that the first encoder can be composed of a convolutional neural network, and the interaction relationship of the sequence is extracted through the neural network, or it can be the following specific structure:

[0086] In some embodiments of the present application, step S130 specifically includes the following process:

[0087] The multi-head attention mechanism uses proportional click attention to ensure the stability of training, where each head The attention matrix is ​​calculated by the following formula (2):

[0088] (2)

[0089] because , is the input of the first encoder, then , , , are the weight matrices for query, key, and value projection respectively. Merge, where For the dimensions of the query vector (query), key vector (key), and value vector (value) in each attention head, the formula is as follows (3):

[0090] (3)

[0091] In order to avoid the sparsity problem of long sequence feature vectors in multi-head attention, this embodiment adds a sub-connection layer when inputting the feature vector into each multi-head attention sub-layer In addition, there are residual connections and normalization to ensure stable training and effective gradient flow. The result is shown in formula (4):

[0092] (4)

[0093] in, is the normalization function, is the Dropout function.

[0094] Then, this embodiment uses a feed-forward neural network (FFN) with a 1x1 convolutional layer to further process the embedded features. By performing nonlinear transformation on the input embedded features, the interactive ability of the convolution kernel is used to enhance the feature representation of entities and relationships. Processing formula It is expressed by formula (5):

[0095] (5)

[0096] in, It is a feed-forward neural network.

[0097] Then, according to formula (6), Perform residual and normalization processing:

[0098] (6)

[0099] In this embodiment, the first encoder part is composed of a multi-head attention mechanism, a feedforward neural network and its sub-connection layer. The purpose is to use the Transformer's multi-head attention mechanism to capture the interactive information in long sequences of entities and relationships, while avoiding gradient disappearance, extracting rich interactive features of entities and relationships, and improving the efficiency and accuracy of link prediction.

[0100] In step S130, a link prediction result is extracted from the first interactive feature vector according to the first decoder corresponding to the first encoder, wherein the present example embeds an encoding-decoding structure based on a multi-layer perceptron in the first decoder, that is, includes a second encoder and a second decoder, to form a prediction model with a dual encoder-decoder architecture, wherein the second encoder mainly performs the following operations:

[0101] The first interaction feature vector is encoded and projected to a second interaction feature vector having the same dimension as the first interaction feature vector.

[0102] Because, the above After being encoded by the first encoder, it is converted into a relatively high-dimensional feature representation .in, (i.e., the first interaction feature vector) contains rich entity and relationship interaction feature information. Now we need to Decoding is performed to achieve link prediction.

[0103] In designing the first decoder, this embodiment uses a multi-layer perceptron as the nested second encoder and second decoder to ensure that the long-distance interaction between entities and relationships (i.e., the first interaction feature vector obtained by the first encoder) can be further captured in a lower dimension. Since the embedding dimensions of entities and relationships are the same, this embodiment uses a fully connected layer to embed After re-encoding, it is then projected onto a feature vector of the same dimension, and then the activation function is applied to obtain the encoded The result is shown in formula (7):

[0104] (7)

[0105] is the first interaction eigenvector, and Generate the weight matrix and bias vector of the second interaction feature vector for the first fully connected layer projection of the second encoder, is the second interaction eigenvector, is the ReLU function.

[0106] Here first After hidden deactivation and normalization, the final link prediction result is obtained by the linear transformation of the second fully connected layer of the second decoder according to formula (8): :

[0107] (8)

[0108] and Generate the weight matrix and bias vector of the link prediction result for the second fully connected layer of the second decoder, Link prediction result.

[0109] This embodiment Apply the sigmoid function to obtain the probability list of predicted entities. The final scoring function of the prediction model is formula (9).

[0110] (9)

[0111] in, is the embedding matrix of each entity, is the number of entities, is the embedding dimension, is the bias term for each entity.

[0112] In some embodiments of the present application, the loss function of the prediction model composed of the first encoder and the first decoder includes:

[0113] (10)

[0114] in, is the predicted probability, is a smooth label, Indicates The predicted probability of a head, Indicates Smooth labels of heads, The total number of heads.

[0115] In step S130, link prediction for the knowledge graph refers to the task of predicting missing entities and relationships in the knowledge graph, and the link prediction result is extracted from the second interactive feature vector through the second decoder.

[0116] This embodiment has the following advantages:

[0117] The prediction method provided in the embodiment of the present application embeds the entities and relationships in the target knowledge graph into entity sequences and relationship sequences of the same dimension respectively. Since the dimensions of the entity sequence and the relationship sequence are the same, the ability of the prediction model composed of the first encoder and the first decoder to capture long-distance interactions can be improved, so that the model can make full use of the interactive feature information of the entity sequence and the relationship sequence, while ensuring effective interaction, reducing the size of the embedding dimension and model parameters, and improving prediction efficiency and accuracy; then, the second encoder and the second decoder based on the multi-layer perceptron are nested in the first decoder, and the fully connected layer in the second encoder can be used to map and restore the encoded feature representation in a unified dimension, so as to further capture the long-distance interactions between entities and relationships in a lower dimension, prevent dimensionality explosion, and improve prediction efficiency and accuracy.

[0118] The prediction method provided in the embodiment of the present application, the encoder part is composed of a multi-head attention mechanism (MHA), a feedforward neural network (FFN) and its sub-connection layer. The purpose is to capture the interactive information in the long sequence of entities and relationships based on the multi-head attention mechanism of Transformer, while avoiding the vanishing gradient, extracting the rich interactive features of entities and relationships, and thus improving the accuracy and efficiency of link prediction.

[0119] Reference Figure 3 For ease of understanding, the present application provides a knowledge graph link prediction method based on a dual encoding and decoding model, and the method includes:

[0120] Step S910: Select the target knowledge graph.

[0121] Step S920: obtain the triple corresponding to each entity in the target knowledge graph, and embed the head entity and relationship corresponding to the triple into an entity sequence and a relationship sequence respectively, where the dimensions of the two sequences are the same (ie, the sequence lengths are the same).

[0122] Step S930: construct a prediction model, where the prediction model includes a first encoder and a first decoder;

[0123] The structure of the first encoder includes: multi-head attention mechanism (MHA), feed-forward neural network (FFN) and its sub-connection layer. The purpose is to use the multi-head attention mechanism of Transformer to capture the interactive information in long sequences of entities and relations, while avoiding gradient vanishing and extracting rich interactive features of entities and relations.

[0124] The structure of the first decoder includes a decoder based on a multi-layer perceptron, which is specifically embedded with an encoder and decoder structure (i.e., including a second decoder and a second encoder). The purpose is to encode the feature vector representation mapped by the first encoder based on the second encoder, so as to extract the interactive feature information of entities and relationships again through a lower dimension. Then, the encoded feature vector is decoded and mapped into a low-dimensional feature vector (i.e., the output of the prediction model) through the second decoder, which can solve the problem of dimensionality explosion.

[0125] The following is a set of experimental data to demonstrate the advantages of building a predictive model:

[0126] First, four data sets and experimental settings are introduced, and a comparison model is proposed; then this embodiment analyzes the impact of dimensions on the prediction model by inputting different dimensions of entities and relationships; finally, this embodiment evaluates the prediction model on four data sets and analyzes the efficiency of the prediction model; the following is a detailed description:

[0127] In order to evaluate the performance of the prediction model, this embodiment uses four typical link prediction task datasets. They are: WN18RR; FB15k-237; DB100k; YAGO3-10, among which WN18RR and FB15k-237 are widely recognized benchmark datasets derived from the original WN18 and FB15k datasets. The main reason for the derivation is the ubiquity of reversible relationships in WN18 and FB15k. DB100k is a subset of DBpedia, a structured data source derived from Wikipedia, which provides a large-scale dataset with multiple relationship types, making it very suitable for evaluating model performance in various relationship contexts. On the other hand, YAGO3-10 is a subset of YAGO3. YAGO is a high-quality knowledge graph that integrates information from multiple sources such as Wikipedia and WordNet. Although YAGO3-10 has fewer relationships, it contains a large number of entities, providing rich context and diversity. The statistics of these four datasets are shown in Table 1 below.

[0128] Table 1

[0129]

[0130] There are two test environments: original and filtered. The filtering setting is for candidate triples. = , in the tail entity prediction The possibility that there may be fact triples in is a one-to-many (1-N) relationship. If such a candidate triple is ranked before the test triple, it does not mean that the test triple is not well predicted. Triples in the training set, validation set, and test set were previously filtered out. This embodiment uses two established evaluation metrics to quantify the link prediction performance of the model: mean reciprocity rating (MRR) and hits@n. The Hits@n metric calculates the percentage of test results that achieve a prediction ranking of n or lower, with common values ​​of n being 1, 3, and 10. Higher values ​​of MRR and Hits@n indicate better performance. The experimental parameters are set as follows: For the four datasets, the entity embedding dimension and relationship embedding dimension of this embodiment are set to 128 and 100, respectively. The smoothing rate is set to 0.1, the learning rate is 0.001, and the batch sizes are 2048 and 4096, respectively. The parameters are adjusted through grid search to verify the performance of the dataset. The input layer dropout rate of the first decoder is {0.0, 0.05, 0.1, 0.15, 0.2}, the hidden layer dropout rate is {0.1, 0.2, 0.3, 0.4, 0.50, 0.55, 0.60, 0.65}, and the sub-layer dropout rate is {0.0, 0.1, 0.2, 0.3, 0.4}. After searching, the best parameter combination is shown in Table 2.

[0131] Table 2

[0132]

[0133] In order to analyze whether the prediction model further enhances the interaction ability between entities and relationships and improves the model performance when the entity and relationship embedding dimensions are unified, and to evaluate the impact of the dual encoding and decoding structure of the prediction model (i.e., the first decoding includes the second encoder and the second decoder) on knowledge representation, this embodiment compares the performance of the prediction model with 19 typical models of 5 categories:

[0134] Translation-based models TransE, TuckER and QuatE;

[0135] Semantic matching-based models DistMult and ComplEx;

[0136] CNN-based models ConvE, KMAE, Hyper, InteractE and JointE;

[0137] GNN-based models R-GCN, KB-GAT, SACN, A2N and CompGCN;

[0138] Transformer-based models LP-BERT, KG-BERT, StAR, and N-MlpE;

[0139] This embodiment sets a relatively high and low embedding dimension combination on FB15K-237, such as , , , and , in order to observe the changes in different embedding dimensions of entities and relations in the prediction model and their impact on the performance of the prediction model, thereby demonstrating its large-scale scalability on large-scale KG (knowledge graph). The results are: When , the prediction model efficiency is quite considerable. The prediction model outperforms most models. When , the performance indicators of the prediction model reached the peak, surpassing all models. Experiments have shown that compared with previous models with high embedding dimensions, the prediction model can use fewer entity and relationship embedding dimensions while showing stronger model performance.

[0140] The experimental results show that the performance of the prediction model is far better than the five models listed in this embodiment. Especially on the FB15k-237 dataset, all indicators of the prediction model have reached the optimal level. In WN18RR, MRR and Hits@1 are far better than other models. Compared with the traditional convolutional models ConvE and HypER, the various indicators of the prediction model far exceed the two. Compared with JointE and InteractE, the MRR indicator of the prediction model on FB15k-237 and WN18RR is improved by 1%. Among them, the performance of the prediction model in the three indicators of Hits@1, Hits@3 and Hits@10 is particularly outstanding. Compared with LP-BERT, KG-BERT, and StAR based on language pre-training models, the prediction model performs very well in FB15K-237, with MRR increasing from 0.310 to 0.362, an increase of nearly 4 percentage points; Hits@10 increasing from 49.0% to 54.9%, an increase of nearly 6%; Hits@3 increasing from 33.6% to 39.6%, an increase of nearly 6%; and Hits@1 recognition performance increasing from 22.3% to 26.8%, an increase of nearly 5%.

[0141] This is because the input features of the language pre-training model are based on text language processing. In the case of many relationships, the familiar entity vectors inferred by the model are abnormally close, resulting in poor recognition performance. The prediction model uses a fully connected layer to capture long-distance interactions, mainly processing entity and relationship vectors. Compared with N-MlpE, the prediction model performs similarly in various indicators on the two datasets, but on FB15K-237, the overall performance of the prediction model is better than N-MlpE. For WN18RR, MRR increased from 0.474 to 0.484, and Hits@1 performance increased from 43.4% to 45.0%. This embodiment believes that N-MlpE introduces too much redundant noise while introducing domain information, resulting in a decrease in model performance. However, the prediction model effectively avoids this problem by focusing on the feature vectors of the entities and relationships themselves and using a multi-head attention mechanism for high-dimensional mapping, while the first decoder outputs a low dimension.

[0142] Therefore, the prediction model further improves the performance of the KGE model while compressing the model dimensional space. The data volume of the typical data set is small. In order to further verify the versatility and robustness of the model, this embodiment selects two larger data sets, DB100k and YAGO3-10, to evaluate the effectiveness of the prediction model. Experiments have shown that the prediction model achieves better results on the two data sets than other models. Especially on DB100k, the prediction model is significantly better than JointE and other convolutional models in all indicators. Compared with JointE, MRR and Hits@3 are improved by nearly 2%, and Hits@10 is improved by nearly 4%.

[0143] Compared with DB100k, YAGO3-10 has fewer relations. The performance of the prediction model of this embodiment on FB15K-237 shows that when the number of relations is large, the prediction model of this embodiment performs better in link prediction performance. In general, on two typical data sets and two large data sets, the prediction model of this embodiment has achieved better results than other models, far exceeding other models. On the WN18RR data set, the performance of the prediction model is very competitive, and on the FB15K-237 and DB100K data sets, the MRR and Hits@1All indicators of the prediction model are far better than other models. Among many models, all indicators of the prediction model are the best, even far exceeding the same series of indicators of N-MlpE and language pre-training models (such as LPBERT and StAR). Experiments show that when the number of relations is large, the expressive power of the prediction model will also be enhanced. The experiments also prove that the prediction model is a high-performance knowledge embedding model that shows strong competitiveness and robustness on four data sets.

[0144] Similar to ConvE and InteractE, the prediction model does not distinguish the size of the entity and relationship embedding dimensions, allowing this embodiment to control the size of the model parameters by adjusting the model parameters. Experiments have shown that, taking FB15K-237 as an example, this embodiment fixes the embedding dimensions of entities and relationships to , the optimal model performance of the prediction model was obtained. The parameter sizes of the other compared models were selected according to their best model performance. The experiments show that the prediction model achieves better performance with fewer model parameters than other convolution-based models. The prediction model has a much smaller number of model parameters than the language pre-training model. For example, the number of parameters of the LP-BERT model is 102M, while the number of model parameters of the prediction model is only 6M. However, the MRR and Hits@10 of the prediction model reached 0.362 and 54.9% respectively, which are the highest indicators among the compared models. The experimental results show that the prediction model model is very efficient. Therefore, compared with these models, the prediction model can achieve better results with very few parameters.

[0145] This embodiment proposes an efficient low-dimensional model framework - a prediction model, which uses an encoding-decoding structure to fully utilize the interactive feature information of entities and relationships. In the encoder part (i.e., the first encoder), the multi-head attention mechanism and the feedforward neural network are fully connected through the sub-connection layer. This setting can further extract the interactive features between entities and relationships while avoiding gradient vanishing and dimensionality explosion. In the decoder part (i.e., the first decoder), an encoding-decoding structure is nested (i.e., the second encoder and the second decoder are nested). In a unified dimension, this embodiment uses a fully connected layer in the second encoder to map and restore the encoded feature representation, thereby capturing the long-range interactions between entities and relationships. The experiments of this embodiment have shown that the prediction model has shown excellent performance on four classic data sets.

[0146] This embodiment has at least the following advantages:

[0147] When entities and relationships have the same embedding dimension, the embodiment provides a lightweight and efficient dual-encoding prediction model, which can effectively capture long-distance interactions and improve link prediction accuracy and efficiency. Moreover, experiments conducted on four commonly used data sets show that the model provided by this embodiment can capture long-distance interactions while using fewer dimensions and model parameters, and has higher model efficiency than the corresponding comparison models.

[0148] like Figure 4 An embodiment of the present application provides a knowledge graph link prediction system based on a dual encoding and decoding model. The system includes a sequence embedding unit 1100, a feature encoding unit 1200 and a link prediction unit 1300, and each component performs the following functions.

[0149] The sequence embedding unit 1100 is used to embed entities and relationships in the target knowledge graph into entity sequences and relationship sequences respectively, and the dimensions of the entity sequences and relationship sequences are equal.

[0150] The feature encoding unit 1200 is used to extract a first interaction feature vector from the entity sequence and the relationship sequence according to the first encoder.

[0151] The link prediction unit 1300 is configured to extract a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder.

[0152] The first decoder includes a second encoder and a second decoder based on a multi-layer perceptron.

[0153] Extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes:

[0154] The first interactive feature vector is encoded according to the second encoder and projected to a second interactive feature vector having the same dimension as the first interactive feature vector.

[0155] A link prediction result is extracted from the second interactive feature vector according to the second decoder.

[0156] It should be noted that the present system embodiment and the above-mentioned method embodiment are based on the same inventive concept and will not be described in detail here.

[0157] like Figure 5 The present application also provides an electronic device, which includes:

[0158] at least one memory;

[0159] at least one processor;

[0160] at least one program;

[0161] The program is stored in the memory, and the processor executes at least one program to implement the knowledge graph link prediction method based on the dual encoding and decoding model implemented in the present disclosure.

[0162] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0163] The electronic device according to the embodiment of the present application is described in detail below.

[0164] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0165] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the knowledge graph link prediction method based on the dual encoding and decoding model of the embodiment of the present application.

[0166] Input / output interface 1800, used to implement information input and output;

[0167] Communication interface 1900, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0168] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0169] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0170] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned knowledge graph link prediction method based on the dual encoding and decoding model.

[0171] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0173] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0174] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0176] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0177] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0182] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A knowledge graph link prediction method based on a dual encoding and decoding model, characterized in that: The method comprises: Embed entities and relations in a target knowledge graph used in a question answering system or semantic search into entity sequences and relationship sequences respectively, wherein the dimensions of the entity sequences and relationship sequences are equal; extracting a first interaction feature vector from the entity sequence and the relationship sequence according to a first encoder; extracting a link prediction result from the first interaction feature vector according to a first decoder corresponding to the first encoder; Wherein, the first decoder includes a second encoder and a second decoder based on a multi-layer perceptron; The extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes: The first interactive feature vector is encoded according to the second encoder and projected to a second interactive feature vector with the same dimension as the first interactive feature vector; the expression for encoding the first interactive feature vector and projecting it to a second interactive feature vector with the same dimension as the first interactive feature vector includes: ； is the first interaction eigenvector, and Generate the weight matrix and bias vector of the second interaction feature vector for the first fully connected layer projection of the second encoder, is the second interaction eigenvector, is the ReLU function; A link prediction result is extracted from the second interaction feature vector according to the second decoder.

2. The knowledge graph link prediction method based on the dual encoding and decoding model according to claim 1 is characterized in that: The entities and relations in the target knowledge graph applied to the question answering system or semantic search are respectively embedded into entity sequences and relationship sequences, including: Determine triples of entities in the target knowledge graph for use in question answering systems or semantic search; Embed the head entity in the triple as an entity sequence of a target dimension, and embed the relationship between the head entity and the tail entity in the triple as a relationship sequence of the target dimension; Before extracting the first interaction feature vector from the entity sequence and the relationship sequence according to the first encoder, the method further includes: Merging the entity sequence and the relationship sequence into a target sequence; The extracting a first interaction feature vector from the entity sequence and the relationship sequence comprises: A first interactive feature vector is extracted from the target sequence.

3. The knowledge graph link prediction method based on the dual encoding and decoding model according to claim 1 is characterized in that: The expression for extracting the first interactive feature vector from the entity sequence and the relationship sequence includes: ； ； ； in, are entity sequence and relationship sequence respectively, is the normalization function, is the multi-head attention function, , , , is the weight matrix of the query projection, is the weight matrix of key projection, is the weight matrix for value projection, is the Dropout function, is a feed-forward neural network, is the first interaction eigenvector.

4. The knowledge graph link prediction method based on the dual encoding and decoding model according to claim 1 is characterized in that: The expression for extracting the link prediction result from the second interactive feature vector includes: ； and Generate the weight matrix and bias vector of the link prediction result for the second fully connected layer of the second decoder, is the link prediction result.

5. The knowledge graph link prediction method based on the dual encoding and decoding model according to claim 3 is characterized in that: The loss functions of the first encoder and the first decoder include: ； in, is the predicted probability, is a smooth label, Indicates The predicted probability of a head, Indicates Smooth labels of heads, The total number of heads.

6. The knowledge graph link prediction method based on the dual encoding and decoding model according to claim 2 is characterized in that: The target dimension is 100.

7. A knowledge graph link prediction system based on a dual encoding and decoding model, characterized in that: The system comprises: A sequence embedding unit, used to embed entities and relations in a target knowledge graph applied in a question answering system or semantic search into entity sequences and relationship sequences respectively, wherein the dimensions of the entity sequences and the relationship sequences are equal; a feature encoding unit, configured to extract a first interaction feature vector from the entity sequence and the relationship sequence according to a first encoder; a link prediction unit, configured to extract a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder; Wherein, the first decoder includes a second encoder and a second decoder based on a multi-layer perceptron; The extracting a link prediction result from the first interactive feature vector according to a first decoder corresponding to the first encoder includes: The first interactive feature vector is encoded according to the second encoder and projected to a second interactive feature vector with the same dimension as the first interactive feature vector; the expression for encoding the first interactive feature vector and projecting it to a second interactive feature vector with the same dimension as the first interactive feature vector includes: ； is the first interaction eigenvector, and Generate the weight matrix and bias vector of the second interaction feature vector for the first fully connected layer projection of the second encoder, is the second interaction eigenvector, is the ReLU function; A link prediction result is extracted from the second interaction feature vector according to the second decoder.

8. An electronic device, characterized in that: It includes at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by at least one of the controllers to enable at least one of the controllers to execute the knowledge graph link prediction method based on the dual codec model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the knowledge graph link prediction method based on the dual encoding and decoding model as described in any one of claims 1 to 6.

Images