A knowledge graph embedding link prediction method, system, device and medium

Abstract

The application discloses a kind of knowledge graph embedding link prediction method, system, equipment and medium, the entity and relation in the method are obtained in knowledge graph training dataset;Entity is embedded into first dimension space, and relation is embedded into second dimension space, obtain entity embedding vector and relation embedding vector;Entity embedding vector and relation embedding vector are embedded into standardization unit vector respectively, obtain the standardization entity vector and standardization relation vector that standardization unit vector output;Splice standardization entity vector and standardization relation vector, obtain input vector;Using first fully connected network, input vector is projected to third dimension space, obtain feature vector;Using second fully connected network, feature vector is projected to first dimension space, obtain output vector;Based on output vector, construct score function;The link prediction result of candidate entity is obtained by score function calculation.The application can capture long-distance interaction, improve the effect of link prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph embedding, in particular to a knowledge graph embedding link prediction method, system, device and medium. BACKGROUND

[0002] Knowledge graph embedding (KGE) is an effective method for predicting missing links in a knowledge graph. Currently, most KGE methods are designed from the perspective of improving the model's ability to capture interactions, and various convolutional neural network models have been developed. Although such models have good results, they are limited by the limited receptive field of the convolution method, making it difficult to capture long-range interactions.

[0003] The prior art ConvR makes further improvements to long-range interactions, ensuring that long-range interactions can be captured. However, the number of ConvR hyperparameters is large, which limits its practical application and affects the effectiveness of link prediction. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a knowledge graph embedding link prediction method, system, device and medium, which can capture long-range interactions and improve the effectiveness of link prediction.

[0005] In a first aspect, the present application provides a knowledge graph embedding link prediction method, which comprises:

[0006] Obtaining entities and relationships in a knowledge graph training data set;

[0007] Embedding the entities into a first-dimensional space and embedding the relationships into a second-dimensional space to obtain entity embedding vectors output by the first-dimensional space and relationship embedding vectors output by the second-dimensional space;

[0008] Embedding the entity embedding vectors and the relationship embedding vectors into standardized unit vectors to obtain standardized entity vectors and standardized relationship vectors;

[0009] Concatenating the standardized entity vectors and the standardized relationship vectors to obtain an input vector;

[0010] Projecting the input vector into a third-dimensional space using a first fully connected network to obtain a feature vector;

[0011] Projecting the feature vector into a first-dimensional space using a second fully connected network to obtain an output vector;

[0012] Constructing a scoring function based on the output vector;

[0013] The link prediction result of the candidate entity is obtained by calculating the score function.

[0014] Compared with the prior art, the first aspect of the present application has the following beneficial effects:

[0015] The method obtains entities and relations in a knowledge graph training data set. In order to avoid parameter explosion, the entities are embedded into a first-dimensional space, and the relations are embedded into a second-dimensional space, to obtain an entity embedding vector output by the first-dimensional space and a relation embedding vector output by the second-dimensional space. The entity embedding vector and the relation embedding vector are respectively embedded into a standardized unit vector to obtain a standardized entity vector and a standardized relation vector output by the standardized unit vector. The standardized entity vector and the standardized relation vector are spliced to obtain an input vector. In order to capture long-distance interaction and improve the effect of link prediction, a first fully connected network is used to project the input vector into a third-dimensional space to obtain a feature vector. A second fully connected network is used to project the feature vector into the first-dimensional space to obtain an output vector. Based on the output vector, a score function is constructed. The link prediction result of the candidate entity is obtained by calculating the score function. The method uses a fully connected network to capture long-distance interaction, reduces the number of hyperparameters, and thus improves the effect of link prediction.

[0016] According to some embodiments of the present application, the standardized entity vector and the standardized relation vector are spliced to obtain an input vector by the following formula:

[0017]

[0018] wherein V represents the input vector, represents the standardized entity vector of entity h, represents the standardized relation vector of relation r, k represents a k-dimensional first-dimensional space, and j represents a j-dimensional second-dimensional space.

[0019] According to some embodiments of the present application, the input vector is projected into a third-dimensional space by a first fully connected network to obtain a feature vector, which includes:

[0020] The k-dimension of the first-dimensional space and the j-dimension of the second-dimensional space are multiplied to obtain a third-dimensional space of m-dimension;

[0021] The input vector V is projected into the third-dimensional space of m-dimension by a first fully connected network to obtain a feature vector.

[0022] According to some embodiments of the present application, the feature vector is projected into the first-dimensional space by a second fully connected network to obtain an output vector, which includes:

[0023] applying an activation function to the feature vector and projecting the feature vector to a first dimensional space by using a second fully connected network to obtain a low-dimensional vector;

[0024] inputting the low-dimensional vector into a dropout layer to obtain an output vector.

[0025] According to some embodiments of the present application, the score function is constructed based on the output vector, including:

[0026] obtaining a standardized entity vector corresponding to the entity t;

[0027] performing dot multiplication on the output vector and the standardized entity vector corresponding to the entity t to construct a score function:

[0028]

[0029] wherein W1 and b1 represent parameters of the first fully connected network for generating the feature vector, W2 and b2 represent parameters of the second fully connected network for generating the output vector, and φ represents an activation function.

[0030] According to some embodiments of the present application, the link prediction method based on knowledge graph embedding further includes:

[0031] processing the score function by a sigmoid(·) function to obtain a probability distribution of the candidate entity;

[0032] obtaining a target probability distribution of an entity label corresponding to the candidate entity from the knowledge graph training data set;

[0033] measuring the gap between the probability distribution and the target probability distribution by a loss function to construct the loss function as:

[0034]

[0035] wherein y represents a label vector, p represents a probability of each label, i represents a label of the label vector, and N represents an upper limit of i.

[0036] In a second aspect, the embodiments of the present application further provide a link prediction system based on knowledge graph embedding, which includes:

[0037] a data acquisition unit configured to acquire entities and relationships in a knowledge graph training data set;

[0038] a data embedding unit configured to embed the entities into a first dimensional space and embed the relationships into a second dimensional space to obtain an entity embedding vector output by the first dimensional space and a relationship embedding vector output by the second dimensional space;

[0039] a vector normalization unit, configured to embed the entity embedding vector and the relation embedding vector into normalized unit vectors respectively, to obtain a normalized entity vector and a normalized relation vector output by the normalized unit vectors;

[0040] an input vector acquisition unit, configured to concatenate the normalized entity vector and the normalized relation vector to obtain an input vector;

[0041] a feature vector acquisition unit, configured to project the input vector to a third dimensional space by using a first full connection network to obtain a feature vector;

[0042] an output vector acquisition unit, configured to project the feature vector to a first dimensional space by using a second full connection network to obtain an output vector;

[0043] a score function construction unit, configured to construct a score function based on the output vector;

[0044] a link prediction unit, configured to calculate a link prediction result of a candidate entity by using the score function.

[0045] According to some embodiments of the present application, the score function construction unit specifically comprises a vector acquisition unit and a dot product calculation unit:

[0046] The vector acquisition unit is configured to acquire a normalized entity vector corresponding to an entity t.

[0047] The dot product calculation unit is configured to perform dot product calculation on the output vector and the normalized entity vector corresponding to the entity t to construct a score function.

[0048]

[0049] wherein W1 and b1 represent parameters of the first full connection network for generating the feature vector, W2 and b2 represent parameters of the second full connection network for generating the output vector, and φ represents an activation function.

[0050] In a third aspect, the embodiments of the present application further provide a link prediction device for knowledge graph embedding, comprising at least one control processor and a memory in communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the above-mentioned link prediction method for knowledge graph embedding.

[0051] ​In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores computer executable instructions for causing a computer to perform the knowledge graph embedding link prediction method.

[0052] It can be understood that the beneficial effects of the second aspect to the fourth aspect compared with the related art are the same as the beneficial effects of the first aspect compared with the related art, which can be seen from the related description in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0053] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the appended drawings, wherein:

[0054] Figure 1 is a flowchart of a knowledge graph embedding link prediction method according to an embodiment of the present application;

[0055] Figure 2 is a model structure diagram according to an embodiment of the present application;

[0056] Figure 3 is a structure diagram of a knowledge graph embedding link prediction system according to an embodiment of the present application;

[0057] Figure 4 is a structure diagram of a score function construction unit according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] Embodiments of the present application are described in detail below with reference to the accompanying drawings, wherein the same or similar notations are used to denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0059] In the description of the present application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features or the sequence of the indicated technical features.

[0060] In the description of the present application, it should be understood that the position description, such as the position or location relationship indicated by up, down, etc., is based on the position or location relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a particular position, be constructed and operated in a particular position, and therefore cannot be understood as limiting the present application.

[0061] In the description of the present application, it should be noted that, unless otherwise explicitly defined, the words such as setting, installing, connecting and the like should be understood broadly, and the person skilled in the art can determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0062] Knowledge graph embedding (KGE) is an effective method for predicting missing links in a knowledge graph. Currently, most KGE methods are designed from the perspective of improving the model's ability to capture interactions, and various convolutional neural network models have been developed. Although such models have good performance, they are limited by the limited receptive field of the convolution method, making it difficult to capture long-range interactions.

[0063] The prior art ConvR further improves long-range interactions, ensuring that long-range interactions can be captured. However, the number of ConvR hyperparameters is large, which limits its practical application and affects the link prediction performance.

[0064] To solve the above problems, the present application obtains entities and relationships in a knowledge graph training data set. To avoid parameter explosion, the entities are embedded into a first-dimensional space, and the relationships are embedded into a second-dimensional space, obtaining entity embedding vectors output by the first-dimensional space and relationship embedding vectors output by the second-dimensional space. The entity embedding vectors and relationship embedding vectors are embedded into standardized unit vectors, respectively, obtaining standardized entity vectors and standardized relationship vectors output by the standardized unit vectors. The standardized entity vectors and standardized relationship vectors are concatenated to obtain input vectors. To capture long-range interactions and improve the performance of link prediction, a first fully connected network is used to project the input vectors into a third-dimensional space to obtain feature vectors. A second fully connected network is used to project the feature vectors into the first-dimensional space to obtain output vectors. Based on the output vectors, a scoring function is constructed. The link prediction result of the candidate entity is obtained by calculating the scoring function. The present application uses a fully connected network to capture long-range interactions, reduces the number of hyperparameters, and thus improves the performance of link prediction.

[0065] Reference Figure 1 The present application provides a knowledge graph embedding link prediction method, which comprises the following steps S100 to S800:

[0066] Step S100, obtaining entities and relationships in a knowledge graph training data set.

[0067] Specifically, the entities and relationships are obtained from the knowledge graph training data set. The knowledge graph training data set of the present application is a data set in the prior art, for example, the WN18RR knowledge graph data set and the FB15K237 knowledge graph data set.

[0068] It should be noted that the embodiment does not limit the knowledge graph training data set to only the two data sets described above.

[0069] Step S200, embedding the entity into a first dimensional space and embedding the relationship into a second dimensional space to obtain an entity embedding vector output by the first dimensional space and a relationship embedding vector output by the second dimensional space.

[0070] Specifically, the entity obtained in step S100 is embedded into a first dimensional space, and the relationship obtained in step S100 is embedded into a second dimensional space, to obtain an entity embedding vector output by the first dimensional space and a relationship embedding vector output by the second dimensional space.

[0071] It should be noted that the dimensions of the first dimensional space and the second dimensional space in the embodiment can be equal or not equal.

[0072] In the embodiment, embedding the entity and the relationship distribution into the first dimensional space and the second dimensional space can avoid parameter explosion.

[0073] Step S300, embedding the entity embedding vector and the relationship embedding vector into a standardized unit vector respectively to obtain a standardized entity vector and a standardized relationship vector output by the standardized unit vector.

[0074] Specifically, the entity embedding vector and the relationship embedding vector are embedded into a standardized unit vector, and the standardized unit vector is a unit vector with a two-norm of 1.

[0075] After the entity embedding vector and the relationship embedding vector are standardized by the unit vector with a two-norm of 1, a standardized entity vector and a standardized relationship vector output by the standardized unit vector are obtained.

[0076] Step S400, concatenating the standardized entity vector and the standardized relationship vector to obtain an input vector.

[0077] Specifically, the standardized entity vector and the standardized relationship vector are concatenated by the following formula to obtain an input vector:

[0078]

[0079] wherein B represents the input vector, represents the standardized entity vector of the entity h, represents the standardized relationship vector of the relationship r, k represents a k-dimensional first dimensional space, and j represents a j-dimensional second dimensional space.

[0080] Step S500, projecting the input vector into a third dimensional space by using a first fully connected network to obtain a feature vector.

[0081] Specifically, the k dimension of the first dimensional space and the j dimension of the second dimensional space are multiplied to obtain a third dimensional space of m dimensions.

[0082] The input vector B is projected into the third dimensional space of m dimensions by using the first fully connected network to obtain a feature vector.

[0083] In this embodiment, the long-distance interaction is captured by using the fully connected network, the number of hyperparameters is reduced, and thus the link prediction effect is improved.

[0084] In step S600, the feature vector is projected into the first dimensional space by using the second fully connected network to obtain an output vector.

[0085] Specifically, the activation function is applied to the feature vector, and the feature vector is projected into the first dimensional space (i.e., a low-dimensional space of the same dimension as the entity embedding space) by using the second fully connected network to obtain a low-dimensional vector; and the low-dimensional vector is input into the dropout layer to obtain the output vector.

[0086] In step S700, a score function is constructed based on the output vector.

[0087] Specifically, a standardized entity vector corresponding to the entity t is obtained.

[0088] The output vector and the standardized entity vector corresponding to the entity t are dot multiplied to construct the score function.

[0089]

[0090] Wherein, W1 and b1 represent the parameters of the first fully connected network for generating the feature vector, W2 and b2 represent the parameters of the second fully connected network for generating the output vector, and φ represents the activation function.

[0091] In step S800, the link prediction result of the candidate entity is obtained by the score function.

[0092] Specifically, the link prediction result of the candidate entity is obtained by the score function. The scores of all candidate entities are obtained by the score function, and the link corresponding to the candidate entity with the highest score is selected as the link prediction result.

[0093] In this embodiment, the score function is processed by the sigmoid(·) function to obtain the probability distribution of the candidate entity.

[0094] The target probability distribution of the entity label corresponding to the candidate entity is obtained from the knowledge graph training data set.

[0095] The difference between the probability distribution and the target probability distribution is measured by the loss function, and the loss function is constructed as: ​

[0096]

[0097] where y represents a label vector, p represents a probability of each label, i represents a label number of the label vector, and N represents an upper limit of i.

[0098] In this embodiment, the earlier the score ranking is, the more accurate the link prediction result is, and the more reliable the link is. It should be noted that this embodiment does not limit the number of links selected from the front of the ranking, and can be changed according to actual needs, for example, the link corresponding to the candidate entity with the first score ranking can be selected as the link prediction result, and the link is established, or the link corresponding to the candidate entity with the top ten score rankings can be selected as the link prediction result, and the link is established.

[0099] To facilitate the understanding of those skilled in the art, a set of best embodiments is provided below:

[0100] A knowledge graph Can be represented as a set consisting of fact triples (h, r, t), where h can represent a head entity or a tail entity, t can also represent a head entity or a tail entity, when h represents a head entity, t represents a tail entity; when h represents a tail entity, t represents a head entity; r represents the relationship between entity h and entity t, is a set of relations r, and ε is a set of entities.

[0101] Link prediction of a knowledge graph is a prediction task for missing relationships between entities, aiming to predict appropriate entities to form reasonable triples based on given entities and relationships, for example, given a tail entity and a relationship, predict a head entity, or given a head entity and a relationship, predict a tail entity. Link prediction can be formally represented as a ranking problem, and the goal is to learn a scoring function The input of the scoring function is any triple, and the output is a real number, which is the score of the triple.

[0102] This embodiment designs a model for link prediction, referring to Figure 2 For all entities and relationships, this embodiment respectively embeds the entities and relationships into k-dimensional space and j-dimensional space to obtain entity embedding vectors output by k-dimensional space and relationship embedding vectors output by j-dimensional space, for example, Figure 2k = 4 (i.e., Entity Embedding size = 4), j = 4 (i.e., Relation Embedding size = 4). ConvE processes entity and relation embeddings by transformation and concatenation, which makes it difficult to capture long-distance interactions and introduces more hyperparameters. The embodiment does not have such limitations. First, the entity embedding vector and the relation embedding vector are respectively embedded into unit vectors with a two-norm of 1 for standardization to obtain a standardized entity vector and a standardized relation vector; and then the standardized entity vector and the standardized relation vector are concatenated into a long column vector and taken as an input vector, where B represents the input vector, represents the standardized entity vector of entity h, represents the standardized relation vector of relation r, k represents a k-dimensional first dimension space, and j represents a j-dimensional second dimension space.

[0103] After concatenation, a fully connected network is used to project B into an m-dimensional space to obtain a feature vector where m is the product of the entity embedding length k-dimensional space and the relation embedding length j-dimensional space, and thus, Figure 3 m = 16 (i.e., Dim = 16) in the above formula. From the perspective of convolution, the feature vector obtained by the embodiment can be regarded as a vector obtained by concatenating features obtained by a convolution operation with a convolution kernel covering the entire input vector. Therefore, long-distance interactions can be captured, and m channels can distinguish the interaction model of any pair of entity and relation elements.

[0104] An activation function is applied to the feature vector V F , and then a fully connected network is used to project it into a low-dimensional space with the same dimension as the entity embedding space, and dropout is applied to obtain an output vector VO, Figure 3 where the dropout is 0.5, but it should be noted that the value of the dropout can be changed according to actual needs, and the embodiment is not specifically limited. Finally, VO is dot multiplied with all candidate entities to construct a scoring function; and the link prediction result of the candidate entity is obtained by calculating the scoring function.

[0105] The scoring function can be represented as:

[0106]

[0107] where W1 and b1 represent the parameters of the first fully connected network for generating the feature vector, W2 and b2 represent the parameters of the second fully connected network for generating the output vector, and φ represents the activation function.

[0108] The embodiment uses a sigmoid (·) processing score function to obtain a probability distribution of the candidate entity; obtains a target probability distribution of an entity label corresponding to the candidate entity from a knowledge graph training data set; and measures the gap between the probability distribution and the target probability distribution through a loss function.

[0109] The loss function can be expressed as:

[0110]

[0111] wherein y represents a label vector, p represents the probability of each label, i represents the label of the label vector, and N represents the upper limit of i.

[0112] In order to avoid parameter explosion, the entity and the relationship are respectively embedded into a k-dimensional space and a j-dimensional space in the embodiment; in order to capture long-distance interaction and improve the effect of link prediction, a fully connected network is used to project an input vector into an m-dimensional space to obtain a feature vector; and a fully connected network is used to project the feature vector into a low-dimensional space of the same dimension of the entity embedding space to obtain an output vector. The embodiment uses a fully connected network to capture long-distance interaction, reduces the number of hyperparameters, and thus improves the effect of link prediction.

[0113] In order to better illustrate, the following experimental analysis is performed in the embodiment:

[0114] In order to evaluate the performance of the link prediction method of the knowledge graph embedding in the embodiment, the embodiment performs a link prediction task experiment on two typical data sets, and the information of the experimental data sets is shown in Table 1, wherein |ε| and |R| respectively represent the number of entities and relationships in the data set, and the last three columns respectively represent the number of triples in the training set, the test set and the validation set.

[0115] Table 1

[0116] The WN18RR knowledge graph data set and the FB15K237 knowledge graph data set are subsets of the two commonly used data sets WN18 and FB15K. Since the data sets WN18 and FB15K are not sufficient to evaluate the extrapolation ability of the model, because they contain too many inverse relationships, and a simple inverse relationship rule can obtain good results on WN18 and FB15K. Therefore, in order to avoid too many inverse relationships affecting the evaluation results of the model, researchers generate the WN18RR knowledge graph data set and the FB15K237 knowledge graph data set from the data sets WN18 and FB15K.

[0117]

[0118] ​The link prediction method of the knowledge graph embedding in the embodiment has fewer hyperparameters than other convolution-based KGE models. The learning rate is set to 0.003, the batchsize is set to 256, and the upper limit of the number of training times is set to 500. The entity and relation embedding lengths are both set to 128, and the smoothing rate is set to 0.1. The link prediction method of the knowledge graph embedding in the embodiment only needs to set the dropout rate of the output vector, and the dropout rate is set to 0.55 on both datasets.

[0119] The embodiment uses two series of indicators to quantitatively represent the effect of link prediction: MR / MRR and Hits@n. MR (Mean Rank) is the average rank of all correctly predicted entities, and MRR (Mean Reciprocal Rank) is a variant of MR, which can more clearly distinguish the ability of the model to accurately predict; Hits@n is the proportion of the predicted results ranked within n, and n is set to 1, 3 and 10 respectively in the embodiment. Lower MR and higher MRR and Hits@n represent higher prediction performance. The embodiment uses the same filtering method as the prior art to evaluate, for example, in a triple (h, r, t), replace h with any entity e i ∈ε, so as to obtain the score of each contaminated triple (e i , r, t), and then sort according to the score, and thus obtain the prediction result of the candidate entity. The triple obtained in this way will have uncertain results due to the arbitrariness of its entity. However, not every contaminated triple is a non-real triple, and experimental results show that the score of the real triple existing in the knowledge graph is higher, which will inevitably affect the evaluation result of the model. Therefore, when finally evaluating, the triple existing in the training set, the validation set and the test set should be excluded from the contaminated triple and not participate in the ranking. Through verification, this method is a fair and objective evaluation method.

[0120] First, the effect of the link prediction method of the knowledge graph embedding in the embodiment (i.e., the scheme in the table) is tested on FB15K237 and WN18RR, and is compared with typical translation-based models, three convolution-based methods, and a bilinear method, as shown in Table 2, where the best result is bolded and the second best result is underlined. Among them, the results of TransE, DistMult and ComplEx on FB15K237 and WN18RR come from JointE and other results come from the original paper.

[0121] Overall, the link prediction method of knowledge graph embedding in this embodiment is very competitive compared with other models on two datasets. The link prediction method of knowledge graph embedding in this embodiment can capture long-distance interactions, so the performance of this solution is better than ConvE on all standards. Although InteractE uses multiple ways to enable the model to capture long-distance interactions, it cannot guarantee to capture all forms of interactions, so its performance is worse than the link prediction method of knowledge graph embedding in this embodiment. JointE combines the ideas of ConvR and ConvKB and can capture long-distance interactions, so its effect is on par with the link prediction method of knowledge graph embedding in this embodiment.

[0122] wherein JointE comes from the document “Zhehui Zhou, Can Wang, Yan Feng, and Defang Chen. 2022. JointE: Jointly utilizing 1D and 2D convolution for knowledge graph embedding. Knowledge-Based Systems 240, C (Mar 2022)”; ConvE comes from the document “Dettmers, T, Minervini, P, Stenetorp, P, Riedel, S: Convolutional 2d knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. pp. 1811-1818. AAAI Press (2018)”; InteractE comes from the document “Vashishth, S, Sanyal, S, Nitin, V, Agrawal, N, Talukdar, P. P: Interacte: Improving convolution-based knowledge graph embeddings by increasing feature interactions.In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. pp.3009-3016. AAAI Press (2020).

[0123]

[0124] Table 2

[0125] The link prediction method of the knowledge graph embedding in the embodiment can achieve good performance and will not introduce too many parameters. Table 3 lists the results of MRR and Hits@10 of typical convolution-based KGE models on the dataset FB15K237 and the corresponding model parameter amount. As can be seen from Table 3, compared with other convolution models, the link prediction method of the knowledge graph embedding in the embodiment achieves good results and has less parameter amount. ConvE is difficult to capture long-distance interaction, while the link prediction method of the knowledge graph embedding in the embodiment can obtain better results with fewer parameters. InteractE uses deformation and cyclic convolution to capture long-distance interaction, but these methods introduce more parameters.

[0126] Model Parameter MRR Hits@10 ConvE 5.05M 0.316 0.491 HypER 4.30M 0.341 0.520 InteractE 10.70M 0.354 0.535 The present solution 4.51M 0.354 0.532

[0127] Table 3

[0128] With reference to Figure 3 The embodiment of the present application also provides a knowledge graph embedding link prediction system. The knowledge graph embedding link prediction system comprises a data acquisition unit 100, a data embedding unit 200, a vector normalization unit 300, an input vector acquisition unit 400, a feature vector acquisition unit 500, an output vector acquisition unit 600, a score function construction unit 700 and a link prediction unit 800, wherein:

[0129] The data acquisition unit 100 is used for acquiring entities and relationships in a knowledge graph training dataset.

[0130] The data embedding unit 200 is configured to embed the entities into a first dimensional space and embed the relations into a second dimensional space, to obtain entity embedding vectors output by the first dimensional space and relation embedding vectors output by the second dimensional space.

[0131] The vector normalization unit 300 is configured to embed the entity embedding vectors and the relation embedding vectors into normalized unit vectors respectively, to obtain normalized entity vectors and normalized relation vectors output by the normalized unit vectors.

[0132] The input vector acquisition unit 400 is configured to concatenate the normalized entity vectors and the normalized relation vectors, to obtain an input vector.

[0133] The feature vector acquisition unit 500 is configured to project the input vector into a third dimensional space by using a first full connection network, to obtain a feature vector.

[0134] The output vector acquisition unit 600 is configured to project the feature vector into the first dimensional space by using a second full connection network, to obtain an output vector.

[0135] The score function construction unit 700 is configured to construct a score function based on the output vector.

[0136] The link prediction unit 800 is configured to obtain a link prediction result of a candidate entity by using the score function.

[0137] In some embodiments, the score function construction unit 700 specifically includes a vector acquisition unit 710 and a dot product calculation unit 720.

[0138] The vector acquisition unit 710 is configured to acquire a normalized entity vector corresponding to the entity t.

[0139] The dot product calculation unit 720 is configured to perform dot product calculation on the output vector and the normalized entity vector corresponding to the entity t. to construct the score function.

[0140]

[0141] wherein W1 and b1 represent parameters of the first full connection network for generating the feature vector, W2 and b2 represent parameters of the second full connection network for generating the output vector, and φ represents an activation function.

[0142] It should be noted that, since the link prediction system of one embodiment of the knowledge graph embedding and the link prediction method of one embodiment of the knowledge graph embedding are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to the system embodiment, which will not be described in detail here.

[0143] The embodiment of the present application also provides a link prediction device for knowledge graph embedding, comprising at least one control processor and a memory connected in communication with the at least one control processor.

[0144] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0145] The non-transitory software programs and instructions required for the link prediction method for knowledge graph embedding of the above embodiment are stored in the memory, and when executed by the processor, the link prediction method for knowledge graph embedding of one of the above embodiments is executed, for example, the method steps S100 to S800 in the above description Figure 1 are executed.

[0146] The system embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0147] The embodiment of the present application also provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more control processors, so that the above one or more control processors execute the link prediction method for knowledge graph embedding in the above method embodiment, for example, execute the functions of the method steps S100 to S800 in the above description Figure 1 .

[0148] As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, and techniques disclosed herein can be embodied in software, firmware, hardware, and / or suitable combination thereof. Some or all of the physical components can be implemented in software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or in hardware, or in integrated circuits, such as application- specific integrated circuits. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As will be appreciated by one of ordinary skill in the art, the term computer storage media includes all physical and tangible computer storage and non-transitory media, such as volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as will be appreciated by one skilled in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0149] The above detailed description of the embodiments of the present application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto.

Claims

1. A method for link prediction of knowledge graph embeddings, characterized in that, The link prediction method of the knowledge graph embedding comprises: obtaining entities and relations in a knowledge graph training dataset; embedding the entities into a first dimensional space and embedding the relations into a second dimensional space to obtain entity embedding vectors output by the first dimensional space and relation embedding vectors output by the second dimensional space; embedding the entity embedding vectors and the relation embedding vectors into standardized unit vectors respectively to obtain standardized entity vectors and standardized relation vectors; concatenating the standardized entity vectors and the standardized relation vectors to obtain an input vector; projecting the input vector into a third dimensional space by using a first fully connected network to obtain a feature vector; projecting the feature vector into the first dimensional space by using a second fully connected network to obtain an output vector; constructing a score function based on the output vector; calculating a link prediction result of a candidate entity by using the score function. 2.The knowledge graph embedding-based link prediction method of claim 1, wherein, The standardized entity vectors and the standardized relation vectors are concatenated by the following formula to obtain an input vector: wherein V represents an input vector, denotes a normalized entity vector of an entity h, denotes a normalized relation vector of a relation r, k denotes a k-dimensional first dimension space, and j denotes a j-dimensional second dimension space.

3. The link prediction method based on knowledge graph embedding according to claim 2, characterized in that: The projecting of the input vector into the third dimensional space by using the first fully connected network to obtain the feature vector comprises: multiplying a k-dimensional space of the first dimensional space and a j-dimensional space of the second dimensional space to obtain a third dimensional space of m-dimensional space; projecting the input vector V into the third dimensional space of m-dimensional space by using the first fully connected network to obtain the feature vector. 4.The knowledge graph embedding-based link prediction method of claim 3, wherein, The projecting of the feature vector into the first dimensional space by using the second fully connected network to obtain the output vector comprises: applying an activation function to the feature vector and projecting the feature vector into the first dimensional space by using the second fully connected network to obtain a low-dimensional vector; inputting the low-dimensional vector into a dropout layer to obtain an output vector.

5. The method of claim 4, wherein, The constructing of the score function based on the output vector comprises: obtaining a standardized entity vector corresponding to an entity t; vector corresponding to the entity t dot product, building a scoring function: wherein W1 and b1 represent parameters of the first fully connected network for generating the feature vector, W2 and b2 represent parameters of the second fully connected network for generating the output vector, and φ represents an activation function. 6.The knowledge graph embedding-based link prediction method of claim 1, wherein, The link prediction method of the knowledge graph embedding further comprises: processing the score function by using a sigmoid(·) function to obtain a probability distribution of the candidate entity; obtaining a target probability distribution of an entity label corresponding to the candidate entity from the knowledge graph training dataset; measuring a gap between the probability distribution and the target probability distribution by using a loss function to construct the loss function as follows: wherein y represents a label vector, p represents a probability of each label, i represents a label of the label vector, and N represents an upper limit of i.

7. A knowledge graph embedding-based link prediction system, characterized in that, The link prediction system of the knowledge graph embedding comprises: a data acquisition unit configured to obtain entities and relations in a knowledge graph training dataset; a data embedding unit configured to embed the entities into a first dimensional space and embed the relations into a second dimensional space to obtain entity embedding vectors output by the first dimensional space and relation embedding vectors output by the second dimensional space; a vector normalization unit, configured to embed the entity embedding vector and the relation embedding vector into normalized unit vectors respectively, to obtain a normalized entity vector and a normalized relation vector output by the normalized unit vectors; an input vector obtaining unit, configured to concatenate the normalized entity vector and the normalized relation vector to obtain an input vector; a feature vector obtaining unit, configured to project the input vector to a third dimensional space by using a first fully connected network to obtain a feature vector; an output vector obtaining unit, configured to project the feature vector to a first dimensional space by using a second fully connected network to obtain an output vector; a score function constructing unit, configured to construct a score function based on the output vector; a link prediction unit, configured to obtain a link prediction result of a candidate entity by using the score function.

8. The knowledge graph-embedded link prediction system of claim 7, wherein, The score function constructing unit specifically includes a vector obtaining unit and a dot product calculation unit: The vector obtaining unit is configured to obtain a normalized entity vector corresponding to an entity t. The point multiplication calculation unit is configured to multiply the output vector by a normalized entity vector corresponding to the entity t The point multiplication is performed to construct a scoring function: Wherein, W1 and b1 represent parameters of the first fully connected network for generating the feature vector, W2 and b2 represent parameters of the second fully connected network for generating the output vector, and φ represents an activation function.

9. A device for link prediction of knowledge graph embeddings, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions are used to make the computer execute the knowledge graph embedding link prediction method as any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions are used to make the computer execute the knowledge graph embedding link prediction method as any one of claims 1 to 6.

Images