Knowledge graph embedding model and application method thereof

Abstract

The invention provides a knowledge graph embedding model and an application method thereof.The embedding model can effectively obtain features between entities and features between relationships by processing a head entity embedding vector, a relationship embedding vector and a tail entity embedding vector, and then shared features containing implicit information are obtained through splicing; rich and comprehensive global shared output features are obtained from the shared features through a shared extraction module; the shared feature is cascaded with a mapping feature obtained by splicing a head entity embedded vector and a relation embedded vector, and then the cascaded shared feature is processed by a relation sensing module to obtain local information related to the relation, namely a relation output feature; the scoring module calculates scores according to the shared output features and the relation output features, and calculates loss through the scores to update the embedding model, so that the embedding model can comprehensively utilize global information and local information, features between entities and relations are effectively mined, and the embedding efficiency is improved. Therefore, the prediction of the links of the corresponding knowledge graph is realized, and the prediction accuracy of the links of the knowledge graph is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more specifically, to a knowledge graph embedding model and an application method thereof. Background Art

[0002] A knowledge graph is a structured semantic knowledge base used to describe concepts in the physical world and the relationships between them. Knowledge graphs have been widely used in various fields, such as ethylene cracking, and provide knowledge for other artificial intelligence fields. Correspondingly, knowledge graph embedding is a popular link prediction method that relies on knowledge graphs and learns representations of all entities and relationships in a continuous vector space. Currently, many knowledge graph embedding models have emerged. However, existing knowledge graph embedding models are typically based on 2D convolutions, which reshape the concatenated embeddings of entities and relationships into two dimensions. The characteristics of convolutions cause the model to ignore global information, while global features in knowledge graphs are important for link prediction tasks. Summary of the Invention

[0003] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide a knowledge graph embedding model and an application method thereof, so as to provide a knowledge graph embedding model that can effectively perform link prediction through global information and local information.

[0004] The technical solution adopted by the present invention is:

[0005] A first aspect of the present invention provides a knowledge graph embedding model, the embedding model comprising:

[0006] The vector conversion module is used to convert the head entity, relationship and tail entity of the triple of the corresponding knowledge graph into the head entity embedding vector, relationship embedding vector and tail entity embedding vector respectively;

[0007] A feature sharing module, configured to obtain shared features based on the head entity embedding vector, the relationship embedding vector, and the tail entity embedding vector;

[0008] A shared extraction module, configured to extract corresponding shared output features based on the shared features;

[0009] a relationship perception module, configured to obtain mapping features by concatenating the head entity embedding vector and the relationship embedding vector, and to obtain relationship output features based on the mapping features and the shared features;

[0010] A scoring module, configured to concatenate the shared output features and the relationship output features, and calculate a score corresponding to the triplet based on the concatenated output features;

[0011] A loss calculation module is used to calculate the loss according to the scores of all triples and update the embedding model according to the loss calculation results.

[0012] By processing the head entity embedding vector, the relationship embedding vector and the tail entity embedding vector, the features between entities and between relationships can be effectively obtained, and then the shared features can be spliced ​​together; the shared features, on the one hand, can directly obtain the rich and comprehensive global shared output features for prediction, and on the other hand, can be cascaded with the mapping features spliced ​​by the head entity embedding vector and the relationship embedding vector, and then processed by the relationship perception module to obtain local information related to the relationship, that is, the relationship output features; and then the scoring module is used to calculate the score based on the shared output features and the relationship output features, and the loss of the embedding model is calculated by the score to update the embedding model, so that the embedding model can comprehensively utilize global information and local information, effectively mine the features between entities and between relationships, and realize the prediction of the corresponding knowledge graph links, thereby greatly improving the accuracy of the knowledge graph link prediction.

[0013] Furthermore, the feature sharing module specifically includes:

[0014] An entity feature extraction layer, configured to obtain entity shared features among all entities based on the head entity embedding vector and the tail entity embedding vector;

[0015] A relation feature extraction layer, configured to obtain relational shared features among all relations based on the relation embedding vectors;

[0016] The shared feature extraction layer is used to combine the entity shared features and the relationship shared features to obtain the shared features.

[0017] By learning entities and relationships separately through the entity feature extraction layer and the relationship feature extraction layer, it is possible to fully learn the deep information between all entities and all relationships.

[0018] Furthermore, the entity feature extraction layer is specifically used to:

[0019] The entity features between all entities are extracted by a dense network set in the entity feature extraction layer, which is performed by extracting the entity features through the following affine function:

[0020] Ω E =W E V ht +b E

[0021] Among them, V ht ∈R k Represents the embedding vector corresponding to the entity; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d E and k are positive integers;

[0022] The extracted entity features are processed through a nonlinear activation function to obtain the entity shared features.

[0023] Furthermore, the relationship feature extraction layer is specifically used to:

[0024] The relationship features between the relationships are extracted by a dense network set in the relationship feature extraction layer, which is performed by extracting the relationship features through the following affine function:

[0025] Ω r =W r V r +b r

[0026] Among them, V r ∈R k Represents the relation embedding vector; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d r and k are positive integers;

[0027] The extracted relationship features are processed through a nonlinear activation function to obtain the relationship sharing features.

[0028] The dense network has the characteristics of strong interpretability and high efficiency, and can specifically, effectively and quickly realize the extraction of the entity shared features and the relationship shared features.

[0029] Furthermore, the shared extraction module is specifically used to:

[0030] The shared features are extracted by a dense network set in the shared extraction module, which is performed by the following affine function:

[0031] Ω c =W c P Er +b c

[0032] in, representing said shared features; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d c d E and d r is a positive integer;

[0033] The features extracted by the dense network are processed by a nonlinear activation function to obtain the shared output features.

[0034] Since the feature sharing module specifically learns and extracts the shared features for entities and relationships, the shared extraction module can further learn the global shared features through a dense network, learn the implicit potential global information of all entities and all relationships, and then obtain the shared output features, so that the subsequent scoring module can combine these global information for calculation based on the shared output features, thereby improving the prediction accuracy of the embedded model.

[0035] Furthermore, the acquiring of the relation output feature according to the mapping feature and the shared feature specifically includes:

[0036] Cascading and concatenating the mapping feature and the shared feature to obtain a cascade feature;

[0037] The cascade features are extracted by a dense network set in the relationship perception module, which is performed by the following affine function:

[0038] Ω g =W g P gEr +b g

[0039] in, represents the cascade feature; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d z and k are positive integers;

[0040] The features extracted by the dense network are processed by a nonlinear activation function to obtain the relational output features.

[0041] By extracting local information related to the relationship through the relationship perception module based on the dense network, the relationship output features focusing on the local information are obtained, so that the subsequent scoring module can combine these local information for calculation when performing calculations based on the relationship output features, so that the scoring module can integrate global information and local information for scoring calculations, thereby effectively improving the prediction accuracy of the embedded model.

[0042] Furthermore, the score corresponding to the triplet is calculated based on the spliced ​​output features obtained by splicing, and is specifically calculated using the following formula:

[0043]

[0044] Where, represents the rating, represents the splicing output feature, Indicates the dimension The learnable projection matrix, f(·) represents the nonlinear activation function, V t T represents the transpose of the tail entity embedding vector; d e d c and d z Is a positive integer.

[0045] Furthermore, the loss calculation is performed based on the scores of all triples, specifically including:

[0046] The scores of all triplets are normalized by the sigmoid function;

[0047] The loss is calculated by the binary cross entropy function It manifests as:

[0048]

[0049] Where y i represents the normalized score of the i-th triplet, Represents the true label value of the i-th triplet, if the i-th triplet actually exists, otherwise,

[0050] A second aspect of the present invention provides an application method of a knowledge graph embedding model, which is applied to a knowledge graph embedding model provided in the first aspect above, and the application method includes:

[0051] Construct a knowledge graph ontology based on ethylene cracking related attributes and obtain entity types and relationship types;

[0052] Acquire data related to ethylene cracking; the data related to ethylene cracking is acquired based on text data related to the ethylene cracking unit and its operation;

[0053] Identifying and naming entities from the ethylene cracking related data according to the entity type and the constructed knowledge graph ontology; the entities include a head entity and a tail entity;

[0054] Obtaining the associated relationships between the entities from the ethylene cracking related data according to the relationship type and the constructed knowledge graph ontology;

[0055] The extracted head entity, tail entity and relationship are used to construct the ethylene cracking triple;

[0056] The knowledge graph is embedded in the model and link prediction is performed on the ethylene cracking knowledge graph according to the ethylene cracking ternary ancestor, and the ethylene cracking knowledge graph is updated according to the prediction results.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. The present invention uses a feature sharing module to separately learn and process the head entity embedding vector, the relationship embedding vector and the tail entity embedding vector, which can effectively obtain the features between entities and between relationships and then splice the shared features containing hidden information. On the one hand, it can directly obtain the rich and comprehensive global shared output features for prediction. On the other hand, it can be cascaded with the mapping features spliced ​​by the head entity embedding vector and the relationship embedding vector and processed by the relationship perception module to obtain local information related to the relationship, that is, the relationship output features; and then the scoring module is used to calculate the score based on the shared output features and the relationship output features, and the loss of the embedding model is calculated by the score to update the embedding model, so that the embedding model can comprehensively utilize global information and local information, effectively mine the features between entities and between relationships, and realize the prediction of the corresponding knowledge graph links, thereby greatly improving the accuracy of the knowledge graph link prediction.

[0059] 2. The present invention realizes feature learning and extraction by adding dense networks to each module, and utilizes the strong interpretability and high efficiency of dense networks to achieve efficient learning of each feature. At the same time, the dense network can effectively learn the information of all features in the learning of cascaded features, thereby realizing the learning and extraction of global information and implicit information of all entities and all relationships of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a model structure diagram of the embedded model of the present invention.

[0061] Figure 2 The figure is a flow chart of the steps of the application method of the present invention.

[0062] Figure 3 4 is a performance curve diagram of each model in Example 2 of the present invention.

[0063] Figure 4 This is a diagram showing the impact of different embedding dimensions on the performance of the model in Example 2 of the present invention.

[0064] Figure annotations: vector conversion module 1, feature sharing module 2, entity feature extraction layer 21, relationship feature extraction layer 22, shared feature extraction layer 23, shared extraction module 3, relationship perception module 4, scoring module 5, loss calculation module 6. DETAILED DESCRIPTION

[0065] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention. To better illustrate the following embodiments, some components in the accompanying drawings may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the accompanying drawings.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] Example 1

[0068] A knowledge graph is a structured semantic knowledge base that describes concepts in the physical world and the relationships between them. Knowledge graphs have been widely used in various fields, such as search engines, question-answering systems, and intelligent recommendation systems, as well as providing knowledge for other artificial intelligence fields. Correspondingly, knowledge graph embedding is a popular link prediction method that relies on knowledge graphs and learns representations of all entities and relationships in a continuous vector space.

[0069] At present, many knowledge graph embedding models have emerged, such as TransE[5], TransH[6], TransR[7], TransD[8], etc., which regard the relationship as the distance between the head entity and the tail entity in the vector space. This type of model judges the authenticity of the triple by comparing the distance between the head and tail entity vectors and the relationship vector. There are also semantic matching models such as Rescal[9], DistMult

[10] , ComplEx

[11] , TuckER

[12] , etc., which use matrices to represent entities and relationships. Semantic matching models capture the complex interactions between entities and relationships by calculating the outer product of matrices. In addition, with the increasing popularity of neural networks, neural network-based embedding models have attracted more and more attention, such as ConvE

[13] , InteractE

[14] , ReInceptionE

[15] , JointE

[16] and other models. This type of neural network-based embedding model embeds entities and relationships and reshapes them into two-dimensional features, using two-dimensional convolutional neural networks to extract information.

[0070] As mentioned above, most of the existing neural network-based embedding models are based on two-dimensional convolutional neural networks to extract information. For example, the aforementioned ConvE

[13] uses 2D convolution to achieve cascade embedding of entities and relationships and perform two-dimensional reshaping. Although convolution operations can extract local features to focus on important features, making it perform well in image processing, in the task of knowledge graph embedding, whether the reshaped features are as suitable for 2D convolution as image features is still unknown. In addition, the characteristics of convolution make the model ignore global information, and global features in knowledge graphs are important for link prediction tasks.

[0071] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below with reference to the accompanying drawings.

[0072] like Figure 1 As shown, this embodiment provides a knowledge graph embedding model, which includes:

[0073] Vector conversion module 1, used to convert the head entity, relationship and tail entity of the triple of the corresponding knowledge graph into the head entity embedding vector, relationship embedding vector and tail entity embedding vector respectively;

[0074] Specifically, in this embodiment, the head entity of the triple of the knowledge graph can be represented as h, the relationship can be represented as r, and the tail entity can be represented as t. The head entity and the tail entity both belong to the entity set, and the relationship belongs to the relationship set; the vector conversion module projects the triple of the knowledge graph, i.e., the head entity, the relationship, and the tail entity, into the k-dimensional space for feature extraction to obtain the corresponding embedding vector V of the head entity. h , relation embedding vector Vr and tail entity embedding vector V t , since the head entity, relation and tail entity are projected into k-dimensional space, the corresponding head entity embedding vector V h , relation embedding vector V r and tail entity embedding vector V t The dimension is R k , denoted as V h 、V r 、V t ∈R k .

[0075] Feature sharing module 2, used to obtain shared features based on the head entity embedding vector, the relationship embedding vector and the tail entity embedding vector;

[0076] Specifically, in this embodiment, the feature sharing module 2 specifically includes:

[0077] An entity feature extraction layer 21 is used to obtain entity shared features between all entities based on the head entity embedding vector and the tail entity embedding vector;

[0078] A relation feature extraction layer 22 is used to obtain relational shared features among all relations based on the relation embedding vectors;

[0079] The shared feature extraction layer 23 is used to combine the entity shared features and the relationship shared features to obtain the shared features.

[0080] The feature sharing module 2 specifically learns and extracts features between entities and between relationships through the entity feature extraction layer 21 and the relationship feature extraction layer 22, so that potential and hidden information can be better extracted based on the features between entities and between relationships.

[0081] In a preferred implementation of this embodiment, the entity feature extraction layer 21 is specifically used for:

[0082] The entity features between all entities are extracted by a dense network set in the entity feature extraction layer, which is performed by extracting the entity features through the following affine function:

[0083] Ω E =W E V ht +b E

[0084] Among them, V ht ∈R k Represents the embedding vector corresponding to the entity; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d E and k are positive integers, d E represents the width of the dense network of the entity feature extraction layer;

[0085] The extracted entity features are processed through a nonlinear activation function to obtain the entity shared features.

[0086] Correspondingly, the relationship feature extraction layer 22 is specifically used for:

[0087] The relationship features between the relationships are extracted by a dense network set in the relationship feature extraction layer, which is performed by extracting the relationship features through the following affine function:

[0088] Ω r =W r V r +b r

[0089] Among them, V r ∈R k Represents the relation embedding vector; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d r and k are positive integers, d r represents the width of the dense network of the relational feature extraction layer;

[0090] The extracted relationship features are processed through a nonlinear activation function to obtain the relationship sharing features.

[0091] Among them, the dense network of the entity feature extraction layer only acts on entities, and the dense network of the relationship feature extraction layer only acts on relationships. The characteristics of the dense network can be used to realize specialized learning of entity and relationship features, effectively improving learning efficiency and learning accuracy.

[0092] In this embodiment, the feature sharing module 2 is used to realize hidden information learning and extraction between all entities and all relationships, and obtains learned global information including features between entities and features between relationships.

[0093] A shared extraction module 3 is configured to extract corresponding shared output features based on the shared features;

[0094] Specifically, in this embodiment, the shared extraction module is specifically used to:

[0095] The shared features are extracted by a dense network set in the shared extraction module, which is performed by the following affine function:

[0096] Ω c =W c P Er +b c

[0097] in, representing said shared features; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d c d E and d r is a positive integer, d c represents the width of the dense network of the shared extraction module, d E +d r represents the width of the feature sharing module;

[0098] The features extracted by the dense network are processed by a nonlinear activation function to obtain the shared output features.

[0099] Relationship perception module 4, configured to obtain mapping features based on the concatenation mapping of the head entity embedding vector and the relationship embedding vector, and obtain relationship output features based on the mapping features and the shared features;

[0100] In the task of knowledge graph embedding, it is crucial to effectively distinguish each relationship in the knowledge graph. In the knowledge graph, the head entity type and the tail entity type of each relationship are often consistent. Therefore, in the process of learning these relationships, more attention should be paid to the types of the head entity and tail entity corresponding to these relationships, which can improve the performance of the embedding model. Therefore, in this embodiment, the relationship perception module 4 is used to effectively extract local information related to the relationship.

[0101] Specifically, in this embodiment, the acquiring of the relationship output feature according to the mapping feature and the shared feature by the relationship perception module 4 specifically includes:

[0102] Cascading and concatenating the mapping feature and the shared feature to obtain a cascade feature;

[0103] The cascade features are extracted by a dense network set in the relationship perception module, which is performed by the following affine function:

[0104] Ω g =W g P gEr+b g

[0105] in, represents the cascade feature; Indicates the dimension The learnable mapping matrix, Indicates the dimension The bias vector of d z and k are positive integers, d z represents the width of the dense network of the relationship-aware module;

[0106] The features extracted by the dense network are processed by a nonlinear activation function to obtain the relational output features.

[0107] Scoring module 5, configured to concatenate the shared output features and the relationship output features, and calculate a score corresponding to the triplet based on the concatenated output features;

[0108] Specifically, in this embodiment, the scoring module 5 can calculate the score of the triplet using the following formula:

[0109]

[0110] Where, represents the rating, represents the splicing output feature, Indicates the dimension The learnable projection matrix, f(·) represents the nonlinear activation function, V t T represents the transpose of the tail entity embedding vector; d e d c and d z is a positive integer, d e represents the output dimension of the scoring module, d c and d z Respectively represent the dimensions of the shared output features and the relationship output features.

[0111] That is, the scoring module 5 projects the spliced ​​output features into the embedding space through the learnable projection matrix W, activates it through a nonlinear activation function, and then multiplies it with the transpose of the tail entity embedding vector to obtain the score of the triplet;

[0112] In this embodiment, the scoring module 5 combines the shared output features containing global information with the relationship output features containing local information, and can more comprehensively learn and score the features between entities and relationships, greatly improving the prediction accuracy of the embedding model.

[0113] The loss calculation module 6 is used to perform loss calculation based on the scores of all triples and update the embedding model based on the loss calculation results.

[0114] Specifically, in this embodiment, the loss calculation of the loss calculation module 6 may include normalizing the scores of all triples using a sigmoid function;

[0115] The loss is calculated by the binary cross entropy function It manifests as:

[0116]

[0117] Where y i Represents the score of the i-th triple after normalization, and the normalized y i Represents the probability that the i-th triple is a fact. The larger the value, the more likely it is a true triple. Represents the true label value of the i-th triplet, if the i-th triplet actually exists, otherwise,

[0118] In addition, in a preferred implementation of this embodiment, the embedding model can be trained according to a 1:N training strategy, where N represents the number of all entities in the corresponding knowledge graph.

[0119] Based on the same inventive concept, this embodiment also provides an application method of a knowledge graph embedding model, such as Figure 2 As shown, the application method includes:

[0120] S1: Construct a knowledge graph ontology based on ethylene cracking related attributes and obtain entity types and relationship types;

[0121] S2: Acquire data related to ethylene cracking; the data related to ethylene cracking is acquired based on text data related to the ethylene cracking unit and its operation, and may specifically include books, papers, etc.

[0122] S3: identifying and naming entities from the ethylene cracking related data according to the entity type and the constructed knowledge graph ontology; wherein the entities include a head entity and a tail entity;

[0123] S4: acquiring the association relationship between the entities from the ethylene cracking related data according to the relationship type and the constructed knowledge graph ontology;

[0124] Specifically, in this embodiment, the entity types of the knowledge extraction include "material", "system", "component", "fault cause", "fault phenomenon", and "solution"; the relationship types include "include", "input", "source", "compose", "occur", "exist", "cause", and "solution". Specifically, the containment relationship is (material, contain, material), the input relationship is (material, input, component) and (material, input, system), the source relationship is (material, source, component), the composition relationship is (system, composition, component) (component, composition, component), the existence relationship is (system, existence, fault phenomenon) and (component, existence, fault phenomenon), the occurrence relationship is (component, occurrence, fault cause), the cause relationship is (fault phenomenon, cause, fault cause), and the solution relationship is (fault phenomenon, solution, solution).

[0125] S5: Construct the ethylene cracking triplet using the extracted head entity, tail entity and relationship;

[0126] S6: performing link prediction on the ethylene cracking knowledge graph according to the ethylene cracking triples through the knowledge graph embedding model, and updating the ethylene cracking knowledge graph according to the prediction results.

[0127] The application method indicates that in this embodiment, the embedding model can be applied to link prediction of the knowledge graph in the field of ethylene cracking knowledge. It should be noted that in some other preferred embodiments, the embedding model can also be applied to link prediction of knowledge graphs in other technical fields.

[0128] Example 2

[0129] Based on the same inventive concept as Example 1, this example mainly verifies the application of a knowledge graph embedding model provided in Example 1 in data in other fields through experiments, and compares it with other types of models to verify the effectiveness of the embedding model in Example 1 (hereinafter referred to as PartDensE).

[0130] In this embodiment, the PartDensE is evaluated mainly through five standard benchmark datasets: FB15k-237

[28] , WN18RR

[13] , YAGO3-10

[29] , AlyawarraKinship

[30] and UMLS. Among them, the dataset FB15k-237 is obtained by filtering the reverse relations in the FB15k[5] dataset, which contains facts in multiple fields such as people, organizations, places, movies, books, music, etc. The dataset WN18RR is built on the basis of WordNet, which stores a large number of words and the relationships between words. The dataset YAGO3-10 is a large-scale knowledge graph that contains a large amount of information about people, places, organizations and other entities, covering a wide range of factual fields such as geography, history, culture, and technology. The dataset Alyawarra Kinship is a dataset released in 2018 and used for link prediction tasks. It is a complex family and social organization system. The dataset UMLS is a biomedical dataset that contains a wealth of medical terms and their relationships. The detailed information of the above five standard base datasets is shown in Table 1 below;

[0131] Table 1

[0132]

[0133] The goal of link prediction is to discover and estimate connections between entities in a knowledge graph. Generally speaking, for a triplet with a head entity h, a relation r, and a tail entity t in a knowledge graph, link prediction typically involves predicting t from (h, r) or h from (r, t). Positive examples are all triples (h, r, t) in the knowledge graph. Negative examples are obtained by replacing the head entity h or the tail entity t in a triplet (h, r, t) with an entity. The candidate triples are then ranked by calculating the scores of the positive and negative examples and sorting them in descending order.

[0134] Link prediction tasks usually use several key evaluation indicators, including mean reciprocal rank (MRR), mean rank (MR), and hit rate (HIT@N). MRR calculates the average reciprocal rank of the real links in the prediction results, MR calculates the average rank of all real links in the prediction results, and Hit@N refers to the percentage of real links hitting the first N positions of the prediction results. The larger the values ​​of MRR and HIT@N, the better the effect of the model, and the opposite is true for MR. In the experiment of this embodiment, the values ​​of MRR and HIT@N (N∈{1, 3, 10}) are selected to comprehensively evaluate the performance of PartDensE. The UMLS dataset is an exception, and only MR and HIT@10 are used for evaluation.

[0135] In the specific experimental process, Adam

[31] was selected as the optimizer, and the corresponding batch size value was set according to the dataset size: the batch size value of FB15k-237 and WN18RR was 128, the batch size value of YAGO3-10 was set to 256, and the batch size value of Kinship and UMLS was 64. Other specific parameters are shown in Table 2 below:

[0136] Table 2

[0137]

[0138] First, the performance of PartDensE is verified on the datasets FB15k-237 and WN18RR. In this embodiment, some classic and latest advanced models are selected as baselines, including translation distance models TransE[5] and RotatE

[21] , semantic matching models DistMult

[10] and ComplEx

[11] , and neural network models ConvE

[13] , HypER

[27] , SACN

[25] , InteractE

[14] , M-DCN

[32] , JointE

[16] and ComDensE

[18] . The results are shown in Table 3 below. The latest convolution-based model JointE and the model ComDensE using dense networks have similar performance, especially on the dataset FB15k-237. However, compared with the two, PartDensE shows more outstanding results. As shown in Table 4 below, in the experiments of this embodiment, the performance of PartDensE ranks in the top two in each indicator of the two datasets. Specifically, on the FB15k-237 dataset, PartDensE achieved first place in MRR, HIT@1, and HIT@3, and second place in HIT@10. On the WN18RR dataset, PartDensE achieved the best results in MRR and HIT@1, followed by HIT@3 and HIT@10. This demonstrates that PartDensE outperforms all baseline models.

[0139] Table 3

[0140]

[0141]

[0142] On this basis, the PartDensE is further compared with other models through the large dataset YAGO3-10, small dataset Kinship and UMLS for comparative experimental verification. Among them, the baseline models used for comparison include: TransE[5], RotatE

[21] , DistMult

[10] , ComplEx

[11] , SimplE

[33] , ConvE

[13] , InteractE

[14] , LASS

[34] , M-DCN

[32] and ComDensE

[18] (some baseline models only have data on one dataset). The experimental results are shown in Table 4 below. On the datasets YAGO3-10, Kinship and UMLS, except for the HIT@10 indicator on Kinship, all other indicators obtained the highest scores. This fully demonstrates the technical leading position of the PartDensE compared with other neural network-based models. In the process of experimental verification, the model ComDensE

[18] using dense network was reproduced and the performance was evaluated using the above three datasets. As shown in Table 4 below, PartDensE has significant advantages over ComDensE

[18] , which also uses a dense network. On the YAGO3-10 dataset, each metric improved by more than 20%. On the Kinship dataset, MRR, HIT@1, and HIT@3 improved by more than 2%. Each metric on the UMLS dataset also saw a slight improvement.

[0143] Table 4

[0144]

[0145]

[0146] Combining Table 3 and Table 4, PartDensE shows very high competitiveness on all five datasets.

[0147] In order to further verify the convergence characteristics of each indicator of PartDensE, in this embodiment, the model performance is visualized. Figure 3 As shown in Figure 2, we selected the convolution-based model InteractE

[14] , the Transformer-based model SAttLE

[35] , and the dense network model ComDensE

[18] as controls, and conducted experiments on the dataset FB15k-237. Figure 3The growth trends of the four indicators are the same. First, the convolution-based model InteractE

[14] and the Transformers-based model SAttLE

[35] converge very quickly, reaching a peak approximately after 100 rounds and then tending to stabilize. The densely connected model has two obvious stages: a rapid growth stage before 150 rounds and a stage where the indicators continue to grow in subsequent rounds. The use of a dense network by PartDensE makes its convergence speed slower than that of the InteractE

[14] and SAttLE

[35] models, but its combination of specific shared layers and relationship perception layers can provide momentum for the growth of various indicators in the later stages of training. Secondly, compared with InteractE

[14] and SAttLE

[35] , although the convergence speed of PartDensE is slow, the final values ​​of various indicators are higher, indicating that PartDensE is more effective. At the same time, the metric values ​​of PartDensE in each round are higher than those of ComDensE

[18] , which also proves that the introduction of directed learning of entities and relationships is effective.

[0148] Furthermore, we use four datasets, FB15k-237, WN18RR, YAGO3-10, and Kinship, to observe the impact of the embedding dimensions of entities and relations on model performance. Figure 4 As shown in Table 3, three control experiments were conducted for each dataset: on the FB15k-237 and YAGO3-10 datasets, the embedding dimension d was set to 128, 256, and 384; on the WN18RR dataset, the embedding dimensions d were set to 384, 512, and 640; and on the Kinship dataset, the embedding dimensions d were set to 80, 160, and 320, respectively. The remaining parameters were consistent with those in Table 3. The MRR, H@1, H@3, and H@10 metrics were also used to measure the performance of the link prediction task at different dimensions.

[0149] from Figure 4Two conclusions were drawn: the model achieves optimal performance at different dimensions on different datasets; and both small and large dimensions degrade model performance. Specifically, when d = 256, PartDensE performs better on the FB15k-237 and YAGO3-10 datasets; on the WN18RR dataset, an embedding dimension d of 512 yields optimal results; and on the Kinship dataset, the best performance is achieved at a dimension of 160. These differences are due to the characteristics of the datasets themselves. Both the FB15k-237 and YAGO3-10 datasets primarily contain relationships among people, organizations, and institutions in various fields. This similarity also leads to the same optimal dimension. The interpersonal network in the Kinship dataset is relatively simple compared to other datasets, allowing it to capture comprehensive and in-depth information at a lower dimension. Experiments also found that on the WN18RR and Kinship datasets, dimensionality changes have little impact on model performance. It's worth noting that increasing the dimensionality may slightly increase the HIT@3 or HIT@10 metrics. For example, on the WN18RR dataset, the HIT@3 value corresponding to 640 dimensions is higher than that corresponding to 512 dimensions. However, this does not affect the selection of the optimal dimensionality, because determining the optimal dimensionality requires a comprehensive evaluation of four metrics, and the MRR and H@1 values ​​are more important than the other two.

[0150] In addition, compared with other datasets, FB15k-237 contains more diverse relations. Therefore, we studied the performance of the PartDensE model on different relation types on the FB15k-237 dataset. According to the average number of tail entities corresponding to each head entity and the average number of head entities per tail entity, four relation types are divided: one-to-one (1:1), one-to-many (1:N), many-to-one (N:1), and many-to-many (N:N). The PartDensE is compared with ConvE

[13] , InteractE

[14] , and ComDensE

[18] . The link prediction results for each relation category are shown in Table 5 below. The results of the PartDensE are generally better than the ConvE

[13] and InteractE

[14] models in various indicators of different relation types. This shows that the model performance of the combination of dense connections with relation-aware functions and specific sharing functions is better than the model using 2D convolution. Compared with the ComDensE

[18] model, the PartDensE has significantly improved performance in relations with multiple head entities or tail entities. This demonstrates the effectiveness of specialized learning of entity and relation embeddings for link prediction. While PartDensE achieves impressive link prediction results for all four relation types, it achieves even higher performance for complex relation types like 1:N, N:1, and N:N. This demonstrates PartDensE's exceptional ability to capture complex relations.

[0151] Table 5

[0152]

[0153] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A knowledge graph embedding model, characterized in that: The embedding model includes: A vector conversion module, used to convert the head entity, relationship and tail entity of the triple of the corresponding knowledge graph into the head entity embedding vector, the relationship embedding vector and the tail entity embedding vector respectively; A feature sharing module, used to obtain shared features according to the head entity embedding vector, the relationship embedding vector and the tail entity embedding vector; A shared extraction module, used for extracting corresponding shared output features according to the shared features; A relationship perception module, configured to obtain mapping features by concatenating and mapping the head entity embedding vector and the relationship embedding vector, and to obtain relationship output features according to the mapping features and the shared features; A scoring module, used for concatenating the shared output features and the relation output features, and calculating a score corresponding to the triplet according to the concatenated output features; A loss calculation module is used to calculate the loss according to the scores of all triples, and update the embedding model according to the loss calculation results.

2. A knowledge graph embedding model according to claim 1, characterized in that: The feature sharing module specifically includes: An entity feature extraction layer, used for obtaining entity shared features among all entities according to the head entity embedding vector and the tail entity embedding vector; A relation feature extraction layer, used for obtaining relational shared features among all relations according to the relation embedding vectors; The shared feature extraction layer is used to concatenate the entity shared features and the relationship shared features to obtain the shared features.

3. A knowledge graph embedding model according to claim 2, characterized in that: The entity feature extraction layer is specifically used for: The entity features between all entities are extracted by a dense network set in the entity feature extraction layer, which is performed by extracting the entity features through the following affine function: Ω E =W E V ht +b E Among them, V ht ∈R k Represents the embedding vector corresponding to the entity; The dimension is The learnable mapping matrix, The dimension is The bias vector of E and k are positive integers; The extracted entity features are processed by a nonlinear activation function to obtain the entity shared features.

4. A knowledge graph embedding model according to claim 2, characterized in that: The relationship feature extraction layer is specifically used for: The relationship features between the relationships are extracted by a dense network arranged in the relationship feature extraction layer, which is performed by extracting the relationship features through the following affine function: Ω r =W r V r +b r Among them, V r ∈R k represents the relation embedding vector; The dimension is The learnable mapping matrix, The dimension is The bias vector of r and k are positive integers; The extracted relationship features are processed by a nonlinear activation function to obtain the relationship sharing features.

5. A knowledge graph embedding model according to claim 1, characterized in that: The shared extraction module is specifically used for: The shared features are extracted by a dense network set in the shared extraction module, which is performed by extracting features through the following affine function: Ω c =W c P Er +b c in, representing said shared features; The dimension is The learnable mapping matrix, The dimension is The bias vector of c d E and d r is a positive integer; The features extracted by the dense network are processed by a nonlinear activation function to obtain the shared output features.

6. A knowledge graph embedding model according to claim 1, characterized in that: The acquiring of the relation output feature according to the mapping feature and the shared feature specifically includes: Cascading and concatenating the mapping feature and the shared feature to obtain a cascade feature; The cascade features are extracted by a dense network set in the relationship perception module, which is performed by extracting features through the following affine function: Ω g =W g P gEr +b g in, represents the cascade feature; The dimension is The learnable mapping matrix, The dimension is The bias vector of z and k are positive integers; The features extracted by the dense network are processed by a nonlinear activation function to obtain the relational output features.

7. A knowledge graph embedding model according to claim 1, characterized in that: The score corresponding to the triplet is calculated based on the spliced ​​output features obtained by splicing, and is specifically calculated by the following formula: In the formula, represents the rating, represents the concatenated output feature, The dimension is The learnable projection matrix, f(·) represents the nonlinear activation function, V t T represents the transpose of the tail entity embedding vector; d e ,d c and d z Is a positive integer.

8. A knowledge graph embedding model according to claim 1, characterized in that: The loss calculation is performed based on the scores of all triples, specifically including: The scores of all triples are normalized by the sigmoid function; The loss is calculated by the binary cross entropy function It manifests as: In the formula, y i represents the normalized score of the ith triplet, represents the true label value of the i-th triplet, if the i-th triplet actually exists, otherwise, 9. A method for applying a knowledge graph embedding model, the method being applied to a knowledge graph embedding model as claimed in any one of claims 1 to 8, characterized in that: The application method comprises: Construct a knowledge graph ontology based on the attributes related to ethylene cracking, and obtain entity types and relationship types; Acquire data related to ethylene cracking; the data related to ethylene cracking is acquired based on text data related to the ethylene cracking unit and operation; Identify and name entities from the ethylene cracking related data according to the entity type and the constructed knowledge graph ontology; the entities include a head entity and a tail entity; Acquire the associated relationships between the entities from the ethylene cracking related data according to the relationship type and the constructed knowledge graph ontology; The extracted head entity, tail entity and relationship are used to construct the ethylene cracking triple; The knowledge graph is embedded in the model and link prediction is performed on the ethylene cracking knowledge graph according to the ethylene cracking triples, and the ethylene cracking knowledge graph is updated according to the prediction results.

Images