Knowledge graph representation learning method and system based on hybrid expert algorithm

Abstract

The invention relates to the technical field of knowledge graphs, and provides a knowledge graph representation learning method and system based on a hybrid expert algorithm, and the method comprises the following steps: obtaining a to-be-processed triple, and carrying out the global embedding, thereby obtaining the global embedded representation of the triple; obtaining an expert embedding representation corresponding to each expert model for the triple to be processed, and connecting the expert embedding representation with global embedding to obtain a connection embedding representation of each expert; carrying out routing selection on the triad to be processed based on a routing mechanism, distributing models for different relationships, and obtaining the weight of each expert model; and carrying out weighted mixing on the obtained connection embedding representation on the basis of a weight obtained by routing selection to generate final triple embedding. According to the invention, through collaborative learning of a plurality of expert models, in combination with a routing mechanism of gate routing and attention routing, appropriate models are allocated for different relation types, so that the performance of representation learning is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field related to knowledge graphs, and specifically, to a knowledge graph representation learning method and system based on a hybrid expert algorithm. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Knowledge graphs express text in a structured form in the form of triples, which is beneficial for knowledge-intensive tasks such as question-answering and dialogue systems, and is also used to enhance the reasoning ability of large language models. Representation learning of knowledge graphs aims to learn implicit embeddings of entities and relations, and has become an indispensable step in knowledge graph applications.

[0004] The inventors found in their research that the existing general representation learning models focus on using linear, bilinear, complex or convolution operations to generate structural features. TransE regards relationships as transfers between head and tail entities, and is mainly applicable to simple transfer relationships, such as one-to-one (1-1), one-to-many (1-n), and many-to-one (n-1) relationships; DistMult is a bilinear model that is good at capturing relationship semantics and is more suitable for processing symmetric relationships; ComplEx uses complex value embedding and can effectively process asymmetric relationships; ConvE uses convolutional neural networks to embed interactions between entities and relationships, thereby showing stronger performance in processing complex relationships. Existing representation learning methods have certain limitations and cannot effectively process diverse and complex relationship types at the same time. Specifically, these methods often perform well in processing simple relationship types (such as one-to-one, symmetric relationships, etc.), but they are insufficient in dealing with complex, diverse and heterogeneous relationships. For example, there may be certain types of relationships that have symmetry, transferability or other more complex characteristics at the same time. At this time, existing representation learning methods may not be able to effectively integrate the characteristics of these complex relationships, resulting in limited reasoning capabilities of the model. Summary of the invention

[0005] In order to solve the above problems, the present disclosure proposes a knowledge graph representation learning method and system based on a hybrid expert algorithm. Through the collaborative learning of multiple expert models, combined with the routing mechanism of gate routing and attention routing, appropriate models are assigned to different relationship types to improve the performance of representation learning.

[0006] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0007] One or more embodiments provide a knowledge graph representation learning method based on a hybrid expert algorithm, comprising the following steps:

[0008] Obtain the triples to be processed, perform global embedding to obtain the global embedding representation of the triples;

[0009] For the triples to be processed, the expert embedding representation corresponding to each expert model is obtained, and connected with the global embedding to obtain the connected embedding representation of each expert;

[0010] For the triples to be processed, routing selection is performed based on the routing mechanism, models are assigned to different relations, and the weight of each expert model is obtained;

[0011] The obtained connection embedding representation is weighted mixed based on the weights obtained from the routing selection to generate the final triple embedding.

[0012] One or more embodiments provide a knowledge graph representation learning system based on a hybrid expert algorithm, including:

[0013] A global embedding module is configured to obtain a triple to be processed and perform global embedding to obtain a global embedding representation of the triple;

[0014] The expert embedding module is configured to obtain the expert embedding representation corresponding to each expert model for the processed triples, and concatenate it with the global embedding to obtain the concatenated embedding representation of each expert;

[0015] The routing selection module is configured to perform routing selection for the triples to be processed based on the routing mechanism, assign models to different relations, and obtain the weight of each expert model;

[0016] The fusion module is configured to perform weighted mixing on the obtained connection embedding representations based on the weights obtained by routing selection to generate the final triplet embedding.

[0017] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above-mentioned knowledge graph representation learning method based on the hybrid expert algorithm are completed.

[0018] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the above-mentioned knowledge graph representation learning method based on a hybrid expert algorithm.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] This embodiment can effectively solve the limitations of existing methods in dealing with diverse and complex relationships through a representation learning method based on a hybrid expert algorithm, and significantly improve the model's modeling and reasoning capabilities for complex relationships. Compared with the existing single model method, the introduction of the routing mechanism realizes adaptive optimization for different relationship types, which is suitable for simple relationships (such as symmetry, one-to-one relationships) and can efficiently handle complex relationships (such as transferability, many-to-many relationships). Through the synergy of expert models and weighted hybrids, the advantages of multiple models are fully utilized to improve the representation capabilities of knowledge graph embedding. In addition, it can be extended to handle other complex relationship types and has a wide range of applicability.

[0021] The advantages of the present disclosure and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0023] Figure 1 It is a schematic diagram of a hybrid expert algorithm model of a knowledge graph representation learning method based on a hybrid expert algorithm in Embodiment 1 of the present disclosure;

[0024] Figure 2 It is a flowchart of the knowledge graph representation learning method based on the hybrid expert algorithm of Example 1 of the present disclosure. DETAILED DESCRIPTION

[0025] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0026] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0027] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0028] Example 1

[0029] In the technical solutions disclosed in one or more embodiments, Figure 1 to Figure 2 As shown in FIG, a knowledge graph representation learning method based on a hybrid expert algorithm includes the following steps:

[0030] Step 1: Get the triple (h, r, t) to be processed, perform global embedding to obtain the global embedding representation of the triple (h g , r g , t g );

[0031] Step 2: For the triples to be processed, use the expert model to obtain the expert embedding representation (h e , r e , t e ), and concatenate with the global embedding to obtain the concatenated embedding representation of each expert;

[0032] Step 3: For the triple (h, r, t) to be processed, routing selection is performed based on the routing mechanism, models are assigned to different relations, and the weight of each expert model is obtained;

[0033] Step 4: The obtained connection embedding representation is weighted mixed based on the weights obtained by routing selection to generate the final triple embedding;

[0034] In this embodiment, in the above implementation, the triple (h, r, t) to be processed is firstly embedded into a global embedding representation (h g , r g , t g ), this process uses existing embedding methods to extract global features of entities and relations to ensure the comprehensiveness of the initial representation. Then, multiple predefined expert models are used, each of which generates a corresponding expert embedding representation (h e , r e , t e ), fully capturing the diverse characteristics of entities and relations. The global embedding representation is connected with the expert embedding representation to obtain the final expert connected embedding representation. The routing selection based on the routing mechanism dynamically calculates the weight of each expert model to assign the most appropriate model to different relations. The routing mechanism automatically selects and adjusts the weight of the expert model according to the characteristics of the relationship, thereby achieving adaptive optimization of the model. Finally, the expert embedding representation is weighted mixed by the routing weight to generate the final embedding representation of the triple, ensuring efficient modeling and reasoning capabilities for complex relations.

[0035] This embodiment can effectively solve the limitations of existing methods in dealing with diverse and complex relationships through a representation learning method based on a hybrid expert algorithm, and significantly improve the model's modeling and reasoning capabilities for complex relationships. Compared with the existing single model method, the introduction of the routing mechanism realizes adaptive optimization for different relationship types, which is suitable for simple relationships (such as symmetry, one-to-one relationships) and can efficiently handle complex relationships (such as transferability, many-to-many relationships). Through the synergy of expert models and weighted hybrids, the advantages of multiple models are fully utilized to improve the representation capabilities of knowledge graph embedding. In addition, it can be extended to handle other complex relationship types and has a wide range of applicability.

[0036] First, let's explain the mixed expert model. In the real world, a medical team is usually composed of experts such as internists, surgeons, and radiologists, each of whom excels in different areas of expertise. When a patient seeks medical care, they are first evaluated at the triage desk and then handed over to the doctor who is most qualified to treat their specific condition. In this process, not every doctor is involved in the treatment of all patients. Instead, each expert focuses on their own area of ​​expertise. This model is called a mixed expert model.

[0037] In some embodiments, the implementation of the above steps 1 to 4 can be achieved by constructing a hybrid expert algorithm model, such as Figure 1 As shown, it includes multiple expert models and gating mechanism modules; multiple representation learning models are selected as expert models to capture different types of relationship features; the expert models are selected and fused through the gating mechanism module;

[0038] In some embodiments, the representation learning model may include but is not limited to a TextEnc model, a TransE model, a DistMult model, a ComplEx model, and a ConvE model, and different types of relational features are captured through different expert models:

[0039] (1) TextEnc: Encodes text sequences through BiGRU to generate text embeddings of entities and relations to supplement semantic information;

[0040] TextEnc learns the sequence features of word sequences of entities and relations. For an entity h, its word sequence is For a relation r, its word sequence is Use BiGRU to encode these sequences:

[0041] I w =BiGUR(w i )for i∈{h,r,t} (1)

[0042] Among them, the last state of the forward and backward directions is concatenated to obtain the sequence embedding, that is, the text embedding of entity h and relation r is generated, which are h w and r w .

[0043] (2) TransE: Translation Embedding, which is a linear translation embedding, assuming h+r≈t. The connection between entity h and relation r can be captured by an addition:

[0044]

[0045] in, h w and r w is the text embedding defined by formula (1).

[0046] (3) DistMult: uses a bilinear formula to capture the bilinear features of entities and relations;

[0047] The connection between entity h and relation r can be obtained by multiplication:

[0048]

[0049] (4) ComplEx: Uses complex-valued embeddings, where the combination of complex-valued embeddings can handle various binary relations, including symmetric and antisymmetric relations. The connection relationship between entity h and relation r can be expressed as:

[0050]

[0051] (5) ConvE: Use a 2D convolutional network to learn the potential connection between the head entity h and the relation r as follows:

[0052]

[0053] in, and Represent entity embedding vectors and relation embedding vector Reshape; the reshape operation converts the vector Convert from one dimension D to two dimensions D = D1.D2;

[0054] Indicates h w and r w The result of concatenating the embedding vector after reshaping. ω The symbol represents a 2D convolutional layer with a filter ω, which returns a feature map tensor Among them, C1 is the number of feature maps with dimension C2×C3. Then Reshape into a vector And project back using a linear layer ρ represents the ReLU activation.

[0055] Furthermore, in order to guide the learning of each expert, this embodiment introduces a global entity embedding for each entity h A global relation embedding is introduced for each relation r Finally, the global embedding and the embedding representation output by each expert model are connected, that is, the connected embedding of the i-th expert is

[0056] In order to assign different triples to the most appropriate experts, the routing mechanism set in this embodiment includes gate routing and attention routing;

[0057] Gate Routing: Based on the input of entities and relations, the weight of each expert is generated through the gate network.

[0058] Furthermore, the gate routing mechanism includes sparse gate routing, dense gate routing and multiple gate routing, and each routing mechanism calculates the weight of experts in a different way to optimize the selection of experts respectively;

[0059] For a triple Its gate routing function is:

[0060] gate(h,r)=Softmax(TopN(f(h,r),n)) (6)

[0061]

[0062] in, They represent the weight matrix and the weight matrix with noise added, σ is the softplus smooth activation function, and softmax is the softmax activation function. i represents the i-th value of f(h,r), gate(h,r) represents the gate routing function, f(h,r) calculates the score of the feature representation of entity h and relationship r, and n is the n expert models set when calculating TopN;

[0063] According to the selection of TopN, the gate routing is divided into sparse gate routing and dense gate routing (n=N), where TopN represents the n expert models with the highest weights, specifically;

[0064] Dense Gate Routing: Each expert is assigned a weight, and experts with higher weights contribute more to the final embedding;

[0065] like Figure 1 As shown in (a), four expert models are set as an example. The dense gate routing calculates weights for all expert models. The weights calculated by the dense gate routing are P = 0.41, P = 0.19, P = 0.32 and P = 0.08, which are multiplied with the corresponding model outputs and then weighted;

[0066] Sparse Gate Routing: Select the TopN expert models (the N expert models with the highest weights), and reset the weights of other experts to zero, thereby reducing computational overhead;

[0067] like Figure 1 As shown in (b), four expert models are set as an example, and the two expert models with the highest weights are selected. The weights calculated by sparse gate routing are P = 0.41 and P = 0.32 respectively. The weights are multiplied by the corresponding model outputs and then weighted to obtain the weighted results;

[0068] Furthermore, this embodiment also proposes a multi-gate routing, designing different gate networks for entities and relationships, respectively, calculating the independent weights of the head entity and relationship, and then fusing them to form the final expert model weight:

[0069]

[0070] f(h,r)=f(h)+f(r) (11)

[0071] in, Represent the weight matrices of the head entity and relation respectively, They represent the noise weight matrices of the head entity and relation, respectively, and f(h) and f(r) represent the feature representation scores of the head entity and relation, respectively.

[0072] like Figure 1 As shown in (c), four expert models are set as an example. After weighting the independent weights of the head entity and the relationship, the two highest expert models are selected. The weights of the independent weights calculated by the multi-gate routing are P = 0.35 and P = 0.45 respectively. The obtained weights are multiplied with the corresponding model outputs and weighted to obtain the weighted result Y;

[0073] In some embodiments, attention routing is configured to calculate the relevance weight of each expert model with the current triplet through an attention network, including sparse attention, dense attention routing, and multiple attention routing;

[0074] Attention routing is an attention network that integrates the correlations between experts. The attention routing function is:

[0075] Att(h,r)=softmax(TopN(f(h,r),k)) (12)

[0076] Q=W Q [h; r], K = W K [h; r], V = W V [h; r] (13)

[0077]

[0078] in, τ represents the temperature hyperparameter;

[0079] W Q : represents the weight matrix used to calculate the query vector Q;

[0080] W K : represents the weight matrix used to calculate the key vector K;

[0081] W V : represents the weight matrix used to calculate the value vector V;

[0082] Q: represents the query vector, which is used to represent the target features in the attention mechanism;

[0083] K: represents the key vector, which is used to represent the context information related to the target feature;

[0084] V: represents the value vector, which represents the eigenvalue associated with the key vector;

[0085] k is the k expert models set when calculating TopN, and TopN is the same as formula (8).

[0086] Furthermore, according to the selection of topN, it can be divided into sparse attention routing and dense attention routing (n=N):

[0087] Dense Attention Routing: All expert models participate in weight calculation;

[0088] Sparse Attention Routing: only assign weights to the TopN experts with the highest relevance, reducing computational redundancy;

[0089] Furthermore, this embodiment also proposes multiple attention routing, designs different attention mechanisms for entities and relationships, calculates the independent weights of the head entity and relationship, and then merges them to form the weight of the final expert model;

[0090] Q=Wh h,K=W r r (15)

[0091] In summary, the probability value of each expert is obtained, which will be used in the subsequent mixing process.

[0092] A further technical solution is to dynamically select a routing mechanism based on the input triples and calculate the weight of the expert model. The method for selecting the routing mechanism includes the following steps:

[0093] Step 31: Conduct experimental verification for triple data of each relationship type, select the optimal routing mechanism for the triple based on the relationship type, and construct a corresponding relationship table between the triple relationship type and the routing mechanism;

[0094] Specifically, we compare different routing mechanisms for representation learning of knowledge graphs, and use the downstream task of link prediction to conduct test experiments, and calculate the index data, where the index data can use AR and H@N indicators;

[0095] Link prediction is a technique for predicting possible connections between unconnected nodes in a network graph (such as a social network or knowledge graph).

[0096] Downstream tasks are specific application tasks based on link prediction results, such as:

[0097] Recommendation system: recommend items of possible interest based on link prediction;

[0098] Knowledge reasoning: predicting possible facts or relationships in a knowledge graph;

[0099] Social network analysis: predicting potential friendships or community connections;

[0100] As a specific example, we conducted experiments on triples of many-to-many relationship types. Through the experiments, we tested six different routing mechanisms and obtained the conclusions shown in Table 1. The top two groups show different routing mechanisms, and the bottom two groups are about the selection of topN. From the results in the table, we can observe that each gate routing mechanism achieves better results than its corresponding attention routing mechanism, and among the gate routing mechanisms, for sparse, dense, and multiple routing mechanisms, the order is dense>sparse>multiple routing, so the dense gate routing mechanism is selected as the optimal routing mechanism.

[0101] Table 1

[0102]

[0103] Step 32: for the acquired triple (h, r, t) to be processed, determine the relationship type of the triple by counting the entity connectivity;

[0104] Specifically, analyze the number of connections between each entity h and t, and calculate the number of tail entities h is connected to through relationship r (degree(h,r)); determine the relationship type based on the number of connected tail entities;

[0105] degree(h,r)=1 and degree(t,r)=1, it is a one-to-one relationship;

[0106] degree(h,r)>1 and degree(t,r)=1, it is a one-to-many relationship;

[0107] degree(h,r)=1 and degree(t,r)>1, it is a many-to-one relationship;

[0108] If degree(h,r)>1 and degree(t,r)>1, it is a many-to-many relationship;

[0109] Step 33: According to the relationship type of the triple to be processed and the correspondence table between the triple relationship type and the routing mechanism, select the routing mechanism corresponding to the triple to be processed;

[0110] In step 4, the routing mechanism is used to mix the representation learning experts and obtain the mixed similarity score β of each triple (h, r, t) m (.), if the routing mechanism is gate routing, the weighted formula is:

[0111]

[0112] Where, “.” represents the dot product similarity function, T∈E;

[0113] Above Figure 1 The training of the hybrid expert algorithm model in the pre-training stage is to generate discriminative embeddings through contrastive learning to obtain the pre-trained model, and in the fine-tuning stage, the model is optimized through link prediction to improve the performance of representation learning.

[0114] A method for pre-training a hybrid expert algorithm model may include the following steps:

[0115] Step S11, obtaining a knowledge graph training data set;

[0116] Step S12: construct positive samples and negative samples for the triples in the data set respectively;

[0117] Specifically, positive samples are constructed by connecting the head entity h to multiple tail entities through relations r, or by connecting the head entity h to the tail entity t through multiple relations;

[0118] Connect the head entity h to multiple tail entities through the relationship r to obtain multiple positive samples pe , expressed as:

[0119]

[0120] Connect the head entity h to the tail entity t through multiple relationships to obtain multiple positive samples p r , expressed as:

[0121]

[0122] Specifically, negative sample construction includes:

[0123] Negative entity samples: Randomly select negative samples from candidate entities that do not meet the true triple conditions as negative entity samples

[0124]

[0125] Where: t j is a negative entity selected from the candidate entity list ε-ε(h,r); ε(h,r) is the entity list of the true answer (tail entity); that is, if the triple Then t i ∈ε(h,r);

[0126] Negative relationship samples: Randomly select negative samples from candidate relationships that do not meet the true triple conditions as negative relationship samples

[0127]

[0128] Among them, the negative relationship From the candidate relationship list Select from is a list of relations that satisfy the condition, that is, if the triple but

[0129] Step S13, inputting the constructed positive samples and negative samples into the hybrid expert algorithm model respectively, capturing the discriminative features of entities and relationships through contrastive learning and minimizing the objective function of contrastive learning, optimizing model parameters, and obtaining the pre-trained hybrid expert algorithm model;

[0130] Among them, the objective function of contrastive learning is defined as:

[0131]

[0132] Where τ is the temperature hyperparameter, n + represents positive samples, n - represents a positive sample;

[0133] A method for fine-tuning a pre-trained hybrid expert algorithm model may include the following steps:

[0134] Step S21: define a fine-tuning task, and use link prediction as a downstream task in the fine-tuning stage;

[0135] Step S22: Match the embeddings of entities and relations with the embeddings of all other entities to obtain the current prediction value

[0136] For a test triple (h, r, t), the embeddings of entity h and relation r are Match with the embeddings of all entities and calculate the prediction value with routing mechanism Take the gate routing weight calculation as an example:

[0137]

[0138] Among them, E i =[E i ; E g ] is the concatenation of the i-th expert embedding and the global embedding for all entities.

[0139] Step S23: Use a gradient descent algorithm (such as Adam optimizer) to update the model parameters and gradually minimize the loss function L FT , get the fine-tuned hybrid expert algorithm model;

[0140] When predicting the head entity h based on a pair (r, t), the method used is to reverse the relationship and obtain the reverse triple (t, r rev ,h) make predictions.

[0141] Use binary cross entropy as the loss function to optimize model parameters. The loss function is as follows:

[0142]

[0143] Among them, if t j ∈ε(h,r), then Y j =1, otherwise Y j = 0, ε(h,r) is the set of all true tail entities of a pair (h,r). When predicting the head entity h based on a pair (r,t), the relationship is reversed to obtain the reverse ternary (t,r rev ,h).

[0144] In this embodiment, the hybrid expert model is introduced into the field of knowledge graph representation learning, and a representation learning method adapted to different types of relations is proposed. A variety of routing mechanisms (including gate routing and attention routing) are designed to select the optimal TopN expert model and assign appropriate models to different relations.

[0145] In order to illustrate the effect of the method of this embodiment, a simulation experiment was carried out, which is specifically described as follows:

[0146] This embodiment uses the ReVerb20K and ReVerb45K OpenKG benchmark test sets, which are built by ReVerb. The data statistics of the data sets are shown in Table 2. Ent, Rel, cluster represent the number of entities, relations, and entity sets. Valid, Test, and Train represent the number of triples in the validation set, test set, and training set, respectively. ReVerb45K, with 27K entities and 21.6K relations, is larger and sparser than ReVerb20K, with 11.1K entities and 11.1K relations. Entity clusters are gold-normalized clusters extracted by the Freebase entity linking tool. Entities in an entity cluster have the same meaning. Typically, entity clusters are only used for evaluation.

[0147] Table 2

[0148]

[0149] To evaluate on a single test triple, this embodiment uses Mention Ranking, referred to as MR, which refers to the minimum ranking position of the answer entity. To evaluate all test triples, this embodiment uses the three most widely used methods to integrate a single MR score:

[0150] (a) H@N: proportion of MR scores not higher than N;

[0151] (b) AR: the average of all MR scores;

[0152] (c) ARR: Calculate the inverse of each MR score and average all the inverses. A better performing model should have higher H@N and ARR scores and lower AR scores.

[0153] In order to illustrate the effectiveness of the method of this embodiment, the model of this embodiment (abbreviated as: MOEE_PFT) is compared with some strong baselines and divided into three groups:

[0154] General: Use general knowledge graph models to capture the structural features of open knowledge graphs, including TransE, DistMult, ComplEx, ConvE, and ConvTransE.

[0155] OpenKG: A baseline designed specifically for open knowledge graphs to capture structural and textual features. Includes CaReTransE, CaReConv, SimKG, OKGITBert, and OKGITRob.

[0156] Single: This includes a single baseline where each model has a kge model and TextEnc, such as TTransE = TransE + TextEnc. It includes TDistMult, TComplEx, TTransE, TConvE, TConvE_PFT, where _PFT means there is both a pre-training stage and a fine-tuning stage, otherwise there is only a fine-tuning stage.

[0157] Table 3 shows the experimental results of the algorithm of this embodiment and the baseline on the ReVerb20K and ReVerb45K datasets. From the results in Table 3, it can be seen that the algorithm of this embodiment has made substantial progress compared with any baseline: first, this algorithm is significantly better than any general method that captures structural features with only a single formula function; second, this algorithm is also better than the open knowledge graph model that captures both structural and text features, including language enhancement methods (OKGITBert and OKGITRob); finally, this algorithm has achieved significant performance improvement over any single baseline.

[0158] Table 3 Experimental results of the model of this embodiment and the existing baseline on the ReVerb20K and ReVerb45K datasets

[0159]

[0160] In Table 2, the explanations of each algorithm are as follows:

[0161] ConvTransE, or Convolutional TransE, combines the ideas of ConvE and TransE, using convolutional networks to process entity relationship embedding while retaining the translation characteristics of TransE.

[0162] CaReTransE: Canonicalization-infused Representations TransE is a variant of the CaRe (Canonicalization-infused Representations) framework that uses TransE as the base model, aiming to improve entity and relation embeddings in open knowledge graphs by integrating canonicalization information.

[0163] CaReConv: Canonicalization-infused Representations Conv, similar to CaReTransE, but the base model uses ConvE;

[0164] SimKG: Similarity-based Knowledge Graph, a similarity-based method for processing open knowledge graphs, focusing on the semantic similarity and matching of entities and relations. SimKGC: Simple Contrastive Knowledge Graph Completion, is a text-based knowledge graph completion method that uses a pre-trained language model to learn embedded representations of entities and relations.

[0165] OKGITBert: Open Knowledge Graph Integration with BERT, uses BERT as a text embedding model to combine text information with the open knowledge graph structure to capture structural and text features. Open Knowledge Graph LinkPrediction with Implicit Types using BERT, uses a pre-trained BERT model to capture the implicit type information of entities.

[0166] OKGITRob: Open Knowledge Graph Integration with RoBERTa, similar to OKGITBert, but uses RoBERTa as the text embedding model. RoBERTa is an improved version of BERT that can capture better text semantic information. Open Knowledge Graph Link Prediction with Implicit Types usingRoBERTa is similar to OKGITBert, using the RoBERTa model instead of Bert.

[0167] TTransE: TransE+TextEnc, which combines the TransE model with a text encoder (Text Encoder, TextEnc) to jointly optimize structure embedding and text embedding;

[0168] TDistMult: DistMult+TextEnc, which combines the knowledge graph embedding model DistMult with the text encoder (TextEnc) to optimize the representation ability of the model;

[0169] TComplEx: ComplEx+TextEnc, which combines the ComplEx model with the text encoder (TextEnc) to handle asymmetric relations and enhance representation capabilities through joint text embedding;

[0170] TConvE: ConvE+TextEnc, which is a combination of the knowledge graph embedding model ConvE based on convolutional neural network and the text encoder (TextEnc);

[0171] TConvE_PFT: ConvE+TextEnc(Pretrain+Finetune), an extended version of TConvE with pre-training (Pretrain) and fine-tuning (Finetune) stages.

[0172] Table 4 Results of different relation types 1-1, 1-M, M-1, MM in ReVerb20K and ReVerb45K

[0173]

[0174] From the analysis of the results (H@1 scores) of the relation categories in Table 4, it can be seen that the algorithm of this embodiment (abbreviated as MOEE in Table 3) achieved the highest scores in almost all subtasks: (1) For the 1-1 category, the performance is significantly better than the baseline method when predicting the head and tail entities and the average of the two datasets; (2) For the M-1 and 1-M categories, the performance is better than (or equal to) the baseline method when predicting the tail entity; (3) For the MM category, better results are achieved when predicting the head entity on the ReVerb20K dataset, and better performance is also achieved when predicting the tail entity on the ReVerb45K dataset. Experiments show that the performance of this method is better than any single representation learning model when dealing with various relation types (such as 1-1, 1-M, M-1, MM).

[0175] Example 2

[0176] Based on Example 1, this embodiment provides a knowledge graph representation learning system based on a hybrid expert algorithm, including:

[0177] A global embedding module is configured to obtain a triple to be processed and perform global embedding to obtain a global embedding representation of the triple;

[0178] The expert embedding module is configured to obtain the expert embedding representation corresponding to each expert model for the processed triples, and concatenate it with the global embedding to obtain the concatenated embedding representation of each expert;

[0179] The routing selection module is configured to perform routing selection for the triples to be processed based on the routing mechanism, assign models to different relations, and obtain the weight of each expert model;

[0180] The fusion module is configured to perform weighted mixing on the obtained connection embedding representations based on the weights obtained by routing selection to generate the final triplet embedding.

[0181] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0182] Example 3

[0183] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps in the knowledge graph representation learning method based on the hybrid expert algorithm of Example 1 are completed.

[0184] Example 4

[0185] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the knowledge graph representation learning method based on a hybrid expert algorithm of Embodiment 1 are completed.

[0186] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0187] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A knowledge graph representation learning method based on a hybrid expert algorithm, characterized in that: The steps include: Obtain the triples to be processed, perform global embedding to obtain the global embedding representation of the triples; For the triples to be processed, the expert embedding representation corresponding to each expert model is obtained, and connected with the global embedding to obtain the connected embedding representation of each expert; For the triples to be processed, routing selection is performed based on the routing mechanism, models are assigned to different relations, and the weight of each expert model is obtained; The obtained connection embedding representation is weighted mixed based on the weights obtained from the routing selection to generate the final triple embedding.

2. The knowledge graph representation learning method based on hybrid expert algorithm according to claim 1, characterized in that: It also includes building a hybrid expert algorithm model, including multiple expert models and a gating mechanism module; selecting multiple representation learning models as expert models to capture different types of relationship features; and selecting and fusing the expert models through the gating mechanism module.

3. The knowledge graph representation learning method based on hybrid expert algorithm as claimed in claim 1, characterized in that: The representation learning model includes a TextEnc model, a TransE model, a DistMult model, a ComplEx model or / and a ConvE model.

4. The knowledge graph representation learning method based on hybrid expert algorithm according to claim 1, characterized in that: Routing mechanisms include gate routing and attention routing; Gate routing mechanisms include sparse gate routing, dense gate routing, and multiple gate routing; Dense gate routing: weights are calculated for all expert models and a weight is assigned to each expert; Sparse gate routing: select the N expert models with the highest weights, and reset the remaining expert weights to zero; Multi-gate routing: Different gate networks are designed for entities and relations respectively, and the independent weights of the head entity and relationship are calculated and then integrated to form the final weight of the expert model.

5. The knowledge graph representation learning method based on hybrid expert algorithm as claimed in claim 4, characterized in that: Attention routing, which calculates the relevance weight of each expert model with the current triplet through the attention network, including sparse attention, dense attention routing, and multiple attention routing; Dense attention routing: weights are calculated for all expert models; Sparse attention routing: assign weights to the top N experts with the highest relevance; Multiple attention routing, design different attention mechanisms for entities and relations respectively, calculate the independent weights of the head entity and relationship respectively, and then merge them to form the weight of the final expert model.

6. The knowledge graph representation learning method based on hybrid expert algorithm according to claim 1, characterized in that: According to the input triples, the routing mechanism is dynamically selected and the weight of the expert model is calculated. The method for selecting the routing mechanism includes the following steps: For each type of triple data, experimental verification is carried out to select the optimal routing mechanism for the triple based on the relationship type, and a corresponding relationship table between the triple relationship type and the routing mechanism is constructed; For the acquired triples to be processed, the relationship type of the triples is determined by counting the entity connectivity; According to the relationship type of the triple to be processed and the correspondence table between the triple relationship type and the routing mechanism, a routing mechanism corresponding to the triple to be processed is selected.

7. The knowledge graph representation learning method based on hybrid expert algorithm as claimed in claim 2, characterized in that: The method for pre-training the hybrid expert algorithm model includes the following steps: Get the knowledge graph training dataset; For the triplets in the data set, construct positive samples and negative samples respectively; Positive sample construction: connect the head entity h to multiple tail entities through the relationship r; or connect the head entity h to the tail entity t through multiple relationships; Randomly select negative samples from candidate entities that do not meet the true triple conditions; or randomly select negative samples from candidate relations that do not meet the true triple conditions; The constructed positive samples and negative samples are input into the hybrid expert algorithm model respectively, and the discriminative features of entities and relations are captured by contrastive learning and minimizing the objective function of contrastive learning, and the model parameters are optimized to obtain the pre-trained hybrid expert algorithm model; Alternatively, the method for fine-tuning the pre-trained hybrid expert algorithm model comprises the following steps: Define the fine-tuning task and use link prediction as the downstream task of the fine-tuning stage; Match the embeddings of entities and relations with the embeddings of all other entities to get the current prediction value; The gradient descent algorithm is used to update the model parameters and gradually minimize the loss function to obtain the fine-tuned hybrid expert algorithm model.

8. A knowledge graph representation learning system based on a hybrid expert algorithm, characterized in that: include: A global embedding module is configured to obtain a triple to be processed and perform global embedding to obtain a global embedding representation of the triple; The expert embedding module is configured to obtain the expert embedding representation corresponding to each expert model for the processed triples, and concatenate it with the global embedding to obtain the concatenated embedding representation of each expert; The routing selection module is configured to perform routing selection for the triples to be processed based on the routing mechanism, assign models to different relations, and obtain the weight of each expert model; The fusion module is configured to perform weighted mixing on the obtained connection embedding representations based on the weights obtained by routing selection to generate the final triplet embedding.

9. An electronic device, characterized in that: It includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the knowledge graph representation learning method based on the hybrid expert algorithm described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the knowledge graph representation learning method based on a hybrid expert algorithm as described in any one of claims 1 to 7.

Images