Knowledge distillation-based double-knowledge graph embedding fusion method, medium and equipment

Abstract

The invention discloses a double-knowledge graph embedding fusion method, medium and equipment based on knowledge distillation, and relates to the technical field of knowledge graph embedding fusion, and the method comprises the steps: obtaining two knowledge graphs, embedding vectors of the two knowledge graphs, and an aligned entity pair set P; the embedded vectors of the two knowledge maps are obtained by training a KGE model; two teacher models are constructed based on the embedded vectors of the two knowledge maps, the two teacher models are connected by a set P, an untrained KGE model is a student model, and an integrated model is constructed; training a student model by using two knowledge maps based on a knowledge distillation framework, inquiring two teacher models by the student model according to the triple score predicted each time, transmitting the corresponding score to an integration model by the teacher model, generating a soft label by the integration model, and distributing a soft label weight; and the student model carries out fusion learning on the two teacher models according to the soft labels to obtain fused graph embedding. According to the method, the knowledge graph embedding fusion efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph embedding and fusion technology, and in particular to a dual knowledge graph embedding and fusion method, medium, and device based on knowledge distillation. Background Art

[0002] Knowledge graphs are composed of triplets that represent facts in the real world. The triples are in the form of (head entity, relationship, tail entity), abbreviated as (h, r, t). Knowledge graphs are now used for various downstream tasks. However, knowledge graphs are generally incomplete, and a commonly used method is knowledge graph embedding (KGE). The classic KGE model represents entities and relationships in the knowledge graph as low-dimensional dense continuous vectors. Then the score of each triple is calculated according to the scoring function given by the model, and this score is used to measure the rationality of the triple. There are many classic embedding models, such as TransE, RotatE, PairRE, etc. According to the embedding technology, the entities and relationships in KG can be represented with higher quality through machine learning. Embedding technology is considered to be an important foundation for knowledge graph applications, such as knowledge question answering, recommendation systems, and semantic search.

[0003] Building knowledge graphs is common in many fields, which naturally leads to the accumulation of a large number of knowledge bases. However, the knowledge contained in a single knowledge graph is limited. In the process of building a knowledge graph, different knowledge graphs are often merged to form a higher quality knowledge base, a process called knowledge fusion. Knowledge fusion can improve the overall representation quality of the knowledge graph, enable a wider range of reasoning and applications, and better perform each other's knowledge graph completion tasks. In recent years, knowledge fusion, especially cross-domain research, has been widely studied. For example, MTransE implements cross-lingual knowledge alignment to handle different expressions of entity relations between different human languages.

[0004] Most KGE models assume that the knowledge graph is static and unchanging. However, in reality, the knowledge graph is changing all the time. Therefore, when new knowledge is added to the knowledge graph, the knowledge graph embedding representation obtained by the KGE model becomes outdated. Relying on the existing representation learning method requires retraining the integrated new knowledge graph, which is not only time-consuming but also inefficient. In addition, knowledge graphs inevitably share a lot of knowledge, which increases unnecessary storage burden. Therefore, it is crucial to fuse knowledge graphs. Among them, entity alignment is a key and effective method for knowledge graph fusion. It tries to find equivalent entities in the same real world in different knowledge graphs. However, the entity alignment method obtains the fused knowledge graph, which cannot be directly applied to most downstream tasks. Therefore, the fused knowledge graph needs to be retrained to obtain its embedding vector representation, which leads to a lot of time and computational overhead. In addition, the two knowledge graphs will have independent embedding training, that is, they will have independent embedding vector representations, and there are certain common entities between them. The original knowledge fusion method did not take into account the reusability of the embedding vectors of the two knowledge graphs, which makes the original embedding vectors of the knowledge graphs inefficient. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the existing technology of knowledge graph fusion requires retraining of the new knowledge graph to be embedded, resulting in low efficiency. A dual knowledge graph embedding fusion method based on knowledge distillation is proposed, which includes the following steps:

[0006] S1. Obtain two knowledge graphs G1 = {E1, R1, T1} and G2 = {E2, R2, T2}, as well as the embedding vectors Em1 and Em2 of the knowledge graphs G1 and G2, and the entity pair set P = {(u, v)|u∈E1, v∈E2} of the two knowledge graphs; where E1 and E2 represent the entity sets of G1 and G2 respectively, R1 and R2 represent the relationship sets of G1 and G2 respectively, and T1 and T2 represent the triple sets of G1 and G2 respectively; where the embedding vectors Em1 and Em2 of the knowledge graphs G1 and G2 are respectively trained by the two knowledge graph embedding models;

[0007] S2. Two teacher models are constructed based on two knowledge graph embedding vectors Em1 and Em2 respectively. The two teacher models are connected by the entity pair set P. An untrained knowledge graph embedding model is used as the student model, and an integrated model is constructed.

[0008] S3. Based on the knowledge distillation framework, the student model is trained using knowledge graphs G1 and G2. The student model asks the two teacher models based on the scores of the predicted triples during each training. The teacher model passes the scores of the corresponding triples to the integration model based on its own knowledge. The integration model generates soft labels and assigns soft label weights. The student model performs fusion learning on the two teacher models based on the soft labels to obtain the final fused graph embedding.

[0009] Furthermore, the student model is trained using the union of knowledge graphs G1 and G2.

[0010] Furthermore, the mechanism of integrating the model to generate soft labels is expressed as:

[0011]

[0012]

[0013] in, represents the integrated soft label passed to the student model, They respectively represent the scores of a given triplet (h, r, t) output by the two teacher models. τ1 and τ2 represent different thresholds for judging the scores of triplet, where the value range of τ1 is {10, 20} and the value range of τ2 is {30, 50}.

[0014] Furthermore, the soft label weights assigned by the integrated model are expressed as:

[0015]

[0016]

[0017] Among them, W pos represents the soft label weight of the positive triplet, W neg represents the soft label weight of the negative triplet, α1, β1, α2 and β2 are all learnable parameters.

[0018] Furthermore, the triplet score difference between the student model and the teacher model is defined as:

[0019]

[0020]

[0021] Among them, D Soft represents the triplet score difference between the student model and the teacher model, l δ () represents the Huber loss function, and δ represents the hyperparameter of the Huber loss function.

[0022] Furthermore, the final soft label loss of the student model is as follows:

[0023]

[0024] Among them, L Soft represents the final soft label loss of the student model, W pos represents the soft label weight of the positive triplet, W neg represents the soft label weight of negative triples, D Soft It represents the difference in triplet scores between the student model and the teacher model, T represents positive triplet, and T' represents negative triplet.

[0025] Furthermore, the final hard label loss of the student model is as follows:

[0026]

[0027] Among them, L Hard represents the final hard label loss of the student model, is the score of the triple (h, r, t) obtained by the student model, σ is the sigmoid function, and y is the true label of (h, r, t), where the y value of the positive sample is 1 and the y value of the negative sample is 0.

[0028] Furthermore, the final total loss of the student model is as follows:

[0029] L=γL Soft +(1-γ)L Hard

[0030] Among them, γ is a hyperparameter of the balance weight.

[0031] The present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned dual knowledge graph embedding fusion method based on knowledge distillation.

[0032] The present invention also proposes an electronic device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the above-mentioned dual knowledge graph embedding fusion method based on knowledge distillation.

[0033] The beneficial effects brought by the technical solution provided by the present invention are:

[0034] The present invention constructs a dual-teacher model based on two knowledge graph embedding vectors Em1 and Em2, uses the alignment entity between the two knowledge graphs as the connecting bridge of the dual-teacher model, uses the untrained KGE model as the student model, and uses the triple score calculated by the KGE model as the soft label. The student model first predicts the score of the triple, and passes the triple to the two teacher models for inquiry. The integration model integrates the output sequences of the two teachers to generate soft labels and soft label weights according to the evaluation results of the teachers. The student model obtains the soft label and updates the parameters through knowledge distillation. The student is guided by the knowledge of the two teachers, thereby learning the fusion result of the embedding of the two knowledge graphs. The knowledge graph embedding fusion method of the present invention fuses the embedding vectors of the two knowledge graphs into a unified vector space, thereby obtaining the embedding of the new knowledge graph, which can be directly applied to downstream tasks, saving the time of retraining the knowledge graph embedding and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a dual knowledge graph embedding fusion method based on knowledge distillation in an embodiment of the present invention;

[0036] Figure 2 It is a block diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0038] The flowchart of the dual knowledge graph embedding fusion method based on knowledge distillation in an embodiment of the present invention is as follows: Figure 1 , specifically including the following steps:

[0039] S1. Obtain two knowledge graphs G1 = {E1, R1, T1} and G2 = {E2, R2, T2} and their embedding vectors Em1 and Em2, and the entity pair set P = {(u, v) | u∈E1, v∈E2} of the two knowledge graphs. The embedding vectors Em1 and Em2 of the knowledge graphs G1 and G2 are trained by two knowledge graph embedding models respectively.

[0040] Among them, E1 and E2 represent the entity sets of G1 and G2 respectively, R1 and R2 represent the relationship sets of G1 and G2 respectively, T1 and T2 represent the triple sets of G1 and G2 respectively, T1={h1,r1,t1}, T2={h2,r2,t2}. The embedding vectors Em1 and Em2 of the two knowledge graphs are trained using classic knowledge graph embedding models, including TransE, RotatE, PairRE, etc. They are all KGE (knowledge graph embedding) models based on translation distance, among which TransE and others regard the relationship as the translation distance from the head entity to the tail entity; RotatE is also based on Euclidean space and models the relationship as a rotation operation from the head entity to the tail entity; PairRE represents the relationship with two vectors, which project the head and tail entities into Euclidean space respectively, minimizing the distance between the projected vectors. For the differences in the way of modeling relationships, the scoring functions of these models are as follows:

[0041] TransE:f r (h,t)=‖h+rt‖ L1 / L2

[0042]

[0043]

[0044] where f r (h, t) represents the score function of the triple (h, r, t) of the KGE model, ‖‖ L1 / L2 represents the L1 norm or L2 norm, represents the Hadmard (or element) product, (h, r, t) represents a triple, h represents the head entity, r represents the relationship, t represents the tail entity, and r represents the H and r T Represents the relationship embedding vector, which is used to project the head entity and the tail entity into the Euclidean space respectively.

[0045] S2. Two teacher models are constructed based on two knowledge graph embedding vectors Em1 and Em2 respectively. The two teacher models are connected with the entity pair set P as a bridge. The untrained knowledge graph embedding model is used as the student model, and an integrated model is constructed. Among them, the two knowledge graph embedding models of the embedding vectors Em1 and Em2 used to train the knowledge graphs G1 and G2 and the knowledge graph embedding model of the student model are set to the same scale.

[0046] S3. Based on the knowledge distillation framework, the student model is trained using knowledge graphs G1 and G2. The student model asks the two teacher models based on the scores of the predicted triples during each training. The teacher model passes the scores of the corresponding triples to the integration model based on its own knowledge. The integration model generates soft labels and assigns soft label weights. The student model performs fusion learning on the two teacher models based on the soft labels to obtain the final fused graph embedding.

[0047] The initialization of the student model can assign certain weights to the embedding vectors of the knowledge graphs G1 and G2, where the weights are determined by the proportion of entities and relationships in the two original graphs to the total entities and relationships. This initialization can be consistent with the direction of subsequent embedding fusion under certain circumstances and play a positive role, but there are also special cases where it conflicts with it.

[0048] In the preferred example of the present invention, in order to obtain the embedding vector of the new graph composed of two knowledge graphs, it is necessary to retrain the knowledge graph embedding model, which will waste a lot of time. The present invention takes into account the reusability of the independent embedding vectors of the two knowledge graphs, uses their embedding vectors as the teacher's knowledge to guide students, and creates a union of the two knowledge graphs for training the student model, so that the student model can quickly learn the embedded representation of the fused graph.

[0049] The integration model connects the teacher model and the student model. When the student model trains the knowledge graph, it receives the embedded representation of entities and relationships as input and outputs a score that reflects the correlation between entities and relationships. The student model continuously asks the two teacher models based on its own predicted triple scores. The teacher model outputs multiple triple scores for the triple scores asked by the student model (using the triple scores output by the teacher model as soft labels). The aggregation model selects the adaptively integrated result as feedback to the student model. The mechanism of generating soft labels by the integration model is expressed as:

[0050]

[0051]

[0052] in, represents the integrated soft label passed to the student model, Respectively represent the scores of a given triple (h, r, t) output by the two teacher models, τ1 and τ2 represent different thresholds for judging the triple score, where τ1 ranges from {10, 20} and τ2 ranges from {30, 50}, and their values ​​are adaptively adjusted according to the quality of the data set. This structure has high flexibility and accuracy in processing output results at multiple error levels or under different conditions, and is particularly suitable for scenarios with different responses to different error ranges.

[0053] The quality of soft labels provided by teachers varies. It is necessary to evaluate the quality of soft labels provided by teachers. Different weights of soft labels and hard labels are assigned to different triplets according to the different quality of soft labels, so as to retain the positive effect of high-quality soft labels and avoid the negative effect of low-quality soft labels. In theory, the KGE model will give higher scores to positive triples and lower scores to negative triples, but for some triplets that are difficult for the KGE model to grasp, the situation is just the opposite. Specifically, if the teacher model gives a high score to a negative triplet, it means that the teacher model tends to judge it as a positive triplet. If the teacher gives a low score to a positive triplet, it means that the teacher tends to judge it as a negative triplet. Then the soft label output by the teacher for this triplet is unreliable and misleading, which may have a negative impact on the student. For both cases, it is necessary to weaken the weight of the soft label and encourage the student model to learn more from the hard label.

[0054] The soft label weights assigned by the integrated model are expressed as:

[0055]

[0056]

[0057] Among them, W pos represents the soft label weight of the positive triplet, W neg Denotes the soft label weight of negative triples, α1, β1, α2, and β2 are all learnable parameters. The quality of the teacher model score for each triple is evaluated through the above method, and different soft label weights and hard label weights are adaptively assigned to different triples, helping the student model to selectively learn knowledge from the teacher and achieve better results.

[0058] When training the student model, there will be soft label loss and hard label loss. The final soft label loss of the student model is as follows:

[0059]

[0060] Among them, L Soft represents the final soft label loss of the student model, W pos represents the soft label weight of the positive triplet, W negrepresents the soft label weight of negative triples, D Soft It represents the difference in triplet scores between the student model and the teacher model, T represents positive triplet, and T' represents negative triplet. Negative triplet is constructed by replacing one of the head entity or tail entity of positive triplet with a random entity, and both cannot be replaced at the same time. This loss function can automatically balance the soft label loss and hard label loss without manually defining any hyperparameters.

[0061] Because the integrated model will adaptively integrate the output results of the teachers, the difference between the student and the triplet score integrated by the aggregation model is used as the soft label supervision target. The difference between the triplet score of the student model and the teacher model is defined as:

[0062]

[0063]

[0064] Among them, D Soft represents the triplet score difference between the student model and the teacher model, l δ () represents the Huber loss function, δ represents the hyperparameter of the Huber loss function, which is used to control the smoothness of the loss function. It usually needs to be adjusted in practical problems. In the present invention, δ=1. Huber loss is a loss function commonly used in regression problems. It is more robust when dealing with outliers (i.e., outliers) in data. Compared with the mean square error (MSE), Huber loss does not give excessive penalties when facing outliers like MSE.

[0065] The final hard label loss of the student model is as follows:

[0066]

[0067] Among them, L Hard represents the final hard label loss of the student model, is the score of the triple (h, r, t) obtained by the student model, σ is the sigmoid function, and y is the true label of (h, r, t), where the y value of the positive sample is 1 and the y value of the negative sample is 0.

[0068] The final total loss of the student model uses the weighted sum of soft label loss and hard label loss as follows:

[0069] L=γL Soft +(1-γ)L Hard

[0070] Among them, γ is a hyperparameter of the balance weight.

[0071] The student model is generally trained in two parts. On the one hand, students make changes based on their own prediction results and try to get closer to the integrated results of the two teachers. On the other hand, students learn independently in order to reach the teacher's level faster. By combining the two parts of training, the student model can learn the fusion embedding of the two teachers at a faster speed.

[0072] The two teacher models and the student model use the Adam optimizer to minimize the overall loss, and the overall loss function is:

[0073]

[0074] Among them, L final represents the overall loss function, is the parameter regularization term.

[0075] In an exemplary embodiment, a computer-readable storage medium is included, which stores a computer program. When the computer program is executed by a processor, the above-mentioned dual knowledge graph embedding fusion method based on knowledge distillation is implemented.

[0076] See also Figure 2 In an exemplary embodiment, an electronic device is also included, including at least one processor, at least one memory, and at least one communication bus.

[0077] Among them, a computer program is stored in the memory, and the computer program includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the above-mentioned dual knowledge graph embedding fusion method based on knowledge distillation.

[0078] In order to verify the effect of the method of the present invention, experiments are conducted on two commonly used knowledge graph completion benchmark datasets WN18RR and FB15k-237, which are subsets of WordNet and Freebase with redundant relations eliminated. Table 1 shows the statistical information of these two datasets.

[0079] Table 1

[0080] Dataset entity relation Training set Validation set Test Set WN18RR 40,943 11 74,403 9,300 9,300 FB15k-237 14,541 237 248,096 31,010 31,010

[0081] The two datasets used above are divided into two sub-datasets of the same size, called Graph1 and Graph2, respectively. The KGE model is used to pre-train Graph1 and Graph2 and build a teacher model. Table 2 shows the statistical information of the sub-datasets. Table 3 shows the connection between the divided sub-datasets, that is, the number of common entities and relationships, which is used to measure the correlation between them.

[0082] Table 2

[0083]

[0084] Table 3

[0085]

[0086] Standard indicators MRR (Mean Reciprocal Rank), MR (Mean Ranking), Hit@k (K=1,3,10), and time efficiency (expressed in time T(s)) are used. For the test triple (h,r,t), the head entity h is replaced with each entity e in the entity set to obtain the candidate triple (e,r,t), and then the score function is used to calculate the scores of all candidate triples. In descending order, the ranking of (h,r,t) can be obtained as the head prediction result. MR can measure the performance by the average ranking of all correct entities. MRR, the average reciprocal ranking of the test triples. Hit@k, the proportion of correct entities ranked in the top k. The higher the MRR and Hit@k, the lower the MR means better performance. The filtering mode is used to delete all triples in the candidate set that exist in the training, validation and test sets. Time efficiency: Compare the time efficiency of the embedding fusion model in this paper and the traditional KGE model, that is, the time spent on training the student model of the two is counted.

[0087] Several common KGE models such as TansE, TransH, RotatE, and PairRE are used as student and teacher models to implement the fusion framework. The fusion framework in this paper is compared with the model retrained without knowledge distillation operation, in which the fusion framework uses two sub-datasets of WN18RR and FB15k-237 as the datasets of the pre-trained teacher model to represent their original embeddings, and then uses the datasets WN18RR and FB15k-237 to train the student model. The model retrained without knowledge distillation operation directly uses the datasets WN18RR and FB15k-237.

[0088] The fusion framework is implemented by extending OpenKE, which is based on the open source KGE framework of PyTorch. The embedding dimensions of the teacher and student models are set to be the same and d = {200, 1024}, where d = 200 is the usual experimental situation. The batch size is set to 2000, and the training rounds are mostly 1000 and the maximum is 6000. For each positive triple, 64 negative triples are generated for WN18RR and 25 negative triples are generated for FB15k-237. Adam is selected as the optimizer, and the learning rate decay and starting decay thresholds are 0.96 and 10 respectively. The learning rate lr is set to {0.00005, 0.01, 0.5, 1}, and the corresponding best learning rate is set for different situations. The balance hyperparameter γ is set in {0.01, 0.1, 0.3, 0.5}. Table 4 shows the link prediction results and training time of each model on WN18RR. Table 5 shows the link prediction results of each model on WN18. Table 6 shows the link prediction results and training time of each model on FB15K-237.

[0089] Table 4

[0090] Model MRR MR Hit@10 Hit@3 Hit@1 T(s) TransE 0.192 4900 0.446 0.365 0.001 756 Dis-TransE 0.196 4124 0.464 0.368 0.000 97(↓87.2％) TransH 0.182 5809 0.418 0.353 0.000 918 Dis-TransH 0.193 5116 0.439 0.367 0.000 132(↓85.6％) RotatE 0.488 1721 0.586 0.507 0.438 3035 Dis-RotatE 0.482 1947 0.587 0.505 0.428 598(↓80.3％)

[0091] Table 5

[0092] Model MRR MR Hit@10 Hit@3 Hit@1 PairRE 0.753 760 0.866 0.821 0.678 Dis-PairRE 0.748 824 0.867 0.821 0.667

[0093] Table 6

[0094]

[0095]

[0096] The teacher model and the student model of the present invention both use the same dimension. Dis-model represents the model of the method of the present invention, and its dual teacher and student models use the same model, and the model without Dis represents the use of the model for retraining.

[0097] As shown in Tables 4, 5, and 6, the fusion framework has shown good results on the datasets FB15K-237 and WN18RR. On WN18RR, Dis-TransE and Dis-TransH can achieve comparable or even better results than the baseline. Dis-RotatE has a slight decrease in the MR index, but other indicators are also very close to the baseline, reaching 98.8%, 100%, and 99.6% of the baseline on MRR, Hit@10, and Hit@3. Since WN18RR may be relatively sparse and not suitable for PairRE, PairRE uses the dataset WN18 for testing. As shown in Table 5, Dis-PairRE performs well on the Hit@K index and can basically reach the level of the baseline. In addition, on the dataset FB15K-237, Dis-TransE is basically close to the results of the baseline, while Dis-RotatE can show better results than the baseline on the MR index, Hit@10, and Hit@3, and is almost the same as the baseline on MRR and Hit@1. Dis-TransH can reach 97.8%, 98.3% and 98.8% of the baseline in MRR, Hit@10 and Hit@3 respectively. The overall effect of Dis-PairRE is slightly lower, but it can reach 97.5%, 98.3% and 97.7% in MRR, Hit@10 and Hit@3 respectively. This shows that the fusion framework in this paper can achieve good performance.

[0098] In addition, the fusion framework can also effectively reduce the training time. As shown in Tables 4 and 6, compared with retraining, on the dataset WN18RR, Dis-TransE, Dis-TransH and Dis-RotatE can reduce the training time by 87.2%, 85.6% and 80.3% respectively, and the overall training time can be reduced by 80-90%. On the dataset FB15K-237, Dis-TransE, Dis-TransH, Dis-RotatE and Dis-PairRE can also reduce by 76.5%, 75.5%, 72.6% and 65.7% respectively, and the overall training time can be reduced to between 70-80%. This shows that the fusion framework in this paper can effectively reduce the training time and reduce the computational overhead.

[0099] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dual knowledge graph embedding fusion method based on knowledge distillation, characterized in that: The following steps are involved: S1, obtain two knowledge graphs G1 and G2 and their embedding vectors Em1 and Em2, and the entity pair set P of the two knowledge graphs G1 and G2 aligned; S2. Two teacher models are constructed based on two knowledge graph embedding vectors Em1 and Em2 respectively. The two teacher models are connected by the entity pair set P. An untrained knowledge graph embedding model is used as the student model, and an integrated model is constructed. S3. Based on the knowledge distillation framework, the student model is trained using knowledge graphs G1 and G2. The student model asks the two teacher models based on the scores of the predicted triples during each training. The teacher model passes the scores of the corresponding triples to the integration model based on its own knowledge. The integration model generates soft labels and assigns soft label weights. The student model performs fusion learning on the two teacher models based on the soft labels to obtain the final fused graph embedding.

2. According to the dual knowledge graph embedding fusion method based on knowledge distillation according to claim 1, it is characterized in that: Based on the knowledge distillation framework, the student model is trained using the union of knowledge graphs G1 and G2.

3. According to the dual knowledge graph embedding fusion method based on knowledge distillation according to claim 1, it is characterized in that: The mechanism of integrating models to generate soft labels is expressed as: in, represents the integrated soft label passed to the student model, They respectively represent the scores of a given triplet (h, r, t) output by the two teacher models. τ1 and τ2 represent different thresholds for judging the scores of triplet, where the value range of τ1 is {10, 20} and the value range of τ2 is {30, 50}.

4. According to the dual knowledge graph embedding fusion method based on knowledge distillation according to claim 3, it is characterized in that: The soft label weights assigned by the integrated model are expressed as: Among them, W pos represents the soft label weight of the positive triplet, W neg represents the soft label weight of the negative triplet, α1, β1, α2 and β2 are all learnable parameters.

5. According to the dual knowledge graph embedding fusion method based on knowledge distillation according to claim 1, it is characterized in that: The triplet score difference between the student model and the teacher model is defined as: Among them, D Soft represents the triplet score difference between the student model and the teacher model, l δ () represents the Huber loss function, and δ represents the hyperparameter of the Huber loss function.

6. A dual knowledge graph embedding fusion method based on knowledge distillation according to claim 5, characterized in that: The final soft label loss of the student model is as follows: Among them, L Soft represents the final soft label loss of the student model, W pos represents the soft label weight of the positive triplet, W neg represents the soft label weight of negative triples, D Soft It represents the difference in triplet scores between the student model and the teacher model, T represents positive triplet, and T' represents negative triplet.

7. A dual knowledge graph embedding fusion method based on knowledge distillation according to claim 6, characterized in that: The final hard label loss of the student model is as follows: Among them, L Hard represents the final hard label loss of the student model, is the score of the triple (h, r, t) obtained by the student model, σ is the sigmoid function, and y is the true label of (h, r, t), where the y value of the positive sample is 1 and the y value of the negative sample is 0.

8. The dual knowledge graph embedding fusion method based on knowledge distillation according to claim 7 is characterized in that: The final total loss of the student model is as follows: L=γL Soft +(1-γ)L Hard Among them, γ is a hyperparameter of the balance weight.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the method according to any one of claims 1 to 8.

Images