Knowledge Graph Completion Method Based on Knowledge Distillation Combined with Endogenous Rule Constraints

Abstract

The present invention belongs to the technical field of bridge detection knowledge graph completion, and particularly relates to a knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints, comprising the following steps: S1. Design n common knowledge graph subgraph structures in the field of bridge detection; S2. Use the knowledge graph subgraph structures in S1 to screen the triples of the original knowledge graph to obtain m triple subsets; S3. Clean the triple subsets screened in S2 to obtain a rule training set; S4. Use the rule training set as a supplement to the training data set, and jointly train the Teacher network of knowledge distillation using the rule training set and the training data set; S5. Use the correct labels in the training data set as supervision jointly, and use the prediction results of the trained Teacher network as another supervision, and jointly train the Student network of knowledge distillation; S6. Use the trained Student network to complete the original knowledge graph. This method can better achieve the completion of the knowledge graph in the field of bridge management and maintenance.

Description

Technical Field

[0001] The present invention belongs to the technical field of bridge detection knowledge graph completion, and particularly relates to a knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints. Background Art

[0002] The knowledge graph was proposed by Google as a knowledge base to enhance its search function and has developed rapidly in recent years. With the continuous discovery of the value of the knowledge graph, various domain knowledge graphs have also been rapidly constructed. Currently, the knowledge graph has been widely applied in various natural language processing tasks, but the lack of relationships between entities in the knowledge graph has also brought many problems to its practical applications. Therefore, a large amount of research work has been carried out in the academic community around the completion of the knowledge graph.

[0003] With the rapid development of big data and artificial intelligence related technologies in the field of bridge detection, a large number of automated and semi-automated intelligent technologies have gradually been introduced as aids to help industry practitioners carry out daily bridge detection work. Among them, the more prominent ones, such as the technical systems based on neural networks and deep learning like automatic bridge detection question answering and bridge detection information visualization, mostly rely on a complete and mature bridge domain knowledge graph as support. Regarding the automatic bridge detection question answering system as a technical system, the construction and completion of the knowledge graph are at the uppermost reaches of the entire system. A complete and mature knowledge graph is the cornerstone for the smooth progress of downstream tasks such as entity extraction, entity disambiguation, knowledge fusion, link prediction, and graph reasoning. Therefore, constructing a vertical industry knowledge graph focusing on the bridge detection field is an essential and crucial step for the continuous development of intelligent technologies in the bridge detection field.

[0004] The construction of a knowledge graph is generally divided into an ontology layer (schema layer) and an instance layer (data layer). The ontology layer is the conceptual model and logical basis of the knowledge graph, which standardizes and constrains the instance layer. In a standardized knowledge graph, an ontology is often used as the schema layer of the knowledge graph, and the instance layer of the knowledge graph is constrained by the rules and axioms defined in the ontology. The knowledge graph can also be regarded as an instantiated ontology, and the data layer of the knowledge graph is an instance of the ontology. Engineering practice and theoretical analysis show that the professional knowledge in the field of bridge inspection is extensive and complex, and it is impossible to complete the construction and improvement of the ontology layer of the knowledge graph in a short time or within a controllable cost range. The main reason is that constructing the ontology layer requires summarizing, inducting, and expressing the domain knowledge patterns with a high degree of generalization that coordinates the entire field, and requires the relevant personnel participating in the construction of the ontology layer to have a large amount of domain knowledge and pay extremely high time costs. Therefore, in the initial stage of constructing most vertical domain knowledge graphs, the top-level design of the ontology layer is bypassed and the instance layer is directly constructed, resulting in the lack of the ontology layer as a guidance and constraint for the instantiated construction of the knowledge graph. As a direct application of knowledge reasoning, the knowledge graph completion technology must require the graph to provide rules and constraints other than the instance layer as the reasoning basis. Objectively, there is a contradiction that the knowledge graph needs the rules provided by the ontology layer for reasoning in the absence of the ontology layer.

[0005] In order to provide reasoning constraints for a knowledge graph lacking an ontology layer while avoiding manual participation in the design and formulation of rules, a common practice in existing related work is to create a set of simple rules as constraints, such as introducing several first-order logic paradigms to form an incomplete rule system, or defining specific subgraph patterns for global search and matching. The commonality of such rules is that although they have strong interpretability. However, this type of method has extremely low tolerance for noisy data, requires a very high degree of data cleanliness for training, and the rule constraints are too fixed. These characteristics determine that this type of method is not suitable for application in vertical domains with relatively low data governance levels. The field of bridge management and maintenance just belongs to such a vertical domain with extensive and complex professional knowledge. For the field of bridge management and maintenance, the rule design of this method is divorced from the knowledge graph and the dataset itself, and in most cases, it cannot well represent the rule characteristics of the dataset itself. Therefore, it is difficult to use this method for knowledge graph completion in the field of bridge management and maintenance.

[0006] In summary, in the absence of an ontology layer and a rule system as guidance, how to better achieve knowledge graph completion in the field of bridge management and maintenance has become an urgent problem to be solved. Summary of the Invention

[0007] Aiming at the above deficiencies of the existing technology, the present invention provides a knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints, which can better achieve knowledge graph completion in the field of bridge management and maintenance.

[0008] To solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints, comprising the following steps:

[0010] S1. According to the experience of constructing the knowledge graph and the professional knowledge in the field of bridge detection, design n common knowledge graph sub-graph structures in the field of bridge detection;

[0011] S2. Use the knowledge graph sub-graph structure in S1 to screen the triples of the original knowledge graph, and obtain m triple subsets as the rule constraints extracted endogenously from the original knowledge graph; where m ≤ n;

[0012] S3. Clean the triple subsets screened in S2 to obtain a rule training set;

[0013] S4. Use the rule training set obtained in S3 as a supplement to the training data set, and jointly train the Teacher network of knowledge distillation using the rule training set and the training data set, so that the Teacher network learns the rule constraints extracted from the original knowledge graph;

[0014] S5. Use the correct labels in the training data set as supervision jointly, and at the same time use the prediction results of the trained Teacher network as another supervision, and jointly train the Student network of knowledge distillation, so that the Student network approaches the learning result after fusing the rules;

[0015] S6. Use the trained Student network to complete the original knowledge graph.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. For the extraction and mining of rules, most of the existing methods design rules according to domain knowledge. These rule designs are separated from the knowledge graph and the data set itself, and in most cases cannot well represent the rule characteristics of the data set itself. The inference constraints introduced by the present invention are based on the common knowledge graph sub-graph structures in the field of bridge detection, and extract the rules endogenous to the original knowledge graph and the data set. These rules are more consistent with the original knowledge graph at the knowledge level, can better represent the rule characteristics of the data set itself, and thus can better help the model learn the domain knowledge representation embedded in the knowledge graph and improve the expression ability of the knowledge graph embedding.

[0018] 2. For rule constraints, existing methods usually adopt methods based on symbolic learning. These methods have extremely low tolerance for noisy data, high requirements for the cleanliness of the training data, and overly fixed rule constraint patterns, and are not applicable to domain knowledge graphs. The present invention introduces a network architecture based on knowledge distillation. The Teacher network fully learns the extracted rules and then forms supervision for the Student network to indirectly complete the integration of the rules. In this way, it not only ensures strong robustness to data noise but also can fully learn the representation of the rules.

[0019] 3. The model of this method can mine targeted rule constraints for the existing knowledge graph in the absence of guidance from the ontology layer and the rule system, and then use these rules for constraint to complete the learning of the rules under the architecture of knowledge distillation, and finally realize the completion of the graph.

[0020] In summary, using this method, the knowledge graph completion in the field of bridge management and maintenance can be better realized.

[0021] Preferably, S2 includes:

[0022] S21. Incorporate all triples constituting the original knowledge graph into a unified triple set;

[0023] S22. Traverse the triple set obtained in S21 to obtain triple subsets that conform to the sub-graph structures of each knowledge graph designed in S1 respectively;

[0024] S23. If the proportion of triples of a certain knowledge graph sub-graph structure reaches a preset threshold, it is determined that the rule represented by this knowledge graph sub-graph structure is endogenous to the original knowledge graph;

[0025] S24. Use the triple subsets corresponding to the m knowledge graph sub-graph structures determined to be endogenous to the original knowledge graph in S23 as the rule constraints extracted from the original knowledge graph.

[0026] In this way, the integrity of the screening of triples in the original knowledge graph can be ensured, thus ensuring the effectiveness of rule extraction and mining.

[0027] Preferably, in S3, the cleaning process includes: deduplication, removing abnormal formats, manually screening incorrect knowledge, and normalizing the output.

[0028] In this way, the effectiveness of the rule constraints for subsequent model training is ensured, and the effect of subsequent model training is ensured.

[0029] Preferably, in S4, the training process of the Teacher network is as follows:

[0030] S41. Use the rule training set of S3 as the training set T2, and fuse it with the training data set T1 in the existing original data set to form a combined training set T.

[0031] S42. Initialize the Teacher network.

[0032] S43. Train T1 to obtain the loss function Loss_train on the training set, and train T2 to obtain the loss function Loss_rule on the rule set.

[0033] S44. Optimize the overall loss function Loss = Loss_train + Loss_rule of the Teacher network to complete the training of the Teacher network.

[0034] In this way, the Teacher network can fully learn the rules extracted from the original knowledge graph, ensuring the effect of subsequent training.

[0035] Preferably, the Teacher network is based on the LineaRE model.

[0036] Preferably, in S5, the training process of the Student network is as follows:

[0037] S51. Initialize the Student network.

[0038] S52. During the training process, for each batch of training samples, the label corresponding to the sample is y_label, the prediction given by the trained Teacher network is y_teacher; the prediction of the Student network is y_student; use y_label as the supervision information generated by the training set itself, and use y_teacher as the supervision information generated by the rule constraint.

[0039] S53. Calculate the loss function Loss_1 by the cross-entropy between y_label and y_student, and calculate the loss function Loss_2 by the cross-entropy between y_student and y_teacher.

[0040] S54. Optimize the overall loss function Loss = Loss_1 + Loss_2 of the Student network to complete the training of the Student network.

[0041] This way of supervision can indirectly complete the integration of the rules extracted from the original knowledge graph. When training the Student network, it can ensure both fully learning the data of the training set and taking into account the incorporated rule constraints.

[0042] Preferably, the Student network is based on the LineaRE model. Description of the Drawings

[0043] To make the objectives, technical solutions, and advantages of the invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings, where:

[0044] Figure 1 It is the flowchart in the embodiment. Specific Embodiments

[0045] The following is a further detailed description through specific embodiments:

[0046] Embodiment:

[0047] As Figure 1 shown, in this embodiment, a knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints is disclosed, including the following steps:

[0048] S1. According to the experience of constructing the knowledge graph and the professional knowledge in the field of bridge detection, n common knowledge graph subgraph structures in the field of bridge detection are designed. Specifically, during implementation, the value of n is determined by the specific design of the knowledge graph subgraph structure.

[0049] S2. Use the knowledge graph subgraph structure in S1 to screen the triples of the original knowledge graph, and obtain m triple subsets as the rule constraints extracted from the original knowledge graph; where m ≤ n. Specifically, during implementation, S2 includes:

[0050] S21. Incorporate all the triples constituting the original knowledge graph into a unified triple set;

[0051] S22. Traverse the triple set obtained in S21 to respectively obtain triple subsets that conform to each knowledge graph subgraph structure designed in S1;

[0052] S23. If the proportion of the triples of a certain knowledge graph subgraph structure reaches a preset threshold, it is determined that the rule represented by this knowledge graph subgraph structure is endogenous to the original knowledge graph;

[0053] S24. Use the triple subsets corresponding to the m knowledge graph subgraph structures determined to be endogenous to the original knowledge graph in S23 as the rule constraints extracted from the original knowledge graph.

[0054] In this way, the integrity of the screening of the triples in the original knowledge graph can be ensured, thereby ensuring the effectiveness of rule extraction and mining.

[0055] S3. Clean the triple subsets screened in S2 to obtain a rule training set. Specifically, during implementation, the cleaning process includes: removing duplicates, removing abnormal formats, manually screening for incorrect knowledge, and normalizing the output. In this way, the effectiveness of the rule constraints for subsequent model training is ensured, and the effect of subsequent model training is ensured.

[0056] S4. Use the rule training set obtained in S3 as a supplement to the training data set, and jointly train the Teacher network of knowledge distillation using the rule training set and the training data set, so that the Teacher network learns the rule constraints extracted from the original knowledge graph.

[0057] Specifically, the Teacher network is based on the LineaRE model. The training process of the Teacher network is as follows:

[0058] S41. Use the rule training set of S3 as the training set T2, and fuse it with the training data set T1 in the existing original data set to form a joint training set T;

[0059] S42. Initialize the Teacher network;

[0060] S43. Train T1 to obtain the loss function Loss_train on the training set, and train T2 to obtain the loss function Loss_rule on the rule set;

[0061] S44. Optimize the overall loss function Loss = Loss_train + Loss_rule of the Teacher network to complete the training of the Teacher network.

[0062] In this way, the Teacher network can fully learn the rules extracted from the original knowledge graph, ensuring the effect of subsequent training.

[0063] S5. Use the correct labels in the training data set jointly as supervision, and at the same time use the prediction results of the trained Teacher network as another kind of supervision, and jointly train the Student network of knowledge distillation, so that the Student network approaches the learning result after fusing the rules.

[0064] Specifically, the Student network is based on the LineaRE model. The training process of the Student network is as follows:

[0065] S51. Initialize the Student network;

[0066] S52. During the training process, for each batch of training samples, the label corresponding to the sample is y_label, and the prediction given by the trained Teacher network is y_teacher; the prediction of the Student network is y_student; use y_label as the supervision information generated by the training set itself, and use y_teacher as the supervision information generated by the rule constraints;

[0067] The loss function Loss_1 is calculated by the cross - entropy between S53, y_label and y_student, and the loss function Loss_2 is calculated by the cross - entropy between y_student and y_teacher;

[0068] S54, optimize the overall loss function Loss of the Student network, Loss = Loss_1+Loss_2, and complete the training of the Student network.

[0069] In this way of supervision, the rules of the original knowledge graph extraction can be indirectly incorporated. When training the Student network, it can not only ensure full learning of the data in the training set but also take into account the incorporated rule constraints.

[0070] The loss function to be optimized by the Student comes from two parts. One part is the difference between the predicted label and the true label (Loss_1), and the other part is the difference between the predicted label generated by the Teacher network and the predicted label generated by the Student network (Loss_2). In this way, it not only ensures that the model can learn the characteristics of the data set itself but also ensures that the model can learn the extracted rule characteristics.

[0071] S6. Use the trained Student network to complete the complement of the original knowledge graph.

[0072] The inference constraints introduced in the present invention are based on the common knowledge graph sub - graph structure in the field of bridge detection, and extract the rules endogenous to the original knowledge graph and the data set. These rules are more consistent with the original knowledge graph at the knowledge level, can better represent the rule characteristics of the data set itself, and thus can better help the model learn the domain knowledge representation embedded in the knowledge graph and improve the expression ability of the knowledge graph embedding. In addition, the present invention introduces a network architecture based on knowledge distillation. The Teacher network fully learns the extracted rules and then forms a supervision method for the Student network to indirectly complete the incorporation of the rules. In this way, it not only ensures strong robustness to data noise but also can fully learn the rule representation. The model of this method can mine targeted rule constraints for the existing knowledge graph in the absence of the guidance of the ontology layer and the rule system, and then use these rules as constraints to complete the learning of the rules under the knowledge distillation architecture, and finally realize the complement of the graph.

[0073] To better illustrate the effect of the present invention, a specific verification example is described below.

[0074] Experimental data

[0075] The research work of this verification example used three datasets for verification, namely the public dataset WN18, the self-built dataset Bridge-KGC in the field of bridge maintenance and management, and the self-built dataset Medical-KGC in the field of medical trials. The dataset statistics are shown in Table 1.

[0076] Table 1 Dataset Statistics

[0077] DataSet relation entity triple(train / valid / test) WN18 18 40943 141442 / 5000 / 5000 Bridge-KGC 34 4605 36818 / 3938 / 3940 Medical-KGC 86 47936 160747 / 8000 / 8000

[0078] Evaluation Metrics

[0079] This verification example uses the general evaluation metrics of the knowledge graph completion task to measure the performance of the proposed method.

[0080] Mean Reciprocal Rank (MRR):

[0081]

[0082] where Q represents the model input, that is, a set of incomplete triples (query set), |Q| represents the number of elements in the query set, and rank i represents the ranking of the i-th target entity in the i-th prediction list given by the model.

[0083] MRR is widely used in sorting problems. In the link prediction task, the results are sorted from high to low according to the scoring function. MRR evaluates the sorting algorithm according to the ranking of the target answer in the sorting. The higher the ranking of the target answer, the better the ranking algorithm.

[0084] Mean-Rank (MR):

[0085]

[0086] Mean-Rank (MR) is similar to MRR and is usually used in Top-K sorting problems. MR represents the average ranking of the target entity in the test set.

[0087] Hits@n:

[0088]

[0089] Hits@n (usually n = 1, 3, 10) represents the proportion of the predicted target entities ranked in the top n in the test set. Considering the continuous feature of entity vectorization, n = 3 and n = 10 are generally considered important indicators for measuring the accuracy of the completion task.

[0090] The performance of the model for the knowledge graph completion task is comprehensively measured by the above indicators. A model with a lower MR indicator and higher MRR and Hits@n simultaneously has better performance.

[0091] Comparative Experiments and Analysis

[0092] In this verification example, representative results in the general domain knowledge graph completion task are selected as the baseline model for comparison.

[0093] ComplEx: This model solves the knowledge graph link prediction problem based on the overall idea of latent factor decomposition. It innovatively expands the embedding space to the complex domain and cleverly uses the mathematical properties of the Hermitian dot product operation to handle more complex binary relations.

[0094] QuatDE: This model uses a dynamic mapping strategy to explicitly capture various relationship patterns of entities and separate different semantic information of entities. It uses a transition vector to adjust the point position of the entity embedding vector in the quaternion space through the Hamilton product, enhancing the feature interaction ability between triple elements.

[0095] ConvE: It uses a convolutional neural network to learn the representations of entities and relations. The representations of entities and relations are regarded as pixels in an image, and then a convolutional neural network is used to learn the relationships between these pixels.

[0096] The results in Table 2 verify that the method proposed in this verification example can fully adapt to the knowledge graph completion task, outperforming the baseline model in the key indicators Hits@3 and Hits@10, and being competitive enough in other indicators.

[0097] Table 2 Comparative Experiments on Datasets

[0098]

[0099]

[0100] Knowledge graph embedding is the basis for solving the link prediction problem. The knowledge graph embedding method based on representation learning is represented by the translation model. From the initial TransE, TransH, RotatE to date, its modeling idea has been continuously refined and complicated on the basis of following translational invariance. By designing a more reasonable embedding space, problems such as entity mapping conflicts are alleviated, but ultimately the problem that the fitting of knowledge lags behind the shallow layer cannot be solved.

[0101] The comparison models ComplEx and QuatDE selected in this verification example are currently the most performant representative models based on representation learning on the WN18 dataset, and they have proposed improvements to previous models in terms of embedding vector operations and embedding space design respectively. Through the comparison of experimental results, it can be seen that the method proposed in this verification example outperforms the representatives based on representation learning on the WN18 dataset in all aspects of the Hit@n metric. For the two vertical domain datasets, there is a slight gap when n = 1, while the accuracy steadily improves when n = 3 and n = 10. This result fully conforms to the theoretical differences between the method proposed in this verification example and a class of models based on representation learning: ComplEx and QuatDE can accurately search for the missing entity with the highest confidence by finely modeling entity embeddings, so they have an advantage in the Hit@1 metric; while the method proposed in this verification example, due to the introduction of knowledge distillation of the rule set, based on the theory of catastrophic forgetting of knowledge distillation, will to some extent blur the model's memory of accurate knowledge. However, due to the characteristics of dense domain knowledge links and complex structures, when expanding the search scope, that is, when n = 3 or n = 10, the distilled rule knowledge can effectively handle structural searches to find entities that do not have high confidence but conform to the rules, so it is superior to this class of translation models in the Hit@3 and Hit@10 metrics.

[0102] ConvE uses a convolutional neural network to learn the representations of entities and relationships. Its core idea is to regard the representations of entities and relationships as pixels in an image, and then use a convolutional neural network to learn the relationships between these pixels. Although it pays more attention to the knowledge at the graph structure level compared to translation models, its disadvantage is that it flattens the knowledge graph, defaulting to the assumption that all relationships and entities in the graph are equally important. Based on the analysis of domain characteristics, the information entropy of the domain knowledge graph shows that the internal distribution of the graph is not uniform, that is, it shows a long-tail effect where a small number of entities occupy a central position at the knowledge level. And the constraints on rules in this verification example are obviously easier to fit the non-flattened graph structure. Therefore, compared with ConvE, the method proposed in this verification example performs better in almost all metrics.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Those of ordinary skill in the art should understand that any modifications or equivalent replacements to the technical solutions of the present invention, without departing from the purpose and scope of the present technical solution, should be covered within the scope of the claims of the present invention.

Claims

1. A knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints, characterized in that, It includes the following steps: S1. Based on the experience of constructing knowledge graphs and the professional knowledge in the field of bridge inspection, design n common knowledge graph sub-graph structures in the field of bridge inspection; S2. Use the knowledge graph sub-graph structures in S1 to screen the triples of the original knowledge graph, and obtain m triple subsets, which are used as the rule constraints extracted endogenously from the original knowledge graph; where m ≤ n; S3. Clean the triple subsets screened in S2 to obtain a rule training set; S4. Use the rule training set obtained in S3 as a supplement to the training data set, and jointly train the Teacher network of knowledge distillation using the rule training set and the training data set, so that the Teacher network learns the rule constraints extracted from the original knowledge graph; S5. Use the correct labels in the training data set as supervision, and at the same time use the prediction results of the trained Teacher network as another supervision, and jointly train the Student network of knowledge distillation, so that the Student network approaches the learning result after integrating the rules; S6. Use the trained Student network to complete the complement of the original knowledge graph, where the original knowledge graph is the knowledge graph in the field of bridge management and maintenance; obtain the complemented knowledge graph in the field of bridge management and maintenance; S7. Use the complemented knowledge graph in the field of bridge management and maintenance to perform entity extraction, entity disambiguation, knowledge fusion, link prediction or graph reasoning tasks on bridge information; Among them, S2 includes: S21. Incorporate all the triples that make up the original knowledge graph into a unified triple set; S22. Traverse the triple set obtained in S21 to obtain triple subsets that conform to each knowledge graph sub-graph structure designed in S1 respectively; S23. If the proportion of the triples of a certain knowledge graph sub-graph structure reaches a preset threshold, it is determined that the rule represented by this knowledge graph sub-graph structure is endogenous to the original knowledge graph; S24. Use the triple subsets corresponding to the m knowledge graph sub-graph structures determined to be endogenous to the original knowledge graph in S23 as the rule constraints extracted from the original knowledge graph.

2. The knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints according to claim 1, wherein: In S3, the cleaning process includes: deduplication, removing abnormal formats, manually screening incorrect knowledge, and normalizing the output.

3. The knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints according to claim 2, wherein: In S4, the training process of the Teacher network is as follows: S41. Use the rule training set in S3 as the training set T2, and fuse it with the training data set T1 in the existing original data set to form a joint training set T; S42. Initialize the Teacher network; S43. Train T1 to obtain the loss function Loss_train on the training set, and train T2 to obtain the loss function Loss_rule on the rule set; S44. Optimize the overall loss function Loss = Loss_train + Loss_rule of the Teacher network to complete the training of the Teacher network.

4. The knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints according to claim 3, wherein: The Teacher network is based on the LineaRE model.

5. The knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints according to claim 3, wherein: In S5, the training process of the Student network is as follows: S51. Initialize the Student network; S52. During the training process, for each batch of training samples, the label corresponding to the sample is y_label, the prediction given by the trained Teacher network is y_teacher; the prediction of the Student network is y_student; y_label is used as the supervision information generated by the training set itself, and y_teacher is used as the supervision information generated by the rule constraint; S53. The cross-entropy between y_label and y_student is calculated to obtain the loss function Loss_1, and the cross-entropy between y_student and y_teacher is calculated to obtain the loss function Loss_2; S54. Optimize the overall loss function Loss = Loss_1 + Loss_2 of the Student network to complete the training of the Student network.

6. The knowledge graph completion method based on knowledge distillation combined with endogenous rule constraints according to claim 5, wherein: The Student network is based on the LineaRE model.

Images