Knowledge graph completion method, system, device and medium based on knowledge distillation
By combining knowledge distillation technology and soft decision trees to train student network models, the problems of scale and interpretability of neural network models deployed on embedded devices are solved, and effective knowledge graph completion and reasoning are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ROCKET FORCE UNIV OF ENG
- Filing Date
- 2023-05-19
- Publication Date
- 2026-07-21
Smart Images
Figure CN116629365B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge graph technology, and in particular to a knowledge graph completion method, system, device and medium based on knowledge distillation. Background Technology
[0002] Knowledge reasoning is an important means of knowledge graph completion. With the development of neural network models, their application in knowledge reasoning has also received widespread attention. Knowledge reasoning methods based on neural network models have stronger reasoning and generalization abilities, make higher utilization of entity and relation information in the knowledge base, and achieve better reasoning results.
[0003] The success of neural network models stems from two main factors: firstly, their greater depth and number of parameters grant them stronger nonlinear fitting capabilities; secondly, continuous advancements in hardware enable rapid training. However, when considering practical applications and porting these neural network models to embedded or mobile devices, several constraints arise: 1) a massive number of model parameters; 2) high computational cost; and 3) high power consumption. Therefore, compressing the model while maintaining its performance has become a key challenge, leading to the rapid development of neural network model compression.
[0004] There are many existing methods for compressing neural network models, and knowledge distillation is one of them. The main idea of knowledge distillation is to learn the class distribution output through softmax, transforming knowledge from a large teacher model to a small student model. However, currently, knowledge distillation is mainly applicable to image classification tasks, and there is no research on more complex knowledge reasoning problems in knowledge graphs. In addition, knowledge distillation often involves transforming large-scale neural network models into small-scale neural network models, lacking interpretability. Summary of the Invention
[0005] The purpose of this invention is to provide a knowledge graph completion method, system, device, and medium based on knowledge distillation, which achieves interpretability while reducing the scale, thus fulfilling the purpose of knowledge graph completion.
[0006] To achieve the above objectives, the present invention provides the following solution:
[0007] In a first aspect, the present invention provides a knowledge graph completion method based on knowledge distillation, comprising:
[0008] Obtain the knowledge graph to be completed;
[0009] Based on the knowledge graph completion model and the knowledge graph to be completed, determine the information to be supplemented in the knowledge graph to be completed;
[0010] Based on the information to be supplemented, the knowledge graph to be completed is supplemented to obtain the completed knowledge graph;
[0011] The training process of the knowledge graph completion model is as follows:
[0012] To meet the target requirements, training data and label data are determined from professional data in the domain to which the knowledge graph to be completed belongs;
[0013] Construct a relational adjacency matrix based on the training data;
[0014] Based on the relational adjacency matrix and the label data, a teacher network model based on a relational graph convolutional network is trained using knowledge distillation techniques.
[0015] Based on the relational adjacency matrix, the label data, and the logical output of the teacher network model based on the relational graph convolutional network, a student network model based on a soft decision tree is trained using knowledge distillation technology to obtain a knowledge graph completion model.
[0016] Secondly, the present invention provides a knowledge graph completion system based on knowledge distillation, comprising:
[0017] The module for acquiring knowledge graphs to be completed is used to acquire knowledge graphs to be completed.
[0018] The module for determining information to be supplemented is used to determine the information to be supplemented in the knowledge graph to be supplemented based on the knowledge graph completion model and the knowledge graph to be supplemented.
[0019] The knowledge graph completion determination module is used to supplement the knowledge graph to be completed based on the information to be supplemented, so as to obtain a completed knowledge graph;
[0020] The training process of the knowledge graph completion model is as follows:
[0021] To meet the target requirements, training data and label data are determined from professional data in the domain to which the knowledge graph to be completed belongs;
[0022] Construct a relational adjacency matrix based on the training data;
[0023] Based on the relational adjacency matrix and the label data, a teacher network model based on a relational graph convolutional network is trained using knowledge distillation techniques.
[0024] Based on the relational adjacency matrix, the label data, and the logical output of the teacher network model based on the relational graph convolutional network, a student network model based on a soft decision tree is trained using knowledge distillation technology to obtain a knowledge graph completion model.
[0025] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to cause the electronic device to perform the knowledge graph completion method based on knowledge distillation as described in the first aspect.
[0026] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge graph completion method based on knowledge distillation as described in the first aspect.
[0027] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0028] For complex knowledge reasoning tasks in knowledge graphs, the teacher network model is based on a relational graph convolutional network, and the student network model is based on a soft decision tree. While achieving a reduction in scale, it also has interpretability, thus achieving the goal of knowledge graph completion. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 A flowchart illustrating the knowledge graph completion method based on knowledge distillation provided in this embodiment of the invention;
[0031] Figure 2 This is a diagram illustrating the training process of the knowledge graph completion model provided in an embodiment of the present invention.
[0032] Figure 3 The overall structure diagram of the teacher network model and student network model trained based on knowledge distillation technology is provided in the embodiments of the present invention;
[0033] Figure 4 This is an overall structure diagram of inference and prediction based on a trained student network model provided in an embodiment of the present invention;
[0034] Figure 5 This invention provides a knowledge graph of a unit's equipment management business as an embodiment of the invention.
[0035] Figure 6 This is an example diagram of interpretable reasoning in a decision tree within a student network model provided in an embodiment of the present invention.
[0036] Figure 7This is a schematic diagram of the knowledge graph completion system based on knowledge distillation provided in an embodiment of the present invention. Detailed Implementation
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0039] Terminology Explanation
[0040] Knowledge distillation is a commonly used model compression method. Unlike pruning and quantization in model compression, knowledge distillation builds a lightweight smaller model and trains it using supervision information from a larger, more powerful model, aiming to achieve better performance and accuracy. The larger model is called the teacher model, and the smaller model is called the student model.
[0041] Soft decision trees (SDTs): As a type of fuzzy decision tree, the structure of a soft decision tree is determined by combining tree growth and pruning, and the tree is modified to improve its generalization ability. The decision criterion of a soft decision tree is a range, rather than a specific value. Compared to standard decision trees, soft decision trees produce more accurate classification results.
[0042] Relational Graph Convolutional Network (R-GCN): Designed to generalize Graph Convolutional Networks (GCN) to handle different relationships between entities in a knowledge base, it can be applied to two tasks: link prediction and entity classification.
[0043] Example 1
[0044] In the field of reasoning for knowledge graphs, neural network-based reasoning methods possess stronger reasoning and generalization capabilities. However, the best-performing models are often very large, imposing strict limitations on latency and computational resources. Therefore, large models are inconvenient to deploy in practice. Furthermore, the black-box nature of neural network models hinders interpretability in both image processing and knowledge graph applications, thus impacting real-world usability.
[0045] Therefore, embodiments of the present invention provide a knowledge graph completion method based on knowledge distillation, such as... Figure 1 As shown, it includes:
[0046] Step 100: Obtain the knowledge graph to be completed.
[0047] The knowledge graph to be completed can be from various application fields, such as medicine, electromechanical engineering, metallurgy, chemical engineering, and architecture. It can be understood that, based on the actual needs corresponding to the knowledge graph, its professional field can be a more detailed classification. Taking the medical field as an example, the specific professional fields to which the knowledge graph belongs could be pharmacy, clinical medicine, basic medicine, public health and preventive medicine, and integrated traditional Chinese and Western medicine.
[0048] In this embodiment of the invention, the knowledge graph to be completed is a knowledge graph constructed based on device management business.
[0049] Step 200: Based on the knowledge graph completion model and the knowledge graph to be completed, determine the information to be supplemented in the knowledge graph to be completed.
[0050] Step 300: Based on the information to be supplemented, supplement the knowledge graph to be completed to obtain a completed knowledge graph. The completed knowledge graph is used to identify the operating status of device management services.
[0051] The training process of the knowledge graph completion model is as follows: Figure 2 As shown, specifically:
[0052] (1) In response to the target requirements, training data and label data are determined from the professional data of the domain to which the knowledge graph to be completed belongs.
[0053] An example: For reasoning tasks, i.e. target requirements, obtain training data for the knowledge graph G = (V, E). Where G represents a directed graph, V represents the set of nodes, E represents the set of edges, and r represents the relationship between the head node and the tail node; r, v i v j Let i and j represent an edge in a directed graph and its head and tail nodes, respectively, where i and j represent the indices of the nodes in the node set. n represents the number of nodes, and also the number of different entity types; m represents the number of edges, and also the number of different relation types; , .
[0054] (2) Construct a relational adjacency matrix A based on the training data, and denote it as follows: ;in, (1).
[0055] Where A is a three-dimensional relational adjacency matrix. This represents an element in the relational adjacency matrix; Let k represent the type of relation corresponding to the edge, and k represent the index of the relation in the relation adjacency matrix.
[0056] (3) Based on the relational adjacency matrix and the label data, a teacher network model based on relational graph convolutional network is trained using knowledge distillation technology.
[0057] An example: A teacher network model based on a relational graph convolutional network includes an input layer, hidden layers, and an output layer.
[0058] The input layer uses the relational adjacency matrix as input features, and the parameter matrix that the model input layer needs to train is: , where d represents the dimension of the encoded vector of the node.
[0059] The hidden layers are relational graph convolutional networks; the hidden layers utilize multi-layer graph convolutional networks to simulate the transmission of information in the network structure, and the transmission framework of each layer is defined as follows:
[0060] (2).
[0061] in, This represents the representation of node i in the (l+1)th hidden layer. This represents the representation of node i in the l-th hidden layer. This represents the representation of all neighboring nodes of node i in the l-th hidden layer. The parameter matrix represents the relation of type r. It represents the number of neighboring nodes of edges of the same type after a linear transformation. In other words, the number of edge types is also the parameter matrix. The number of It is also a parameter matrix, referring to its own feature weight matrix. For neighboring nodes that have an edge of type r connected to node i, These are normalization constants for specific problems, which can be learned or preset (e.g., W is a coefficient matrix that performs a linear transformation on neighboring nodes of the same type of edge. It is the logistic sigmoid activation function, denoted as: (3).
[0062] The output layer is used to predict the probability of multi-class classification results based on the updated softmax function. The updated softmax function is obtained by adding a hyperparameter T to soften the softmax function in the teacher network model, based on knowledge distillation. The specific formula of the updated softmax function is as follows:
[0063] (4).
[0064] in, It is the probability of the classification result for category i. Let z represent the input vector of the output layer, K represent the total number of categories, and z represent the input vector of the output layer. j Let i represent the input vector of category j, where i represents one of the categories.
[0065] The loss function of the teacher network model based on graph convolutional networks is defined as follows:
[0066] (5).
[0067] Where L is the loss of the teacher network model based on a relational graph convolutional network, and Y is the training data set with real labeled data. It is the predicted output of the i-th labeled input data at the k-th position. It is the actual label data of the i-th labeled input data at the k-th position.
[0068] (4) Based on the relational adjacency matrix, the label data and the logical output of the teacher network model based on the relational graph convolutional network, the knowledge distillation technique is used to train the student network model based on the soft decision tree to obtain the knowledge graph completion model.
[0069] An example: 1) Construct a student network model based on a soft decision tree, specifically as follows:
[0070] Construct a binary soft decision tree, where the learning probability distribution of each leaf node j (where j represents one of the classes) is as follows: Each internal node i (where i represents the index of the node in the set of internal nodes) has a learned filter. and a bias Then, at each internal node, the probability of choosing the rightmost branch is... for:
[0071] (6).
[0072] Where x is the input data of the student network model based on soft decision trees, i.e., the relational adjacency matrix. It is the sigmoid function.
[0073] The student network model based on soft decision trees includes an input layer, a binary soft decision tree, and an output layer. The input and output layers of the student network model based on soft decision trees are the same as those of the teacher network model based on relational graph convolutional networks, and will not be described in detail here.
[0074] 2) Based on the knowledge distillation technique and the logical output of the teacher network model based on a relational graph convolutional network, the loss function of the student network model based on a soft decision tree is determined, and its specific loss function is as follows:
[0075] (7).
[0076] in, Let represent the loss of the student network model based on a soft decision tree, where u and z are the probability values of the output layers of the teacher and student network models, respectively, and T is a pre-defined hyperparameter used to soften the value of the softmax activation function. and The teacher network model and the student network model, respectively, are softened class probabilities after being processed by the softmax function (4), which is called the soft target; the subscript i represents the probability value of the i-th class.
[0077] 3) Based on the relational adjacency matrix, the label data, and the loss function of the student network model based on the soft decision tree, train the student network model based on the soft decision tree to obtain the knowledge graph completion model.
[0078] In summary, the overall structure of training teacher and student network models based on knowledge distillation technology is as follows: Figure 3 As shown.
[0079] (5) Using the trained student network model, interpretable inference is performed based on the test dataset. Specifically, when using the trained student network model for prediction, T=1 is set in formula (4) and the result is taken as the output value. At this time, formula (4) is updated as follows:
[0080] (8).
[0081] In summary, the overall structure for inference and prediction based on a trained student network model is as follows: Figure 4 As shown.
[0082] The following specific examples illustrate the methods described in the embodiments of the present invention.
[0083] 1. Construct and obtain a knowledge graph based on a unit's equipment management business, such as... Figure 5 As shown in Table 1, the constructed training dataset is shown in Table 2.
[0084] Table 1. Training dataset for a knowledge graph of equipment management in a certain unit, oriented towards reasoning tasks.
[0085] Head node Border relations Tail node Equipment 1 Equipped with Part 1 Equipment 1 run Location 1 Location 1 Storage Equipment 1 Personnel 1 Responsible Location 1 Personnel 1 Responsible Equipment 1 Personnel 1 Responsible Part 1 Equipment 2 run Location 2 Equipment 2 Equipped with Part 2 Personnel 1 Responsible Equipment 2 Personnel 1 Responsible Location 2 Equipment 3 run Location 3 Location 3 Storage Equipment 3 Personnel 1 Responsible Location 3 … … … … … …
[0086] Table 2. Test dataset of a knowledge graph for equipment management in a certain unit, oriented towards reasoning tasks.
[0087] Head node Border relations Tail node Location 1 Storage Part 1 Personnel 1 Responsible Part 2 Personnel 1 Responsible Equipment 3 … … … … … …
[0088] 2. Based on the training dataset, construct the adjacency relations representing the knowledge graph structure, as shown in Table 3, and convert them into a relational adjacency matrix A.
[0089] Table 3. Adjacency Relationships of an Equipment Management Knowledge Graph in a Certain Unit
[0090] Node 1 / Device 1 Node 2 / Personnel 1 Node 3 / Location 1 Node 4 / Part 1 … … Node 1 / / Device 1 Relationship 1 / Run Relationship 2 / with … … Node 2 / Personnel 1 Relationship 3 / Responsible Relationship 3 / Responsible Relationship 1 / Responsible … … Node 3 / Location 1 Relationship 4 / Storage … … Node 4 / Part 1 … … … … … … … … … … … …
[0091] 3. The training of the teacher network model based on the relational graph convolutional network is completed based on the training dataset. Specifically: 1) Input the relational adjacency matrix A and set the parameter d representing the vector dimension of the node encoding; 2) Construct the teacher network model based on the relational graph convolutional network according to formulas (2)-(5) and complete the model training based on the training dataset.
[0092] 4. Train the student network model based on the soft decision tree, specifically: 1) Based on the pre-trained teacher network model, obtain the probability distribution Q of its output layer, and construct a binary soft decision tree according to formula (6); 2) Complete the training of the student network model according to formula (7).
[0093] 5. Based on formula (8), use the trained student network model to predict and evaluate the test set. The interpretability of the model is achieved through the decision tree of the student network model, such as... Figure 6 As shown.
[0094] Example 2
[0095] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a knowledge graph completion system based on knowledge distillation is provided below.
[0096] like Figure 7 As shown, an embodiment of the present invention provides a knowledge graph completion system based on knowledge distillation, comprising:
[0097] The knowledge graph acquisition module 701 is used to acquire the knowledge graph to be completed. The knowledge graph to be completed is a knowledge graph constructed based on equipment management business.
[0098] The module 702 for determining information to be supplemented is used to determine the information to be supplemented in the knowledge graph to be supplemented based on the knowledge graph completion model and the knowledge graph to be supplemented.
[0099] The knowledge graph completion determination module 703 is used to supplement the knowledge graph to be completed based on the information to be supplemented, thereby obtaining a completed knowledge graph. The completed knowledge graph is used to identify the operating status of device management services.
[0100] The training process of the knowledge graph completion model is as follows:
[0101] (1) In response to the target requirements, training data and label data are determined from the professional data of the domain to which the knowledge graph to be completed belongs; (2) A relational adjacency matrix is constructed based on the training data; (3) According to the relational adjacency matrix and the label data, a teacher network model based on a relational graph convolutional network is trained using knowledge distillation technology; (4) According to the relational adjacency matrix, the label data and the logical output of the teacher network model based on the relational graph convolutional network, a student network model based on a soft decision tree is trained using knowledge distillation technology to obtain the knowledge graph completion model.
[0102] Example 3
[0103] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the knowledge graph completion method based on knowledge distillation as described in Embodiment 1.
[0104] Alternatively, the aforementioned electronic device may be a server.
[0105] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge graph completion method based on knowledge distillation as described in Embodiment 1.
[0106] Compared with the prior art, the present invention has the following advantages:
[0107] 1. For more complex knowledge reasoning tasks in knowledge graphs, a teacher network model based on a relational graph convolutional network (RPN) yields better reasoning performance. While maintaining model performance, the RPN-based teacher network model is compressed using knowledge distillation techniques.
[0108] 2. For more complex knowledge reasoning tasks in knowledge graphs, the teacher network model is based on a relational graph convolutional network, and the student network model is based on a soft decision tree. While achieving a reduction in scale, it also has interpretability, thus achieving the goal of knowledge graph completion.
[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0110] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A knowledge graph completion method based on knowledge distillation, characterized in that, include: Obtain the knowledge graph to be completed; the knowledge graph to be completed is a knowledge graph constructed based on equipment management business. Based on the knowledge graph completion model and the knowledge graph to be completed, determine the information to be supplemented in the knowledge graph to be completed; Based on the information to be supplemented, the knowledge graph to be completed is supplemented to obtain a completed knowledge graph; the completed knowledge graph is used to identify the operating status of equipment management services; The training process of the knowledge graph completion model is as follows: To meet the target requirements, training data and label data are determined from professional data in the domain to which the knowledge graph to be completed belongs; Construct a relational adjacency matrix based on the training data; Based on the relational adjacency matrix and the label data, a teacher network model based on a relational graph convolutional network is trained using knowledge distillation techniques. The teacher network model includes an input layer, hidden layers, and an output layer. The input layer uses the relational adjacency matrix as input features. The hidden layers are relational graph convolutional networks. The hidden layers utilize multi-layer graph convolutional networks to simulate the transmission of information within the network structure, with each layer's transmission framework defined as follows: ;in, This represents the representation of node i in the (l+1)th hidden layer. This represents the representation of node i in the l-th hidden layer. This represents the representation of all neighboring nodes of node i in the l-th hidden layer. The parameter matrix represents the relation of type r. It represents the number of neighboring nodes of edges of the same type after a linear transformation. In other words, the number of edge types is also the parameter matrix. The number of It is also a parameter matrix, referring to its own feature weight matrix; For neighboring nodes that have an edge of type r connected to node i, It is a normalization constant for a specific problem, which can be learned or preset. W is a coefficient matrix that performs a linear transformation on the neighbor nodes of the same type of edge. It is the logistic sigmoid activation function, denoted as: The output layer is used to predict the probability of multi-class classification results based on the updated softmax function. The updated softmax function is obtained by adding a hyperparameter T to soften the softmax function in the teacher network model, based on knowledge distillation. The formula for the updated softmax function is: ;in, It is the probability of the classification result for category i. Let z represent the input vector of the output layer, K represent the total number of categories, and z represent the input vector of the output layer. j Let represent the input vector of category j, i represent one of the categories, and T represent a pre-defined hyperparameter; the loss function of the teacher network model based on the relational graph convolutional network is defined as follows: Where L is the loss of the teacher network model based on a relational graph convolutional network, and Y is the training dataset with real labeled data. It is the predicted output of the i-th labeled input data at the k-th position. It is the actual label data of the i-th labeled input data at the k-th position; Based on the relational adjacency matrix, the label data, and the logical output of the teacher network model based on a relational graph convolutional network, a student network model based on a soft decision tree is trained using knowledge distillation techniques to obtain a knowledge graph completion model. Specifically, this includes: constructing a student network model based on a soft decision tree; determining the loss function of the student network model based on the soft decision tree based on the knowledge distillation techniques and the logical output of the teacher network model based on a relational graph convolutional network; training the student network model based on a soft decision tree based on the relational adjacency matrix, the label data, and the loss function of the student network model based on the soft decision tree to obtain a knowledge graph completion model; and constructing the student network model based on a soft decision tree, specifically by constructing a binary soft decision tree, where the learning probability distribution of each leaf node j is... This leaf node j represents one of the categories; each internal node i has a learned filter. and a bias Then, at each internal node, the probability of choosing the rightmost branch is... for: Where x is the input data of a student network model based on a soft decision tree, i.e., the relational adjacency matrix. It is the sigmoid function; the internal node i represents the index of the node in the set of internal nodes; the student network model based on the soft decision tree includes an input layer, a binary soft decision tree, and an output layer; the input and output layers of the student network model based on the soft decision tree are the same as those of the teacher network model based on the relational graph convolutional network; the loss function of the student network model based on the soft decision tree is: ;in, Let represent the loss of the student network model based on a soft decision tree, u and z represent the probability values of the output layers of the teacher network model and the student network model, respectively, and T represent a pre-defined hyperparameter. and Let i and y represent the softened class probabilities of the teacher network model and the student network model, respectively; the subscript i indicates the probability value of the i-th class.
2. A knowledge graph completion system based on knowledge distillation, characterized in that, include: The module for acquiring knowledge graphs to be completed is used to acquire knowledge graphs to be completed. The knowledge graph to be completed is a knowledge graph constructed based on equipment management business; The module for determining information to be supplemented is used to determine the information to be supplemented in the knowledge graph to be supplemented based on the knowledge graph completion model and the knowledge graph to be supplemented. The knowledge graph completion determination module is used to supplement the knowledge graph to be completed based on the information to be supplemented, so as to obtain a completed knowledge graph; the completed knowledge graph is used to identify the operating status of the device management business; The training process of the knowledge graph completion model is as follows: To meet the target requirements, training data and label data are determined from professional data in the domain to which the knowledge graph to be completed belongs; Construct a relational adjacency matrix based on the training data; Based on the relational adjacency matrix and the label data, a teacher network model based on a relational graph convolutional network is trained using knowledge distillation techniques. The teacher network model includes an input layer, hidden layers, and an output layer. The input layer uses the relational adjacency matrix as input features. The hidden layers are relational graph convolutional networks. The hidden layers utilize multi-layer graph convolutional networks to simulate the transmission of information within the network structure, with each layer's transmission framework defined as follows: ;in, This represents the representation of node i in the (l+1)th hidden layer. This represents the representation of node i in the l-th hidden layer. This represents the representation of all neighboring nodes of node i in the l-th hidden layer. The parameter matrix represents the relation of type r. It represents the number of neighboring nodes of edges of the same type after a linear transformation. In other words, the number of edge types is also the parameter matrix. The number of It is also a parameter matrix, referring to its own feature weight matrix; For neighboring nodes that have an edge of type r connected to node i, It is a normalization constant for a specific problem, which can be learned or preset. W is a coefficient matrix that performs a linear transformation on the neighbor nodes of the same type of edge. It is the logistic sigmoid activation function, denoted as: The output layer is used to predict the probability of multi-class classification results based on the updated softmax function. The updated softmax function is obtained by adding a hyperparameter T to soften the softmax function in the teacher network model, based on knowledge distillation. The specific formula for the updated softmax function is as follows: ;in, It is the probability of the classification result for category i. Let z represent the input vector of the output layer, K represent the total number of categories, and z represent the input vector of the output layer. j Let represent the input vector of category j, and represent one of the categories; the loss function of the teacher network model based on the relational graph convolutional network is defined as follows: Where L is the loss of the teacher network model based on a relational graph convolutional network, and Y is the training dataset with real labeled data. It is the predicted output of the i-th labeled input data at the k-th position. It is the actual label data of the i-th labeled input data at the k-th position; Based on the relational adjacency matrix, the label data, and the logical output of the teacher network model based on a relational graph convolutional network, a student network model based on a soft decision tree is trained using knowledge distillation techniques to obtain a knowledge graph completion model. Specifically, this includes: constructing a student network model based on a soft decision tree; determining the loss function of the student network model based on the soft decision tree based on the knowledge distillation techniques and the logical output of the teacher network model based on a relational graph convolutional network; training the student network model based on a soft decision tree based on the relational adjacency matrix, the label data, and the loss function of the student network model based on the soft decision tree to obtain a knowledge graph completion model; and constructing the student network model based on a soft decision tree, specifically by constructing a binary soft decision tree, where the learning probability distribution of each leaf node j is... This leaf node j represents one of the categories; each internal node i has a learned filter. and a bias Then, at each internal node, the probability of choosing the rightmost branch is... for: Where x is the input data of a student network model based on a soft decision tree, i.e., the relational adjacency matrix. It is the sigmoid function; the internal node i represents the index of the node in the set of internal nodes; the student network model based on soft decision trees includes an input layer, a binary soft decision tree, and an output layer; the input and output layers of the student network model based on soft decision trees are the same as those of the teacher network model based on graph convolutional networks; the loss function of the student network model based on soft decision trees is as follows: ;in, Let represent the loss of the student network model based on a soft decision tree, where u and z are the probability values of the output layers of the teacher network model and the student network model, respectively, and T is a pre-defined hyperparameter. and Let i and y represent the softened class probabilities of the teacher network model and the student network model, respectively; the subscript i indicates the probability value of the i-th class.
3. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform the knowledge graph completion method based on knowledge distillation according to claim 1.
4. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the knowledge graph completion method based on knowledge distillation as described in claim 1.