Knowledge graph embedding hyperbolic mutual distillation method fusing topological and semantic information

Abstract

The application relates to the field of artificial intelligence, and discloses a knowledge graph embedding hyperbolic mutual distillation method fusing topological and semantic information, which comprises the following steps: acquiring knowledge graph data containing text labels and a text label query to be completed; performing graph neural network coding on hyperbolic embedding vectors corresponding to the text labels of entities and relations; inputting the text labels of the entities and the relations into a pre-trained language model for coding; performing joint training and distribution alignment based on bidirectional mutual knowledge distillation on the structure-perceived vector representation and the semantic-perceived vector representation in the same hyperbolic geometric space to generate a text-enhanced joint representation; calculating the scores of each candidate answer corresponding to the text label query through a preset hyperbolic translation scoring function; and outputting a completion prediction result according to the scores of the candidate answers. The application realizes deep and balanced fusion of the topological structure of a knowledge graph and semantic information of text labels, and maintains high prediction reliability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a hyperbolic inter-distillation method for embedding knowledge graphs that integrates topological and semantic information. Background Technology

[0002] A knowledge graph is a structured textual semantic knowledge base that typically stores information in the form of triples (head entity, relation, tail entity). However, real-world knowledge graphs are often massive and highly sparse, containing a large number of missing links. Knowledge graph completion tasks aim to predict these missing textual triples and are a key technology for improving the quality and completeness of knowledge graphs.

[0003] Knowledge graph embedding technology provides an effective solution for knowledge graph completion by mapping entities and relations to a continuous vector space. Existing methods can be mainly divided into three categories: structure-based methods, text-based methods, and hybrid methods. Structure-based methods mainly rely on the topological connection patterns of the graph for representation learning, but fail to fully utilize the rich semantic information carried by entity and relation names. Text-based methods use pre-trained language models to understand label semantics, but often underutilize the structural regularities of the graph itself and are susceptible to noisy labels or overly generic names. Existing hybrid methods attempt to fuse two types of information, but often do so in an asymmetric manner (e.g., using one type of information as an aid to the other), resulting in the final learned representation still potentially biased towards one side of structure or semantics, failing to achieve a balance and deep complementarity between the two information sources.

[0004] Therefore, how to design a model that can fully and evenly integrate the rich structural patterns and semantic signals in knowledge graphs, and improve the completion performance in sparse, long-tailed and noisy scenarios, has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a hyperbolic inter-distillation method for knowledge graph embedding that integrates topological and semantic information, thereby solving the technical problems of unbalanced information fusion and insufficient generalization ability for sparse and noisy data in existing technologies.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A hyperbolic inter-distillation method for knowledge graph embedding that integrates topological and semantic information includes the following steps: Obtain knowledge graph data containing text tags and queries with text tags to be completed; the knowledge graph data includes multiple entities, multiple relations, and triples representing the relationships between entities, all represented by text tags. Based on the topological connection structure defined by the triples, the hyperbolic embedding vectors corresponding to the text tags of the entities and relations are encoded by a graph neural network to generate structure-aware vector representations. The text labels of the entities and relationships are input into a pre-trained language model for encoding to generate semantically aware vector representations; The structure-aware vector representation and the semantic-aware vector representation are jointly trained and aligned in the same hyperbolic geometric space based on bidirectional mutual knowledge distillation to generate a text-enhanced joint representation. Based on the joint representation of the text enhancement, the score of each candidate answer corresponding to the text-labeled query is calculated by a preset hyperbolic translation scoring function; The prediction results are completed based on the scores of each candidate answer.

[0007] Preferably, generating a structure-aware vector representation includes: The hyperbolic embedding vectors of the entities and relations are projected from the hyperbolic embedding space to the tangent space via a Riemann logarithmic map. In the tangent space, the attention weights of each entity's neighboring entities under different relation types are calculated; The representations of neighboring entities are weighted and aggregated according to the attention weights to obtain the aggregated tangent space message; The aggregated tangent space message is combined with the entity and then mapped back to the hyperbolic embedding space through the Riemann exponent mapping to update the structure-aware vector representation.

[0008] Preferably, when weighting and aggregating the representations of neighboring entities according to the attention weights, a multi-head attention mechanism is used. In the slicing space, attention weights and attention-weighted slicing space messages are calculated separately for each head, and then spliced ​​or averaged for multi-head output.

[0009] Preferably, combining the aggregated tangent space message with the entity and mapping it back to the hyperbolic embedding space using a Riemann exponent mapping includes: In the tangent space, the aggregated tangent space message is added to the tangent space vector of the entity, and then the addition result is nonlinearly transformed and mapped back to the hyperbolic embedding space through the Riemann exponent mapping.

[0010] Preferably, the process includes the following steps before generating the structure-aware vector representation: The knowledge graph data is expanded by adding a corresponding inverse relation to each relation in the knowledge graph to form a symmetrical message passing path.

[0011] Preferably, generating semantically aware vector representations includes: Semantic feature vectors for each entity text label and each relation text label are extracted using a pre-trained language model. The semantic feature vectors are mapped to the tangent space at the origin of the hyperbolic embedding space through a trainable projection network. The mapped tangent space vector is mapped into the hyperbolic embedding space through the Riemann exponent mapping, serving as the vector representation of the semantic awareness.

[0012] Preferably, the semantic feature vector is the CLS tag embedding vector or the mean-pooled word embedding vector corresponding to the text tag after being encoded by a pre-trained language model.

[0013] Preferably, joint training and distribution alignment based on bidirectional mutual knowledge distillation includes: The first prediction distribution and the second prediction distribution are calculated using the hyperbolic translation scoring function based on the structure-aware vector representation and the semantic-aware vector representation, respectively. Temperature scaling is applied to the first and second predicted distributions to obtain the corresponding first and second softening distributions; Calculate the first KL divergence loss from the second softened distribution to the first softened distribution, and the second KL divergence loss from the first softened distribution to the second softened distribution; The weighted sum of the first KL divergence loss and the second KL divergence loss is used as the bidirectional mutual knowledge distillation loss.

[0014] Preferably, the total loss function of the joint training is the sum of the supervision loss and the bidirectional knowledge distillation loss.

[0015] Preferably, the supervised loss is the cross-entropy loss calculated based on the predicted distributions generated by the structure-aware vector representation and the semantic-aware vector representation, respectively, using the true answer labels.

[0016] The present invention has the following beneficial effects: This invention achieves a deep and balanced fusion of knowledge graph topology and text tag semantic information by constructing a symmetrical bi-branch framework operating within a shared hyperbolic space and introducing a bidirectional distillation mechanism. Its symmetrical bi-branch design and bidirectional distillation mechanism avoid view bias during information fusion, enabling semantic priors to effectively guide structural reasoning. Simultaneously, structural constraints regularize semantic predictions, resulting in slow performance degradation of the overall model and maintaining high prediction reliability. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a hyperbolic inter-distillation method for embedding knowledge graphs that integrates topological and semantic information, according to the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0019] like Figure 1 As shown, an embodiment of the present invention provides a hyperbolic inter-distillation method for knowledge graph embedding that integrates topological and semantic information, comprising the following steps S1 to S6: S1. Obtain knowledge graph data containing text tags and queries with text tags to be completed; the knowledge graph data includes multiple entities, multiple relations, and triples representing the relationships between entities, all represented by text tags. S2. Based on the topological connection structure defined by the triplet, perform graph neural network encoding on the hyperbolic embedding vectors corresponding to the text tags of the entity and relation to generate a structure-aware vector representation; S3. Input the text labels of the entities and relationships into a pre-trained language model for encoding to generate semantically aware vector representations; S4. Jointly train and align the structure-aware vector representation and the semantic-aware vector representation in the same hyperbolic geometric space based on bidirectional mutual knowledge distillation to generate a text-enhanced joint representation. S5. Based on the joint representation of the text enhancement, calculate the score of each candidate answer corresponding to the text-labeled query using a preset hyperbolic translation scoring function; S6. Output the complete prediction results based on the scores of each candidate answer.

[0020] This embodiment provides a Hyperbolic Mutual-Distillation (HMD) model. This model is a symmetric bi-branch framework that jointly models structural and semantic views in a shared hyperbolic latent space: the structural branch uses a hyperbolic relation graph attention network to encode typed neighborhood information; the semantic branch uses a pre-trained language model to encode entity and relation labels and projects them onto the same Poincaré sphere space. Both branches share a unified scoring function based on hyperbolic translation, enabling compatible representation and fusion of evidence. To achieve balanced fusion of the two views, HMD introduces a bi-directional mutual distillation mechanism: aligning the prediction distributions of the two branches under joint supervision of ground truth labels. This design allows the model to capture semantic information from text labels and learn structural patterns from topological connections.

[0021] In this embodiment, the knowledge graph constructed from the text data is represented as follows: Here, E and R represent the entity set and relation set, respectively, and T is the set of observed triples (h, r, t), where h is the head entity, r is the relation, and t is the tail entity. As one of the most important downstream tasks in knowledge graph embedding, knowledge graph completion (KGC) is typically modeled as a task of solving entity prediction queries (h, r, ?), where ? represents the unknown tail entity that the model needs to predict. For each observed triple (h, r, t), a supervised query is constructed, and the query set with text labels is denoted as . For queries Its true answer entity is denoted as .

[0022] This embodiment uses knowledge graph completion as the main evaluation task, with the goal of maximizing the conditional log-likelihood of the correct entity for each query:

[0023] in, To find the maximum value function, For the set of all trainable parameters of the model, The conditional probability distribution is determined by the model parameter Θ. This is the predicted answer entity for query q. The model parameters Θ contain all trainable parameters for both branches, i.e., the structure branch parameters. And semantic branch parameter θ.

[0024] The hyperbolic inter-distillation model jointly models structural and semantic branches on a unified hyperbolic manifold, enabling it to capture both topological regularity and label semantics in a compatible manner. Given a query q, the two branches output predicted distributions on the entities:

[0025] in, For structural branches, This is a semantic branch.

[0026] The hyperbolic inter-distillation model does not use either view as an aid, but instead couples two predictors at the distribution level through bidirectional inter-distillation and anchors them with supervised loss. This symmetric design allows semantic priors to guide topology-based reasoning, while structural constraints regularize text-based predictions.

[0027] The hyperbolic space used in this embodiment is a non-Euclidean space with negative curvature, whose volume grows exponentially with the radius, naturally suitable for modeling tree-like, locally dense but rapidly expanding structures. This is precisely the pattern presented by multi-hop relationships and hierarchical semantics in knowledge graphs. The hyperbolic interdistillation model jointly embeds entities and relationships into a Poincaré sphere. In this way, it can maintain the hierarchical and clustering characteristics more compactly in a lower dimension, reduce parameter overhead, and improve the representation ability and generalization performance of entity prediction.

[0028] This embodiment embeds entities and relations into a d-dimensional Poincaré sphere with curvature c>0:

[0029] in, For the space of the Poincaré ball, Let Poincaré be a point in the space of the sphere. For Euclidean space This is the norm notation. To bridge the gap between hyperbolic geometry and Euclidean neural network operations, this embodiment uses the tangent space at the origin. and the Riemannian logarithmic mapping And exponential mapping :

[0030]

[0031] in, Let be a vector in the tangent space. For the tangent space. And let 0. All trainable network parameters (such as linear mappings and attention vectors) are defined in Euclidean space, while entity and relation embeddings are located in Euclidean space. The above formula is used to map the values ​​as needed.

[0032] To avoid numerical instability near the boundary, this embodiment embeds arbitrary hyperbolic embeddings after each update. Reprojected to radius Inside the sphere, among which It is a small constant. This embodiment uses the standard geodesic distance on the Poincaré sphere, denoted as . (·,·), and its squared form is used in the translation scoring function of the two branches.

[0033] In addition to modeling the structural branches in hyperbolic space, this embodiment also embeds the semantic branches into the same Poincaré sphere. This is because the hyperbolic inter-distillation model uses a unified hyperbolic translation scoring template for both branches and aligns their predicted distributions through inter-distillation. If the semantic branches remain in Euclidean space while the structural branches are in hyperbolic space, the two predictors will measure similarity under different geometric criteria (such as distance growth and calibration methods), leading to weakened distribution alignment and decreased stability. Projecting the pre-trained language model representation onto the same manifold eliminates this geometric mismatch, enabling the two branches to generate comparable logits under a consistent hyperbolic metric. Furthermore, hyperbolic geometry is well-suited for capturing latent hierarchical structures and tree-like extensions, which are prevalent not only in knowledge graph topologies but also in the vocabulary and ontological semantics encoded by labels. Therefore, embedding semantic information into hyperbolic space provides a more structured prior, particularly beneficial for long-tailed entities and ambiguous labels.

[0034] In this embodiment, the Hyperbolic Relational Graph Attention Network (HRGAT) structure branches are used to model the connection patterns and global topology of the knowledge graph. Node attributes largely depend on connection patterns such as their neighbors, multi-hop paths, and higher-order neighborhoods. Embedding this topology into a hyperbolic space allows HRGAT to compactly represent these hierarchical and branching structures, while the graph attention mechanism can distinguish between information-rich and less relevant neighbors and relationships. HRGAT, acting as a topology-aware predictor, injects structure-based inductive bias into the entity representation.

[0035] Specifically, HRGAT encodes knowledge graphs using a multi-layer hyperbolic relation graph attention network. A multi-relation structure. This embodiment expands the relation set by adding inverse relations to achieve bidirectional message passing. Specifically defined as: , All neighborhood sets are in The definition above, where For the expanded relation set, For the original relation set, It is an inverse relation set. For the expanded set of triples, The original triplet, This is the inverse relation. In the l-th layer (l∈{0,...,L-1}), where L is the total number of layers in the model, each entity i corresponds to a hyperbolic embedding vector. For each relation r and its neighbor node j, first map the embeddings to the tangent space:

[0036]

[0037] in, Let r be the set of neighboring nodes of entity i under relation r. Let E be the mathematical representation of the entity set. Let be the vector representation of entity i in the l-th layer after being projected onto the tangent space at the origin via a Riemann logarithmic mapping. Let be the tangent space vector representation of the neighboring entity j at the l-th layer. Let i be the original embedding vector of entity i at layer l. Let j be the original embedding vector of the neighboring entity j at layer l. It is the Riemann logarithmic mapping function.

[0038] And apply relation-specific linear transformations: ; in, Let i be the hidden state of entity i at level l after the specific linear transformation of relation r. Let r be the learnable weight matrix of the l-th layer relation. Let be the hidden state of neighbor entity j at layer l after the relation-specific linear transformation of relation r. The relation-specific attention score and corresponding attention weight are: ; ; in, Let i be the original attention score between entity i and its neighbor j at layer l and relation r. For a non-linear activation function with leakage, Let T be the learnable attention vector, T be the transpose symbol, and [·||·] denote the concatenation operation. The final attention weight when aggregating information about neighboring entity j for entity i. It is an exponential function. For entity i in relation The following is a collection of neighbors. For entity i and its neighboring entity k at level l, the relationship is... The attention score is then calculated. Subsequently, the transformed neighbor features are aggregated in the tangent space to achieve permutation-invariant and numerically stable message passing. Specifically, attention-weighted tangent space messages are calculated: ; in, Let i be the new feature vector of entity i in the tangent space after aggregation at layer l+1.

[0039] And map it back to a hyperbolic manifold using an exponential mapping: ; in, Let i be the final embedding vector of entity i after being updated and mapped back to hyperbolic space at layer l+1. It is an exponential mapping function.

[0040] Then, residual updates and nonlinear transformations are performed in the tangent space: ; in, Let be the new embedding vector of entity i in the tangent space after the nonlinear transformation at layer l+1. These are non-linear activation functions, such as ReLU and Tanh.

[0041] This tangent space aggregation constructively satisfies permutation invariance, avoiding the order-sensitivity problem of the sequential Möbius summation method, and improving optimization stability in practice. Multi-head attention can be achieved by computing each head separately in the tangent space. and and in application This is achieved through pre-concatenation or average multi-head output. After propagation through L layers, the final structural entity embedding of all entities e is obtained. Embedding of relation r It can be optimized directly on the Poincaré sphere as a learnable parameter.

[0042] For tail entity prediction queries q = (h,r,?) and candidate tail entities t, a hyperbolic translation scoring function is used: ; in, For hyperbolic translation scoring function, For hyperbolic distance, For entities Embedded vectors in hyperbolic space This is a hyperbolic translation operation. Let be the embedding vector of relation r in hyperbolic space. Candidate tail entity Embedded vectors in hyperbolic space. Head entity prediction query q=(?,r,t) can be commutated. and Symmetrical treatment.

[0043] In this embodiment, the hyperbolic text representation (HTR) semantic branch utilizes text labels of entities and relations as semantic evidence. Unlike structure-driven KGE models that treat symbols as anonymous IDs, labels typically encode lexical semantics, type cues, and latent ontological patterns, serving as a supplement to graph topology, especially when neighborhood information is scarce. However, directly combining pre-trained language model embeddings in Euclidean space with hyperbolic structure embeddings introduces geometric mismatch: the two branches will represent similarity at different metrics and scales, violating the goal of using a unified scoring format and performing distribution-level inter-distillation. To address this issue, HTR maps the pre-trained language model representation to a tangent space and then embeds it into the same Poincaré sphere as the structural branch, enabling both predictors to operate under consistent hyperbolic geometry.

[0044] For each entity e, the present invention uses its label. Construct the text input. Labels are typically entity names; if a brief description is provided, it can be added as additional context without altering the overall form. Obtain the representation using a pre-trained language model: ; in, For entities The text representation vector is LM(·), which represents a pre-trained language model encoder, such as BERT or GPT, trained on large-scale text corpora. Its function is to process the input text sequence (i.e., labels) into a vector. This is encoded as a high-dimensional dense numerical vector. In practice, [CLS] tag embeddings or mean-pooled word embeddings are acceptable. Relationship processing is similar: given relation labels... ,calculate .

[0045] Then a projection network was used. Map the output of the pre-trained language model to the origin tangent space: ; in, For entities The feature vector after projection For relationship The feature vector after projection For relationship The text representation vector.

[0046] This lightweight MLP projector adapts the pre-trained language model space to a task-specific hyperbolic embedding space. It uses exponential mapping to map tangent space vectors to a Poincaré sphere. ; in, For entities The final embedding vector in hyperbolic space, For relationship The final embedding vector in hyperbolic space.

[0047] To improve numerical stability, clipping can be performed in the tangent space. And use the same throwing operation as the structural branches to avoid boundary problems.

[0048] Given a query q=(h,r,?) and candidate tail entities t, HTR uses the same hyperbolic translation scoring method as structural branching:

[0049] in, For the scoring function of semantic branches, For entities Text embedding vectors in hyperbolic space Let be the text embedding vector of relation r in hyperbolic space. Candidate tail entity Text embedding vectors in hyperbolic space.

[0050] This design enables semantic and structural evidence to be represented compatiblely under the same hyperbolic metric and a unified scoring template, which is crucial for the interdistillation between the two prediction distributions.

[0051] The training objective of the hyperbolic inter-distillation model is not only to fit labeled queries, but also to couple structural and semantic predictors, enabling them to exchange complementary inductive biases. From a multi-view learning perspective, the knowledge graph topology and labeled text constitute two information-rich but incomplete views: the structural branch is stronger in relational patterns and multi-hop constraints, while the semantic branch provides lexical priors and label associations, remaining effective even with limited neighborhood information. A prescriptive approach to combining these views is collaborative regularization: encouraging consistent outputs from both predictors while maintaining constraints from supervised signals. This consistency reduces variance, alleviates view-specific overfitting, and is particularly beneficial for sparse neighborhoods and long-tailed entities.

[0052] To this end, the hyperbolic inter-distillation model optimizes two types of objectives: (1) bidirectional inter-distillation: forcing the two branches to achieve distribution level alignment under the same hyperbolic scoring template; (2) supervised loss: anchoring each branch to the true answer.

[0053] (1) Bidirectional mutual knowledge distillation: Since the two branches use the same hyperbolic translation scoring template on the shared manifold, their logit has consistent geometric semantics, making the distribution level alignment meaningful rather than just numerical matching.

[0054] Given a query q=(h,r,?), HRGAT and HTR score the candidate entities respectively. and This invention does not merely align hard labels, but rather aligns the prediction distributions generated by the two branches. This design follows the classic distillation principle: soft distributions contain richer information than one-hot supervision, revealing relative preferences (often referred to as dark knowledge) between candidate entities. This is crucial for entity prediction tasks with a large number of semantically or structurally similar candidates.

[0055] This embodiment uses a temperature τ>0 to construct a softening distribution:

[0056]

[0057] in, To predict the probability distribution of a soft target with tail entity e based on structural features, given a query q, For candidate tail entities, To predict the probability distribution of a soft target with tail entity e based on text features, given a query q.

[0058] The higher the temperature τ, the smoother the distribution, and the more fine-grained the similar structures between candidates can be exposed. This embodiment uses τ... 2 The distillation objective is scaled to maintain comparable gradient magnitudes at different temperatures. Two distillation losses based on KL divergence are then defined:

[0059]

[0060] in, The knowledge distillation loss is used to transition from semantic branches to structural branches. For querying the set, Let KL divergence be the KL divergence. To determine the semantic branch probability distribution under query condition q, To query the structural branch probability distribution under condition q, The loss is for knowledge distillation from structural branches to semantic branches.

[0061] Using both directions simultaneously is crucial for symmetry. One-way distillation implicitly treats one branch as the master signal and the other as a secondary student, contradicting the goal of balanced fusion. In contrast, the two-way form can be viewed as imposing a more symmetric consistency constraint on the two prediction distributions (similar to minimizing symmetric divergence), allowing each branch to act as a teacher when it is more reliable for a particular query. Furthermore, since the KL divergence itself is asymmetric, the two directions provide complementary regularization for the distribution shape, rather than simply matching a single modality. The overall cross-distillation loss is:

[0062] in, For overall interdistillation losses, These are the weighting coefficients for the structural branches. These are the weight coefficients for semantic branches. and Control cross-view Figure 1 Consistency strength can be set to equal weights, with an initial value of 1. Intuitively, LKD encourages structural branches to absorb semantic priors and label associations from semantic branches, while also encouraging semantic branches to conform to the topological constraints distilled from structural branches.

[0063] (2) Supervised anchoring of true values: Mutual distillation can promote consistency, but consistency alone is not enough: two predictors may become consistent but converge to an incorrect consensus. Therefore, this invention anchors each branch to the true answer through standard supervised learning, ensuring that mutual learning revolves around the labeled solution.

[0064] For each labeled query q∈Q L Structural branches and semantic branches define probability distributions on the entity set:

[0065]

[0066] in, To predict the conditional probability distribution of tail entity e given query q and graph structure G, For structural branch parameters, The conditional probability distribution for semantic branches. For semantic branch parameters.

[0067] The supervision loss for the two branches is the cross-entropy:

[0068]

[0069] in, The supervision loss for structural branches, The supervision loss is for semantic branches.

[0070] The overall training objective of the hyperbolic inter-distillation model combines supervised anchoring with bidirectional inter-distillation:

[0071] in, For the total loss function, This is due to knowledge distillation loss.

[0072] Overall, and To ensure the model's accuracy on labeled data, LKD, as a cross-view method... Figure 1 Consistency regularization terms transfer complementary knowledge between topology and text. This combination achieves balanced integration and improves generalization ability in sparse and long-tail scenarios, eliminating the problems of overfitting or insufficient constraints that can easily occur when relying on a single view in these scenarios.

[0073] In summary, this invention provides a hyperbolic mutual distillation framework (HMD) for knowledge graph embedding, achieving a symmetrical fusion of structural topology and label semantics. This addresses the problem that existing hybrid methods often fuse structural and semantic information asymmetrically, leading to a bias in the final representation towards one side of the structure or semantics. HMD fully leverages the rich structural patterns and semantic signals within the knowledge graph. Secondly, HMD models structural and semantic branches on a shared Poincaré sphere and employs a unified hyperbolic translation scoring template, ensuring that both predictors represent evidence under the same geometric criteria, generating comparable logits outputs. To achieve balanced fusion rather than unidirectional enhancement, a bidirectional mutual distillation objective is designed: while anchoring the two branches with supervised loss, their prediction distributions are aligned. This design allows semantic priors and label associations to guide topology-based reasoning, while topology-induced constraints can regularize text-driven predictions, proving particularly effective in sparse scenarios, with long-tailed entities, and noisy labels. Extensive experiments on multiple benchmark datasets validate the effectiveness of HMD. HMD consistently achieves state-of-the-art overall performance across various datasets. Further analysis reveals that bidirectional inter-distillation, shared hyperbolic geometry, and geometric consistency significantly enhance the model's robustness to label perturbations and its generalization ability in sparse scenarios. It is demonstrated that symmetric coupling on shared hyperbolic manifolds is a standard and effective approach for mining the complementarity between knowledge graph structure and label semantics.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0078] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A hyperbolic inter-distillation method for knowledge graph embedding that integrates topological and semantic information, characterized in that, Includes the following steps: Obtain knowledge graph data containing text tags and queries with text tags to be completed; the knowledge graph data includes multiple entities, multiple relations, and triples representing the relationships between entities, all represented by text tags. Based on the topological connection structure defined by the triples, the hyperbolic embedding vectors corresponding to the text tags of the entities and relations are encoded by a graph neural network to generate structure-aware vector representations. The text labels of the entities and relationships are input into a pre-trained language model for encoding, generating semantically aware vector representations, including: Semantic feature vectors for each entity text label and each relation text label are extracted using a pre-trained language model. The semantic feature vectors are mapped to the tangent space at the origin of the hyperbolic embedding space through a trainable projection network. The mapped tangent space vector is mapped into the hyperbolic embedding space through the Riemann exponent mapping, and serves as the vector representation of the semantic awareness. The structure-aware vector representation and the semantic-aware vector representation are jointly trained and aligned in the same hyperbolic geometric space based on bidirectional mutual knowledge distillation to generate a text-enhanced joint representation; wherein the joint training and distribution alignment based on bidirectional mutual knowledge distillation includes: The first prediction distribution and the second prediction distribution are calculated using the hyperbolic translation scoring function based on the structure-aware vector representation and the semantic-aware vector representation, respectively. Temperature scaling is applied to the first and second predicted distributions to obtain the corresponding first and second softening distributions; Calculate the first KL divergence loss from the second softened distribution to the first softened distribution, and the second KL divergence loss from the first softened distribution to the second softened distribution; The weighted sum of the first KL divergence loss and the second KL divergence loss is used as the bidirectional mutual knowledge distillation loss. Based on the joint representation of the text enhancement, the score of each candidate answer corresponding to the text-labeled query is calculated by a preset hyperbolic translation scoring function; The prediction results are completed based on the scores of each candidate answer.

2. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 1, characterized in that, Generate structure-aware vector representations including: The hyperbolic embedding vectors of the entities and relations are projected from the hyperbolic embedding space to the tangent space via a Riemannian logarithmic map. In the tangent space, the attention weights of each entity's neighboring entities under different relation types are calculated; The representations of neighboring entities are weighted and aggregated according to the attention weights to obtain the aggregated tangent space message; The aggregated tangent space message is combined with the entity and then mapped back to the hyperbolic embedding space through the Riemann exponent mapping to update the structure-aware vector representation.

3. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 2, characterized in that, When weighting and aggregating the representations of neighboring entities based on the attention weights, a multi-head attention mechanism is used. In the slicing space, attention weights and attention-weighted slicing space messages are calculated separately for each head, and then spliced ​​or averaged for multi-head output.

4. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 2, characterized in that, The aggregated tangent space message, combined with the entity, is then mapped back to the hyperbolic embedding space via a Riemann exponent mapping, including: In the tangent space, the aggregated tangent space message is added to the tangent space vector of the entity, and then the addition result is nonlinearly transformed and mapped back to the hyperbolic embedding space through the Riemann exponent mapping.

5. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 2, characterized in that, Before generating structure-aware vector representations, the following steps are also included: The knowledge graph data is expanded by adding a corresponding inverse relation to each relation in the knowledge graph to form a symmetrical message passing path.

6. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 1, characterized in that, The semantic feature vector is either the CLS tag embedding vector or the mean-pooled word embedding vector corresponding to the text tag after being encoded by a pre-trained language model.

7. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 1, characterized in that, The total loss function of the joint training is the sum of the supervision loss and the bidirectional knowledge distillation loss.

8. The knowledge graph embedding hyperbolic inter-distillation method that integrates topological and semantic information according to claim 7, characterized in that, The supervised loss is the cross-entropy loss calculated based on the predicted distributions generated by the structure-aware vector representation and the semantic-aware vector representation, respectively, using the true answer labels.

Images