Construction method of knowledge graph embedding model based on hyperbolic geometry

Abstract

The invention provides a hyperbolic geometry-based knowledge graph embedding model construction method, which comprises the following steps of: carrying out primary basic translation change on a head entity in a hyperbolic space based on Mobius addition, embedding the head entity through two-dimensional rotation and three-dimensional rotation, mapping a vector back to a tangent space based on logarithm mapping, and constructing a hyperbolic geometry-based knowledge graph embedding model. And calculating score functions of different embedding results. According to the method, the hierarchical structure and the complex relation in the knowledge graph are more efficiently expressed and processed by utilizing the characteristics of the hyperbolic space, the defects in the aspects of high-dimensional data processing and hierarchical information capturing in the prior art are overcome, and the method is convenient to use.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph embedding technology, and in particular to a method for constructing a knowledge graph embedding model based on hyperbolic geometry. Background Art

[0002] As a structured knowledge representation method, knowledge graphs are widely used in natural language processing, recommendation systems and other fields. Existing knowledge graph embedding models are mainly based on Euclidean geometry to represent nodes and edges. However, knowledge graphs usually have hierarchical structures and complex relationship networks. Euclidean geometry has certain limitations when processing high-dimensional data and it is difficult to effectively capture the hierarchical and multi-relational information in knowledge graphs. Therefore, it is necessary to design a method for constructing a knowledge graph embedding model based on hyperbolic geometry. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for constructing a knowledge graph embedding model based on hyperbolic geometry.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] The present invention provides a method for constructing a knowledge graph embedding model based on hyperbolic geometry, comprising:

[0006] Based on the Möbius addition method, a basic translation change is performed on the head entity in the hyperbolic space, where r H and are the hyperbolic embeddings of the head entity, relation, and tail entity, respectively;

[0007] Embed the head entity through two-dimensional rotation and three-dimensional rotation;

[0008] The vector is mapped back to the tangent space based on the logarithmic mapping, and the score function of different embedding results is calculated.

[0009] Preferably, a basic translation change is performed on the head entity in the hyperbolic space based on the Mobius addition method, specifically:

[0010] Based on the Möbius addition method, a basic translation change is performed on the head entity in the hyperbolic space, which is:

[0011]

[0012] In the formula, x and y are the vectors participating in the Möbius addition, c r is the curvature of the hyperbolic space, Addition for Mobius.

[0013] Preferably, the two-dimensional rotation is specifically:

[0014] Based on the Givens transformation simulation relationship, the two-dimensional rotation operation in hyperbolic space with different curvatures is realized. The number of dimensions is set to an even number, and the block diagonal rotation matrix is ​​used to parameterize the rotation, which is:

[0015]

[0016] In the formula, are parameters specific to each relationship type.

[0017] Preferably, the three-dimensional rotation is specifically:

[0018] The rotation operation in three-dimensional space is simulated based on Hamilton product matrix operation:

[0019]

[0020] In the formula, is the Hamilton product. The quaternion q consists of a real number and three imaginary units, and is defined as: q = a + bi + cj + dk, where a, b, c, d are real numbers, and i, j, k are imaginary units.

[0021] Preferably, the head entity is embedded by two-dimensional rotation and three-dimensional rotation, specifically:

[0022] According to the formulas of two-dimensional rotation and three-dimensional rotation, the head entity is embedded as follows:

[0023]

[0024] Preferably, the vector is mapped back to the tangent space based on a logarithmic mapping, and the score functions of different embedding results are calculated, specifically:

[0025] Calculate the attention scores and weighted averages of different rotation transformations based on the attention mechanism;

[0026] Based on the logarithmic mapping, the vector is mapped back to the tangent space, and the residual idea is introduced according to the attention score calculation formula and the weighted average calculation formula. H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function.

[0027] Preferably, the attention scores and weighted averages of different rotation transformations are calculated based on the attention mechanism, specifically:

[0028] set up and For two vectors that have been embedded in the hyperbolic space through different rotation transformations, use the logarithmic mapping to map and to the tangent space respectively, and calculate the attention scores of the two rotation transformations, which are:

[0029]

[0030] In the formula, v is the attention vector, and the weighted average is calculated as:

[0031]

[0032] Preferably, the vector is mapped back to the tangent space based on the logarithmic mapping, and the residual idea is introduced according to the attention score calculation formula and the weighted average calculation formula, using the initial r H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function, which is:

[0033] Mapping the vector back to tangent space based on the logarithmic mapping is:

[0034]

[0035] According to the attention score calculation formula and the weighted average calculation formula, the residual idea is introduced, and the initial r H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function:

[0036]

[0037] Where b h and b t is the entity deviation that serves as the boundary of the scoring function.

[0038] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] The present invention provides a method for constructing a knowledge graph embedding model based on hyperbolic geometry, the method comprising performing a basic translation change on the head entity in the hyperbolic space based on the Mobius addition method, embedding the head entity through two-dimensional rotation and three-dimensional rotation, mapping the vector back to the tangent space based on logarithmic mapping, and calculating the score function of different embedding results. The present invention uses the characteristics of the hyperbolic space to more efficiently represent and process the hierarchical structure and complex relationships in the knowledge graph, solves the deficiencies of the prior art in high-dimensional data processing and hierarchical information capture, and is easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0042] Figure 2 This is the structural decomposition diagram of the score function. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] First, let's introduce hyperbolic geometry, which is a type of geometry with constant negative curvature. Curvature reflects the degree of deviation from a plane. When the curvature is equal to 0, it is Euclidean geometry. In hyperbolic space, the concept corresponding to the straight line in Euclidean space is called geodesic, which is expressed in the form of a curve. In theory, any finite tree can be almost perfectly embedded in a two-dimensional hyperbolic space, but Euclidean space cannot do this.

[0046] Unlike the embedding model in Euclidean space, which has a fixed curvature of 0, the embedding model in hyperbolic space will get different results depending on the curvature of the embedded hyperbolic space. Compared with the previous model where all relations are embedded in the hyperbolic space with the same curvature, the model of the present invention learns different exclusive absolute curvatures C for each different relation. r , enabling the model to represent different hierarchical structures. The curvature of the hyperbolic space determines whether the relationship is embedded in a more curved, less flat geometry or in a flatter, more flat geometry.

[0047] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a method for constructing a knowledge graph embedding model based on hyperbolic geometry, comprising:

[0048] Step 100: Perform a basic translation change on the head entity in the hyperbolic space based on the Möbius addition method. r H and are the hyperbolic embeddings of the head entity, relation, and tail entity, respectively;

[0049] Step 200: embedding the head entity through two-dimensional rotation and three-dimensional rotation;

[0050] Step 300: Map the vector back to the tangent space based on the logarithmic mapping, and calculate the score function of different embedding results.

[0051] In step 100, a basic translation change is performed on the head entity in the hyperbolic space based on the Mobius addition method, specifically:

[0052] Translation, as a basic operation in geometric transformation, plays a significant role in many embedding models based on Euclidean space. Its actual operation is realized by vector addition, but there is no clear definition of vector addition in hyperbolic space. The present invention performs a basic translation change on the head entity in hyperbolic space based on the Möbius addition method, which is:

[0053]

[0054] In the formula, x and y are the vectors participating in the Möbius addition, c r is the curvature of the hyperbolic space, Addition for Mobius.

[0055] In step 200, the two-dimensional rotation is specifically:

[0056] Two-dimensional rotation can simultaneously model inversion, combination, symmetry or antisymmetry and other relational patterns, and has been successfully applied in the RotatE model, and plays an important role in capturing the multiplicity of data. The present invention implements two-dimensional rotation operations in hyperbolic spaces of different curvatures based on the Givens transformation simulation relationship. The Givens transformation is a 2×2 rotation matrix commonly used in numerical linear algebra. The number of dimensions is assumed to be an even number, and the block diagonal rotation matrix is ​​used to parameterize the rotation, which is:

[0057]

[0058] In the formula, are parameters specific to each relationship type.

[0059] In step 200, the three-dimensional rotation is specifically:

[0060] The non-commutative nature of three-dimensional rotation enables the graph embedding model to perform non-commutative combinations. The QuatE model successfully proves that three-dimensional rotation is more expressive than two-dimensional rotation in some data. The present invention simulates the rotation operation in three-dimensional space based on Hamilton product matrix operation, which is:

[0061]

[0062] In the formula, is the Hamilton product. The quaternion q consists of a real number and three imaginary units, and is defined as: q = a + bi + cj + dk, where a, b, c, d are real numbers, and i, j, k are imaginary units.

[0063] In step 200, the head entity is embedded by two-dimensional rotation and three-dimensional rotation, specifically:

[0064] Combining formula (2) and formula (3), the embedded head entity is:

[0065]

[0066] In step 300, the vector is mapped back to the tangent space based on logarithmic mapping, and the score functions of different embedding results are calculated, specifically:

[0067] Since the two rotation transformations will be applied to hyperbolic spaces with different curvatures, one of the rotation transformations may perform better in some cases. In order to combine the advantages of the two rotation transformations at the same time, the present invention uses an attention mechanism to learn the importance concepts of different rotation transformations, specifically:

[0068] set up and For two vectors that have been embedded in the hyperbolic space through different rotation transformations, use the logarithmic mapping to map and to the tangent space respectively, and calculate the attention scores of the two rotation transformations, which are:

[0069]

[0070] In the formula, v is the attention vector, and the weighted average is calculated as:

[0071]

[0072] We add skip connections to the modules obtained above to further improve the representation capability of the embedding model, and finally derive a scoring function based on hyperbolic distance. Specifically, for triples set up r H and They are the hyperbolic embeddings of the head entity, relation, and tail entity respectively. The vectors are mapped back to the tangent space based on the logarithmic mapping, which are:

[0073]

[0074] According to the attention score calculation formula (5) and the weighted average calculation formula (6), the residual idea is introduced and the initial relationship moment r is used. H The matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function:

[0075]

[0076] Where b h and b t is the entity deviation that serves as the boundary of the scoring function.

[0077] The score function structure decomposition diagram of the entire model is as follows Figure 2 As shown. Intuitively, the combination of translation and different rotation transformations encodes logical patterns, while the hyperbolic space can provide enough space to capture the tree structure in a limited dimension. Finally, the embedding result is compared with the target tail entity embedding by hyperbolic distance to obtain the score function shown in formula (9).

[0078] The present invention provides experimental results of link prediction using two public datasets WN18RR and FB15k-237 in a low-dimensional space with a dimension of 32 dimensions, as shown in Table 1, where the best score is indicated by bold and the second best score is indicated by underline.

[0079] Table 1 Experimental results

[0080]

[0081] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0082] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for constructing a knowledge graph embedding model based on hyperbolic geometry, characterized in that: include: Based on the Möbius addition method, a basic translation change is performed on the head entity in the hyperbolic space, where r H and are the hyperbolic embeddings of the head entity, relation, and tail entity, respectively; Embed the head entity through two-dimensional rotation and three-dimensional rotation; The vector is mapped back to the tangent space based on the logarithmic mapping, and the score function of different embedding results is calculated.

2. The method according to claim 1, characterized in that Based on the Möbius addition method, the head entity undergoes a basic translation change in the hyperbolic space, specifically: Based on the Möbius addition method, a basic translation change is performed on the head entity in the hyperbolic space, which is: In the formula, x and y are the vectors participating in the Möbius addition, c r is the curvature of the hyperbolic space, Addition for Mobius.

3. The method according to claim 1, characterized in that The specific two-dimensional rotation is: Based on the Givens transformation simulation relationship, the two-dimensional rotation operation in hyperbolic space with different curvatures is realized. The number of dimensions is set to an even number, and the block diagonal rotation matrix is ​​used to parameterize the rotation, which is: In the formula, are parameters specific to each relationship type.

4. The method according to claim 3, characterized in that The three-dimensional rotation is as follows: The rotation operation in three-dimensional space is simulated based on Hamilton product matrix operation: In the formula, is the Hamilton product. The quaternion q consists of a real number and three imaginary units, and is defined as: q = a + bi + cj + dk, where a, b, c, d are real numbers, and i, j, k are imaginary units.

5. The method according to claim 4, characterized in that The head entity is embedded through two-dimensional rotation and three-dimensional rotation, specifically: According to the formulas of two-dimensional rotation and three-dimensional rotation, the head entity is embedded as follows:

6. The method according to claim 5, characterized in that Based on the logarithmic mapping, the vector is mapped back to the tangent space, and the score function of different embedding results is calculated, specifically: Calculate the attention scores and weighted averages of different rotation transformations based on the attention mechanism; Based on the logarithmic mapping, the vector is mapped back to the tangent space, and the residual idea is introduced according to the attention score calculation formula and the weighted average calculation formula. H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function.

7. The method according to claim 6, characterized in that The attention scores and weighted averages of different rotation transformations are calculated based on the attention mechanism, specifically: set up and For two vectors that have been embedded in the hyperbolic space through different rotation transformations, use the logarithmic mapping to map and to the tangent space respectively, and calculate the attention scores of the two rotation transformations, which are: In the formula, v is the attention vector, and the weighted average is calculated as:

8. The method according to claim 6, characterized in that Based on the logarithmic mapping, the vector is mapped back to the tangent space, and the residual idea is introduced according to the attention score calculation formula and the weighted average calculation formula. H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function, which is: Mapping the vector back to tangent space based on the logarithmic mapping is: According to the attention score calculation formula and the weighted average calculation formula, the residual idea is introduced, and the initial r H The relationship matrix is ​​translated in the hyperbolic space, and the hyperbolic distance between the embedding result and the target tail entity is calculated to obtain the final score function: Where b h and b t is the entity deviation that serves as the boundary of the scoring function.

Images