Knowledge Graph Embedding Representation Method and System Based on Neighborhood Relationship Representation Vector

Abstract

The present invention relates to a knowledge graph embedding representation method and system based on a neighborhood relationship representation vector, including the generation of a neighborhood relationship representation vector of an entity, the division of the same type of entity set, and the extraction and fusion of the same type of entity features. The purpose of the present invention is to numerically characterize the neighborhood relationship characteristics of an entity in the form of a neighborhood relationship representation vector based on the type and quantity of the entity's neighborhood relationships, laying the foundation for using a clustering method to measure the similarity of the entity neighborhood relationship vectors. During the training process of the knowledge graph embedding model, the extraction of the same type of entity features and the fusion of the same type of entity features with the entity embedding vector are completed. The present invention lays the foundation for extracting high-quality same type of entity features and enhancing the accuracy of the knowledge graph embedding representation, and provides a new knowledge graph embedding representation method. The method of the present invention also has the advantages of low computational complexity, good applicability, and good interpretability of the same type of entity division results.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph embedding representation, and specifically relates to a method and system for knowledge graph embedding representation based on neighborhood relationship representation vectors. Background Art

[0002] In the related research on knowledge graphs, knowledge graph embedding (KGE, also known as knowledge representation learning KRL) is a representation learning technology that represents the semantic information of relationships and entities in a knowledge graph as dense low-dimensional real-valued vectors, and efficiently calculates the semantic connections between entities and relationships in the low-dimensional space. It is one of the key basic technologies throughout the entire process of knowledge graph construction and application.

[0003] The accuracy or quality of the embedding representation of entities and relationships in a knowledge graph directly affects the performance of many knowledge acquisition and knowledge graph application models, such as link prediction, triple classification, entity classification, entity alignment, as well as knowledge graph-based intelligent question answering, information retrieval, system recommendation, multi-source information fusion, etc. Therefore, improving the accuracy of knowledge graph embedding representation has important fundamental significance for the performance of knowledge acquisition and application models based on knowledge graphs.

[0004] In the related research work on knowledge graph embedding technology driven by knowledge graph completion or link prediction tasks, how to better model triple information and how to incorporate more information other than triple facts to better improve the accuracy of knowledge graph embedding representation has always been the main line of research in this direction.

[0005] In order to pursue better embedding representation quality, in addition to fact-based embedding methods, in recent years, knowledge graph embedding technologies that integrate auxiliary semantic information other than fact triples have become a new research trend. Among them, in the embedding methods that integrate auxiliary information, methods that enrich the embedding representation of entities and relationships with semantic feature information contained in semantic description information, entity attribute information, etc. can improve the accuracy of their embedding representation to a certain extent. However, this type of method highly depends on the existence of auxiliary descriptive information, resulting in certain limitations in its application; methods that enrich the embedding representation of entities and relationships with feature information contained in graph structures, relationship paths, etc. have problems of high computational complexity and poor adaptability. Therefore, there is an urgent need to study a new method for representing knowledge graph embedding. Summary of the Invention

[0006] To solve the deficiencies of the above-mentioned existing technologies, the present invention provides a knowledge graph embedding representation method based on neighborhood relation representation vectors, which can solve the limitations, high computational complexity, and poor adaptability existing in the existing knowledge graph embedding methods that fuse auxiliary information. Starting from the types and quantities of neighborhood relations of entities, it fully excavates the structural feature information contained in this heterogeneous network of the knowledge graph, so as to accurately represent the knowledge graph embedding.

[0007] The technical solution of the present invention first starts from the perspective of the types and quantity characteristics of the neighborhood relations of entities in the knowledge graph, and gives a method for generating the neighborhood relation representation vectors of entities to numerically characterize the types and quantity characteristics of the neighborhood relations of entities; then uses a clustering method to cluster the neighborhood relation representation vectors in the set of normalized entity neighborhood relation representation vectors, and completes the division of the entity sets of the same type according to the clustering results; finally, along with the training of the knowledge graph embedding model, the extraction of the features of entities of the same type and the fusion of the features of entities of the same type and the entity embedding vectors are completed, so as to enhance the accuracy of the knowledge graph embedding representation.

[0008] Specifically, the present invention is implemented as follows:

[0009] The present invention provides a knowledge graph embedding representation method based on neighborhood relation representation vectors, which includes the following sub-steps:

[0010] S1. Generate the neighborhood relation representation vectors of entities to obtain the set Ω of neighborhood relation representation vectors of entities;

[0011] S2. Generate entity sets of the same type, specifically including the following sub-steps:

[0012] S21. Define vectors max and min with a dimension of 2*m;

[0013] S22. Assign values to the max and min vectors;

[0014] S23. Perform normalization processing on any entity neighborhood relation representation vector v in the set Ω i to obtain the set Ω' of normalized neighborhood relation representation vectors;

[0015] S24. Use a clustering method to cluster the entity neighborhood relation representation vectors in the set Ω' of neighborhood relation representation vectors into K clusters, and then select the corresponding entities e from the entity set E according to the neighborhood relation representation vectors of the same cluster i to complete the division of the entity sets C k (k ∈ [1, K]);

[0016] S3. Extract and fuse the features of entities of the same type, specifically including the following sub-steps:

[0017] S31. Determine the entity set C of the same type to which the entity e belongs according to the entity set of the same type generated in step S2 i ; k ;

[0018] S32. Select the corresponding entity embedding vectors from the entity embedding matrix of the baseline knowledge graph embedding model according to the entity set C of the same type, and splice the entity embedding vectors into the entity embedding matrix E of the same type according to formula (2) k ; k ;

[0019]

[0020] where || represents the splicing operation, and h i is the embedding vector of the entity e k in the entity set C of the same type i ;

[0021] S33. During the training process of the baseline knowledge graph embedding model, adopt the linear dimensionality reduction method shown in formula (3) to extract the entity type feature t k implied in the entity embedding matrix E of the same type k ;

[0022] t k =σ(WE k E k ) (3)

[0023] where σ(·) is the mish activation function, W k is the linear transformation matrix. After the model training is completed, the finally obtained t k is the type feature of all entities in the entity set C of the same type k ;

[0024] S34. After concatenating the entity type feature t k with the embedding vector h i of the entity e i , input the concatenated feature vector into the fully connected layer according to formula (4), so as to obtain the entity embedding vector e' k after fusing the entity type feature t i .

[0025] e′ i =concat(h i ,t k )W + b (4)

[0026] where concat is the operation of serially concatenating two feature vectors, and h i is the entity e iThe embedding vector, t k is the entity e i The feature vector representation of the entity type to which it belongs, W ∈ R (dim1 +dim2)×dim1 is the weight matrix, dim1 is the entity e i The embedding vector h i The dimension of, dim2 is the entity type feature vector t k The dimension of, e' i is the entity e i The embedding vector h i Fusing the entity type feature vector t k The embedding vector representation after that, b is the bias parameter;

[0027] S35. Update the embedding vector h of the entity e in the entity embedding matrix according to Equation (5) i The embedding vector h i ;

[0028] h i = e′ i (5)

[0029] S36. In the training process constrained by the scoring function and loss function of the benchmark knowledge graph embedding model, continuously update the model parameters through backpropagation, and output the result when the model training ends to obtain the final entity type feature t k And the entity embedding vector h after fusing the entity type feature i .

[0030] Preferably, step S1 specifically includes the following steps:

[0031] S11. Represent a certain knowledge graph as G = (E, R), where E is the entity set and R is the relationship set, E = {e0,..., e i ,..., e n-1}, where i ∈ [0, n - 1] and n is the number of entities; R = {r0,..., r j ,..., r m-1}, where j ∈ [0, m - 1] and m is the number of relationships;

[0032] S12. Define the neighborhood relationship characterization vector v of length m * 2 of the entity e according to the number of relationships m in the knowledge graph G i = [d0,..., d i ,..., d j ,..., d m+j ,..., d 2m-1 , and initialize each of its elements to 0; among them, the first m dimensions are used to store the entity e iThe number of various relationships directly associated with it when it is the head entity, and the latter m dimensions are used to store the entity e i The number of various relationships directly associated with it when it is the tail entity;

[0033] S13. Count the entity e i The number of various relationships directly associated with it when it is the head entity. If the relationship r j appears p j times, then update the d i dimension of the neighborhood relationship representation vector v j to p j ;

[0034] S14. Count the entity e i The number of various relationships directly associated with it when it is the tail entity. If the relationship r j appears pj' times, then update the d i dimension of the neighborhood relationship representation vector v m+j to p j ';

[0035] S15. Perform the operations of S12 to S14 on any entity in the entity set E to obtain the neighborhood relationship representation vector set Ω = {v0,..., v i ,..., v n-1}.

[0036] Preferably, the specific method of step S22 is to assign the maximum value in the d j dimension of all entity neighborhood relationship representation vectors in the neighborhood relationship representation vector set Ω of the entity generated in step S1 to the d j dimension in the vector max; assign the minimum value in the d j dimension of all entity neighborhood relationship representation vectors in the set Ω to the d j dimension in the vector min, where j ∈ [0, 2*m - 1].

[0037] Preferably, the specific method of the normalization process in step S23 is as follows: Normalize any entity neighborhood relationship representation vector v i in the set Ω according to formula (1), so that the numerical values of each dimension of all neighborhood relationship representation vectors in the normalized neighborhood relationship representation vector set Ω' are all in the interval [0, 1].

[0038]

[0039] where, v i * is the vector after the neighborhood relationship representation vector v i of the entity e i is normalized.

[0040] On the other hand, the present invention also provides a knowledge graph embedding representation system based on neighborhood relationship representation vectors, which includes an entity neighborhood relationship representation vector generation module, a same-type entity set generation module, and a same-type entity feature extraction and fusion module, wherein the entity neighborhood relationship representation vector generation module is used to obtain an entity neighborhood relationship representation vector set Ω, and the same-type entity set generation module is used to generate a same-type entity set, thereby completing the same-type entity set C k (k∈[1,K]), the module for extracting and fusing entity features of the same type is used to extract and fuse entity features of the same type, thereby completing the embedded representation of the knowledge graph.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The knowledge graph embedding representation method based on the neighborhood relationship representation vector of the present invention fully exploits the structural feature information contained in the knowledge graph, a heterogeneous network, and numerically characterizes the neighborhood relationship characteristics of entities in the form of neighborhood relationship representation vectors based on the neighborhood relationship types and quantities of entities, laying the foundation for using clustering methods to measure the similarity of entity neighborhood relationship vectors, and on this basis, realizes the division of entity sets of the same type, completes the extraction of entity features of the same type and the fusion of entity features of the same type with entity embedding vectors during the training process of the knowledge graph embedding model. The knowledge graph embedding method proposed in the present invention can adapt to the embedding representation of various knowledge graphs, and its overall computational complexity is low, the accuracy is high and the adaptability is strong.

[0043] (2) The present invention better measures entity similarity from the perspective of entity neighborhood relationship similarity, thus laying a foundation for extracting high-quality features of entities of the same type and further enhancing the accuracy of knowledge graph embedding representation. In addition, compared with other methods, the present invention also has the advantages of low computational complexity, good applicability, and good interpretability of entity segmentation results of the same type, and proposes a new feasible method for knowledge graph embedding representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0045] Figure 2 A schematic diagram of a specific method flow in an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a model structure of an embodiment of the present invention;

[0047] Figure 4 This is an example of a knowledge graph according to an embodiment of the present invention;

[0048] FIG. 5 is a loss change curve of the FTKGE-ConvKB model on the FB15K-237 dataset in the embodiment of the present invention, where Figure 5a is the loss change curve of the FTKGE encoder, Figure 5b is the loss change curve of the ConvKB decoder. Detailed implementation manners

[0049] Hereinafter, with reference to the accompanying drawings and embodiments, the working principle of the present invention will be further explained.

[0050] Intelligent search is one of the typical applications of knowledge graphs, and the knowledge graph provides rich structured results for search engines. After a user inputs a query keyword, the search engine matches the keyword with entities in the knowledge graph, and then obtains the association relationships between the entity and other entities. The knowledge graph embedding representation method based on neighborhood relationship representation vectors provided by this patent is applied in the knowledge base involved in intelligent search.

[0051] In the intelligent search task, in addition to providing content directly related to the query entity to the user, it is also necessary to provide some content that should be related to the query entity but is missing due to the sparsity of the knowledge graph, so as to improve the performance of the intelligent search engine and thus provide a better search experience for the query user. In order to obtain the missing association relationships in the knowledge graph, the knowledge graph embedding representation method is usually adopted to represent the semantic information of the entities and relationships in the knowledge graph as dense low-dimensional real-valued vectors, and efficiently calculate the semantic connections between entities and relationships in the low-dimensional space, so as to realize the prediction of the association relationships between entities. It can be seen that the accuracy or quality of the embedding representation of entities and relationships in the knowledge graph directly affects the performance of the intelligent search engine.

[0052] When the present invention represents the embedding of entities and relationships in the knowledge graph involved in intelligent search, it incorporates the entity type features constructed based on neighborhood relationship representation vectors, making the embedding representations of entities of the same type closer in the vector space, so as to make the relationship prediction between entities more accurate. Freebase is a large general knowledge graph, containing 68 million entities about the real world, 1 billion relationship information, and more than 2.4 billion fact triple information. The FB15k-237 knowledge graph is a subset of the Freebase knowledge graph. Hereinafter, taking the improvement of the performance of predicting the relationship between entities in intelligent search based on the FB15k-237 knowledge graph as an example, the implementation manner of the application of the present invention will be described with reference to the accompanying drawings:

[0053] As Figure 1 shown, the method of the present invention mainly includes the following parts:

[0054] First, count the neighborhood relationships of each entity in the required FB15k-237 knowledge graph involved in intelligent search, and then generate the neighborhood relationship representation vectors of each entity to obtain the set of neighborhood relationship representation vectors of entities in the knowledge graph.

[0055] Secondly, cluster based on the normalization of the representation vectors in the set of neighborhood relationship representation vectors. Consider the clusters obtained from the clustering as entity types, and select the entities in the knowledge graph involved in intelligent search corresponding to the neighborhood relationship representation vectors in the same cluster to form sets of entities of the same type.

[0056] Finally, select the corresponding entity embedding vectors from the entity embedding matrix of the knowledge graph embedding model according to the entities in the sets of entities of the same type, and splice the entity embedding vectors into an entity type embedding matrix. During the training process of the knowledge graph embedding model, based on the entity type embedding matrix, use linear dimensionality reduction to complete the extraction of the common features of entities of the same type, that is, the extraction of entity type features.

[0057] The following further describes the method of the present invention in combination with specific embodiments:

[0058] In this embodiment, a knowledge graph embedding representation method and system based on neighborhood relationship representation vectors are proposed. The system includes a neighborhood relationship representation vector generation module for entities, a set generation module for entities of the same type, and an extraction and fusion module for features of entities of the same type. The neighborhood relationship representation vector generation module for entities is used to obtain the set of neighborhood relationship representation vectors Ω of entities. The set generation module for entities of the same type is used to generate sets of entities of the same type, thereby completing the division of sets of entities of the same type C k (k∈[1,K]), and the extraction and fusion module for features of entities of the same type is used to extract and fuse features of entities of the same type, thereby completing the embedding representation of the knowledge graph. As Figure 1 and Figure 2 shown, the knowledge graph embedding representation method specifically includes the following sub-steps:

[0059] S1. Generate the neighborhood relationship representation vectors of entities. Represent the FB15k-237 knowledge graph involved in intelligent search as G=(E,R), where E={e0,...,e i ,...,e n-1} is the set of entities, where i∈[0,n-1] and n is the number of entities; R={r0,...,r j ,...,r m-1} is the set of relationships, where j∈[0,m-1] and m is the number of relationships. Then, for the knowledge graph G involved in intelligent search, the method for generating the neighborhood relationship representation vector of entity e i is specifically as follows:

[0060] S11. Define an entity e with a length of m * 2 based on the number of relationships m in the knowledge graph G involved in intelligent search i of the neighborhood relationship representation vector v i = [d0,..., d j ,..., d m+j ,..., d 2m-1 , and initialize each of its elements to 0; where the first m dimensions are used to store the number of various direct relationships when the entity e i is the head entity, and the last m dimensions are used to store the number of various direct relationships when the entity e i is the tail entity;

[0061] S12. Respectively count the number of various direct relationships with the entity e i when it is the head entity. If the relationship r j appears p j times, then update the d i dimension of the neighborhood relationship representation vector v j to p j ;

[0062] S13. Respectively count the number of various direct relationships with the entity e i when it is the tail entity. If the relationship r j appears p j ' times, then update the d i dimension of the neighborhood relationship representation vector v m+j to p j ';

[0063] S14. Perform the S1 - S3 operations on all entities in the knowledge graph G to obtain a set of neighborhood relationship representation vectors of entities Ω = {v0,..., v i ,..., v n-1}.

[0064] Figure 3 shows a schematic diagram of the model structure of the entity in this embodiment. To better understand the above process of generating the neighborhood relationship representation vector of the entity, taking the knowledge graph shown in Figure 4 as an example, further explanation is carried out.

[0065] In Figure 4In the knowledge graph example shown, there are 8 entities (e1, e2, e3, e4, e5, e6, e7, e8) and 5 relationships (r1, r2, r3, r4, r5). It can be seen from step S11 that the dimension of the neighborhood relationship representation vector to be defined is 10. Taking entity e1 as an example, when it is the head entity, there is a relationship r1 between e1 and e2, and the number of r4 relationships is 1; when it is the tail entity, there is a relationship r1 between e3 and e1 and a relationship r1 between e5 and e1, so the number of r1 relationships is 2.

[0066] It can be seen from S12 - step S13 that the neighborhood relationship representation vector of entity e1 should be (0, 0, 0, 1, 0, 2, 0, 0, 0, 0).

[0067] According to step S14, the set of neighborhood relationship representation vectors Ω of this knowledge graph is Ω = {(0, 0, 0, 1, 0, 2, 0, 0, 0, 0), (0, 0, 0, 0, 0, 0, 0, 0, 2, 0), (1, 0, 1, 1, 0, 0, 0, 0, 0, 0), (0, 0, 0, 0, 0, 0, 0, 0, 0, 1), (3, 0, 0, 0, 0, 0, 0, 2, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0, 0, 0), (0, 0, 1, 0, 1, 0, 0, 0, 0, 0)}.

[0068] S2. Generate a set of entities of the same type, which specifically includes the following sub - steps:

[0069] S21. Define vectors max and min with a dimension of 2 * m;

[0070] S22. Assign values to the max and min vectors. The method is to assign the maximum value of the d - th dimension of all entity neighborhood relationship representation vectors d in the set Ω of entity neighborhood relationship representation vectors generated in step S1 to the d - th dimension of vector max; assign the minimum value of the d - th dimension of all entity neighborhood relationship representation vectors d in the set Ω to the d - th dimension of vector min, where j ∈ [0, 2 * m - 1]; j dimension to the d - th dimension in vector max; j dimension; assign the d - th dimension of all entity neighborhood relationship representation vectors d in set Ω j dimension to the d - th dimension in vector min, j where j ∈ [0, 2 * m - 1];

[0071] S23. Process any entity neighborhood relationship representation vector v in set Ω i according to formula (1), so that the numerical values of each dimension of all neighborhood relationship representation vectors in the normalized neighborhood relationship representation vector set Ω' are all within the interval [0, 1].

[0072]

[0073] where, vi * For entity e i Neighborhood relation representation vector v i A vector after normalization processing;

[0074] S24. Use the clustering method to cluster the entity neighborhood relation representation vectors in the neighborhood relation representation vector set Ω' into K clusters, and then select the corresponding entity e from the entity set E according to the neighborhood relation representation vectors in the same cluster i , thus completing the partitioning of the same-type entity set C k (k ∈ [1, K]);

[0075] In order to verify the feasibility of the key steps in the assumption that "the higher the similarity of the neighborhood relation representation vectors of two entities, the greater the possibility that the two entities are of the same type" implemented by the clustering method, on the FB15k-237 knowledge graph, cluster the neighborhood relation representation vectors of entities (the clustering method selects hierarchical clustering, and the number of clustering categories is 100), and part of the results obtained are shown in Table 1

[0076] Table 1 Partial results table of entity clustering in the FB15k-237 knowledge graph

[0077]

[0078] As can be seen from Table 1, clustering the neighborhood relation representation vectors in the FB15k-237 knowledge graph can basically cluster entities of the same type into the same cluster. Through analysis, it is not difficult to find that in Table 1, the entity type (cluster) numbered 1 corresponds to the award name entity, the entity type numbered 2 corresponds to the country name entity, the entity type numbered 3 corresponds to the occupation name entity, the entity type numbered 4 corresponds to the musical instrument name entity, the entity type numbered 5 corresponds to the month name entity, and so on. It is particularly worth mentioning that this clustering method accurately clusters the entities representing 12 months in the knowledge graph into the same cluster, which is consistent with the expected results

[0079] S3. Extract and fuse the features of the same-type entities, specifically including the following sub-steps:

[0080] S31. According to the same-type entity set generated in step S2, determine the same-type entity set C i to which the entity e belongs k ;

[0081] S32. According to the same-type entity set C k select the corresponding entity embedding vectors from the entity embedding matrix of the benchmark knowledge graph embedding model, and splice the entity embedding vectors into the same-type entity embedding matrix E according to formula (2) k ;

[0082]

[0083] Among them, || represents the concatenation operation, h i is the entity set C of the same type k entity e in i embedding vector;

[0084] S33. During the training process of the baseline knowledge graph embedding model, the linear dimensionality reduction method shown in Equation (3) is adopted to extract the entity type features t contained in the embedding matrix E of entities of the same type k ; k ;

[0085] t k = σ(W k E k ) (3)

[0086] Among them, σ(·) is the mish activation function, and W k is the linear transformation matrix. After the model training is completed, the finally obtained t k is the type feature of all entities in the entity set C of the same type k ;

[0087] S34. After concatenating the entity type feature t k with the embedding vector h of the entity e i , the concatenated feature vector is input into the fully connected layer according to Equation (4), so as to obtain the entity embedding vector e' after fusing the entity type feature t i . k e′ i .

[0088] e′ i = concat(h i , t k )W + b (4)

[0089] Among them, concat is the operation of serially concatenating two feature vectors, h i is the embedding vector of the entity e obtained through the knowledge graph embedding model i , t k is the feature vector representation of the entity type of the category to which the entity e i belongs, W ∈ R (dim1 +dim2)×dim1 is the weight matrix, dim1 is the dimension of the embedding vector h of the entity e i , dim2 is the dimension of the entity type feature vector t i , and e' k is the embedding vector h of the entity e i i i ​​The embedded vector representation after fusing the entity type feature vector t, where b is the bias parameter; k

[0090] S35. Update the embedding vector h of entity e in the entity embedding matrix according to Equation (5); i i ;

[0091] h i = e' i (5)

[0092] S36. During the training process constrained by the scoring function and loss function of the benchmark knowledge graph embedding model, continuously update the model parameters through backpropagation. When the model training ends, the final entity type feature t k and the entity embedding vector h after fusing the entity type feature can be obtained; i .

[0093] In the embodiments of this patent, the model is also corrected by means of the loss function. The loss function is a key element in machine learning that controls the learning efficiency and results of the model. It is used to measure the difference between the output result of each iteration of the model and the true value, and makes the output result value of the model approach the true value to reduce the difference between the true value and the output result, so as to achieve the purpose of learning.

[0094] For a triple (e i , r k , e j ), the FTKGE model adopts the scoring function d = ||e i + r k - e j ||, and defines the hinge-based loss function (Hinge Loss), as shown in Equation (6).

[0095]

[0096] In Equation (6), γ > 0 is the margin hyperparameter, S is the set of correct triples, S' is the set of incorrect triples, is the score of the correct triple, is the score of the incorrect triple. During the model training process, using this loss function can maximize the score gap between the correct triples and the incorrect triples as much as possible, so as to better classify the triples.

[0097] Generally, the method of randomly replacing the head entity or tail entity in the correct triple is used to obtain the incorrect triple, and its definition is shown in Equation (7).

[0098]

[0099] ​​The so-called error triples randomly constructed by this method may coincide with the correct triples already existing in the dataset. Therefore, it is necessary to remove them from the error triples to avoid affecting the test results.

[0100] Verify the calculation results of the above knowledge graph

[0101] The link prediction task can be regarded as a ranking search problem. For example, given a (h, r,?) triple, it is necessary to predict its tail entity. In the test phase, the triples to be evaluated corresponding to each triple (h, r, t) are composed of all entities in the knowledge graph sequentially replacing the head entity or the tail entity in the triple (h, r, t). Then, the model scores these triples to be evaluated sequentially through a scoring function and sorts their scores in descending order. The higher the ranking, the greater the possibility that the triple holds.

[0102] In this embodiment, MR (Mean Rank), MRR (Mean Reciprocal Rank), and Hit@N (N takes 10, 3, 1 in this embodiment) are selected as indicators to measure the performance of each model in the link prediction task.

[0103] MR represents the average rank of the correct triples in the ranking result, and its calculation formula is shown in Equation (8).

[0104]

[0105] Among them, n represents the number of triples in the test set, and rank i represents the position of the correct triple in the ranking result when testing the i-th triple in the test set.

[0106] MRR represents the average of the reciprocals of the ranks of the correct triples in the ranking result, and its value range is 0 to 1. The calculation is shown in Equation (9).

[0107]

[0108] Hit@N represents the probability that the correct triple ranks among the top N in the ranking result, and its value range is 0 to 1. The calculation is shown in Equation (10).

[0109]

[0110] In Equation (10), ind(·) is a flag function, as shown in (11).

[0111]

[0112] From the calculation formulas of the above evaluation indicators, it can be seen that the lower the MR, the higher the MRR, and the higher the Hit@N, the better the performance of the knowledge graph embedding model.

[0113] Among the evaluation metrics for these link prediction tasks, the MR and MRR metrics evaluate the position of the correct triples in the ranking results. Among them, the MRR, as a general evaluation metric in the field of search ranking algorithms, can better reflect the accuracy of the model's prediction results. The Hit@N metric focuses more on whether there are correct triples among the top N ranked triples, reflecting the recall rate of the model's prediction results.

[0114] Therefore, when evaluating the performance of a knowledge graph embedding model, it is necessary to comprehensively consider various evaluation metrics to judge the model's link prediction performance.

[0115] Next, the performance verification of the FTKGE model for link prediction tasks is carried out:

[0116] During the training process of the FTKGE-ConvKB model on the FB15k-237 knowledge graph, the loss change curve is as Figure 5a - Figure 5b .

[0117] As can be seen from Figure 5a , on the FB15K-237 knowledge graph, the loss of the FTKGE encoder model drops rapidly in the early stage and tends to be stable after 1000 epochs; as can be seen from Figure 5b , the ConvKB decoder model reaches stability after 100 epochs, thus enabling the model to converge.

[0118] To verify the performance of the FTKGE model, a performance comparison experiment of the FTKGE-ConvKB model with 7 other representative models is carried out on the FB15k-237 knowledge graph, and the obtained results are shown in Table 2.

[0119] Table 2 Experimental results on the FB15k-237 knowledge graph

[0120]

[0121]

[0122] As can be seen from Table 2, the values of the five metrics of MR, MRR, Hit@1, Hit@3, and Hit@10 of the FTKGE-ConvKB model on the FB15k-237 knowledge graph are all better than those of other models, and its superiority is highlighted on the FB15k-237 knowledge graph, thus verifying to a certain extent the effectiveness of the entity type feature extraction method based on neighborhood relation representation vectors and the knowledge graph embedding model that fuses entity type features in improving the embedding representation performance.

[0123] As can be seen from the entity type feature extraction process based on neighborhood relation representation, on the basis of representing entities based on neighborhood relations, the purpose of clustering is to cluster similar entities into the same cluster from the perspective of the similarity of the neighborhood relation structure of entities and their relation characteristics. The quality of clustering directly affects the generation quality of the entity type embedding matrix, which is exactly an important basis for extracting high-quality entity type features and improving the accuracy of the knowledge graph embedding representation.

[0124] In the FB15k-237 knowledge graph, the proportion of the relation types of 1-N, N-1, and N-N between its entities is as high as 78%; the neighborhood relation representation vector v i of this dataset has been significantly improved in terms of data sparsity problems, which enables entities with high neighborhood structure and relation similarity to be clustered into the same cluster during the clustering process, and further enables the embedding representation accuracy and prediction performance of the FTKGE-ConvKB model on this dataset to be significantly improved.

[0125] As can be seen from the above analysis, for a knowledge graph dataset, the more complex the neighborhood relation structure and its relations of its entities are, the more obvious the performance improvement of the FTKGE-ConvKB model on it is. Thus, it can be known that the entity type feature extraction method based on the neighborhood relation representation vector is more suitable for the problem scenarios where the neighborhood relation structure and its relations of entities are relatively complex.

[0126] (1) Analysis of the reasons for the differential increase in Hit@N of the FTKGE-ConvKB model

[0127] As can be seen from Table 2, compared with the KBGAT model, the improvement amplitude of the FTKGE-ConvKB model in terms of the Hit@1, Hit@3, and Hit@10 indicators shows a downward trend.

[0128] Although the KBGAT model does not explicitly fuse the entity type features described in this embodiment, in its training process, it uses the scoring function of the formula d = ||e i + r k - e j ||. During the training process, this scoring function makes the embedding vectors of semantically similar entities also similar, and can implicitly depict the common features of the same type of entities to a certain extent, that is, it has fused some features of the entity type features described in this embodiment to a certain extent.

[0129] The FTKGE-ConvKB model explicitly integrates entity type features during link prediction, enhancing the characterization of entity type features for candidate entities, enabling more accurate matching of candidate entities, and thus achieving a significant improvement of around 2% in the most precise metric, Hit@1. In terms of the relatively broad metrics of Hit@3 and Hit@10, the improvement of the FTKGE-ConvKB model is weaker, and there is a trend that the broader the evaluation metric, the weaker the improvement.

[0130] (2) Performance difference analysis of graph neural network and convolutional neural network models

[0131] Regarding the Hit@1 metric in Table 2, the prediction performance of KBGAT based on graph neural network and the FTKGE-ConvKB model proposed in this embodiment is lower than that of ConvE, InteractE, and AcrE models based on convolutional neural network.

[0132] The advantage of graph neural network is that it is suitable for graph-structured data and can make full use of the complex topological structure in the graph for embedded representation learning of node features. The advantage of convolutional neural network is that it can better extract local features according to the receptive field of the convolutional kernel. Table 2 shows the performance of each model on the FB15k-237 knowledge graph with complex neighborhood relationship structure and relationships of entities. By comparing various metrics, it can be found that the superiority of each model based on graph neural network is reflected, and almost all Hit@N metric items exceed those of each model based on convolutional neural network.

[0133] It can be seen from the embodiments of this patent that this patent better realizes the measurement of entity similarity from the perspective of entity neighborhood relationship similarity globally, thus laying a foundation for extracting high-quality entity features of the same type and further enhancing the accuracy of knowledge graph embedded representation. In addition, compared with other methods, the present invention also has the advantages of low computational complexity, good applicability, and good interpretability of the division results of entities of the same type.

[0134] The above-described embodiments are only used to describe the preferred embodiments of the present invention and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A knowledge graph embedding representation method based on neighborhood relation representation vectors, characterized in that, It includes the following sub-steps: S1. Generate the neighborhood relationship representation vector of the entity to obtain the set Ω of the neighborhood relationship representation vectors of the entity; S2. Generate the set of entities of the same type, which specifically includes the following sub-steps: S21. Define the vectors max and min with the dimension of 2*m; S22. Assign values to the max and min vectors; S23. Normalize any entity neighborhood relation representation vector v in the set Ω i to obtain a normalized neighborhood relation representation vector set Ω'; S24. Use the clustering method to cluster the entity neighborhood relationship representation vectors in the neighborhood relationship representation vector set Ω' into K clusters, and then select the corresponding entity e from the entity set E according to the neighborhood relationship representation vectors in the same cluster, so as to complete the partitioning of the same type of entity set C i , thus completing the partitioning of the same type of entity set C k (k ∈ [1, K]); S3. Extract and fuse the features of entities of the same type, which specifically includes the following sub-steps: S31. Determine the entity set C of the same type to which the entity e belongs according to the entity set of the same type generated in step S2 i belonging to the same type of entity set C k ; S32. According to the entity set C of the same type k Pick the corresponding entity embedding vectors from the entity embedding matrix of the baseline knowledge graph embedding model, and splice each entity embedding vector into the entity embedding matrix E of the same type according to Equation (2) k ; Among them, || represents the concatenation operation, h i is the entity set C of the same type k entity e in i embedding vector of S33. During the training process of the benchmark knowledge graph embedding model, the linear dimensionality reduction method shown in Equation (3) is used to extract the entity type feature t contained in the homogeneous entity embedding matrix E k ; k ; t k = σ(W k E k ) (3) Among them, σ(·) is the mish activation function, and W k is the linear transformation matrix. After the model training is completed, the finally obtained t k is the type feature of all entities in the entity set C k of the same type; S34. Concatenate the entity type feature t k with the embedding vector h i of the entity e i . After concatenation, input the concatenated feature vector into the fully connected layer according to formula (4) to obtain the entity embedding vector e' k after fusing the entity type feature t i ; e′ i = concat(h i , t k )W + b (4) Among them, concat is the operation of serially concatenating two feature vectors, h i is the entity e obtained through the knowledge graph embedding model i The embedding vector of k For entity e i The feature vector representation of the entity type of the category, W∈R (dim1+dim2)×dim1 is the weight matrix, dim1 is the entity e i Embedding vector h i The dimension of dim2 is the entity type feature vector t k The dimension, e' i For entity e i The embedding vector h i Fusion entity type feature vector t k The embedded vector after is represented by , and b is the bias parameter; S35. Update the embedding vector h of entity e in the entity embedding matrix according to formula (5). i i ;​ h i = e' i (5) S36. During the training process constrained by the scoring function and loss function of the baseline knowledge graph embedding model, the model parameters are continuously updated through backpropagation, and the output result is obtained when the model training ends to obtain the final entity type feature t k and the entity embedding vector h after fusing the entity type feature i .

2. The knowledge graph embedding representation method based on the neighborhood relationship representation vector according to claim 1, wherein Step S1 specifically includes the following steps: S11. Represent a certain knowledge graph as G = (E, R), where E is the entity set, R is the relationship set, E = {e0,..., e i ,..., e n-1}, where i ∈ [0, n - 1] and n is the number of entities; R = {r0,..., r j ,..., r m-1}, where j ∈ [0, m - 1] and m is the number of relationships; S12. Define an entity e with a length of m * 2 according to the number of relationships m in the knowledge graph G i 's neighborhood relationship representation vector v i = [d0,..., d j ,..., d m+j ,..., d 2m-1 , and initialize each of its elements to 0; among them, the first m dimensions are used to store the number of various relationships directly associated with entity e i when it is the head entity, and the last m dimensions are used to store the number of various relationships directly associated with entity e i when it is the tail entity; S13. Count the entity e i When it is the head entity, count the number of various relationships directly associated with it. If the relationship r j appears p j times, then update the d i dimensions of the neighborhood relationship representation vector v j to p j ; S14. Count the entity e i When it is the tail entity, count the number of various relationships directly associated with it. If the relationship r j appears p j ' times, then update the d i -dimensional neighborhood relationship representation vector v m+j to p j '; S15. Perform the operations of S12 to S14 on any entity in the entity set E to obtain the neighborhood relation representation vector set Ω = {v0,..., v i ,..., v n-1}.

3. The knowledge graph embedding representation method based on the neighborhood relationship representation vector according to claim 1, characterized in that: The specific method of step S22 is to assign the maximum value among all the entity neighborhood relation representation vectors d in the set Ω of entity neighborhood relation representation vectors generated in step S1 to d in the vector max j in dimension; assign the minimum value among all the entity neighborhood relation representation vectors d in the set Ω to d in the vector min j in dimension, where j ∈ [0, 2*m - 1]. j in dimension; assign the minimum value among all the entity neighborhood relation representation vectors d in the set Ω to d in the vector min j in dimension, where j ∈ [0, 2*m - 1].

4. The knowledge graph embedding representation method based on the neighborhood relation representation vector according to claim 1, wherein: The specific method of normalization in step S23 is as follows: for any entity neighborhood relationship representation vector v in the set Ω i perform normalization processing according to formula (1), so that the numerical values of each dimension of all neighborhood relationship representation vectors in the neighborhood relationship representation vector set Ω' after normalization processing are all within the interval [0, 1]. Among them, v i * is the neighborhood relationship representation vector v i of entity e i after normalization processing.

5. A system for the knowledge graph embedding representation method based on the neighborhood relationship characterization vector described in claim 1, characterized in that: It includes a neighborhood relationship representation vector generation module for entities, a same-type entity set generation module, and an extraction and fusion module for same-type entity features. The neighborhood relationship representation vector generation module for entities is used to obtain a set of neighborhood relationship representation vectors Ω of entities. The same-type entity set generation module is used to generate a same-type entity set, thereby completing the partitioning of the same-type entity set C k (k ∈ [1, K]), and the extraction and fusion module for same-type entity features is used to extract and fuse same-type entity features, thereby completing the embedding representation of the knowledge graph.

Images