Knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation
Through the self-interleaving attention mechanism and diffusion aggregation method, the problem of insufficient capture of structural features and semantic information in knowledge graph embedding is solved, and a more accurate knowledge graph embedding representation is achieved, especially when dealing with complex relationship inference and relationship nodes. There are advantages when it is.
Patent Information
- Application Number
- CN202410986681.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2023-12-08
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-23
AI Technical Summary
Existing knowledge graph embedding methods are difficult to effectively capture the structural characteristics and semantic information of knowledge graphs, especially when dealing with complex relationship reasoning and relationship nodes.
Using a method based on self-interleaved attention mechanism and diffusion aggregation, the node embedding representation is initialized by obtaining the neighbor matrix and diffusion path dictionary of the knowledge graph, and the diffusion attention calculation module, the node attention calculation module and the semantic distance attention module are used. Learn information propagation and semantic context information between nodes to obtain the final embedding representation of the node.
This method can capture the structural information and semantic context of the knowledge graph more comprehensively, improving the accuracy of the embedded representation of the knowledge graph, especially when dealing with the connection between entities and relationship nodes.
Smart Images

Figure CN118520943B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence knowledge graphs, and specifically is a knowledge graph embedding method based on self-interleaving attention mechanism and diffusion aggregation. Background Art
[0002] Knowledge graph is a large-scale semantic network knowledge base that uses "entity-relationship-entity" triples to abstractly represent the relationships between entities in the real world and store information in the form of directed graphs. This technology has the advantages of rich semantics, friendly structure, and easy to understand. It has a wide range of application scenarios and development prospects in the era of big data. At present, the construction of large-scale knowledge graphs inevitably requires manual annotation and maintenance of a large amount of real data to ensure its correctness and reliability. Despite this, the sparsity of information, lack of facts and weak generalization ability are still the biggest challenges facing knowledge graphs, which also makes it difficult for knowledge graphs to cope with complex relational reasoning. In order to solve the problem of knowledge graph completion, with the rapid development of deep learning, a large number of knowledge graph embedding methods have been born in academia. It embeds entities and relationships in knowledge graphs into continuous vector spaces, thereby facilitating calculations while retaining the structural information in knowledge graphs. Among them, research methods based on pre-trained models and graph neural networks are still the main research direction. The method based on pre-trained models aims to use the pre-trained model's ability to analyze sequence information such as natural language to process knowledge graphs. The labels or descriptive texts of entities and relationships in the knowledge graph are input into the pre-trained model after certain pre-processing, and the corresponding embedded representation is obtained after calculation by modules such as self-attention. The method based on graph neural networks aims to use the powerful graph encoding capabilities of graph neural networks to fully learn the graph structure information of the knowledge graph, so as to more effectively learn the embedded representation of entities and relationships. The above two methods have performed well in the fields of intelligent search engines, biomedical engineering, and intelligent customer service.
[0003] In the fields of intelligent search engines, biomedical engineering, and intelligent customer service, knowledge graph embedding still faces the following challenges and shortcomings: (1) The pre-trained model is mainly based on the self-attention mechanism and is often used to process data sequences. In order to convert the graph structure information of the knowledge graph into a sequence, the corresponding entities and relations in the triples need to be converted into text sequences containing semantic information, which often loses or weakens the local or structural features of the knowledge graph, and thus cannot effectively capture the information dependency relationship over a longer distance; (2) The graph neural network-based method embeds the knowledge graph information into various high-dimensional spaces by learning triple representation, relation representation, and graph (or subgraph) representation. Although it can effectively capture the structural information of the knowledge graph, it is difficult to analyze the semantic information contained in each entity node in different contexts. (3) In the method that uses triple representation as the data processing object, the entity nodes are still the main focus, and only the relationship between entities is paid attention to, while ignoring the potential connection between the "relationship" nodes in the knowledge graph. The network composed of relationship nodes can better capture the association pattern characteristics in the knowledge graph. Summary of the invention
[0004] In view of the above, the purpose of the present invention is to provide a knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation to achieve accurate embedding representation of the knowledge graph.
[0005] To achieve the above-mentioned purpose of the invention, the embodiment provides a knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation, comprising the following steps:
[0006] Obtaining a knowledge graph and extracting a neighbor matrix and a diffusion path dictionary from the knowledge graph, wherein the neighbor matrix includes an entity neighbor matrix of an entity node and a relationship neighbor matrix of a relationship node, and the diffusion path dictionary includes an entity diffusion path dictionary and a relationship diffusion path dictionary;
[0007] Initialize the embedded representation of the node, where the node includes an entity node and a relationship node, and map the embedded representation of the relationship node to the feature space of the entity node according to the current diffusion distance to obtain the mapped embedded representation;
[0008] At each diffusion distance, the diffusion attention embedding of each node and all K-neighboring neighbor nodes under the corresponding path list is calculated through the diffusion attention calculation module based on the neighbor matrix, diffusion path dictionary and the mapped embedding representation. The node attention embedding of each node is calculated through the node attention calculation module based on the self-interlaced attention mechanism based on all the diffusion attention embeddings.
[0009] According to the node attention embedding of each node, the semantic distance attention calculation module is used to perform fusion at different diffusion distances to obtain the final embedding representation of the node.
[0010] Preferably, the step of acquiring the knowledge graph and extracting a neighbor matrix and a diffusion path dictionary therefrom comprises:
[0011] Extracting entity neighbor matrix of entity nodes from knowledge graph and entity diffusion path dictionary ,
[0012] ;
[0013] ;
[0014] ;
[0015] in, represents the maximum diffusion distance considered when aggregating neighbor node features, Indicates i The distance between entity nodes is The set of neighbor nodes of Represents the set of all entity nodes, Indicates the distance k The set of all neighbor nodes of the entity node, Indicates the length , starting from i entity nodes, the end point is j The path set of entity nodes stores the relationship nodes that the path passes through in the order of traversal;
[0016] Analyze the knowledge graph to get the relational knowledge graph, in which the relations are nodes and entities are edges. Extract the relational neighbor matrix of the relational nodes based on the relational knowledge graph and relation diffusion path dictionary ,
[0017] ;
[0018] ;
[0019] ;
[0020] in, Represents the set of all relationship nodes, Indicates The distance between the relationship nodes is The set of neighbor nodes of Indicates the distance k The set of all neighbor nodes of the relationship node, Indicates the length , starting from i relationship nodes, the end point is A path set of relationship nodes, storing the entity nodes passed by the path in the order of traversal.
[0021] Preferably, the embedding representation of the initialization node includes:
[0022] The embedding representation of the node is initialized by using a randomly distributed embedding method, where the random distribution includes a Gaussian distribution.
[0023] Preferably, mapping the embedded representation of the relationship node to the feature space of the entity node according to the current diffusion distance to obtain the mapped embedded representation includes:
[0024] ;
[0025] ;
[0026] in, and Respectively represent the entity node set No. i The initial embedding representation and the embedded representation after mapping of entity nodes, and Respectively represent the set of nodes belonging to the relationship No. i The initial embedding representation and the embedded representation after mapping of the relationship nodes. Indicates that the diffusion distance is k The corresponding projection transformation matrix.
[0027] Preferably, the step of calculating the diffuse attention embedding of each node and all K-neighboring neighbor nodes under the corresponding path list by the diffuse attention calculation module includes:
[0028] ;
[0029] in, Indicates diffusion distance The diffuse attention calculation module under Represents a splicing operation, It represents the embedded representation of all heterogeneous nodes diffused from the target node to be calculated to the K-neighboring neighbor nodes. When the target node is an entity node, the corresponding heterogeneous node is a relationship node. When the target node is a relationship node, the corresponding heterogeneous node is an entity node. represents the diffuse attention of each node;
[0030] In the diffuse attention calculation module, query the parameter matrix , key parameter matrix and the value parameter matrix , remember the tensor , first calculate the tensor The corresponding query tensor , key tensor With value tensor :
[0031] ;
[0032] ;
[0033] ;
[0034] The diffusion distance is then calculated Corresponding diffuse attention under a single diffuse path :
[0035] ;
[0036] in, Representing a tensor The dimension size of express Activation function;
[0037] Finally, combine each node in the diffusion distance The corresponding diffuse attention under all diffuse paths obtains the diffuse attention embedding under the corresponding path list.
[0038] Preferably, the node attention embedding of each node is calculated by a node attention calculation module based on a self-interleaving attention mechanism according to all diffuse attention embeddings, comprising:
[0039] ;
[0040] in, Indicates diffusion distance The node attention calculation module based on the self-staggered attention mechanism under and Indicates the first i Nodes and j The mapped embedding representation of the nodes, the operator Represents a splicing operation, Indicates the diffusion distance Next i Nodes and j The diffuse attention embedding between nodes, represents the node attention of each node;
[0041] In the node attention calculation module based on the self-staggered attention mechanism, the query parameter matrix , key parameter matrix and the value parameter matrix , remember the tensor , first calculate the tensor The corresponding query tensor , key tensor With value tensor :
[0042] ;
[0043] ;
[0044] ;
[0045] The diffusion distance is then calculated Node attention with a single endpoint :
[0046] ;
[0047] Finally, combine each node in the diffusion distance The node attention of all corresponding terminal nodes obtains the node attention embedding of the node.
[0048] Preferably, the node attention embedding of each node is fused at different diffusion distances through a semantic distance attention calculation module to obtain a final embedding representation of the node, including:
[0049] Complete all nodes within the diffusion distance After calculating the node attention under , in the semantic distance attention calculation module, first, the diffusion distance is calculated The semantic distance attention parameter under :
[0050] ;
[0051] in, Indicates the diffusion distance The following belongs to the node set No. i The node attention embedding of nodes, , , is the parameter of the semantic distance attention calculation module, the superscript T represents transposition, Indicates the total number of nodes in the node set. Any activation function;
[0052] After all semantic distance attention parameters are calculated, the diffusion distance is recorded The node attention embedding of all nodes of the same type is a tensor , the semantic distance attention parameter , get the updated semantic distance attention parameter , and then the updated semantic distance attention parameters under different propagation distances are Tensor corresponding to node attention Multiply them together to get the final feature embedding of the node :
[0053] ;
[0054] ;
[0055] in, K Indicates the maximum diffusion distance.
[0056] Preferably, the diffusion attention calculation module, the node attention calculation module based on the self-interlaced attention mechanism, and the semantic distance attention calculation module need to be parameter optimized before being referenced, and the loss function used for the specific optimization is:
[0057] ;
[0058] ;
[0059] in, Indicates that it belongs to an entity set Entity node The final embedding representation of Entity tags in knowledge graphs The cross entropy of Indicates that it belongs to a relation set Relationship Node The final embedding representation of Relationship labels in knowledge graphs The cross entropy of Represents a multilayer perceptron.
[0060] To achieve the above-mentioned purpose of the invention, an embodiment of the present invention provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation.
[0061] To achieve the above-mentioned purpose of the invention, an embodiment of the present invention provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the above-mentioned knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation is implemented.
[0062] Compared with the prior art, the present invention has the following beneficial effects:
[0063] The present invention proposes a knowledge graph embedding method based on self-interleaving attention mechanism and diffusion aggregation. The method can learn the structural representation of the knowledge graph under different levels of self-attention mechanism. At the same time, by considering the knowledge graph as a knowledge graph containing two types of nodes, entity and relationship, and using the self-interleaving attention mechanism, the method not only learns the connection between entities, but also captures the connection between relationship nodes, thereby more comprehensively capturing the structural information of the knowledge graph. In addition, the present invention combines the idea of diffusion graph neural network, learns the attention weights at different semantic propagation distances, and learns the implicit semantic context information in the knowledge graph while retaining the structural features, thereby improving the accuracy of the embedded representation of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0065] Figure 1 is a flow chart of a knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation provided by an embodiment;
[0066] Figure 2 This is an example diagram of extending the knowledge graph provided by the embodiment to obtain a relational knowledge graph;
[0067] Figure 3 is a schematic diagram of the principle of a diffuse attention calculation module provided in an embodiment;
[0068] Figure 4 It is a schematic diagram of the principle of a node attention calculation module based on a self-interlaced attention mechanism provided in an embodiment. DETAILED DESCRIPTION
[0069] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.
[0070] The technical concept of the present invention is as follows: the knowledge graph embedding method based on the self-interlaced attention mechanism and diffusion aggregation proposed in the embodiment of the present invention first regards the entities and relationships in the knowledge graph as two different types of nodes, stretches the knowledge graph into a heterogeneous directed graph, and analyzes the relationship knowledge graph from the entity knowledge graph contained in the original knowledge graph. Then, in order to unify the entity nodes and the relationship nodes into the same feature space, a relationship node embedding projection method based on the propagation path is proposed. Then, three levels of attention calculation modules are proposed: a diffusion attention calculation module, a node attention calculation module, and a semantic distance attention calculation module. Among them, the diffusion attention calculation module uses the self-attention mechanism to calculate the diffusion path embedding of information propagation between nodes, and uses the feature vectors of entities and relationships as calculation parameters of the diffusion attention weights. By dynamically adjusting the entity embedding space and the relationship embedding space, the structural characteristics of the knowledge graph can be more comprehensively fitted. Then, after obtaining the diffusion path embedding between nodes, the node attention calculation module uses the head and tail nodes and the diffusion path embedding as parameters, and inputs the self-interlaced attention mechanism to perform node attention calculation of the target node. The semantic distance attention calculation module uses the node embedding sets under different distance paths as parameters to calculate the embedding vectors of different semantic propagation distances, and further calculates the final embedding representation of entity nodes and relationship nodes.
[0071] like Figure 1 As shown, the embodiment provides a knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation, which is applied to the field of natural language processing, including the following steps:
[0072] Obtaining a knowledge graph and extracting a neighbor matrix and a diffusion path dictionary from the knowledge graph, wherein the neighbor matrix includes an entity neighbor matrix of an entity node and a relationship neighbor matrix of a relationship node, and the diffusion path dictionary includes an entity diffusion path dictionary and a relationship diffusion path dictionary;
[0073] Initialize the embedded representation of the node, where the node includes an entity node and a relationship node, and map the embedded representation of the relationship node to the feature space of the entity node according to the current diffusion distance to obtain the mapped embedded representation;
[0074] At each diffusion distance, the diffusion attention embedding of each node and all K-neighboring neighbor nodes under the corresponding path list is calculated through the diffusion attention calculation module based on the neighbor matrix, diffusion path dictionary and the mapped embedding representation. The node attention embedding of each node is calculated through the node attention calculation module based on the self-interlaced attention mechanism based on all the diffusion attention embeddings.
[0075] According to the node attention embedding of each node, the semantic distance attention calculation module is used to perform fusion at different diffusion distances to obtain the final embedding representation of the node.
[0076] Among them, extracting the neighbor matrix and diffusion path dictionary from the knowledge graph is the preparation work. The specific knowledge graph is the knowledge graph in the field of intelligent search, which is defined as ,in, They represent entity sets and relationship sets respectively. Entities in the entity set include real data such as proper nouns in various disciplines, names of public figures such as celebrities, games or film and television works, names of historical events, etc. Relationships in the relationship set include various descriptive predicates, such as logical relationships such as "belong to", "antonym", "synonym", or verbs such as "invent" and "direct". More specifically, for example, in the knowledge graph commonly used in the field of intelligent search, the task is to perform knowledge search based on the embedded representation after embedding the knowledge graph. Represents the set of valid triples contained in the knowledge graph, Represents the head and tail entities, Indicates the relationship between the head and tail entities, when calculating the entity node The distance is When the neighbor list of the entity node is calculated, it is calculated recursively, that is, the entity node is first calculated. Distance is Neighbor list. During the traversal process, the traversal path is recorded at the same time, and the distance between each pair of entity nodes has been obtained. After obtaining the neighbor list and propagation path dictionary of the entity node, the relationship-entity-relationship triples can be constructed according to the original triple set to obtain the relationship knowledge graph, and the relationship node neighbor list and propagation path dictionary can be calculated.
[0077] Specifically, the entity neighbor matrix of the entity node is obtained by traversing the triple set :
[0078] ;
[0079] ;
[0080] in, represents the maximum diffusion distance considered when aggregating neighbor node features, Indicates i The distance between entity nodes is The set of neighbor nodes of Represents the set of all entity nodes, Indicates the distance k The set of all neighbor nodes of the entity node is composed of the knowledge graph entity set During the traversal process, the path information is recursively recorded to obtain the entity propagation path dictionary of the entity node. :
[0081] ;
[0082] Indicates the length , starting from i entity nodes, the end point is j The path set of entity nodes is stored in the form of a list array in the order of traversal. Dictionary of propagation paths with entities After that, the relational knowledge graph is obtained by analyzing the entity knowledge graph (knowledge graph), for example Figure 2 As shown in the relational knowledge graph, with relations as nodes and entities as edges, the relational neighbor matrix of the relational nodes can be further obtained based on the triple set. and relation propagation path dictionary :
[0083] ;
[0084] ;
[0085] ;
[0086] in, Represents the set of all relationship nodes, Indicates The distance between the relationship nodes is The set of neighbor nodes of Indicates the distance k The set of all neighbor nodes of the relationship node, Indicates the length , starting from i relationship nodes, the end point is The path set of relationship nodes is stored in the form of a list array in the order of traversal.
[0087] Based on the above preparations, the embedding process of the knowledge graph includes:
[0088] Step 1: Set the maximum transmission distance , It means that a node can reach its neighbor node through at most several heterogeneous nodes (when the starting point is an entity node, the heterogeneous node refers to the relationship node, otherwise it refers to the entity node). Needs to be satisfied , initialize the variable , initialize the embedding representation of all entity nodes Embedding representation of relation nodes , the embedding dimension of the entity node is , the embedding dimension of the relationship node is , Represents an embedding method under any random distribution, such as a Gaussian distribution:
[0089] ;
[0090] ;
[0091] Step 2: If , the calculation ends and proceed directly to step 19; otherwise, proceed to step 3;
[0092] Step 3: Since entity nodes and relationship nodes are in different feature spaces, and usually the number of entity nodes is much larger than that of relationship nodes, that is, , so we choose to embed the initial relationship node Project to the feature space of sparser entity nodes and keep the embedding representation of entity nodes constant:
[0093] ;
[0094] ;
[0095] Indicates that the semantic distance is The corresponding projection transformation matrix is randomly initialized as A tensor of size, which is one of the model parameters;
[0096] Step 4: Initialize variables ;
[0097] Step 5: If , the calculation ends and proceed directly to step 16; otherwise, take out the entity node , take the diffusion distance from the entity neighbor matrix as Neighbor list ;
[0098] Step 6: Initialize variables ;
[0099] Step 7: If , the calculation ends and proceeds directly to step 14; otherwise, the entity node is taken out from the diffusion path dictionary , The diffusion distance is List of paths ;
[0100] Step 8: Initialize variables ;
[0101] Step 9: If , the calculation ends and proceed directly to step 11; otherwise, the diffuse attention embedding of the entity node is performed Calculation:
[0102] ;
[0103] in, Indicates diffusion distance The diffuse attention calculation module of the entity node under express Included in The mapped embedded representation of the relationship nodes; operator Represents a tensor concatenation operation. Specifically, Figure 3 As shown, the diffuse attention calculation module contains three model parameters, the query parameter matrix , key parameter matrix and the value parameter matrix , remember tensor , first calculate The corresponding query tensor , key tensor With value tensor :
[0104] ;
[0105] ;
[0106] ;
[0107] Then calculate the entity node , On the path The diffuse attention embedding of:
[0108] ;
[0109] Step 10: Variables , return to step 9;
[0110] Step 11: Record entity nodes , In the path list All diffuse attention under is a tensor :
[0111] ;
[0112] First, the tensor Perform layer normalization:
[0113] ;
[0114] Specifically, for a tensor One of the No. In terms of dimension:
[0115] ;
[0116] Then the tensor Add by column , and conduct a , get the entity node , In the path list Diffuse Attention Embedding :
[0117] ;
[0118] Step 12: Perform node attention on entity nodes Calculation:
[0119] ;
[0120] in Indicates diffusion distance The node attention calculation module based on the self-interleaving attention mechanism is as follows: Figure 4 As shown in the figure, the node attention calculation module based on the self-interlaced attention mechanism contains three model parameters, the query parameter matrix , key parameter matrix and the value parameter matrix , remember tensor , first calculate The corresponding query tensor , key tensor With value tensor :
[0121] ;
[0122] Then calculate the entity node , In the diffusion distance The following node attention:
[0123] ;
[0124] Step 13: Variables , return to step 7;
[0125] Step 14: Record entity nodes and distance The attention of all neighbor nodes is a tensor :
[0126]
[0127] First, the tensor Perform layer normalization:
[0128] ;
[0129] Specifically, for a tensor One of the No. In terms of dimension:
[0130] ;
[0131] Then the tensor Add by column and do one , get the entity node and distance The node attention embedding of all neighboring nodes :
[0132] ;
[0133] in, Represents a multidimensional tensor Each term in is summed along the first dimension;
[0134] Step 15: Variables , return to step 5;
[0135] Step 16: Perform the calculations from step 5 to step 15 on the relationship node. The calculation process is the same as that of the entity node. You only need to replace the operation data and parameters of the relationship node with the entity node. I will not repeat them here. Finally, you can get the relationship node Diffusion distance The node attention embedding of all neighboring nodes ;
[0136] Step 17: Calculate the diffusion distance Entity semantic distance attention parameters under and relational semantic distance attention parameters :
[0137] ;
[0138] ;
[0139] in , , , , , are model parameters, , , , is a column vector, , is the parameter matrix; It can be any activation function, for example , Indicates i Node attention embedding of entity nodes, For the i Node attention embedding of relation nodes;
[0140] Step 18: Variables , return to step 2;
[0141] Step 19: Note the distance The node attention embedding of all entity nodes is a tensor , the node attention embeddings of all relationship nodes are tensors , and the entity semantic distance attention parameter Attention parameters with relational semantic distance conduct , get the updated entity semantic distance attention parameter and relational semantic distance attention parameters ;
[0142] ;
[0143] ;
[0144] The final feature embedding of the entity node is obtained by multiplying the semantic distance attention parameter at different propagation distances with the tensor Final feature embedding with relation nodes , respectively:
[0145] ;
[0146] ;
[0147] It should also be noted that the above-mentioned diffusion attention calculation module, the node attention calculation module based on the self-interlaced attention mechanism, and the semantic distance attention calculation module need to be parameter optimized before being referenced. The specific loss function used for optimization is:
[0148] ;
[0149] ;
[0150] in, Indicates that it belongs to an entity set Entity node The final embedding representation of and the entity label in the knowledge graph The cross entropy of Indicates that it belongs to a relation set Relationship Node The final embedding representation of and the relationship label in the knowledge graph The cross entropy of Represents a multilayer perceptron that projects node embeddings into a dictionary space for cross entropy calculation.
[0151] The knowledge graph embedding method provided in the above embodiment can be used as an upstream method of a natural language processing module such as a pre-training model, and the entity embedding representation and relationship embedding representation obtained by the present invention are reprocessed to fine-tune the downstream model for various tasks, including triple classification of the knowledge graph in the field of biomedical engineering, link prediction and other knowledge graph completion tasks in the field of intelligent customer service; on the other hand, the present invention is not limited to the traditional knowledge graph. For nodes mixed with other description types, such as time labels, etc., the knowledge graph can still be extended into a heterogeneous graph for subsequent processing according to the method described in the present invention, and all other types of node features except themselves are used as heterogeneous nodes of the self-interlaced attention mechanism as diffuse attention calculation parameters. Finally, the present invention is applicable to one-to-many, many-to-one, many-to-many and multiple relationship graph structures. Therefore, compared with the prior art, the present invention has better extensibility and application prospects.
[0152] Based on the same inventive concept, an embodiment also provides a computing device, including a memory and one or more processors, wherein executable code is stored in the memory, and when one or more processors execute the executable code, the steps of the above-mentioned knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation are performed.
[0153] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the steps of the above-mentioned knowledge graph embedding method based on self-interleaving attention mechanism and diffusion aggregation. Of course, in addition to the software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0154] Based on the same inventive concept, an embodiment also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the steps of the knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation mentioned in the claim are implemented.
[0155] In an embodiment, computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0156] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation, characterized in that: For intelligent search, the following steps are included: Obtaining a knowledge graph, which is a knowledge graph in the field of intelligent search, including an entity set and a relationship set, wherein the entities in the entity set include real data, which include proper nouns of various disciplines, names of public figures, games or film and television works, and names of historical events, and the relationships in the relationship set include various descriptive predicates, specifically including logical relationships or verbs, wherein the logical relationships include belonging, antonyms, and synonyms, and the verbs include invention and director; and extracting a neighbor matrix and a diffusion path dictionary from the knowledge graph, wherein the neighbor matrix includes an entity neighbor matrix of an entity node and a relationship neighbor matrix of a relationship node, and the diffusion path dictionary includes an entity diffusion path dictionary and a relationship diffusion path dictionary; Initialize the embedded representation of the node, where the node includes an entity node and a relationship node, and map the embedded representation of the relationship node to the feature space of the entity node according to the current diffusion distance to obtain the mapped embedded representation; At each diffusion distance, the diffusion attention embedding of each node and all K-neighboring neighbor nodes under the corresponding path list is calculated through the diffusion attention calculation module based on the neighbor matrix, diffusion path dictionary and the mapped embedding representation. The node attention embedding of each node is calculated through the node attention calculation module based on the self-interlaced attention mechanism based on all the diffusion attention embeddings. According to the node attention embedding of each node, the semantic distance attention calculation module is used to perform fusion at different diffusion distances to obtain the final embedding representation of the node, including: Complete all nodes within the diffusion distance After calculating the node attention under , in the semantic distance attention calculation module, first, the diffusion distance is calculated The semantic distance attention parameter under : ; in, Indicates the diffusion distance The following belongs to the node set No. i The node attention embedding of nodes, , , is the parameter of the semantic distance attention calculation module, the superscript T represents transposition, Indicates the total number of nodes in the node set. Any activation function; After all semantic distance attention parameters are calculated, the diffusion distance is recorded The node attention embedding of all nodes of the same type is a tensor , the semantic distance attention parameter , get the updated semantic distance attention parameter , and then the updated semantic distance attention parameters under different propagation distances are Tensor corresponding to node attention Multiply them together to get the final feature embedding of the node : ; ; in, K Indicates the maximum diffusion distance.
2. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The step of obtaining a knowledge graph and extracting a neighbor matrix and a diffusion path dictionary therefrom includes: Extracting entity neighbor matrix of entity nodes from knowledge graph and entity diffusion path dictionary , ; ; ; in, represents the maximum diffusion distance considered when aggregating neighbor node features, Indicates i The distance between entity nodes is The set of neighbor nodes of Represents the set of all entity nodes, Indicates the distance k The set of all neighbor nodes of the entity node, Indicates the length , starting from i entity nodes, the end point is j The path set of entity nodes stores the relationship nodes that the path passes through in the order of traversal; Analyze the knowledge graph to get the relational knowledge graph, in which the relations are nodes and entities are edges. Extract the relational neighbor matrix of the relational nodes based on the relational knowledge graph and relation diffusion path dictionary , ; ; ; in, Represents the set of all relationship nodes, Indicates The distance between the relationship nodes is The set of neighbor nodes of Indicates the distance k The set of all neighbor nodes of the relationship node, Indicates the length , starting from i relationship nodes, the end point is A path set of relationship nodes, storing the entity nodes passed by the path in the order of traversal.
3. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The embedded representation of the initialization node includes: The embedding representation of the node is initialized by using a randomly distributed embedding method, where the random distribution includes a Gaussian distribution.
4. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The step of mapping the embedded representation of the relationship node to the feature space of the entity node according to the current diffusion distance to obtain the mapped embedded representation includes: ; ; in, and Respectively represent the entity node set No. i The initial embedding representation and the embedded representation after mapping of entity nodes, and Respectively represent the set of nodes belonging to the relationship No. i The initial embedding representation and the embedded representation after mapping of the relationship nodes. Indicates that the diffusion distance is k The corresponding projection transformation matrix.
5. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The diffuse attention calculation module calculates the diffuse attention embedding of each node and all K-neighboring neighbor nodes under the corresponding path list, including: ; in, Indicates diffusion distance The diffuse attention calculation module under Represents a splicing operation, It represents the embedded representation of all heterogeneous nodes diffused from the target node to be calculated to the K-neighboring neighbor nodes. When the target node is an entity node, the corresponding heterogeneous node is a relationship node. When the target node is a relationship node, the corresponding heterogeneous node is an entity node. represents the diffuse attention of each node; In the diffuse attention calculation module, query the parameter matrix , key parameter matrix and the value parameter matrix , remember the tensor , first calculate the tensor The corresponding query tensor , key tensor With value tensor : ; ; ; The diffusion distance is then calculated Corresponding diffuse attention under a single diffuse path : ; in, Representing a tensor The dimension size of express Activation function; Finally, combine each node in the diffusion distance The corresponding diffuse attention under all diffuse paths obtains the diffuse attention embedding under the corresponding path list.
6. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The node attention embedding of each node is calculated by a node attention calculation module based on a self-interleaving attention mechanism according to all diffuse attention embeddings, including: ; in, Indicates diffusion distance The node attention calculation module based on the self-staggered attention mechanism under and Indicates the first i Nodes and j The mapped embedding representation of the nodes, the operator Represents a splicing operation, Indicates the diffusion distance Next i Nodes and j The diffuse attention embedding between nodes, represents the node attention of each node; In the node attention calculation module based on the self-staggered attention mechanism, the query parameter matrix , key parameter matrix and the value parameter matrix , remember the tensor , first calculate the tensor The corresponding query tensor , key tensor With value tensor : ; ; ; The diffusion distance is then calculated Node attention with a single endpoint : ; Finally, combine each node in the diffusion distance The node attention of all corresponding terminal nodes obtains the node attention embedding of the node.
7. The knowledge graph embedding method based on self-staggered attention mechanism and diffusion aggregation according to claim 1 is characterized in that: The diffusion attention calculation module, the node attention calculation module based on the self-interlaced attention mechanism, and the semantic distance attention calculation module need to be parameter optimized before being referenced. The specific loss function used for optimization is: ; ; in, Indicates that it belongs to an entity set Entity node The final embedding representation of Entity tags in knowledge graphs The cross entropy of Indicates that it belongs to a relation set Relationship Node The final embedding representation of Relationship labels in knowledge graphs The cross entropy of Represents a multilayer perceptron.
8. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation as described in any one of claims 1-7.
9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the knowledge graph embedding method based on self-interlaced attention mechanism and diffusion aggregation as described in any one of claims 1-7 is implemented.
