Relation perception learning method of time sequence knowledge graph based on attention mechanism

Abstract

The invention relates to a relationship perception learning method of a time sequence knowledge graph based on an attention mechanism, which comprises the following steps of: firstly, acquiring a target relationship, a neighbor entity and an embedding vector of time sequence information of the neighbor entity through an embedding layer, and dynamically adjusting a weight according to the similarity of the target relationship and the neighbor relationship in a time dimension by utilizing the attention mechanism; therefore, the time sequence information of the neighbors is finely aggregated. Then, a fitness function is introduced to score negative samples, and the most challenging negative sample is selected to realize local optimization; in addition, a specially designed positive and negative sample processing mechanism effectively improves the time sequence discrimination capability of the relation perception embedding model on the negative sample, so that the modeling effect on the time sequence relation is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph representation learning, and in particular to a relationship-aware learning method of temporal knowledge graphs based on an attention mechanism. Background Art

[0002] In the study of temporal knowledge graphs, entities and relations are usually represented by embedding vectors in order to capture the interactions and associations between them in high-dimensional space. However, as the scale and complexity of temporal knowledge graphs increase, entities and relations become complex not only in space, but also affected by the temporal dimension. It is difficult to accurately capture the fine-grained changes of entities and relations at different time points by simply relying on global structure or local neighborhood information. In this case, a single global or local feature cannot fully express the dynamic interactions between entities and their neighbors in the temporal knowledge graph, which may lead to a decrease in prediction accuracy.

[0003] Therefore, how to effectively aggregate the temporal information of the target relation and its neighbor entities and relations becomes a key challenge to improve the prediction performance of the relation-aware embedding model. This requires the relation-aware embedding model to not only capture the temporal correlation between the target entity and its direct neighbors, but also to dynamically aggregate the temporal information related to these neighbors and reflect the complexity and temporal changes of the relation through deep aggregation techniques. Traditional methods may not be able to handle this kind of spatiotemporal dynamic aggregation and perform poorly when faced with complex temporal relations. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a relationship-aware learning method for a temporal knowledge graph based on an attention mechanism. By introducing embedding vectors of target relations and neighbor relations and combining them with embedding vectors of neighbor entities, the relationship-aware embedding model can more effectively utilize the complex interactions between entities and relations in the graph structure; in particular, the attention mechanism allows the relationship-aware embedding model to dynamically assign weights according to different neighbor relations, thereby capturing more fine-grained relationship information between entities; this helps to improve the performance of the relationship-aware embedding model on complex dynamic knowledge graphs, especially in link prediction and relationship prediction tasks.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: The temporal knowledge graph relationship perception learning and optimization method based on the attention mechanism has the following steps: Step 1: Initialize the embedding of entities and relations and generate the corresponding embedding matrix; the embedding vector of each entity and relation will be used as input for the subsequent attention mechanism operation; Step 2: By randomly sampling entities as negative samples, historical information with similar structures to positive samples is generated for these negative samples, so that the relation-aware embedding model can better learn to distinguish between positive and negative samples; Step 3: Combine negative samples with periodic and non-periodic temporal features and static features, and normalize the temporal features and static features to generate time-related entity embeddings; Step 4: Locally optimize the entity embedding of negative samples, that is, introduce a fitness function, score the negative samples, and select the most challenging negative samples. The most challenging negative samples refer to those negative samples that are closest to the positive samples in the feature space; then generate historical information related to the negative samples, including historical relations, historical entities, and historical timestamps; Step 5: Computation of relation-aware embeddings for positive and negative samples: The relation-aware embeddings for positive and negative samples are calculated by aggregating the embeddings of the target relation with its neighbor relations and neighbor entities. First, the relation-aware embedding model calculates the similarity based on the target relation and the neighbor relations, and weights the embeddings of the neighbor entities through the attention mechanism. Then, the relation-aware embedding model aggregates these weighted embedding vectors to generate relation-aware embedding representations. This process is applied to positive and negative samples respectively to obtain the relation-aware embeddings of positive and negative samples, which are used for subsequent score calculations and relation-aware embedding model optimization. Step 6: Calculate the loss function: Calculate the relationship-aware scores of positive and negative samples, and use the cross-entropy loss function to measure the ability to distinguish between positive and negative samples. The loss function combines the basic score and the relationship-aware score in a weighted manner to obtain a global loss value to guide the optimization of the relationship-aware embedding model. Step 7: Relation-aware embedding model optimization: Use the back-propagation algorithm to optimize the relationship-aware embedding model and learn the optimal embedding representations of entities and relations by minimizing the loss function; these embedding representations are ultimately used for representation learning of the knowledge graph. Preferably, step 2 comprises the following sub-steps: Step 2.1: First, find all timestamps earlier than the current time from the entity's history, and ensure that the number of timestamps returned is fixed; Step 2.2: If the entity has no associated history at a certain point in time, fill in the default entities and relationships; Step 2.3: Finally, convert all historical relations, entities, and timestamps into tensors to ensure that their dimensions are consistent with the positive samples.

[0006] Preferably, step 3 includes the following sub-steps: Step 3.1: By combining the time-related embedding vector with time t and transforming it using the sine function sin, the periodic changes of entities and relations in the time dimension are captured, thereby achieving the process of periodic embedding. The formula is: ; in, , , Represent the embedding of year, month, and day respectively. The normalization function F.normalize is used to maintain the unit length of the vector. Represents the periodic embedding of entities and relations in the time dimension; Step 3.2: By combining the time-related embedding vector with time t and transforming it using the hyperbolic tangent function tanh, we can capture the changes in entities and relationships that are not periodic in the time dimension but have long-term trends, thereby achieving the process of non-periodic embedding. The formula is: ; in, , , Represent the non-periodic year, month, and day embeddings respectively, and use Represents aperiodic embeddings of entities and relations in the time dimension; Step 3.3: The static embedding process is achieved by assigning a fixed embedding vector to each entity, which does not change over time and reflects the inherent characteristics of the entity or relationship. The formula is: ; in, For static embedding of entities, use Represents a static embedding of an entity or relation.

[0007] Preferably, step 4 includes the following sub-steps: Step 4.1: For a given negative sample embedding and , first calculate the L2 norm of each embedding vector: , ; in, and are the subject and object embedding vectors of the negative sample, respectively, and d is the dimension of the embedding; the L2 norm measures the length of the vector, using represents the L2 norm of each embedding vector; Step 4.2: You need to sort all the calculated norms and select the smallest top k negative samples. The formula is: , ; in, is the subject negative sample, is the object negative sample, k is the number of optimal negative samples to be selected, Indicates selecting k indexes that minimize the norm; Step 4.3: Get the embedding vector of the corresponding best negative sample by index, the formula is: , ; Among them, the embedding vectors of the best negative sample subject and object are and ; Step 4.4: Generate historical information for a given negative sample entity before a specific time point, where the historical information includes historical relations, historical entities, and historical timestamps.

[0008] Preferably, step 5 includes the following sub-steps: Step 5.1: The formula for calculating the similarity between the target relationship and the neighbor relationship using the attention mechanism is: ; Among them, S represents the target relationship Relationship with neighbors The similarity score matrix between is the embedding vector of the target relation, with the shape , that is, the target relationship vector of each sample, with dimension d, is the set of embedding vectors of neighbor relationships, with the shape , T is the time step or sequence length, N is the number of neighbor relations, and d is the dimension of the embedding vector; Step 5.2: The formula for nonlinear processing of the similarity score using the LeakyReLU activation function is: ; That is, when the similarity When it is positive, it remains unchanged; when the similarity When it is negative, press Reduce; among them, is the similarity score between the target relation and the neighbor relation, is the similarity score after LeakyReLU activation, is the negative slope of LeakyReLU, which is used to control the scaling of the negative part; Step 5.3: Normalize the similarity scores through Softmax, and get the formula of attention weight for each neighbor relationship: ; in, It's a neighbor relationship The attention weight, is the similarity score after LeakyReLU activation, N is the number of neighbor relations, indicating the total number of neighbor relations connected to the entity, Softmax normalizes all similarity scores of neighbor relations to ensure that the weight of each neighbor relation is between 0 and 1, and the sum of all weights is 1; after Softmax normalization, the weight of each neighbor relation is It can indicate the degree of attention of the target relationship to different neighbor relationships, which is used in the subsequent weighted summation process; Step 5.4: Use the local optimization strategy to fine-tune the attention weight and embedding aggregation strategy to achieve better aggregation effect. The formula is: ; in, Embedding for target relation With each neighbor embedding The cosine similarity between , is the original attention weight, is the adjusted attention weight; Step 5.5: The formula for weighted summing of the neighbor entity embeddings to get the final aggregate embedding vector is: ; in, is the aggregated embedding representation, is the attention weight of the i-th neighbor relationship, indicating the strength of association between the target entity and the i-th neighbor, is the embedding vector of the i-th neighbor entity, and N is the total number of neighbor entities; this formula indicates that through the attention weight Embedding for each neighbor entity Perform weighted summation to finally obtain the aggregate representation of the target entity , which integrates the relationships between the target entity and all neighbor entities.

[0009] Preferably, the process of calculating the loss function in step 6 includes: Step 6.1: The formula for the subject and object scores of the positive sample is: ; ; in, is the embedding vector of the subject entity of the i-th positive sample, is the embedding vector of the object entity of the i-th positive sample, is the aggregate embedding of the subject neighbor entities of the i-th positive sample, is the aggregate embedding of the object neighbor entities of the i-th positive sample, is the Sigmoid activation function; Step 6.2: The formula for the subject and object scores of negative samples is: ; ; in, is the embedding vector of the subject entity of the i-th negative sample, is the embedding vector of the object entity of the i-th negative sample, is the aggregate embedding of the subject neighbor entities of the i-th negative sample, is the aggregate embedding of the object neighbor entities of the i-th negative sample; Step 6.3: Use the Sigmoid activation function and take the logarithm of the scores of the subject and object of the positive sample. The formula for calculating the loss is: ; Where M is the number of positive samples, is the score of the i-th positive sample subject, is the score of the i-th positive sample object; Step 6.4: Use the Sigmoid activation function to the scores of the subject and object of the negative sample and take the logarithm, and negate the scores of the negative samples. The formula for calculating the loss of the negative samples is: ; Among them, N is the number of positive samples, and K is the number of negative samples corresponding to each positive sample; and are the subject and object scores of the jth negative sample corresponding to the ith positive sample.

[0010] A relation-aware learning system for temporal knowledge graphs based on an attention mechanism, comprising: Relation and neighbor attention aggregation module: used for the aggregation of target relations and neighbor information, weighted aggregation is performed through the embedding vector of the target relation and the embedding of its related neighbor relations and neighbor entities to obtain an enhanced target entity representation; Similarity and attention weight calculation module: used to generate attention weights for neighbor relations, calculate the similarity between target relations and neighbor relations, and assign attention weights based on the similarity; Attention weight adjustment module: This module makes the relationship-aware embedding model pay more attention to neighbors with stronger relationships with the target entity. It integrates the local optimization strategy and adjusts the original attention weights based on the cosine similarity between the neighbor entity and the target relationship to further strengthen the focus on important neighbors. Relation-aware embedding calculation module: used to calculate the perceptual embedding of the target entity under a given relationship, including the perceptual embedding of the subject and object of positive and negative samples, which is used for subsequent score calculation and loss function optimization; Embedding expansion and comparison module for positive and negative samples: used to expand the embedding vector of the positive sample in the positive and negative samples so that it can be compared with each negative sample, thereby distinguishing the characteristics of the positive and negative samples.

[0011] A computer device comprising: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the relation-aware learning of the temporal knowledge graph based on the attention mechanism as described is implemented.

[0012] A computer-readable storage medium having a computer program stored thereon, which, when executed, implements the relation-aware learning of a temporal knowledge graph based on an attention mechanism as described above.

[0013] The present invention can achieve the following beneficial effects: 1. By introducing the embedding vectors of target relations and neighbor relations, combined with the embedding vectors of neighbor entities, the relationship-aware embedding model can more effectively utilize the complex interactions between entities and relations in the graph structure. In particular, the attention mechanism allows the relationship-aware embedding model to dynamically assign weights according to different neighbor relations, thereby capturing more fine-grained relationship information between entities. This helps to improve the performance of the relationship-aware embedding model on complex dynamic knowledge graphs, especially in link prediction and relationship prediction tasks.

[0014] 2. By aggregating historical neighbor entities and their relationships related to the target entity, a more detailed representation can be constructed for the target entity. This multi-level representation method enhances the expressiveness of the relationship-aware embedding model and can better capture the evolutionary patterns of entities and relationships in the temporal knowledge graph. Based on the relationship-aware embedding of positive and negative samples, the relationship-aware embedding model can distinguish the different characteristics of positive and negative samples by comparing the embeddings of target relations and neighbor relations in positive and negative samples. The added calculation of relationship-aware scores for positive and negative samples further improves the ability of the relationship-aware embedding model to distinguish between positive and negative samples, helping the relationship-aware embedding model to converge faster during training and improve prediction accuracy.

[0015] 3. The introduction of local optimization strategies can enhance the global search capability of knowledge graph embedding learning and improve the exploratory nature of relationship-aware embedding models in complex optimization spaces. At the same time, by combining with the attention mechanism, more adaptive and dynamic optimization can be achieved, thereby effectively improving the quality and performance of knowledge graph representation.

[0016] 4. Based on the aggregation method of relationship and neighbor information, the loss function not only includes the basic positive and negative sample comparison, but also adds the relationship-aware score. This multi-task loss design idea can help the relationship-aware embedding model to more comprehensively optimize the dynamic evolution of entity-relationships, so that the relationship-aware embedding model can capture global and local structural changes while maintaining good generalization capabilities. By using the attention mechanism to weight neighbor relationships, the relationship-aware embedding model can flexibly select the neighbor information that is most helpful for the current task and avoid information redundancy. This dynamic adjustment mechanism enables the relationship-aware embedding model to more efficiently utilize the embedded information of neighbor relationships when processing knowledge graphs with dynamic evolution characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 This is a flow chart of the relationship-aware learning method of the temporal knowledge graph based on the attention mechanism of the present invention; Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0018] The preferred solution is Figure 1 to Figure 2 As shown in the figure, the relation-aware learning method of temporal knowledge graph based on attention mechanism is shown in the figure. Figure 1 , including the following steps: Step 1: Initialize the embedding vector, embed the entities and relations, and generate the corresponding embedding matrix. The embedding vector of each entity and relation will be used as input for the subsequent attention mechanism operation.

[0019] Step 2: By randomly sampling entities as negative samples, historical information with similar structures to positive samples will be generated for these negative samples, so that we can better learn to distinguish between positive and negative samples.

[0020] Step 3: Combine negative samples with periodic and non-periodic temporal features and static features, and normalize these features to generate time-dependent entity embeddings.

[0021] Step 4: Locally optimize the entity embedding of negative samples, that is, introduce a fitness function, score the negative samples, and select the most challenging negative samples, that is, those negative samples that are closest to the positive samples in the feature space; then generate historical information related to the negative samples, including historical relations, historical entities, and historical timestamps.

[0022] Step 5: Computation of relation-aware embeddings for positive and negative samples. Computing relation-aware embeddings for positive and negative samples is done by aggregating the embeddings of the target relation with its neighbor relations and neighbor entities. First, the relation-aware embedding model calculates the similarity based on the target relation and the neighbor relations, and weights the embeddings of the neighbor entities through the attention mechanism. Then, the relation-aware embedding model aggregates these weighted embedding vectors to generate a relation-aware embedding representation. This process is applied to positive and negative samples respectively to obtain their relation-aware embeddings, which are used for subsequent score calculations and relation-aware embedding model optimization.

[0023] Step 6: Calculate the loss function, calculate the relationship-aware scores of positive and negative samples, and use the cross-entropy loss function to measure the ability to distinguish between positive and negative samples; the loss function combines the basic score and the relationship-aware score in a weighted manner to finally obtain a global loss value to guide the optimization of the relationship-aware embedding model.

[0024] Step 7: Relation-aware embedding model optimization. Use the back-propagation algorithm to optimize the relationship-aware embedding model. By minimizing the loss function, the optimal embedding representations of entities and relationships are learned. These embedding representations are ultimately used for representation learning of knowledge graphs and play an important role in downstream tasks (link prediction, relationship prediction, and future prediction).

[0025] Specifically, in step 2, historical information with a similar structure to that of the positive samples is generated for these negative samples. The positive samples directly obtain the actual relationships and entities from the historical records of the entities, while the negative samples are different, specifically including: Step 2.1: First, find all timestamps earlier than the current time from the entity's history, and ensure that the number of timestamps returned is fixed; Step 2.2: If the entity has no associated history at a certain point in time, fill in the default entities and relationships; Step 2.3: Finally, convert all historical relations, entities, and timestamps into tensors to ensure that their dimensions are consistent with the positive samples.

[0026] Specifically, in step 3, the negative samples are combined with periodic and non-periodic time features and static features, and these features are normalized to generate time-related entity embeddings, which specifically includes: Step 3.1: By combining the time-related embedding vector (such as year, month, day) with time t and transforming it using the sine function sin, the periodic changes of entities and relationships in the time dimension are captured, thereby achieving the process of periodic embedding. The formula is:

[0027] in, , , Represent the embedding of year, month, and day respectively. The normalization function F.normalize is used to maintain the unit length of the vector. Represents the periodic embedding of entities and relationships in the time dimension;.

[0028] Step 3.2: By combining the time-related embedding vector (such as year, month, day) with time t and transforming it using the hyperbolic tangent function tanh, we can capture the changes in entities and relationships that are not periodic in the time dimension but have long-term trends, thereby achieving the process of non-periodic embedding. The formula is:

[0029] in, , , Represent the non-periodic year, month, and day embeddings respectively, and use Represents aperiodic embeddings of entities and relations in the time dimension.

[0030] Step 3.3: The static embedding process is achieved by assigning a fixed embedding vector to each entity, which does not change over time and reflects the inherent characteristics of the entity or relationship. The formula is:

[0031] in, For static embedding of entities, use Represents a static embedding of an entity or relation.

[0032] Step 3.4: The final embedding vector is obtained by adding the periodic, non-periodic and static parts. This embedding captures the periodic and non-periodic changes of the entity in different time dimensions as well as the time-invariant static features. The formula is:

[0033] Specifically, the process of performing local optimization in step 4, i.e., introducing a fitness function, scoring negative samples, and selecting the most challenging negative samples specifically includes: Step 4.1: For a given negative sample embedding (negative embedding of the subject) and (negative embeddings of the object), we first calculate the L2 norm of each embedding vector: ,

[0034] in, and are the subject and object embedding vectors of the negative sample, respectively, and d is the dimension of the embedding. The L2 norm measures the length of the vector, using represents the L2 norm of each embedding vector.

[0035] Step 4.2: You need to sort all the calculated norms and select the smallest top k negative samples. The formula is: ,

[0036] in, is the subject negative sample, is the object negative sample, k is the number of optimal negative samples to be selected, It means to select k indices that minimize the norm.

[0037] Step 4.3: Get the embedding vector of the corresponding best negative sample by index, the formula is: ,

[0038] Among them, the embedding vectors of the best negative sample subject and object are and .

[0039] Step 4.4: Generate historical information for a given negative sample entity before a specific time point, including historical relations, historical entities, and historical timestamps.

[0040] Specifically, the neighbor aggregation based on the graph neural network mechanism in step 5 uses the idea of ​​graph neural network (GNN) to aggregate the target relationship and its neighbor relationship to generate the representation of the target entity. This process ensures that the relationship-aware embedding model can dynamically focus on the neighbor relationship that is most relevant to the current task, thereby giving these relationships greater weight when aggregating information. Specifically, it includes: Step 5.1: Here, the formula for calculating the similarity between the target relationship and the neighbor relationship using the attention mechanism is:

[0041] Among them, S represents the target relationship Relationship with neighbors The similarity score matrix between is the embedding vector of the target relation, with the shape , that is, the target relationship vector of each sample, with dimension d, is the set of embedding vectors of neighbor relationships, with the shape , T is the time step or sequence length, N is the number of neighbor relations, and d is the dimension of the embedding vector.

[0042] Step 5.2: The formula for nonlinear processing of the similarity score using the LeakyReLU activation function is:

[0043] That is, when the similarity When it is positive, it remains unchanged; when the similarity When it is negative, press Narrow down. Among them, is the similarity score between the target relation and the neighbor relation, is the similarity score after LeakyReLU activation, is the negative slope of LeakyReLU (set to 0.2 in the code), which is used to control the scaling of the negative part.

[0044] Step 5.3: Normalize the similarity scores through Softmax, and get the formula of attention weight for each neighbor relationship:

[0045] in, It's a neighbor relationship The attention weight, is the similarity score after LeakyReLU activation, N is the number of neighbor relationships, indicating the total number of neighbor relationships connected to the entity, and Softmax normalizes all similarity scores of neighbor relationships to ensure that the weight of each neighbor relationship is between 0 and 1, and the sum of all weights is 1. After Softmax normalization, the weight of each neighbor relationship is It can indicate the degree of attention of the target relationship to different neighbor relationships and is used in the subsequent weighted summation process.

[0046] Step 5.4: Use the local optimization strategy to fine-tune the attention weight and embedding aggregation strategy to achieve better aggregation effect. The formula is:

[0047] in, Embedding for target relation With each neighbor embedding The cosine similarity between , is the original attention weight, is the adjusted attention weight.

[0048] Step 5.5: The formula for weighted summing of the neighbor entity embeddings to get the final aggregate embedding vector is:

[0049] in, is the aggregated embedding representation (weighted embedding of neighbor entities), is the attention weight of the i-th neighbor relationship, indicating the strength of association between the target entity and the i-th neighbor, is the embedding vector of the i-th neighbor entity, and N is the total number of neighbor entities. This formula indicates that through the attention weight Embedding for each neighbor entity Perform weighted summation to finally obtain the aggregate representation of the target entity , which integrates the relationships between the target entity and all neighbor entities.

[0050] Specifically, the process of calculating the loss function in step 6 specifically includes: Step 6.1: The formula for the subject and object scores of the positive sample is:

[0051]

[0052] in, is the embedding vector of the subject entity of the i-th positive sample, is the embedding vector of the object entity of the i-th positive sample, is the aggregate embedding of the subject neighbor entities of the i-th positive sample (the embedding obtained through the attention mechanism), is the aggregate embedding of the object neighbor entities of the i-th positive sample, is the Sigmoid activation function.

[0053] Step 6.2: The formula for the subject and object scores of negative samples is:

[0054]

[0055] in, is the embedding vector of the subject entity of the i-th negative sample, is the embedding vector of the object entity of the i-th negative sample, is the aggregate embedding of the subject neighbor entities of the i-th negative sample, is the aggregate embedding of the object neighbor entities of the i-th negative sample.

[0056] Step 6.3: Use the Sigmoid activation function and take the logarithm of the scores of the subject and object of the positive sample. The formula for calculating the loss is:

[0057] Where M is the number of positive samples, is the score of the i-th positive sample subject, is the score of the i-th positive sample object.

[0058] Step 6.4: Use the Sigmoid activation function to log the scores of the subject and object of the negative sample, and negate the scores of the negative sample (i.e. and represents a negative sample), the formula for calculating the loss of negative samples is:

[0059] Among them, N is the number of positive samples, and K is the number of negative samples corresponding to each positive sample. and are the subject and object scores of the jth negative sample corresponding to the ith positive sample.

[0060] Step 6.5: The final total loss is the weighted sum of the positive and negative sample losses. The scores are processed through the Sigmoid activation function and normalized and logarithmized to calculate the final loss formula of the relation-aware embedding model:

[0061] The above is a description of the proposed relationship-aware learning method for a temporal knowledge graph based on an attention mechanism and the specific implementation methods of each module in combination with the accompanying drawings. The beneficial effect of the present invention is that by introducing the embedding vectors of the target relationship and the neighbor relationship, combined with the embedding vectors of the neighbor entity, the relationship-aware embedding model can more effectively utilize the complex interaction between entities and relationships in the graph structure. By aggregating the historical neighbor entities and their relationships related to the target entity of the positive and negative samples, a more detailed representation can be constructed for the target entity. This multi-level representation method enhances the expressive power of the relationship-aware embedding model, further improves the ability of the relationship-aware embedding model to distinguish between positive and negative samples, and can better capture the evolutionary pattern of entities and relationships in the temporal knowledge graph. By using the attention mechanism to weight the neighbor relationship, the relationship-aware embedding model can flexibly select the neighbor information that is most helpful to the current task and avoid information redundancy. A local optimization strategy is introduced to optimize the attention weight distribution, and the weight of the neighbor relationship and entity in the embedded representation is further improved. The attention mechanism can enhance the representation of the relationships and entities most relevant to the target entity by adaptively weighting the local information, and achieve better generalization effect by optimizing these weight distributions locally.

[0062] The present invention provides a relationship-aware learning system for a temporal knowledge graph based on an attention mechanism, comprising the following modules: The relation and neighbor attention aggregation module is used to aggregate target relation and neighbor information. It performs weighted aggregation on the embedding vector of the target relation and the embeddings of its related neighbor relations and neighbor entities to obtain an enhanced target entity representation.

[0063] The similarity and attention weight calculation module is used to generate the attention weight of the neighbor relationship, calculate the similarity between the target relationship and the neighbor relationship, and assign the attention weight according to the similarity.

[0064] The attention weight adjustment module makes the relationship-aware embedding model pay more attention to neighbors with stronger relationships with the target entity. It integrates the local optimization strategy and adjusts the original attention weights based on the cosine similarity of the relationship between the neighbor entity and the target to further strengthen the focus on important neighbors.

[0065] The relation-aware embedding calculation module calculates the perceptual embedding of the target entity under a given relation, including the perceptual embedding of the subject and object of positive and negative samples, for subsequent score calculation and loss function optimization.

[0066] The embedding expansion and comparison module of positive and negative samples expands the embedding vector of the positive sample so that it can be compared with each negative sample to distinguish the characteristics of the positive and negative samples.

[0067] A computer device, characterized in that it comprises: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the relation-aware learning of the temporal knowledge graph based on the attention mechanism as described is implemented.

[0068] A computer-readable storage medium having a computer program stored thereon, which, when executed, implements the relation-aware learning of a temporal knowledge graph based on an attention mechanism as described above.

[0069] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A relation-aware learning method for temporal knowledge graphs based on an attention mechanism, characterized by: The following steps are involved: Step 1: Initialize the embedding of entities and relations and generate the corresponding embedding matrix; the embedding vector of each entity and relation will be used as input for the subsequent attention mechanism operation; Step 2: By randomly sampling entities as negative samples, historical information with similar structures to positive samples is generated for these negative samples, so that the relation-aware embedding model can better learn to distinguish between positive and negative samples; Step 3: Combine negative samples with periodic and non-periodic temporal features and static features, and normalize the temporal features and static features to generate time-related entity embeddings; Step 4: Locally optimize the entity embedding of negative samples, that is, introduce a fitness function, score the negative samples, and select the most challenging negative samples. The most challenging negative samples refer to those negative samples that are closest to the positive samples in the feature space; then generate historical information related to the negative samples, including historical relations, historical entities, and historical timestamps; Step 5: Computation of relation-aware embeddings for positive and negative samples: The relation-aware embeddings for positive and negative samples are calculated by aggregating the embeddings of the target relation with its neighbor relations and neighbor entities. First, the relation-aware embedding model calculates the similarity based on the target relation and the neighbor relations, and weights the embeddings of the neighbor entities through the attention mechanism. Then, the relation-aware embedding model aggregates these weighted embedding vectors to generate relation-aware embedding representations. This process is applied to positive and negative samples respectively to obtain the relation-aware embeddings of positive and negative samples, which are used for subsequent score calculations and relation-aware embedding model optimization. Step 6: Calculate the loss function: Calculate the relationship-aware scores of positive and negative samples, and use the cross-entropy loss function to measure the ability to distinguish between positive and negative samples. The loss function combines the basic score and the relationship-aware score in a weighted manner to obtain a global loss value to guide the optimization of the relationship-aware embedding model. Step 7: Relation-aware embedding model optimization: Use the back-propagation algorithm to optimize the relationship-aware embedding model and learn the optimal embedding representations of entities and relations by minimizing the loss function; these embedding representations are ultimately used for representation learning of the knowledge graph.

2. The relationship-aware learning method of the temporal knowledge graph based on the attention mechanism according to claim 1 is characterized in that: The step 2 includes the following sub-steps: Step 2.1: First, find all timestamps earlier than the current time from the entity's history, and ensure that the number of timestamps returned is fixed; Step 2.2: If the entity has no associated history at a certain point in time, fill in the default entities and relationships; Step 2.3: Finally, convert all historical relations, entities, and timestamps into tensors to ensure that their dimensions are consistent with the positive samples.

3. The relationship-aware learning method of the temporal knowledge graph based on the attention mechanism according to claim 1 is characterized in that: The step 3 includes the following sub-steps: Step 3.1: By combining the time-related embedding vector with time t and transforming it using the sine function sin, the periodic changes of entities and relations in the time dimension are captured, thereby achieving the process of periodic embedding. The formula is: ； in, , , Represent the embedding of year, month, and day respectively. The normalization function F.normalize is used to maintain the unit length of the vector. Represents the periodic embedding of entities and relations in the time dimension; Step 3.2: By combining the time-related embedding vector with time t and transforming it using the hyperbolic tangent function tanh, we can capture the changes in entities and relationships that are not periodic in the time dimension but have long-term trends, thereby achieving the process of non-periodic embedding. The formula is: ； in, , , Represent the non-periodic year, month, and day embeddings respectively, and use Represents aperiodic embeddings of entities and relations in the time dimension; Step 3.3: The static embedding process is achieved by assigning a fixed embedding vector to each entity, which does not change over time and reflects the inherent characteristics of the entity or relationship. The formula is: ； in, For static embedding of entities, use Represents a static embedding of an entity or relation.

4. The relationship-aware learning method of the temporal knowledge graph based on the attention mechanism according to claim 1 is characterized in that: The step 4 includes the following sub-steps: Step 4.1: For a given negative sample embedding and , first calculate the L2 norm of each embedding vector: ， ； in, and are the subject and object embedding vectors of the negative sample, respectively, and d is the dimension of the embedding; the L2 norm measures the length of the vector, using represents the L2 norm of each embedding vector; Step 4.2: You need to sort all the calculated norms and select the smallest top k negative samples. The formula is: ， ； in, is the subject negative sample, is the object negative sample, k is the number of optimal negative samples to be selected, Indicates selecting k indexes that minimize the norm; Step 4.3: Get the embedding vector of the corresponding best negative sample by index, the formula is: ， ； Among them, the embedding vectors of the best negative sample subject and object are and ; Step 4.4: Generate historical information for a given negative sample entity before a specific time point, where the historical information includes historical relations, historical entities, and historical timestamps.

5. The relationship-aware learning method of the temporal knowledge graph based on the attention mechanism according to claim 1 is characterized in that: The step 5 comprises the following sub-steps: Step 5.1: The formula for calculating the similarity between the target relationship and the neighbor relationship using the attention mechanism is: ； Among them, S represents the target relationship Relationship with neighbors The similarity score matrix between is the embedding vector of the target relation, with the shape , that is, the target relationship vector of each sample, with dimension d, is the set of embedding vectors of neighbor relationships, with the shape , T is the time step or sequence length, N is the number of neighbor relations, and d is the dimension of the embedding vector; Step 5.2: The formula for nonlinear processing of the similarity score using the LeakyReLU activation function is: ； That is, when the similarity When it is positive, it remains unchanged; when the similarity When it is negative, press Reduce; among them, is the similarity score between the target relation and the neighbor relation, is the similarity score after LeakyReLU activation, is the negative slope of LeakyReLU, which is used to control the scaling of the negative part; Step 5.3: Normalize the similarity scores through Softmax, and get the formula of attention weight for each neighbor relationship: ； in, It's a neighbor relationship The attention weight, is the similarity score after LeakyReLU activation, N is the number of neighbor relations, indicating the total number of neighbor relations connected to the entity, Softmax normalizes all similarity scores of neighbor relations to ensure that the weight of each neighbor relation is between 0 and 1, and the sum of all weights is 1; after Softmax normalization, the weight of each neighbor relation is It can indicate the degree of attention of the target relationship to different neighbor relationships, which is used in the subsequent weighted summation process; Step 5.4: Use the local optimization strategy to fine-tune the attention weight and embedding aggregation strategy to achieve better aggregation effect. The formula is: ； in, Embedding for target relation With each neighbor embedding The cosine similarity between , is the original attention weight, is the adjusted attention weight; Step 5.5: The formula for weighted summing of the neighbor entity embeddings to get the final aggregate embedding vector is: ； in, is the aggregated embedding representation, is the attention weight of the i-th neighbor relationship, indicating the strength of association between the target entity and the i-th neighbor, is the embedding vector of the i-th neighbor entity, and N is the total number of neighbor entities; this formula indicates that through the attention weight Embedding for each neighbor entity Perform weighted summation to finally obtain the aggregate representation of the target entity , which integrates the relationships between the target entity and all neighbor entities.

6. The method for learning and optimizing temporal knowledge graph relationships based on attention mechanism according to claim 1, characterized in that: The process of calculating the loss function in step 6 includes: Step 6.1: The formula for the subject and object scores of the positive sample is: ； ； in, is the embedding vector of the subject entity of the i-th positive sample, is the embedding vector of the object entity of the i-th positive sample, is the aggregate embedding of the subject neighbor entities of the i-th positive sample, is the aggregate embedding of the object neighbor entities of the i-th positive sample, is the Sigmoid activation function; Step 6.2: The formula for the subject and object scores of negative samples is: ； ； in, is the embedding vector of the subject entity of the i-th negative sample, is the embedding vector of the object entity of the i-th negative sample, is the aggregate embedding of the subject neighbor entities of the i-th negative sample, is the aggregate embedding of the object neighbor entities of the i-th negative sample; Step 6.3: Use the Sigmoid activation function and take the logarithm of the scores of the subject and object of the positive sample. The formula for calculating the loss is: ； Where M is the number of positive samples, is the score of the i-th positive sample subject, is the score of the i-th positive sample object; Step 6.4: Use the Sigmoid activation function to the scores of the subject and object of the negative sample and take the logarithm, and negate the scores of the negative samples. The formula for calculating the loss of the negative samples is: ； Among them, N is the number of positive samples, and K is the number of negative samples corresponding to each positive sample; and are the subject and object scores of the jth negative sample corresponding to the ith positive sample.

7. A relation-aware learning system for temporal knowledge graphs based on an attention mechanism, characterized by: include: Relation and neighbor attention aggregation module: used for the aggregation of target relations and neighbor information, weighted aggregation is performed through the embedding vector of the target relation and the embedding of its related neighbor relations and neighbor entities to obtain an enhanced target entity representation; Similarity and attention weight calculation module: used to generate attention weights for neighbor relations, calculate the similarity between target relations and neighbor relations, and assign attention weights based on the similarity; Attention weight adjustment module: This module makes the relationship-aware embedding model pay more attention to neighbors with stronger relationships with the target entity. It integrates the local optimization strategy and adjusts the original attention weight based on the cosine similarity of the relationship between the neighbor entity and the target, further strengthening the focus on important neighbors. Relation-aware embedding calculation module: used to calculate the perceptual embedding of the target entity under a given relationship, including the perceptual embedding of the subject and object of positive and negative samples, which is used for subsequent score calculation and loss function optimization; Embedding expansion and comparison module for positive and negative samples: used to expand the embedding vector of the positive sample in the positive and negative samples so that it can be compared with each negative sample, thereby distinguishing the characteristics of the positive and negative samples.

8. A computer device, characterized in that: include: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the relation-aware learning of the temporal knowledge graph based on the attention mechanism as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed, implements relation-aware learning of a temporal knowledge graph based on an attention mechanism as described in any one of claims 1-6.

Images