Knowledge representation learning model construction method and system based on periodic perception contrast graph attention network

Abstract

The invention provides a method and a system for constructing a knowledge representation learning model based on a periodic perception contrast graph attention network. The method comprises the steps of constructing a sub-graph sampler, constructing a graph attention network and performing time sequence perception contrast learning. According to the method, firstly, adjacent neighbor entities are screened and queried through a period time sensing sub-graph sampler in a period dynamic weighting mode, consumption of computing resources is reduced, and key time sequence context and period information are reserved; then, the graph attention network with the time perception function generates dynamic representation of time perception by using graph attention in the constructed query related sub-graph, and entity embedding representation fused with neighbor information is obtained; then, time sequence perception contrast learning improves robustness in time knowledge graph reasoning through dynamic and static representation contrast learning of an entity, and entity prediction is carried out through embedded representation obtained through learning. Finally, the entity representation is decoded using a decoder.

Description

Technical Field

[0001] This invention proposes a method and system for constructing a knowledge representation learning model based on a periodic perception contrast graph attention network, which relates to the field of computer technology. Background Technology

[0002] Temporal knowledge graphs (TKGs) store massive amounts of knowledge in the form of quadruplets (s, r, o, t), effectively enhancing the dynamic representation capabilities of static knowledge graphs by incorporating a temporal dimension. However, TKGs generally suffer from incompleteness, severely limiting their application potential in fields such as intelligent question answering and event prediction. Temporal knowledge graph reasoning (TKGR) aims to predict missing quadruplets (s, r, o, t) by learning the joint representation of entities, relations, and timestamps in a temporal knowledge graph. Since facts in a temporal knowledge graph are only meaningful at specific moments or time intervals, TKGR needs to learn not only structural and semantic information but also model the temporal information within the knowledge.

[0003] One of the strategies of existing TKGR methods is to integrate temporal information into the static knowledge graph reasoning framework. By designing a time encoder to obtain the semantic features of timestamps, they can then interact with entity and relation representations to achieve more efficient reasoning. For example, TTrassE[1] and TA-DistMult[2] extend the classic static KGR model, namely TransE[3] and DistMult[4]. They encode temporal information into relation-like translations by adding temporal constraints and use RNN[5] to move the representations of these translations in the embedding space. TComplEx and TNTComplEx[6] are both based on ComplEx[7]. The difference with ComplEx is that the fourth-order tensor space of the additional temporal information is modeled by them. ChronoR[8] is built on RotatE[9] and connects relation embeddings and temporal embeddings to make the overall rotation embedding applied to the final entity embedding. TuckERTNT

[10] proposes a new tensor decomposition model inspired by fourth-order tensor Tucker decomposition for temporal knowledge graphs and introduces additional temporal embeddings. HyTE

[11] represents each timestamp as a learnable hyperplane in the embedding space and projects entity and relation embeddings onto the hyperplane, then evaluates these embeddings using the TransE scoring function on the projection. TRHyTE

[12] uses GRU to transform entities into a latent space and then encodes facts into a temporal relation hyperplane for temporal relation-aware representation generation. ATiSE

[13] maps entity and relation embeddings of a temporal knowledge graph to a Gaussian space based on timestamps. To mine the dynamic graph evolution of a temporal knowledge graph, DyERNIE

[14] defines a velocity vector in the tangent space over time for embedding.

[0004] TeMP

[15] uses message-passing graph neural networks (MPNNs) to learn structured entity representations for each timestamp, and then the encoder combines the representations of all timestamps. TPmod

[16] uses a finer-grained mechanism to utilize the dynamic interaction of entities and relations. T-GAP

[17] uses an attention mechanism to iteratively update the query-related subgraph to achieve path-based reasoning. TARGCN

[18] connects facts at different timestamps by sampling to capture the interaction of facts at different timestamps. TARGAT

[19] designs a time-aware relation graph attention network to model relation and timestamp information, and uses a time-aware graph attention layer to generate a transformation matrix to capture the multi-fact interaction at different timestamps. GLARGCN

[20] proposes to combine global historical event frequency and local event relative displacement to capture important neighbors to reduce the number of training parameters, and combines attention mechanism and graph convolutional network to fuse entity neighborhood information.

[0005] Furthermore, knowledge representation learning for the inference task of temporal knowledge graphs has been widely applied in many fields. However, due to the current limitations of the extrapolation model's performance, which is insufficient to support downstream applications, the following shortcomings still exist and require further improvement.

[0006] 1) Low training efficiency. Although models that incorporate temporal information into static knowledge representation learning frameworks can easily model time dependencies, they often require the introduction of redundant time encoding modules for complex cycles (such as the Olympics being held every four years, futures delivery date rules, etc.), leading to an increase in training parameters and a decline in training efficiency.

[0007] 2) Limited ability to model long-distance temporal dependencies. Existing graph neural network models capture dynamic structural features by aggregating domain information from multiple time snapshots, but they usually need to traverse multi-hop neighbors across the entire graph and have limited ability to model long-distance temporal dependencies, which exacerbates computational complexity and memory consumption.

[0008] 3) Poor robustness. In temporal knowledge graph reasoning tasks, model robustness is one of the key factors determining its practical application value. Mainstream temporal knowledge graph reasoning methods focus on optimizing temporal feature representation and structural dependency modeling to improve prediction accuracy. However, model robustness as a guarantee of accurate prediction is rarely considered. Existing models are weak in noise resistance and cannot effectively guarantee accurate predictions with noisy data.

[0009] Therefore, how to improve the training efficiency of the model, enhance the ability to capture long-distance temporal dependencies, and improve the robustness of the model are the key research contents of this invention. Summary of the Invention

[0010] In view of this, to fill the gaps and deficiencies in existing technologies, this invention proposes a method and system for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network. This method and system first uses a periodic time-aware subgraph sampler to filter and query neighboring entities through periodic dynamic weighting, reducing computational resource consumption while retaining key temporal context and periodic information. Next, a time-aware graph attention network generates a time-aware dynamic representation using graph attention in the constructed query-related subgraph, obtaining an entity embedding representation that integrates neighbor information. Then, time-aware contrastive learning improves the robustness of temporal knowledge graph reasoning by comparing the dynamic and static representations of entities, and uses the learned embedding representation for entity prediction. Finally, a decoder decodes the entity representation.

[0011] This invention proposes a knowledge representation learning model based on a periodic-aware contrastive graph attention network, characterized by the following: the knowledge representation learning model follows an encoder-decoder architecture; the knowledge representation learning model includes the following modules:

[0012] Subgraph sampler: The periodic time-aware subgraph sampler filters and queries neighboring entities through periodic dynamic weighting, reducing computational resource consumption while preserving key temporal context and periodic information;

[0013] Graph Attention Network: A time-aware graph attention network generates a time-aware dynamic representation by using graph attention in a constructed query-related subgraph. It learns the semantic contributions of neighbors at different time distances through the graph attention network and obtains an entity embedding representation that integrates neighbor information.

[0014] Temporal-aware contrastive learning: Temporal-aware contrastive learning improves the robustness of temporal knowledge graph reasoning by learning through the contrast of dynamic and static representations of entities, and finally uses the learned embedded representations for entity prediction.

[0015] This invention proposes a method for constructing a knowledge representation learning model based on a periodic perception contrast graph attention network. The method for constructing this model includes the following steps:

[0016] Step S1: Construct a subgraph sampler; the construction of the subgraph sampler includes the following:

[0017] Given a query (s) q ,r q ,? ,tq This provides periodic time-aware sampling to obtain a query-related subgraph G. sub This allows the component's subgraph sampler to model periodic events while preserving temporal proximity preferences;

[0018] Step S2: Construct a graph attention network; wherein constructing a graph attention network includes using a time encoder to encode time information and learning time-aware entity representations of different time neighbors; after obtaining time-aware entity and relation representations, graph attention is used to aggregate information of domain nodes;

[0019] Step S3: Constructing time-aware contrastive learning; Constructing time-aware contrastive learning includes, for each query (s q ,r q ,? ,t q Temporally-aware contrastive learning learns static and dynamic contrastive representations of queries by minimizing supervised contrastive loss;

[0020] Step S4: Construct the decoder and loss function.

[0021] Further, step S1 includes the following:

[0022] Step S11: Given a query (s) q ,r q , ? ,t q ), obtain s from the time knowledge graph q The set N of all neighboring nodes sq Then, the sampling probability is calculated from two aspects: monotonic time decrease and periodic recurrence, to obtain the query relevance subgraph G. sub Furthermore, the subgraph sampler first uses Fourier basis functions to encode periodicity in order to capture periodic events, including the following:

[0023]

[0024] Where Δt represents the query timestamp t q The absolute time difference with the current timestamp t |t q -t|, f represents the preset period length, and the choice of f is based on the statistical characteristics of the dataset;

[0025] Step S12: Calculate the final sampling probability by exponentially changing and normalizing the time difference, and combining this with the obtained periodic frequency value, including the following:

[0026]

[0027] Where p(o,t) represents the query entity s q The sampling probability of the neighboring entity o.

[0028] Further, step S2 includes the following:

[0029] Step S21: Employ a time encoder to encode time information and learn time-aware entity representations of different time neighbors; further, for a fact at time t, the time interval between it and the query time tq is Δt = |t|q. q If -t|, then its time embedding φ(Δt) is expressed as:

[0030]

[0031] Among them, w s w s ,β s ,β t For learnable network parameters, d represents the time embedding dimension;

[0032] Step S22: Obtain the time-aware expressions for entities and relations by combining time embedding, entity embedding, and relation embedding, including the following:

[0033]

[0034] Where || represents the vector concatenation operation, and W1 and W2 represent linear transformation matrices. These represent the time-aware dynamic representation and the static representation of an entity, respectively.

[0035] Step S23: Use graph attention to aggregate information from the domain nodes, including calculating the attention coefficient between each node for a quadruple (s,r,o,t):

[0036]

[0037] in It is a linear transformation matrix. This represents the attention parameter vector, and || represents the concatenation operation;

[0038] Step S24: For the set of neighbor nodes N s After normalization, we get:

[0039]

[0040] in,

[0041] Step S25: Update the features of the center node s using normalized attention coefficients, including the following:

[0042]

[0043] in This represents the time-aware representation of entity s at timestamp t. It includes information about time neighbors and also reflects the impact of dynamic changes over time.

[0044] Further, step S3 includes the following:

[0045] Step S31: Define static embedding z static and dynamic embedding z time , where static embedding z static By learning the inherent attributes of entities and relationships through a time-unconstrained embedding layer, z is dynamically embedded. time An embedding representation with temporal information is generated through a time-aware graphical attention network; where the static embedding z static and dynamic embedding z time The expressions include the following:

[0046]

[0047] z static =MLP[e||r]

[0048] MLP stands for Projection Layer Mapped to Unit Hypersphere.

[0049] Furthermore, step S3 also includes the following:

[0050] Step S32: Static and dynamic representations can be viewed as dual-view representations encoding facts in a temporal knowledge graph. The static and dynamic representations of the same query are used as positive samples to obtain... Static and dynamic representations of different queries were used as negative samples.

[0051] Step S33: Use the static representation as the anchor point to query time t q Supervision and comparison of losses The calculations include the following:

[0052]

[0053] Q tq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product;

[0054] Step S34: Using the dynamic representation as the anchor point, the loss can be obtained. Includes the following:

[0055]

[0056] Qtq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product;

[0057] Step S35: Calculate the average of the two losses to obtain the following:

[0058]

[0059] in, This represents the final comparative learning loss.

[0060] Further, step S4 includes the following:

[0061] Step S41: Use the ConvTransE decoder to execute the timestamp t q The entity prediction task, further, targets the query quadruple (s q r q ,o,t q The entity prediction score is:

[0062]

[0063] in This indicates the embedding of query relationships that incorporate time information. The query entity embedding σ(·) represents the sigmoid function.

[0064] Further, step S4 includes the following:

[0065] Step S42: Train using cross-entropy loss. For a query (s) q r q ,o,t q The training objective for tail entity prediction is to maximize the reduction of the loss, thus obtaining the entity prediction loss. Represented as:

[0066]

[0067] Where P(s) q r q ,o,t q ) represents the entity prediction probability score;

[0068] Step S43: Fusing the entity prediction loss and the temporal-aware contrastive learning loss, we obtain:

[0069]

[0070] in Indicates the final loss

[0071] According to a second aspect of the present invention, a system for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network includes an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements a method for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network as described in any one of the present invention.

[0072] According to a third aspect of the present invention, a system for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network includes a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a method for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network as described in any one of the present invention.

[0073] The present invention has the following advantages:

[0074] 1) This invention optimizes the sampling strategy by introducing a periodic dynamic weighted optimization sampling strategy, which reduces computational overhead and improves training efficiency while retaining key temporal context information and periodic information.

[0075] 2) This invention implements a time-aware graph attention network. It enables the model to adaptively distinguish the semantic contributions of neighbors at different temporal distances, integrating temporal information into the graph aggregation process to obtain dynamic representations of entities.

[0076] 3) This invention introduces a contrastive learning framework and designs a time-aware contrastive learning method. It performs contrastive learning through static and dynamic representations, thereby adapting to the laws of temporal evolution while preserving the core semantics of entities, thus improving the robustness of the model in temporal knowledge graph reasoning. Attached Figure Description

[0077] Figure 1 This is the overall framework diagram of the present invention.

[0078] Figure 2 This is a schematic diagram of the steps of the present invention.

[0079] Figure 3 This is a schematic diagram of a sampling case of the present invention.

[0080] Figure 4 This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention on ICEWS14 when conducting embedding dimension analysis experiments.

[0081] Figure 5This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention in YAGO11K when performing embedding dimension analysis experiments.

[0082] Figure 6 This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention on ICEWS14 when conducting sampling quantity analysis experiments.

[0083] Figure 7 This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention on YAGO11K when conducting sampling quantity analysis experiments.

[0084] Figure 8 This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention for analyzing the temperature coefficient τ on ICEWS14.

[0085] Figure 9 This is a schematic diagram of the experimental results of the knowledge representation learning model of the present invention for analyzing the temperature coefficient τ at YAGO11K. Detailed Implementation

[0086] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0087] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0088] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0089] like Figure 1 As shown, this invention proposes a knowledge representation learning model based on a periodic-aware contrastive graph attention network, comprising the following: the knowledge representation learning model follows an encoder-decoder architecture; the knowledge representation learning model includes the following modules:

[0090] Subgraph sampler: The periodic time-aware subgraph sampler filters and queries neighboring entities through periodic dynamic weighting, reducing computational resource consumption while preserving key temporal context and periodic information;

[0091] Graph Attention Network: A time-aware graph attention network generates a time-aware dynamic representation by using graph attention in a constructed query-related subgraph. It learns the semantic contributions of neighbors at different time distances through the graph attention network and obtains an entity embedding representation that integrates neighbor information.

[0092] Temporal-aware contrastive learning: Temporal-aware contrastive learning improves the robustness of temporal knowledge graph reasoning by learning through the contrast of dynamic and static representations of entities, and finally uses the learned embedded representations for entity prediction.

[0093] like Figure 2 As shown, this invention proposes a method for constructing a knowledge representation learning model based on a periodic perception contrast graph attention network, used to construct the knowledge representation learning model based on a periodic perception contrast graph attention network described in this invention. The method for constructing the knowledge representation learning model based on a periodic perception contrast graph attention network includes the following steps:

[0094] Step S1: Construct a subgraph sampler; the construction of the subgraph sampler includes the following:

[0095] Given a query (s) q ,r q , ? , t q This provides periodic time-aware sampling to obtain a query-related subgraph G. sub This allows the component's subgraph sampler to model periodic events while preserving temporal proximity preferences;

[0096] Step S2: Construct a graph attention network; wherein constructing a graph attention network includes using a time encoder to encode time information and learning time-aware entity representations of different time neighbors; after obtaining time-aware entity and relation representations, graph attention is used to aggregate information of domain nodes;

[0097] Step S3: Constructing time-aware contrastive learning; Constructing time-aware contrastive learning includes, for each query (s q r q , ? , t q Temporally-aware contrastive learning learns static and dynamic contrastive representations of queries by minimizing supervised contrastive loss;

[0098] Step S4: Construct the decoder and loss function.

[0099] In one embodiment of the present invention, step S1 includes the following:

[0100] Step S11: Given a query (s) q ,r q ,? ,t q ), obtain s from the time knowledge graphq The set N of all neighboring nodes sq Then, the sampling probability is calculated from two aspects: monotonic time decrease and periodic recurrence, to obtain the query relevance subgraph G. sub Furthermore, the subgraph sampler first uses Fourier basis functions to encode periodicity in order to capture periodic events, including the following:

[0101]

[0102] Where Δt represents the query timestamp t q The absolute time difference with the current timestamp t |t q -t|, f represents the preset period length, and the choice of f is based on the statistical characteristics of the dataset;

[0103] Step S12: Calculate the final sampling probability by exponentially changing and normalizing the time difference, and combining this with the obtained periodic frequency value, including the following:

[0104]

[0105] Where p(o,t) represents the query entity s q The sampling probability of the neighboring entity o.

[0106] In one embodiment of the present invention, step S2 includes the following:

[0107] Step S21: Employ a time encoder to encode time information and learn time-aware entity representations of different time neighbors; further, for a fact at time t, the time interval between it and the query time tq is Δt = |t|q. q If -t|, then its time embedding φ(Δt) is expressed as:

[0108]

[0109] Among them, w s w s ,β s ,β t For learnable network parameters, d represents the time embedding dimension;

[0110] Step S22: Obtain the time-aware expressions for entities and relations by combining time embedding, entity embedding, and relation embedding, including the following:

[0111]

[0112] Where || represents the vector concatenation operation, and W1 and W2 represent linear transformation matrices. These represent the time-aware dynamic representation and the static representation of an entity, respectively.

[0113] Step S23: Use graph attention to aggregate information from the domain nodes, including calculating the attention coefficient between each node for a quadruple (s,r,o,t):

[0114]

[0115] in It is a linear transformation matrix. This represents the attention parameter vector, and || represents the concatenation operation;

[0116] Step S24: For the set of neighbor nodes N s After normalization, we get:

[0117]

[0118] in,

[0119] Step S25: Update the features of the center node s using normalized attention coefficients, including the following:

[0120]

[0121] in This represents the time-aware representation of entity s at timestamp t. It includes information about time neighbors and also reflects the impact of dynamic changes over time.

[0122] In one embodiment of the present invention, step S3 includes the following:

[0123] Step S31: Define static embedding z static and dynamic embedding z time , where static embedding z static By learning the inherent attributes of entities and relationships through a time-unconstrained embedding layer, z is dynamically embedded. time An embedding representation with temporal information is generated through a time-aware graphical attention network; where the static embedding z static and dynamic embedding z time The expressions include the following:

[0124]

[0125] z static =MLP[e||r]

[0126] MLP stands for Projection Layer Mapped to Unit Hypersphere.

[0127] In one embodiment of the present invention, step S3 further includes the following:

[0128] Step S32: Static and dynamic representations can be viewed as dual-view representations encoding facts in a temporal knowledge graph. The static and dynamic representations of the same query are used as positive samples to obtain... Static and dynamic representations of different queries were used as negative samples.

[0129] Step S33: Use the static representation as the anchor point to query time t q Supervision and comparison of losses The calculations include the following:

[0130]

[0131] Q tq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product;

[0132] Step S34: Using the dynamic representation as the anchor point, the loss can be obtained. Includes the following:

[0133]

[0134] Q tq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product;

[0135] Step S35: Calculate the average of the two losses to obtain the following:

[0136]

[0137] in, This represents the final comparative learning loss.

[0138] In one embodiment of the present invention, step S4 includes the following:

[0139] Step S41: Use the ConvTransE decoder to execute the timestamp t q The entity prediction task, further, targets the query quadruple (s q r q ,o,t q The entity prediction score is:

[0140]

[0141] in This indicates the embedding of query relationships that incorporate time information. The query entity embedding σ(·) represents the sigmoid function.

[0142] In one embodiment of the present invention, step S4 includes the following:

[0143] Step S42: Train using cross-entropy loss. For a query (s) q r q ,o,t q The training objective for tail entity prediction is to maximize the reduction of the loss, thus obtaining the entity prediction loss. Represented as:

[0144]

[0145] Where P(s) q r q ,o,t q ) represents the entity prediction probability score;

[0146] Step S43: Fusing the entity prediction loss and the temporal-aware contrastive learning loss, we obtain:

[0147]

[0148] in Indicates the final loss

[0149] According to a second aspect of the present invention, a system for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network includes an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements a method for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network as described in any one of the present invention.

[0150] According to a third aspect of the present invention, a system for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network includes a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a method for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network as described in any one of the present invention.

[0151] In addition to the above, the present invention also provides some embodiments.

[0152] The present invention also provides the following related embodiments:

[0153] In one embodiment of the present invention, query(s) q r q ,? ,t q () represents a case that the model needs to predict. Examples are as follows:

[0154] For query(s) q r q , ? , t q For example: (Olympics, held in, ?, xx year), where "?" represents the host country, we get (Olympics, held in, country A, xx year); that is, we get a complete quadruple (s, r, o, t).

[0155] In one embodiment of the present invention, the minibatch in step S33 means: when training a neural network or other machine learning model, the entire training dataset is usually divided into multiple small subsets, each of which is called a "minibatch".

[0156] In one embodiment of the present invention, the ConvTransE decoder in step S4 is a decoder structure that combines convolution operations and the TransE concept, and is mainly used for tasks such as knowledge graph completion.

[0157] In one embodiment of the present invention, in order to comprehensively evaluate the performance of the temporal knowledge representation (PCGAT) model based on the periodic awareness contrast graph attention network, the present invention selected three datasets, namely ICEWS14, ICEWS05-15 and YAGO11K

[22] . Each dataset was divided into training set, validation set and test set according to the proportion. The specific division results and statistical data are shown in Table 1.

[0158] Entity Number Relationships training set Validation set test set ICEWS14 6,869 230 74,826 8,941 8,963 ICEWS05-15 10,094 251 366,962 46,275 46,092 YAGO11K 10,623 10 16,408 2,050 2,051

[0159] Table 1. Statistical data of the time knowledge graph dataset.

[0160] In one embodiment of the present invention, two evaluation metrics are used: mean regression ranking (MRR) and the hit rate of the top N rankings (Hits@N). These two metrics are widely used to evaluate the effectiveness of temporal knowledge representation learning methods on temporal knowledge graph reasoning tasks.

[0161] MRR is the inverse average of the rankings of the true values ​​of all queries. A higher MRR indicates a higher ranking for the model and better predictive performance. The specific formula is as follows:

[0162]

[0163] Where |Q| is the size of the query set, and rank k i This represents the rank of the correct answer to the i-th query in the inference results. Hits@N calculates whether the model contains the correct answer in the first N inference results. If the correct answer is within the first N predictions, it is counted as a hit. Hit@N measures the model's ability to capture the correct answer in the first N positions. The specific formula is as follows:

[0164]

[0165] Where is the indicator function when rank i The value is 1 when ≤ N, indicating that the correct answer is among the first N reasoning results; otherwise, it is 0.

[0166] In one embodiment of the present invention, in order to verify the effectiveness of the PCGAT method proposed in this invention in temporal knowledge graph reasoning, the present invention compares the proposed model order with some classic static knowledge graph reasoning (SKGR) and temporal knowledge graph reasoning (TKGR), and also with some latest temporal knowledge graph reasoning models.

[0167] In one embodiment of the invention, the model is implemented using PyTorch, and experiments are run on an RTX3090. For all datasets, the embedding size d for entities and relations is set to 300, and the learning rate is set to 0.001. To avoid oversampling entity neighbors and introducing noise, and to ensure parameter efficiency, the number of neighbor samples for each entity in the three datasets is set to 40. Adam is used to optimize the model parameters during training. The temperature coefficients τ for the comparative learning of ICEWS14, ICEWS05-15, and YAGO11K are 0.05, 0.07, and 0.07, respectively. For the decoder on all datasets, the number of kernels is set to 50, and the size of the one-dimensional convolutional kernel is 5×5.

[0168] In one embodiment of the present invention, Tables 2 and 3 summarize the results of PCGAT and baseline models on three public datasets for link prediction tasks. PCGAT achieved the best performance compared with other baselines, verifying the effectiveness of the method proposed in this invention.

[0169] Traditional SKGR methods are not well-suited for temporal knowledge graphs containing dynamic event information, often resulting in poor performance in temporal knowledge graph reasoning. However, the method of this invention utilizes temporal encoding to cleverly integrate temporal information into entities and relationships, allowing entities and relationships to carry different semantic information at different timestamps.

[0170]

[0171]

[0172] Table 2. Link prediction experimental results on the ICEWS14 and ICEWS05-15 datasets.

[0173]

[0174] Table 3 shows the experimental results of link prediction on the YAGO11K dataset.

[0175] In summary, PCGAT overcomes the limitation of traditional sampling methods in capturing periodic events through its periodic-aware subgraph sampling strategy. Simultaneously, it effectively integrates neighborhood information using time-aware graph attention, resulting in better entity representation. Furthermore, the time-aware contrastive learning proposed in this invention accelerates model convergence and significantly improves training efficiency. This innovative method provides a new research approach for temporal knowledge graph reasoning tasks, demonstrating significant application value.

[0176] In one embodiment of the present invention, such as Figure 3 It is for query (s) q r q ,? ,t q The specific sampling case is shown below, where darker colors indicate a higher probability of being sampled. Specifically, the subgraph sampler first uses Fourier basis functions to encode periodicity to capture periodic events, as shown in the following formula:

[0177]

[0178] Where Δt represents the query timestamp t q The absolute time difference with the current timestamp t |t q -t|, where f represents the preset period length, and the choice of f is based on the statistical characteristics of the dataset. For example, in the ICEWS14 dataset, events tend to exhibit a more weekly pattern, so the value of f is 7. In addition, to accommodate multiple period scales, multiple variables can be stacked to ensure that the weights are non-negative.

[0179] In one embodiment of the present invention, to verify the effectiveness of the periodic time-aware subgraph sampler, time-aware graph attention network, time encoding, and time-aware contrastive learning designed in this invention, four variants of PCGAT were designed: "PCGAT w / o Sam" replaces the periodic time-aware subgraph sampler proposed in this invention with random sampling; "PCGAT w / o TGAT" replaces the time-aware graph attention network with simple average aggregation of neighbor information; "PCGAT w / o CL" removes the time-aware contrastive learning module; and "PCGAT w / o Time" does not use time difference encoding but uses absolute time encoding, for example, for (s... q ,t q Learn the representation of t from the time neighbor (e,t) instead of |t.q -t|.

[0180]

[0181]

[0182] Table 4 Results of the four variants on ICEWS14 and YAGO11K

[0183] The experimental results show that the complete PCGAT model achieved the best performance across all metrics, indicating that each module of the PCGAT model contributes to the model's performance. Replacing random sampling with periodic time-aware subgraph sampling reduced the MMR (Mean Relationship Ratio) of the model on the two datasets by 5.7% and 15.1%, respectively. This fully demonstrates that the sampling strategy proposed in this invention, by sampling more and closer neighbors in the temporal context and periodically sampling neighbors, allows the model to benefit more in learning entity embeddings. After removing the time-aware graph attention network and adopting the average aggregation method, the model's MMR decreased by 2.8% and 13%, respectively. This is because graph attention aggregation can dynamically allocate the weights of different temporal neighbor nodes and relationships, accurately capturing time-sensitive event dependencies, while average aggregation of neighbors struggles to adapt to dynamic semantic changes in the event dimension. Removing the temporal difference encoding resulted in a decrease in both MMR and Hits@1 on both datasets, demonstrating the effectiveness of temporal difference modeling. After removing the time-aware contrastive learning module, the MRR metric decreased by 2.1% and 4.2% respectively, while the Hits@1 metric decreased by 2.1% and 10% respectively. This is mainly attributed to the fact that contrastive learning enhances the model's robustness to noise and maintains the stability of the core semantics by constraining static and dynamic embeddings.

[0184] In one embodiment of the present invention, to verify the noise resistance capability of PCGAT, Gaussian noise was added to the PCGAT, PCGAT w / o CL, and TARGCN models for experiments. During the model input phase, Gaussian noise with a mean of 0 and a standard deviation σ∈[0.1,0.5] was applied to the entity embedding vectors to simulate observation errors or data acquisition noise in real-world scenarios. The noise intensity was dynamically normalized according to the embedding dimension to avoid introducing biases based on different entity sizes. Since the main goal is link prediction, noise was only introduced to entities, not relationships. The results are shown in Table 5, where the results marked with ▲ are the experimental results after introducing noise. Under the same noise intensity, the PCGAT, PCGAT w / o CL, and TARGCN models showed a decrease in MRR of 1.6%, 7.5%, and 10%, respectively, and a decrease in Hits@1 of 2.4%, 9.1%, and 10.6%, respectively. The experimental results clearly demonstrate that, on the ICEWS14 dataset, the PCGAT model outperforms PCGAT w / o CL and TARGCN across all metrics. This experiment verifies the effectiveness of temporal contrastive learning from the perspective of noise robustness, providing a methodological reference for temporal knowledge reasoning in complex noisy environments.

[0185]

[0186]

[0187] Table 5. Experimental Results of Gaussian Noise

[0188] In one embodiment of the present invention, different embedding dimensions affect the model's performance. This section presents experiments with different embedding dimensions on two datasets, ICEWS14 and YAGO11K. The experimental results are as follows: Figure 4 , Figure 5 As shown.

[0189] Furthermore, such as Figure 4 , Figure 5 As shown, the model's representational ability exhibits a significant variation across different dimensional spaces. When the embedding dimension gradually increases from its initial value to 200, the evaluation metrics for both independent datasets show continuous growth, reaching their optimal performance at this critical point; however, beyond this threshold, the metric curves exhibit a reverse fluctuation. This may be because excessively high dimensionality leads to overfitting, resulting in a decrease in the model's generalization ability. The number of samples in the subgraph sampler affects the model's parameters and performance. Too many samples increase the number of parameters, impacting training efficiency, while too few samples fail to capture crucial neighborhood information, leading to information loss and affecting model performance. Therefore, experiments were conducted with different sampling numbers, and the results are shown below. Figure 6 , Figure 7 As shown.

[0190] from Figure 6 , Figure 7 The experimental results clearly show that as the number of samples increases, the model's performance gradually improves. When the number of samples reaches 40, further increasing the number of samples no longer significantly improves the model. This may be because excessive sampling introduces some noise, resulting in a slow improvement in the model's performance. Therefore, the number of samples was set to 40 in the experiments.

[0191] In one embodiment of the present invention, the temperature coefficient τ in time-aware contrastive learning also affects the model's performance. Therefore, this section presents experimental analyses of different values ​​of τ, and the specific experimental results are as follows: Figure 8 , Figure 9 As shown.

[0192] like Figure 8 , Figure 9 The experimental data shown indicate that different datasets are affected differently by the temperature coefficient. The best results are achieved when τ = 0.05 on the ICEWS14 dataset, while the best results are achieved when τ = 0.07 on the YAGO11K dataset.

[0193] In one embodiment of the present invention, the proposed periodic time-aware subgraph sampler can reduce the cost of model training and improve training efficiency, while time-aware contrastive learning can accelerate model convergence, achieving more efficient training. To verify the contributions of these two modules to model training efficiency, the proposed PCGAT model is tested on the ICEWS14 dataset, and compared with the baseline model TARGCN and the ablation version PCGAT w / o CL. The experiment mainly compares the MRR and Hits@1 metrics, while recording the number of training epochs required for model convergence and the time required for a single epoch to measure efficiency. Specific experimental results are shown in Table 6.

[0194]

[0195] Table 6 Comparison of Model Training Efficiency

[0196] Experimental results show that the proposed time-aware contrastive learning accelerates model convergence. The complete PCGAT model converges approximately 4.7 times faster than the ablation-based version of PCGAT with CL, validating the promoting effect of contrastive learning on feature representation optimization. Furthermore, the training time per epoch for TARGCN without periodic time-aware subgraph sampling is nearly 10 times that of PCGAT, indicating that the subgraph sampling method in this invention significantly contributes to model training efficiency.

[0197] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A knowledge representation learning model based on a periodic-aware contrastive graph attention network, characterized in that, The knowledge representation learning model includes the following: it follows an encoder-decoder architecture; the knowledge representation learning model includes the following modules: Subgraph sampler: The periodic time-aware subgraph sampler filters and queries neighboring entities through periodic dynamic weighting, reducing computational resource consumption while preserving key temporal context and periodic information; Graph Attention Network: A time-aware graph attention network generates a time-aware dynamic representation by using graph attention in a constructed query-related subgraph. It learns the semantic contributions of neighbors at different time distances through the graph attention network and obtains an entity embedding representation that integrates neighbor information. Temporal-aware contrastive learning: Temporal-aware contrastive learning improves the robustness of temporal knowledge graph reasoning by learning through the contrast of dynamic and static representations of entities, and finally uses the learned embedded representations for entity prediction.

2. A method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network, characterized in that, The method for constructing a knowledge representation learning model based on a periodic-aware contrast-map attention network as described in claim 1 includes the following steps: Step S1: Construct a subgraph sampler; the construction of the subgraph sampler includes the following: Given a query (s) q ,r q ,? ,t q This provides periodic time-aware sampling to obtain a query-related subgraph G. sub This allows the component's subgraph sampler to model periodic events while preserving temporal proximity preferences; Step S2: Construct a graph attention network; wherein constructing a graph attention network includes using a time encoder to encode time information and learning time-aware entity representations of different time neighbors; after obtaining time-aware entity and relation representations, graph attention is used to aggregate information of domain nodes; Step S3: Constructing a time-aware contrastive learning model; Constructing a time-aware contrastive learning model includes, for each query (s q ,r q ,? ,t q Temporally-aware contrastive learning learns static and dynamic contrastive representations of queries by minimizing supervised contrastive loss; Step S4: Construct the decoder and loss function.

3. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 2, characterized in that, Step S1 includes the following: Step S11: Given a query (s) q ,r q ,? ,t q ), obtain s from the time knowledge graph q The set N of all neighboring nodes sq Then, the sampling probability is calculated from two aspects: monotonic time decrease and periodic recurrence, to obtain the query relevance subgraph G. sub Furthermore, the subgraph sampler first uses Fourier basis functions to encode periodicity in order to capture periodic events, including the following: Where Δt represents the query timestamp t q The absolute time difference with the current timestamp t |t q -t|, f represents the preset period length, and the choice of f is based on the statistical characteristics of the dataset; Step S12: Calculate the final sampling probability by exponentially changing and normalizing the time difference, and combining this with the obtained periodic frequency value, including the following: Where p(o,t) represents the query entity s q The sampling probability of the neighboring entity o.

4. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 2, characterized in that, Step S2 includes the following: Step S21: Employ a time encoder to encode time information and learn time-aware entity representations of different time neighbors; further, for a fact at time t, the time interval between it and the query time tq is Δt = |t|q. q If -t|, then its time embedding φ(Δt) is expressed as: Among them, w s ,w s ,β s ,β t For learnable network parameters, d represents the time embedding dimension; Step S22: Obtain the time-aware expressions for entities and relations by combining time embedding, entity embedding, and relation embedding, including the following: Where || represents the vector concatenation operation, and W1 and W2 represent linear transformation matrices. These represent the time-aware dynamic representation and the static representation of an entity, respectively. Step S23: Use graph attention to aggregate information from the domain nodes, including calculating the attention coefficient between each node for a quadruple (s,r,o,t): in It is a linear transformation matrix. This represents the attention parameter vector, and || represents the concatenation operation; Step S24: For the set of neighbor nodes N s After normalization, we get: in, Step S25: Update the features of the center node s using normalized attention coefficients, including the following: in This represents the time-aware representation of entity s at timestamp t. It includes information about time neighbors and also reflects the impact of dynamic changes over time.

5. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 2, characterized in that, Step S3 includes the following: Step S31: Define static embedding z static and dynamic embedding z time , where static embedding z static By learning the inherent attributes of entities and relationships through a time-unconstrained embedding layer, z is dynamically embedded. time An embedded representation with time information is generated through a time-aware graphical attention network; Where static embedding z static and dynamic embedding z time The expressions include the following: With static =MLP[e||r] MLP stands for Projection Layer Mapping to the Unit Hypersphere.

6. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 5, characterized in that, Step S3 also includes the following: Step S32: Static and dynamic representations can be viewed as dual-view representations encoding facts in a temporal knowledge graph. The static and dynamic representations of the same query are used as positive samples to obtain... Static and dynamic representations of different queries were used as negative samples. Step S33: Use the static representation as the anchor point to query time t q Supervision and comparison of losses The calculations include the following: Q tq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product; Step S34: Using the dynamic representation as the anchor point, the loss can be obtained. Includes the following: Q tq and N tq These represent the word timestamp t. q The query set and the minibatch of the query set are given, where τ is the temperature coefficient and (·) represents the dot product; Step S35: Calculate the average of the two losses to obtain the following: in, This represents the final comparative learning loss.

7. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 2, characterized in that, Step S4 includes the following: Step S41: Use the ConvTransE decoder to execute the timestamp t q The entity prediction task, further, targets the query quadruple (s q r q ,o,t q The entity prediction score is: in This indicates the embedding of query relationships that incorporate time information. The query entity embedding σ(·) represents the sigmoid function.

8. The method for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network according to claim 7, characterized in that, Step S4 includes the following: Step S42: Train using cross-entropy loss; for a query (s) q ,r q ,o,t q The training objective for tail entity prediction is to maximize the reduction of the loss, thus obtaining the entity prediction loss. Represented as: Where P(s) q ,r q ,o,t q ) represents the entity prediction probability score; Step S43: Fusing the entity prediction loss and the temporal-aware contrastive learning loss, we obtain: in Indicates the final loss 9. A system for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a method for constructing a knowledge representation learning model based on a periodic-aware contrast graph attention network as described in any one of claims 2 to 8.

10. A system for constructing a knowledge representation learning model based on a periodic-aware contrastive graph attention network, comprising a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a method for constructing a knowledge representation learning model based on a periodic perceptual contrastive graph attention network as described in any one of claims 2 to 8.

Images