Timing sequence heterogeneous network link prediction method and system based on hierarchical comparative learning

Abstract

The invention discloses a time sequence heterogeneous network link prediction method and system based on hierarchical comparative learning, and relates to the technical field of time sequence heterogeneous network prediction. The invention aims to mine and model the space complexity and the time sequence complexity in the network from three different microscopic perspectives of node level, edge level and time level so as to predict the node connection behavior in the time sequence heterogeneous network in the real world. According to the technical key points, heterogeneous structure information of node representation vectors is stored through a proposed link prediction method, the time sequence evolution process of the heterogeneous network is captured, meanwhile, the topological dependency relationship between heterogeneous snapshots is captured, and the distribution mode in the complex time sequence heterogeneous network is described, so that the dynamic and complex connection relationship between entities is predicted. The fine-grained difference relation between different nodes and edges and the difference of evolution paradigms directly affect representation learning, and the performance of link prediction is affected to a great extent. The method is used for predicting dynamic and complex connection relations between entities in social recommendation and traffic management.

Description

Technical Field

[0001] The present invention relates to the technical field of time series heterogeneous network prediction, and in particular to a time series heterogeneous network link prediction method and system. Background Art

[0002] Contemporary information networks (such as social networks and biological systems) are developing rapidly and becoming increasingly complex. These networks are usually composed of various types of nodes and link relationships, and evolve continuously over time. Traditional static homogeneous networks can no longer fully abstract these systems, and the emergence of time-series heterogeneous networks can more accurately simulate the complex systems of the real world. Link prediction in these time-series heterogeneous complex networks is a long-term challenge.

[0003] The key challenges in link prediction research on temporally heterogeneous networks are modeling heterogeneous entity relationships and dynamic snapshot change patterns. We refer to these challenges as “spatial complexity” and “temporal complexity”. Existing link prediction methods in the literature mainly divide dynamic temporal networks into static snapshot sequences according to time points in chronological order and characterize the complex entity relationships in each snapshot. Among them, (1) “spatial complexity” is mainly reflected in the complex heterogeneous relationships between multi-type nodes. The various co-occurrence patterns in complex networks are mainly modeled by heterogeneous graph embedding methods. They can be roughly divided into meta-path-based and node-attribute-based methods. Meta-path-based methods construct meta-paths within each snapshot to mine heterogeneous information. Node-attribute-based methods aim to make full use of rich multiple attributes and fuse adjacent attributes together before inputting into the node embedding process. (2) “Temporal complexity” mainly utilizes the dynamic distribution changes in the snapshot sequence. The temporal method focuses on capturing the continuous evolution process. It can be roughly divided into sequence methods and graph methods. Sequence methods take input from a sequence arranged in chronological order and capture the evolutionary dependencies between different snapshots based on recurrent neural networks (RNNs) and attention mechanisms. Graph methods aggregate the embeddings of each dynamic node and continuously encode features that appear or disappear over time in the network based on graph neural networks (GNNs). Although these methods excel in efficiency, a significant drawback is that these methods model dynamic heterogeneous representations in a coarse-grained manner, focusing only on the representation paradigm but ignoring the fine-grained differential relations that are prevalent in temporal heterogeneous networks, which leads to poor performance in link prediction tasks.

[0004] For example, the authorized invention document discloses a method for predicting time-series network links using GraphSAGE (patent number ZL202010746960.8, authorization announcement number CN111756587B), which uses the time slicing method to divide the time-series network into a series of network snapshots, and then performs data preprocessing on the number of edge connections and edge duration information of node pairs in the network snapshots; the preprocessed data is used as the input of the GraphSAGE algorithm and trained to obtain a node embedding generation model; a node similarity index is constructed by combining the node embedding similarity and the node topology structure similarity, thereby predicting the future connection status of the corresponding node pairs.

[0005] In summary, among the existing related network link prediction methods, no one has proposed how to use this differential relationship to bridge the spatial and temporal heterogeneity to comprehensively depict detailed dynamic and diverse features to improve the performance of link prediction. No one has proposed that the fine-grained differential relationship (space) between different nodes and edges and the difference in evolution paradigm (time) will affect representation learning and affect the performance of link prediction. Therefore, it is very necessary to provide a temporal heterogeneous network link prediction method based on hierarchical contrastive learning. Summary of the invention

[0006] The technical problems to be solved by the present invention are:

[0007] The purpose of the present invention is to provide a time-heterogeneous network link prediction method (CLP) and system based on hierarchical contrastive learning, which mines and models the spatial complexity and temporal complexity in the network from three different microscopic perspectives: node level, edge level and time level, so as to realize the prediction of node connection behavior in time-heterogeneous networks in the real world.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: a time series heterogeneous network link prediction method based on hierarchical contrastive learning, and the formal definition of time series heterogeneous network link prediction is:

[0009] Heterogeneous Networks: Given is an undirected graph, where V = {v 1 , v 2 , …, v N}, E={e 1 , e 2 ,…,e M}, A v and A e Represents node set, edge set, node type and edge type respectively; each node v i ∈V and edge e j ∈E has a corresponding node type associated with it and edge type And |A v |+|A e |>2, then is a heterogeneous graph;

[0010] Temporal Heterogeneous Network: Given an undirected heterogeneous graph It consists of a series of heterogeneous network snapshot sequences at different times, namely in is a static snapshot of the network at time t, where and denote the node set and edge set at time t, respectively, T denotes the total number of snapshots, and For any node a∈V, its node representation is a feature vector of fixed size Then X={x a} a∈V Represents the feature matrix of all nodes;

[0011] The method is used to predict the probability of the existence of a target link e = (a, b) between two nodes a, b∈V in a time series heterogeneous network at the future time T+1, and obtain the user representation by training the learning model CLP and The similarity between them is used to obtain the existence of e in the snapshot graph probability;

[0012] An end-to-end model based on contrastive learning, CLP, is proposed. It obtains the deep representation of temporal heterogeneous networks by learning heterogeneous information of different types of nodes and edges and dynamic dependencies to express spatial and temporal differences. The existence of the target link is predicted by the similarity of the vector representations of two target users.

[0013] CLP mainly consists of three parts: (1) spatial feature learning, (2) temporal information modeling, and (3) output layer;

[0014] (1) Spatial feature learning: First, a graph attention network (GAT) is set up to preserve the temporal heterogeneous network at the node level and edge level. The heterogeneous structure information is obtained, thereby obtaining the node-level representation vector of the heterogeneous structure and edge-level representation vector At the same time, it is regarded as a homogeneous network and GNN is deployed to capture its inherent structural information to generate the corresponding node-level representation vector and edge-level representation vector right and and It performs two-layer contrastive learning at the node level and edge level respectively to enhance the acquisition of network structure differences;

[0015] (2) Temporal information modeling: Secondly, GRU and LSTM are used to save each static snapshot graph. The hidden state of , thus obtaining the temporal level representation vector and For node representation and Perform contrastive learning to enhance the capture of temporal differences in temporally heterogeneous networks;

[0016] (3) Output layer: Finally, the similarity between node a and node b is calculated to represent the target link e = (a, b), which is fed into an overall loss function to map the probability of the target link existing.

[0017] The present invention has the following beneficial technical effects:

[0018] The present invention proposes a new temporal heterogeneous network link prediction model (CLP) based on hierarchical contrastive learning to solve the technical problems in the background technology, and performs multi-perspective differential representation of nodes to model heterogeneous semantic relationships and dynamic dependencies in temporal heterogeneous networks, thereby predicting whether two nodes will generate a target link. Scientific and reasonable decision-making reference plans can be formulated for applications such as social network friend recommendation, precise targeted advertising delivery, traffic route optimization, and drug effect monitoring. The main contributions of the present invention are: (1) A three-layer hierarchical contrastive differential relationship extraction module is proposed to realize the multi-perspective difference elimination function, which provides a difference bridging effect for spatial and temporal heterogeneity in link prediction scenarios. (2) A heterogeneous temporal graph network is designed to model structural and temporal distribution paradigms and eliminate differences from different perspectives. Specifically, we learn node-level and edge-level distribution paradigms through a graph attention network, and propose a dual-channel temporal network to capture the temporal dependencies between different snapshots at the temporal level. (3) Extensive experiments are conducted on four real time-heterogeneous information networks, and the performance of the CLP model is compared with existing advanced time-heterogeneous network link prediction models. The experiments show that the proposed model is significantly superior to existing models in terms of AUC and AP.

[0019] The present invention believes that if differential relationships (fine-grained differential relationships between different nodes and edges) can be used to bridge spatial and temporal heterogeneity, more comprehensive and detailed dynamic and diverse features can be depicted, thereby improving the performance of link prediction. Specifically, focusing on the aforementioned "temporal complexity" and "spatial complexity", the fine-grained differential relationships (space) between different nodes and edges and the difference in evolutionary paradigms (time) directly affect representation learning, which can greatly affect the performance of link prediction. The present invention provides a temporal heterogeneous network link prediction method (CLP) based on hierarchical contrastive learning, which mines and models the spatial complexity and temporal complexity in the network from three different microscopic perspectives: node level, edge level, and time level, to achieve the prediction of node connection behavior in temporal heterogeneous networks in the real world. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall framework of the CLP model (the overall framework of the temporal heterogeneous network link prediction method based on hierarchical contrastive learning);

[0021] Figure 2 The graph for the parameter selection experiment for the node embedding dimension d is shown in the figure: Figure 2 (a) shows the impact of node embedding dimension d on AUC. Figure 2 (b) The impact of node embedding dimension d on AP;

[0022] Figure 3 Experimental curve diagram for selecting the number of multi-head attention heads (h) parameter. In the figure: Figure 3 (a) is the impact of the number of attention heads h on AUC, Figure 3 (b) The effect of the number of attention heads h on AP;

[0023] Figure 4 is the weight adjustment factor (λ 1 ,λ 2 ,λ 3 ) Parameter selection experiment curve diagram, in the figure: Figure 4 (a) is the weight coefficient λ 1 Impact on AUC, Figure 4 (b) is the weight coefficient λ 1 Impact on AP Figure 4 (c) is the weight coefficient λ 2 Impact on AUC, Figure 4 (d) is the weight coefficient λ 2 Impact on AP Figure 4 (e) is the weight coefficient λ 3 Impact on AUC, Figure 4 (f) is the weight coefficient λ 3 Impact on AP.

[0024] Figure 5 This is the experimental curve diagram for parameter selection of temperature coefficient (τ), in which: Figure 5 (a) is the effect of temperature coefficient τ on AUC, Figure 5 (b) is the effect of temperature coefficient T on AP. DETAILED DESCRIPTION

[0025] 1. The present invention proposes a time-series heterogeneous network link prediction method based on hierarchical contrastive learning, whose task is to predict the probability of future connection between any two nodes in a time-series heterogeneous network, namely, the link prediction task. The proposed link prediction method stores the heterogeneous structural information of the node representation vector, captures the temporal evolution process of the heterogeneous network, and at the same time, captures the topological dependencies between heterogeneous snapshots, characterizes the distribution pattern in the complex time-series heterogeneous network, and thus predicts the dynamic and complex connection relationship between entities. It has greatly promoted the development of many real-life applications including social recommendation, traffic management, medical health and network biology.

[0026] For example, recommending people you know on social networking platforms, optimizing route management in traffic planning, completing and restoring case knowledge graphs in medical health, and predicting protein interactions in network biology, link prediction helps to understand the complexity and functions of the above applications and provides new ideas and methods for the above applications.

[0027] 2. Combined with attachment Figure 1 , the implementation of the technical solution of the temporal heterogeneous network link prediction method based on hierarchical contrastive learning proposed in the present invention is described as follows:

[0028] 2.1 Overall Approach

[0029] Formal definition of link prediction in time-series heterogeneous networks:

[0030] Heterogeneous Networks: Given is an undirected graph, where V = {v 1 , v 2 , …, v N}, E={e 1 , e 2 ,…,e M}, A v and A e Represents node set, edge set, node type and edge type respectively. Each node v i ∈V and edge e j ∈E has a corresponding node type associated with it and edge type And |A v |+|A e |>2. Then is a heterogeneous graph.

[0031] Temporal Heterogeneous Network: Given an undirected heterogeneous graph It consists of a series of heterogeneous network snapshot sequences at different times, namely in is a static snapshot of the network at time t, where and denote the node set and edge set at time t, respectively, T denotes the total number of snapshots, and For any node a∈V, its node representation is a feature vector of fixed size Then X={x a} a∈V Represents the feature matrix of all nodes.

[0032] The task of the present invention is to predict the probability of the existence of a target link e = (a, b) between two nodes a, b∈V in a time series heterogeneous network at the future time T+1, and obtain the user representation by training the learning model CLP and The similarity between them is used to obtain the existence of e in the snapshot graph probability.

[0033] This paper proposes an end-to-end model CLP based on contrastive learning. The main idea is to obtain the deep representation of the temporal heterogeneous network by learning the heterogeneous information of different types of nodes and edges and the dynamic dependencies to express the spatial and temporal differences, and predict the existence of the target link by the similarity of the vector representation of two target users. Figure 1 As shown in Figure 2, CLP is mainly divided into three parts: (1) spatial feature learning, (2) temporal information modeling, and (3) output layer.

[0034] (1) Spatial feature learning: First, we set up a graph attention network (GAT) to preserve temporal heterogeneous networks at the node level and edge level. The heterogeneous structure information is obtained, thereby obtaining the node-level representation vector of the heterogeneous structure and edge-level representation vector At the same time, it is regarded as a homogeneous network and GNN is deployed to capture its inherent structural information to generate the corresponding node-level representation vector and edge-level representation vector right and and It performs two-layer contrastive learning at the node level and edge level respectively to enhance the acquisition of network structure differences.

[0035] (2) Temporal Information Modeling: Secondly, we use GRU and LSTM to save each static snapshot graph The hidden state of , thus obtaining the temporal level representation vector and For node representation and Perform contrastive learning to enhance the capture of temporal differences in temporally heterogeneous networks.

[0036] (3) Output layer: Finally, we calculate the similarity between node a and node b to represent the target link e = (a, b), which is fed into an overall loss function to map the probability of the target link existing.

[0037] 2.2 Spatial Feature Learning

[0038] To capture the heterogeneous structural features, we first transform the static snapshot The network is divided into specific sub-networks according to the edge type. Subsequently, we use node-level and edge-level GAT to represent different types of edges and nodes in the temporal heterogeneous graph. The two-level GAT modeling effectively captures the different importance of different types of nodes and edges, and promotes the processing of heterogeneous semantics and structures in the temporal heterogeneous graph. Finally, we design node-level and edge-level contrastive learning to enhance the representation of nodes from the node view and edge view, respectively.

[0039] 2.2.1 Node-level heterogeneous feature learning

[0040] For each snapshot According to the edge type r∈R, it is divided into |R| subgraphs. Then it belongs to the static snapshot at time t The attention score between any two nodes (nodes a and b) of the r-type subgraph It is expressed as:

[0041]

[0042] Among them, x a and x b Used to initialize nodes a and b. rt and W rt yes The attention weight vector and mapping matrix corresponding to the r-type subgraph are updated during the training process. σ represents the activation function, and || is the concatenation operation. Then calculate The weight between nodes a and b in the r-type subgraph for:

[0043]

[0044] in, Representation diagram The rth neighbor of node a in . Then the node The representation of is obtained by weighted summation:

[0045]

[0046] In addition, to enhance the representation of each node, we adopt a node-level contrastive learning method to ensure that multiple enhanced representations of the same node have inherent similarities and are different from the representations of other nodes. Specifically, we ignore the categories of nodes and edges and use GNN to linearly aggregate all unclassified node embeddings:

[0047]

[0048] In order to ensure that the same node a is aggregated by GAT and GNN, the two representations and Similar to the representation obtained by GAT aggregation of different nodes b We use the node-level InfoNCE loss function to is regarded as a positive sample pair, are regarded as negative sample pairs, as shown in formulas (5) and (6):

[0049]

[0050]

[0051] Where τ is the temperature coefficient. sim() is represented by the dot product operation and is used to measure the similarity between two vectors, i.e. In addition, |V t |, T, and R represent the graphs respectively. The number of snapshots and edge types.

[0052] 2.2.2 Edge-level heterogeneous feature learning

[0053] The node-level feature learning module is able to capture specific information of a specific edge type. However, heterogeneous networks usually contain multiple edge types. In order to effectively obtain information from all edge types of each node, we design an edge-level feature learning module to obtain the importance weights of different edge types. Specifically, we aggregate these different types of specific edge information to generate node embeddings with heterogeneous edge information. Specifically, the embedding vector of each node is generated through a nonlinear mapping. In the t-th snapshot graph, the attention weight between node a and edge type r is The output is obtained through the softmax activation function, as shown in formulas (7) and (8):

[0054]

[0055]

[0056] Among them, z, Wt and b represent the trainable attention weight vector, weight matrix, and bias vector, respectively. σ is a nonlinear activation function. Then, by Nodes of a specific edge type Perform weighted summation to get its representation:

[0057]

[0058] In addition, to enhance the representation of each specific type of edge node, we adopt an edge-level contrastive learning method to ensure that different enhanced representations of the same subgraph have intra-similarity, but have inter-dissimilarity when compared with other types of subgraph nodes. Specifically, we ignore the graph G t The node and edge types in the network are considered as a homogeneous network, and GNN is used to linearly aggregate the node representations in the homogeneous network:

[0059]

[0060] in Representation diagram The set of neighbors of node a whose edge type is r.

[0061] To ensure that a specific edge type of node a is embedded In the same graph Embedding in Keep similarity between them, and make it aggregated with node b There is a significant difference between them. We set the edge-level InfoNCE loss function. This loss function considers the heterogeneous and homogeneous aggregate representations of the same node as positive sample pairs. The aggregate representations from different nodes in the heterogeneous graph are used as negative samples. As shown in formulas (11) and (12):

[0062]

[0063]

[0064] in, Representation diagram All adjacent nodes of node a cover all edge types.

[0065] 2.3 Temporal Information Modeling

[0066] The temporal information modeling layer aims to address the heterogeneity of temporal series information in different sequence modeling scenarios, and different modeling techniques can capture different sequence patterns. Specifically, we adopt a dual-channel architecture to learn different sequence dependency paradigms: (1) In the long-term channel, we deploy LSTM to explore the intrinsic interdependencies in the long-term temporal evolution process; (2) In the short-term channel, we apply GRU to analyze the interactions between adjacent snapshots in the short-term evolution process. However, existing studies have ignored the heterogeneity between long-term and short-term dependencies. To this end, we propose a contrastive learning method to emphasize the differences between different sequence learning paradigms, thereby focusing on modeling temporal heterogeneity. We use LSTM and GRU to represent the spatiotemporal patterns of learning long-term and short-term dependencies, respectively, denoted as and

[0067]

[0068]

[0069] in, Represents the embedding of node a in all snapshots at all time points, and the embedding vector of node a in all snapshots is generated. In order to solve the temporal inhomogeneity between different semantic spaces, we design a temporal contrastive learning method to achieve the closeness of the latent vector representation obtained by LSTM and GRU.

[0070]

[0071]

[0072] Among them, V T This is the last snapshot The collection of nodes in .

[0073] 2.4 Output Layer

[0074] We use the last snapshot Embedding vector of node a To perform the link prediction task, we aim to infer the potential connection between node i and other nodes. Therefore, the problem is transformed into evaluating the relationship between node i and the last snapshot. The similarity between neighbor nodes in . We use binary cross entropy minimization as the objective function, which is defined as shown in formula (17):

[0075]

[0076] Among them, for the final snapshot For any node a in for and The average pooling of At the same time, in Figure In the example, the neighbor i of node a is considered as a positive example, and the randomly selected non-neighbor node j is considered as a negative example, thus forming a triple (a, i, j). is defined as the set of all possible triples, namely but is a set of positive sample pairs, is the set of negative sample pairs.

[0077] Then, by using the cross entropy loss Node-level contrastive learning loss Edge-level contrastive learning loss and temporal contrastive learning loss Weighted to get the total loss

[0078]

[0079] Among them, λ 1 ,λ 2 ,λ 3 is a learnable weight adjustment factor used to coordinate the three losses.

[0080] The temporal heterogeneous network link prediction method based on hierarchical contrastive learning described in the present invention is applied to social recommendation or traffic management scenarios. Social networks are dynamic platforms with diverse interaction characteristics between users. In the rapidly evolving landscape of social networks, accurately predicting potential connections between users is crucial to improving user engagement and satisfaction. The present invention studies temporal heterogeneous network link prediction technology, which can significantly improve the accuracy of potential friend recommendations in social networks by utilizing the heterogeneity of social interactions and temporal dependency information. Specifically, the social network at time T is regarded as a temporal heterogeneous graph Form T static heterogeneous snapshot sequences. (1) In terms of heterogeneity, social networks are inherently heterogeneous, containing various types of node sets V t (e.g. users, posts) and the edge set E t (e.g. likes, comments, and follows). The present invention utilizes a spatial feature learning layer to learn and reflect rich node embeddings with diversity, capturing the complex interdependencies within the network. (2) In terms of temporal dynamics, temporal information is crucial to understanding the evolution of social relationships. The present invention fully utilizes the recent and historical interaction data of social networks through the temporal information modeling layer, captures the long-term and short-term interaction features between social network snapshots, and captures the temporal evolution of user behavior, thereby generating more timely and relevant friend recommendations. (3) Finally, the overall loss function in the output layer of the present invention is used to The embedding vectors of social network nodes are trained to optimize the possibility that there is a link e between the node pair (a, b) formed by any two users (user a and user b) in the social network. By accurately modeling heterogeneous information and time series information in the social network, the present invention can provide more personalized and timely friend recommendations for the social network, ultimately improving user satisfaction and participation, and bringing huge potential and opportunities in promoting interpersonal relationship expansion, knowledge sharing, and offline event organization.

[0081] 2.5 Algorithm Description

[0082] The pseudo code of a time-series heterogeneous network link prediction method and system based on hierarchical contrastive learning is shown in Algorithm 1, which describes the learning, training and convergence process of the CLP model. The node embedding dimension, number of nodes and number of snapshots of the model are denoted as d, N and T respectively, and the time complexity of the model is O(TNd 2 ).

[0083]

[0084] 3. Verification of the Effect of the Present Invention 3.1 Model Implementation Details

[0085] Model parameter settings: During the training process, the model uses a batch size of 1024 and achieves convergence within 5 iterations. The learning rate is set to 1e-4. In addition, the three balance parameters λ 1 , 2 and λ 3 , are all assigned a value of 1e-08. In the three InfoNCE loss functions, τ is set to 0.1. The dimension of node embedding is set to 32, and the number of attention heads in the spatial feature learning module is set to 4. All experiments are performed on the Ubuntu 18.04 operating system and the NVIDIA RTX A200012G graphics card.

[0086] 3.2 Dataset

[0087] To verify the efficiency and generalization of the model, this paper conducted experiments in four different heterogeneous time series scenarios, namely Math-overflow, Taobao, OGBN-MAG, and COVID-19. The details of the dataset are shown in Table 3-1.

[0088] Table 3-1 Dataset statistics

[0089]

[0090] Math-overflow: The Math-overflow dataset is collected by the Math Overflow community on the Stack Exchange website and published on the SNAP platform. It contains three different types of interactions between users over 2,350 days (questions and answers, questions and comments, and answers and comments). In the experiment, 124 days is used as the time window, and this dataset is divided into 11 snapshots in subsequent experiments.

[0091] Taobao: The Taobao dataset consists of records of users clicking on cloud-themed products in the Taobao App from April 1 to May 31, 2008. It contains three types of nodes: user (U), product (I), and theme (T), and three types of links: UI, UT, and IT. In the experiment, this dataset is divided into five snapshots with a 12-day time window.

[0092] OGBN-MAG: The OGBN-MAG dataset is a subset of the Microsoft Academic Graph (MAG). The dataset contains four types of nodes (papers, authors, institutions, and research fields) and four types of relationships between them (authors' institutions, authors' papers, papers citing papers, and papers with research fields as their subjects). In this experiment, a THN was extracted from OGBN-MAG. The time span of the THN was from January 1 to 10, 2010. The time slices were divided into days, and 1,000 edges of each type were taken from each time slice to form the dataset used in the experiment.

[0093] COVID-19: The COVID-19 dataset comes from the 1point3acres platform, which contains daily case reports at the state and county levels (e.g., confirmed cases, new cases, deaths, and recovered cases). This model uses daily new COVID-19 cases as time series data for each state and county. The dataset consists of two types of nodes (states and counties) and four relationships between them, namely two administrative affiliations (states include counties, and counties belong to states) and two geospatial relationships (states are close to states, and counties are close to counties). The THN constructed in this experiment spans from May 1 to 21, 2020, and contains 21 time slices. Each time slice is limited to a maximum of 2,000 edges of each type.

[0094] 3.3 Experimental Evaluation

[0095] 3.3.1 The proposed CLP model is experimented and evaluated on four data sets: Math-overflow, Taobao, OGBN-MAG, and COVID-19. For the problem of predicting the link between two nodes in a time-series heterogeneous network, the present invention uses two indicators, Area Under Curve (AUC) and Average Precision (AP), to evaluate the prediction effect of this model. AUC refers to the area under the Receiver Operating Characteristic Curve (ROC), with the false positive rate displayed on the x-axis and the true positive rate displayed on the y-axis. AP refers to the area under the Precision-Recall curve, where the recall rate is the horizontal coordinate and the precision is the vertical coordinate. The larger the values ​​of AUC and AP are, the closer they are to 1, and the better the prediction effect of this model.

[0096] 3.3.2 Comparison with Baseline Methods

[0097] To measure the performance of the CLP model, CLP is compared with several classic link prediction methods, which are mainly distributed in four categories: (1) static homogeneous networks, (2) static heterogeneous networks, (3) dynamic homogeneous networks, and (4) dynamic heterogeneous networks. The details are as follows.

[0098] (1) Static homogeneous network:

[0099] SEAL: Extract local graphs from target links and learn local graph features to map the probability of target link existence.

[0100] VGNAE: Combining variational inference, graph convolutional networks, and normalization techniques to effectively learn probabilistic node embeddings from graph-structured data.

[0101] (2) Static heterogeneous network:

[0102] Metapath2Vec: The random walk sequences generated under the guidance of metapaths are input into the skip-gram model to learn low-dimensional embeddings of nodes in heterogeneous information networks, thereby capturing the structural and semantic relationships in the network.

[0103] GATNE: We propose a comprehensive framework to learn node representations in multi-attribute networks, addressing the challenges of integrating multiple types of relations and node attributes into a unified representation.

[0104] (3) Dynamic homogeneous network:

[0105] TGAT: We propose an inductive representation learning method on temporal graphs, combining temporal encoding and G-graph neural networks to capture the dynamic properties of such graphs. This method can generalize to new nodes and future graph snapshots.

[0106] TDGNN: We design a method for continuous-time link prediction in dynamic graphs, using temporal coding techniques and attention mechanisms to enhance the prediction capabilities of graph neural networks.

[0107] (4) Dynamic Heterogeneous Networks

[0108] THAN: Representation Learning on Graphs with Temporal Heterogeneity, exploiting memory mechanisms and transformer architectures to capture complex temporal and structural information in temporally heterogeneous graphs.

[0109] THAT: We propose a comprehensive dynamic heterogeneous graph representation learning framework that combines neighborhood type modeling, neighborhood information aggregation, and temporal dynamics integration to achieve accurate and efficient node representation.

[0110] Table 3-2 Experimental results compared with the Baseline method on four datasets

[0111]

[0112] Table 3-2 summarizes the performance of the CLP model and eight baseline models on four datasets: Math-overflow, Taobao, OGBN-MAG, and COVID-1. The best results are in bold and the suboptimal results are underlined. The following conclusions can be drawn from the table: (1) The static homogeneous network models SEAL and VGNAE perform poorly on all four datasets, mainly because they cannot exploit temporal dynamics and heterogeneous information. (2) The static heterogeneous network models Metapath2Vec and GATNE perform better than SEAL and VGNAE due to their exploitation of heterogeneous information. However, their limitation is that they ignore network dynamics. For example, a link that is considered a negative link during training may become a positive link in the test set due to the dynamic characteristics of the original network. The inability to capture these temporal changes causes the node representation vector to convey opposite meanings, thereby reducing model performance. (3) The dynamic homogeneous networks TGAT and TDGNN have limitations in exploiting heterogeneous information. In a network, different types of nodes and edges have different importance, but TGAT and TDGNN cannot distinguish them. (4) The dynamic heterogeneous networks THAN and THAT effectively integrate and model the temporal dynamics and heterogeneous types in graph data, as reflected by their superior performance over other baseline models. (5) On the four datasets, our CLP model significantly outperforms all baseline models in terms of AUC and AP. Our CLP model effectively learns heterogeneous features through node-level and edge-level contrastive learning techniques, while effectively modeling temporal information through a temporal-level contrastive learning method. Our method successfully captures the dynamic patterns and semantic information in dynamic heterogeneous graph data. Compared with the best baseline (i.e., THGAT), our model shows significant performance improvement in terms of AUC and AP, with an average improvement of 10.10% and 13.44% in AUC and AP, respectively. (6) The results of all models in the COVID-19 dataset generally decline. One possible explanation for this phenomenon is the scarcity of observed nodes and the resulting data instability, which leads to poor link prediction results.

[0113] 3.3.3 Ablation Experiment Setup

[0114] In order to verify the effectiveness of each module in the CLP model, the present invention sets up an ablation experiment:

[0115] CLP -N : Remove the node-level contrastive learning module to evaluate its importance in modeling structural differences.

[0116] CLP -E : The edge-level contrastive learning module is removed to verify its effectiveness in modeling structural differences.

[0117] CLP -T : Remove the time-level comparative learning module to observe the contribution of time-series differences to the improvement of system prediction performance.

[0118] Table 3-3 Experimental results compared with variant methods on four datasets

[0119]

[0120] Ablation experiments are conducted on four datasets: Math-overflow, Taobao, OGBN-MAG, and COVID-1 to evaluate the performance of the three key modules of the CLP model: node-level contrastive learning, edge-level contrastive learning, and time-level contrastive learning. The comparison results are shown in Table 3-3. It can be seen from the table that CLP performs best compared with other variant models. After removing any of the three modules, the prediction performance decreases significantly. In particular, (1) when we remove the node-level contrastive learning module (i.e., CLP1 -N ), both AUC and AP showed a significant decrease, ranging from 2.35% to 25.30%. This further confirms the key role of node-level contrastive learning representation in improving the efficiency of link prediction. (2) When removing the edge-level contrastive learning module (i.e., CLP) -E ), the most significant drop in prediction performance occurs, ranging from 3.36% to 35.65%. This emphasizes the indispensable role of edge-level contrastive learning in our model. (3) Removing the temporal-level contrastive learning module (i.e., CLP -T ), which also leads to the decrease of AUC and AP, ranging from 4.28% to 26.58%. In summary, node-level, edge-level, and temporal-level contrastive learning losses constitute important elements of CLP, fundamentally improving the accuracy of predicting temporally heterogeneous links.

[0121] 3.3.4 Parameter comparison experiment

[0122] The present invention conducts comparative experiments on the selection of key parameters. We select the node representation vector dimension (d), the number of multi-head attention heads (h), the loss function weight adjustment factor (λ 1 ,λ 2 ,λ 3 ) and temperature coefficient (τ) for comparative analysis. Figure 2-Figure 5 The prediction indicators AUC and AP change with these parameters. (1) For the node representation vector dimension (d). We set d = 8, 16, 32, 64, 128. As the vector dimension increases, the model performance increases because the training capacity of the learnable parameters is enhanced under a larger training scale. However, too many dimensions will lead to a decrease in model performance, which may be due to overfitting and increased noise in the representation vector. Figure 2 As shown in , when d = 32, the optimal model performance can be obtained. (2) Number of multi-head attention heads (h). The h-head attention mechanism in the node-level and edge-level feature learning modules divides the sub-semantic space, enabling our model to direct attention to different heterogeneous information dimensions. Figure 3 As shown in Figure 2, when h = 4, the expression ability and attention allocation ability of our model are significantly improved. (3) Loss function weight adjustment factor (λ 1 ,λ 2 ,λ 3 ). The weights of the node-level, edge-level, and time-level contrast losses (respectively expressed as λ 1 , 2 and λ 3 A good optimization balance can be achieved by adjusting Figure 4 As shown, when λ 1 , 2 and λ 3 When both are set to e-08, the best prediction performance can be obtained. (4) Temperature coefficient (τ). τ affects the training process and the performance of the final model by adjusting the sensitivity and attention to sample similarity. Figure 5 As shown, when τ is 0.1, our model effectively completes the contrastive learning training.

[0123] In summary, the method, experiments and practical applications proposed in the present invention have verified the technical effects and practicality claimed by the present invention.

[0124] The algorithm (method) proposed in the present invention is the underlying technical core of the present invention. Based on the algorithm, various products can be derived to be applicable to the dynamic and complex connection relationships between entities in the prediction of related fields. It has greatly promoted the development of many applications in real life, including social recommendation, traffic management, medical health and network biology. For example, recommending acquaintances in social networking platforms, optimizing route management in traffic planning, completing and restoring case knowledge graphs in medical health, predicting protein interactions in network biology, and link prediction help to understand the complexity and functions of the above applications.

[0125] Based on the algorithm (method) proposed in the present invention, a time-series heterogeneous network link prediction system based on hierarchical contrastive learning is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps of the above-mentioned time-series heterogeneous network link prediction method based on hierarchical contrastive learning during operation.

[0126] The computer program of the developed system (software) is stored on a computer-readable storage medium, and the computer program is configured to implement the steps of the above-mentioned method for predicting time-series heterogeneous network links based on hierarchical contrastive learning when called by a processor. That is, the present invention is materialized on a carrier to become a computer program product.

[0127] The present invention also proposes a time-series heterogeneous network link prediction device based on hierarchical contrastive learning, the device comprising at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the time-series heterogeneous network link prediction method based on hierarchical contrastive learning to realize the prediction of dynamic and complex connection relationships between entities in the time-series heterogeneous network.

[0128] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The computer programs (also referred to as programs, software, software applications, or codes) of the present invention include machine instructions for programmable processors, and these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used in the present invention, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, they are all within the scope of protection of the present invention.

Claims

1. A time series heterogeneous network link prediction method based on hierarchical contrastive learning, the formal definition of time series heterogeneous network link prediction: Heterogeneous Networks: Given is an undirected graph, where V = {v1,v2,…,v N }, E={e1,e2,…,e M }, A v and A e Represents node set, edge set, node type and edge type respectively; each node v i ∈V and edge e j ∈E has a corresponding node type associated with it and edge type And |A v |+|A e |>2, then is a heterogeneous graph; Temporal Heterogeneous Network: Given an undirected heterogeneous graph It consists of a series of heterogeneous network snapshot sequences at different times, namely in is a static snapshot of the network at time t, where and denote the node set and edge set at time t, respectively, T denotes the total number of snapshots, and For any node a∈V, its node representation is a feature vector of fixed size Then X={x a } a∈V Represents the feature matrix of all nodes; The method is used to predict the probability of the existence of a target link e = (a, b) between two nodes a, b∈V in a time series heterogeneous network at the future time T+1, and obtain the user representation by training the learning model CLP and The similarity between them is used to obtain the existence of e in the snapshot graph probability; The feature is that an end-to-end model CLP based on contrastive learning is proposed. By learning the heterogeneous information of different types of nodes and edges and dynamic dependencies, the deep representation of the temporal heterogeneous network is obtained to express spatial and temporal differences. The existence of the target link is predicted by the similarity of the vector representations of two target users. CLP mainly consists of three parts: (1) spatial feature learning, (2) temporal information modeling, and (3) output layer; (1) Spatial feature learning: First, a graph attention network (GAT) is set up to preserve the temporal heterogeneous network at the node level and edge level. The heterogeneous structure information is obtained, thereby obtaining the node-level representation vector of the heterogeneous structure and edge-level representation vector At the same time, it is regarded as a homogeneous network and GNN is deployed to capture its inherent structural information to generate the corresponding node-level representation vector and edge-level representation vector right and and It performs two-layer contrastive learning at the node level and edge level respectively to enhance the acquisition of network structure differences; (2) Temporal information modeling: Secondly, GRU and LSTM are used to save each static snapshot graph. The hidden state of , thus obtaining the temporal level representation vector and For node representation and Perform contrastive learning to enhance the capture of temporal differences in temporally heterogeneous networks; (3) Output layer: Finally, the similarity between node a and node b is calculated to represent the target link e = (a, b), which is fed into an overall loss function to map the probability of the target link existing.

2. According to claim 1, a temporal heterogeneous network link prediction method based on hierarchical contrastive learning is characterized in that: The specific spatial feature learning is as follows: To capture heterogeneous structural features, firstly, the static snapshot The network is divided into specific sub-networks according to the edge types. Then, node-level and edge-level GATs are used to represent different types of edges and nodes in the temporal heterogeneous graph. The importance of different types of nodes and edges is effectively captured through two-level GAT modeling, which promotes the processing of heterogeneous semantics and structures in the temporal heterogeneous graph. Finally, node-level and edge-level comparative learning is designed to enhance the representation of nodes from the node view and edge view, respectively.

3. According to claim 2, a temporal heterogeneous network link prediction method based on hierarchical contrastive learning is characterized in that: Node-level heterogeneous feature learning is specifically as follows: For each snapshot According to the edge type r∈R, it is divided into |R| subgraphs; then it belongs to the static snapshot at time t The attention score between any two nodes (nodes a and b) of the r-type subgraph It is expressed as: Among them, x a and x b Used to initialize nodes a and b; A rt and W rt yes The attention weight vector and mapping matrix corresponding to the r-type sub-graph are updated during the training process; σ represents the activation function, ∥ is the concatenation operation; then the calculation The weight between nodes a and b in the r-type subgraph for: in, Representation diagram The rth neighbor of node a in The representation of is obtained by weighted summation: A node-level contrastive learning method is used to enhance the representation of each node, ensuring that multiple enhanced representations of the same node have inherent similarities and are different from the representations of other nodes; the categories of nodes and edges are ignored, and GNN is used to linearly aggregate all unclassified node embeddings:

4. According to claim 3, a temporal heterogeneous network link prediction method based on hierarchical contrastive learning is characterized in that: In the process of learning node-level heterogeneous features, to ensure that the same node a is aggregated by GAT and GNN, the two representations and Similar to the representation obtained by GAT aggregation of different nodes b Dissimilar, using the node-level InfoNCE loss function, is regarded as a positive sample pair, are regarded as negative sample pairs, as shown in formulas (5) and (6): Among them, τ is a hyperparameter called temperature; sim() is represented by the dot product operation, which is used to measure the similarity between two vectors, that is, T, and R represent the graphs respectively. The number of snapshots and edge types.

5. According to claim 4, a temporal heterogeneous network link prediction method based on hierarchical contrastive learning is characterized in that: Edge-level heterogeneous feature learning is specifically as follows: Build an edge-level feature learning module to obtain information from all edge types of each node to obtain the importance weights of different edge types; Different types of specific edge information are aggregated to generate node embeddings with heterogeneous edge information. The embedding vector of each node is generated by nonlinear mapping. In the t-th snapshot graph, the attention weight between node a and edge type r is The output is obtained through the softmax activation function, as shown in formulas (7) and (8): Among them, z, W t and b represent the trainable attention weight vector, weight matrix and bias vector respectively; σ is a nonlinear activation function; then, by Nodes of a specific edge type Perform weighted summation to get its representation: We use edge-level contrastive learning to enhance the representation of each specific type of edge node, ensuring that different enhanced representations of the same subgraph have intra-similarity and inter-dissimilarity when compared with other types of subgraph nodes. We ignore the graph G. t The node and edge types in the network are considered as a homogeneous network, and GNN is used to linearly aggregate the node representations in the homogeneous network: in Representation diagram The neighbor set of node a with edge type r; Set the edge-level InfoNCE loss function to ensure that the specific edge type of node a is embedded In the same graph Embedding in Keep similarity between them, and make it aggregated with node b. There is a significant difference; this loss function considers heterogeneous and homogeneous aggregate representations of the same node as positive sample pairs The aggregate representations from different nodes in the heterogeneous graph are used as negative samples. As shown in formulas (11) and (12): in, Representation diagram All adjacent nodes of node a cover all edge types.

6. The method for predicting time series heterogeneous network links based on hierarchical contrastive learning according to claim 5, characterized in that: The specific modeling of time series information is as follows: LSTM and GRU are used to represent the spatiotemporal patterns of learning long-term dependencies and short-term dependencies, respectively, as and in, Represents the embedding of node a in all snapshots at all time points, and thus the embedding vector of node a in all snapshots is generated; a time-level contrastive learning method is designed to solve the temporal inhomogeneity between different semantic spaces, so as to achieve the closeness of the latent vector representation obtained by LSTM and GRU, specifically: Among them, V T This is the last snapshot The collection of nodes in .

7. The method for predicting time series heterogeneous network links based on hierarchical contrastive learning according to claim 6, characterized in that: The output layer is specifically: Use the last snapshot Embedding vector of node a To perform the link prediction task, infer the potential connection between node i and other nodes; this problem is transformed into evaluating the relationship between node i and the last snapshot The similarity between neighbor nodes in the binary cross entropy is minimized as the objective function, which is defined as shown in formula (17): Among them, for the final snapshot For any node a in for and The average pooling of At the same time, in Figure In , the neighbor i of node a is regarded as a positive example, and the randomly selected non-neighbor node j is regarded as a negative example, thus forming a triple (a, i, j); the set is defined as the set of all possible triples, namely but is a set of positive sample pairs, is a set of negative sample pairs; Then, by using the cross entropy loss Node-level contrastive learning loss Edge-level contrastive learning loss and temporal contrastive learning loss Weighted to get the total loss Among them, λ1, λ2, λ3 are learnable weight adjustment factors used to coordinate the three losses.

8. The method for predicting time series heterogeneous network links based on hierarchical contrastive learning according to claim 7, characterized in that: The method is applied in social recommendation scenarios. Consider the social network at time T as a temporally heterogeneous graph Form T static heterogeneous snapshot sequences; node set V t Corresponding to users and / or posts, the edge set E t corresponding likes, comments and / or follows; through the temporal information modeling layer, the recent and historical interaction data of the social network are fully utilized to capture the long-term and short-term interaction characteristics between social network snapshots, and the time evolution of user behavior, so as to generate more timely and relevant friend recommendations; through the overall loss function in the output layer of the present invention, The embedding vectors of social network nodes are trained to optimize the possibility that there is a link e between the node pair (a, b) formed by any two users (user a and user b) in the social network.

9. A temporal heterogeneous network link prediction system based on hierarchical contrastive learning, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 8, and executes the steps in the temporal heterogeneous network link prediction method based on hierarchical contrastive learning when running.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of a temporal heterogeneous network link prediction method based on hierarchical contrastive learning according to any one of claims 1 to 8 when called by a processor.

Images