Methods and apparatus for dynamic graph node embedding via optical convolution

Abstract

A method includes extracting, by an analysis computer, a plurality of first data sets from a plurality of graph snapshots using a graph structure learning module. The analysis computer can then extract, using a temporal convolution module, a plurality of second data sets from the plurality of first data sets across the plurality of graph snapshots. The analysis computer can then perform graph context prediction with the plurality of second data sets.

Description

[0001] Cross-referencing related applications

[0002] This application is a PCT application claiming priority to U.S. Provisional Application No. 63 / 080,559, filed on September 18, 2020, which is incorporated herein by reference. Background Technology

[0003] Learning implicit representations (e.g., embeddings) of nodes in graphs is an important and widespread task with broad applications, such as link prediction, node classification, and visualization. However, the vast majority of real-world graphs are dynamic and evolve over time, such as email communications, collaboration, and interaction graphs. Despite recent successes in neural graph representation learning, almost all existing methods focus on static graphs, neglecting temporal dynamics.

[0004] In some cases, when considering the temporal dynamics of the graph, the embedding at the first time step can be determined, and then the embedding at the second time step can be determined based on the first embedding at the first time step. For example, a temporal regularizer can be used to enhance the smoothness of embeddings from neighboring time steps.

[0005] However, doing so requires determining the embeddings sequentially for each time step, since the embeddings depend on each other. Furthermore, due to this dependency on previous embeddings, any errors, biases, etc., will propagate through each subsequent embedding.

[0006] The embodiments of the present invention individually and collectively address these and other problems. Summary of the Invention

[0007] One embodiment relates to a method comprising: extracting multiple first datasets from multiple graph snapshots by an analysis computer using a graph structure learning module; extracting multiple second datasets from the multiple first datasets by the analysis computer using a temporal convolution module across the multiple first datasets; and performing graph context prediction by the analysis computer using the multiple second datasets.

[0008] Another embodiment relates to an analysis computer, including: a processor; and a computer-readable medium coupled to the processor, the computer-readable medium including code executable by the processor for implementing a method comprising: extracting a plurality of first datasets from a plurality of graph snapshots using a graph structure learning module; extracting a plurality of second datasets from the plurality of first datasets using a temporal convolution module across the plurality of first datasets; and utilizing the plurality of second datasets to perform graph context prediction.

[0009] More detailed information about embodiments of the present invention can be found in the detailed description and accompanying drawings. Attached Figure Description

[0010] Figure 1 A block diagram of a dynamic graph representation learning system according to some embodiments is shown.

[0011] Figure 2 A block diagram of an analysis computer according to some embodiments is shown.

[0012] Figure 3 A diagram illustrating an example of graph embedding according to some embodiments.

[0013] Figure 4 A flowchart illustrating a structured self-attention method according to some embodiments is shown.

[0014] Figure 5 A flowchart illustrating a depth-temporal convolution method according to some embodiments is shown.

[0015] Figure 6 A flowchart illustrating a lightweight temporal convolution method according to some embodiments is shown.

[0016] Figure 7 A flowchart illustrating a time-series learning method with additional module processes according to some embodiments is shown.

[0017] Figure 8 A flowchart illustrating the DyCLN architecture according to some embodiments is shown.

[0018] Figure 9 A flowchart illustrating a learning process using dynamic graph representations according to some embodiments is shown.

[0019] Figure 10 The performance comparison for a first benchmark dataset is shown according to some embodiments.

[0020] Figure 11 The performance comparison for a second benchmark dataset is shown according to some embodiments. Detailed Implementation

[0021] Before describing the embodiments of this disclosure, some terms may be described in detail.

[0022] A "machine learning model" can include applications of artificial intelligence that provide a system with the ability to automatically learn and improve from experience without being explicitly programmed. A machine learning model can include a set of software routines and parameters that can predict the output of a process based on "feature vectors" or other input data (e.g., identifying a computer network attacker, authenticating a computer, providing appropriate recommendations based on a user's search query, etc.). The structure of the software routines (e.g., the number of subroutines and the relationships between them) and / or the values ​​of the parameters can be determined during training, using the actual results of the process being modeled, such as the identification of input data from different classes. Examples of machine learning models include: Support Vector Machines (SVMs); models that classify data by establishing gaps or boundaries between inputs of different classes; and neural networks, which are collections of artificial "neurons" that execute functions by activating in response to inputs. In some embodiments, neural networks can include convolutional neural networks, recurrent neural networks, etc.

[0023] A "model database" can include a database that can store machine learning models. Machine learning models can be stored in the model database in various forms, such as a set of parameters or other values ​​defining the machine learning model. Models in the model database can be stored in association with keywords that convey a certain aspect of the model. For example, a model used to evaluate news articles can be stored in the model database in association with the keywords "news," "advertisement," and "information." The analysis computer can access and retrieve models from the model database, modify models in the model database, delete models from the model database, or add new models to the model database.

[0024] A "feature vector" can include a set of measurable attributes (or "features") representing an object or entity. A feature vector can include a collection of data represented numerically as an array or vector structure. A feature vector can also include a collection of data that can be represented as mathematical vectors, on which vector operations such as scalar products can be performed. Feature vectors can be determined or generated from input data. Feature vectors can be used as input to a machine learning model, causing the model to produce a certain output or classification. Based on the nature of the input data, feature vectors can be constructed in various ways. For example, for a machine learning classifier that classifies words as correctly spelled or incorrectly spelled, the feature vector corresponding to a word such as "LOVE" can be represented as a vector (12, 15, 22, 5), corresponding to the alphabetical index of each letter in the input data word. For more complex "inputs," such as human entities, exemplary feature vectors can include features such as a person's age, height, weight, numerical representations of relative happiness, etc. Feature vectors can be represented and stored electronically in a feature storage area. Furthermore, feature vectors can be normalized, that is, given unit values. For example, the feature vector (12, 15, 22, 5) corresponding to "LOVE" can be normalized to approximately (0.40, 0.51, 0.74, 0.17).

[0025] "Interaction" can include reciprocal effects or influences. "Interaction" can include communication, contact, or exchange between parties, devices, and / or entities. Example interactions include transactions between two parties and data exchange between two devices. In some embodiments, an interaction can include a user requesting access to secure data, a secure webpage, a secure location, etc. In other embodiments, an interaction can include a payment transaction in which two devices can interact to facilitate the payment.

[0026] A "topology graph" can be a representation of a graph in a plane connected by edges. The different vertices in the topology graph can be referred to as "nodes." Each node can represent specific information about an event, or specific information about a profile of an entity or object. Nodes can be related to each other by a set of edges E. An "edge" can be described as an unordered pair of nodes that is a subset of a graph G=(V,E), where G is a graph comprising a set of vertices (nodes) V connected by a set of edges E. For example, a topology graph can represent a transaction network in which nodes representing transactions can be connected by edges to one or more nodes associated with the transaction, such as nodes representing information about devices, users, transaction types, etc. Edges can be associated with numerical values ​​called "weights," which can be assigned to the pairwise connections between two nodes. Edge weights can be identified as the strength of the connection between two nodes and / or can be related to cost or distance, as it often represents the amount required to move from one node to the next. In some embodiments, the graph can be a dynamic graph that can change over time. For example, nodes and / or edges can be added to and / or removed from the graph.

[0027] A "subgraph" can be a graph formed from a subset of the elements of a larger graph. Elements can include vertices and connecting edges, and a subset can be a set of nodes and edges selected from the entire set of nodes and edges in the larger graph. For example, multiple subgraphs can be formed by randomly sampling graph data, where each random sample can be a subgraph. Each subgraph can overlap with another subgraph formed from the same larger graph.

[0028] A "community" can include a densely connected group of nodes within a graph. A community can be a subgraph or a part / derivative thereof, and a subgraph may or may not be a community and / or include one or more communities. Communities can be identified from a graph using graph learning algorithms, such as those used to map protein complexes. Communities identified using historical data can be used to classify new data for prediction. For example, identifying communities can be used as part of a machine learning process where predictions can be made about information elements based on their relationships to each other.

[0029] The term "node" can include discrete data points representing specified information. Nodes can be connected to each other in a topology graph via edges, which can be assigned values ​​called edge weights to describe the strength of the connection between two nodes. For example, a first node can be a data point representing a first device in the network, and the first node can be connected in the graph to a second node representing a second device in the network. The connection strength can be defined by the edge weights, which correspond to the degree to which information can be sent quickly and easily between two nodes. Edge weights can also be used to express the cost or distance required to move from one state or node to the next state or node. For example, a first node can be a data point representing a first location of a machine, and the first node can be connected in the graph to a second node representing a second location of the machine. The edge weight can be the energy required to move from the first location to the second location.

[0030] “Graph data” can include data represented as a topological graph. For example, graph data can include data represented by multiple nodes and edges. Graph data can include any suitable data (e.g., interaction data, communication data, review data, network data, etc.).

[0031] A "graph snapshot" can include graph data within a time frame. For example, a graph snapshot can include graph data that occurred during time periods such as 3 days, 1 week, or 2 months.

[0032] “Graph context prediction” can include any suitable prediction based on graph data. In some embodiments, the prediction may relate to the context of the graph or at least a portion of the graph data. For example, if the graph data is formed from weather data, the prediction may involve predicting the weather at a specific location. In some embodiments, graph context prediction can be performed using a machine learning model that forms a final node representation (also known as the final vector representation of a node), which may correspond to data from a second dataset. In some embodiments, graph context prediction may be a classification of some input data by a machine learning model.

[0033] A "vector representation" can be a vector that represents something. In some embodiments, a vector representation may include vectors representing nodes from graph data in a vector space. In some embodiments, a vector representation may include embeddings.

[0034] A “dataset” can include a collection of related information sets, which may consist of individual elements but can be manipulated by a computer as a unit. In some embodiments, a dataset may include multiple vectors. For example, in some embodiments, a first dataset may include multiple intermediate vector representations, and a second dataset may include multiple final node representations.

[0035] A "kernel" may include a set of values. The kernel can have any suitable length, such as two, three, four, five, or any other suitable number of values. In some embodiments, the kernel may include a series of weight parameter values ​​that can be normalized. The weight parameter values ​​can be trained using historical data and machine learning processes. In some embodiments, the kernel is unique for a specific feature dimension of the vector. In other embodiments, the kernel can be used for multiple feature dimensions of the vector.

[0036] A "server computer" can include a powerful computer or cluster of computers. For example, a server computer can be a mainframe, a small cluster of computers, or a group of servers that work like cells. In one example, a server computer can be a database server coupled to a web server. A server computer can include one or more computing devices and can use any of a variety of computing architectures, arrangements, and compilations to serve requests from one or more client computers.

[0037] "Memory" can include any suitable one or more devices capable of storing electronic data. Suitable memory can include non-transient computer-readable media storing instructions executable by a processor to implement the desired method. Examples of memory can include one or more memory chips, disk drives, etc. Such memory can be operated using any suitable electrical, optical, and / or magnetic modes of operation.

[0038] "Processor" can include any suitable one or more data computing devices. A processor can include one or more microprocessors working together to achieve more efficient training of the desired function. A processor can include a CPU, which includes at least one high-speed data processor sufficient to execute program components for performing user- and / or system-generated requests. The CPU can be a microprocessor such as AMD's Athlon, Duron, and / or Opteron; IBM and / or Motorola's PowerPC; IBM and Sony's Cell processors; Intel's Celeron, Itanium, Pentium, Xeon, and / or XScale; and / or similar processors.

[0039] I. Introduction

[0040] In some cases, embodiments of this disclosure are also referred to as Dynamic Graph Optical Convolutional Networks (DGLC), which operate on dynamic graphs and learn node representations that capture both structural features and temporal evolution patterns. Embodiments allow the analysis computer to first employ graph structure learning layers to efficiently capture structural adjacency information, and then employ temporal convolutional layers to efficiently capture the temporal evolution of the graph sequence, thereby computing node representations. Compared to prior art, the temporal convolution of these embodiments enables the learning of adaptive temporal evolution patterns at a fine-grained node-level granularity. Further temporal convolution can achieve processing efficiency and avoid unnecessary computational costs by leveraging a single-dimensional focus on input features within a temporal kernel window.

[0041] Learning latent representations (or embeddings) of nodes in graphs has been considered a fundamental learning problem due to its widespread use in various fields such as biology (Grover and Leskovec, 2016), social media (Perozzi et al., 2014), and knowledge bases (Wang et al., 2014). The idea is to encode the structural characteristics (and possible attributes) of a node's neighbors as low-dimensional vectors. Such low-dimensional representations can be beneficial for a wide range of graph analysis tasks, such as node classification, link prediction, and graph visualization (Perozzi et al., 2014; Grover and Leskovec, 2016; Wang et al., 2016; Tang et al., 2015).

[0042] Previous work on graph representation learning has primarily focused on static graphs, which contain a fixed set of nodes and edges. However, many graphs in real-world applications are inherently dynamic, where the graph structure can evolve over time. Dynamic graphs can be represented as a sequence of graph snapshots from different time steps (Leskovec et al., 2007). Examples include academic co-authorship networks, where authors can periodically change their collaborative behavior; and email communication networks, whose structure may change drastically due to sudden events. In such contexts, modeling temporal evolution patterns can be important for accurately predicting node characteristics and future links.

[0043] Learning dynamic node representations is challenging compared to static settings due to the complex, time-varying structure of graphs. For example, nodes can appear and disappear, links (e.g., edges) can emerge and vanish, and communities can merge and split. This necessitates that learned embeddings not only preserve the structural proximity of nodes but also collectively capture temporal dependencies over time. While some recent work has attempted to learn node representations in dynamic networks, they primarily impose temporal regularizers to enhance the smoothness of node representations from neighboring snapshots (see Zhu et al., 2016; Li et al., 2017; Zhou et al., 2018). However, these approaches fail when nodes exhibit significantly different evolutionary behaviors. Trivedi et al. (2017) employed recurrent neural structures for temporal reasoning in multi-relational knowledge graphs. However, this approach learns temporal node representations by focusing only on chain-level evolution while ignoring the structure of local graph neighbors.

[0044] Attention mechanisms have recently achieved great success in many sequential learning tasks, such as machine translation (Bahdanau et al., 2015) and reading comprehension (Yu et al., 2018). The basic principle of attention mechanisms can be learning a function that aggregates variable-sized inputs while focusing on the parts most relevant to a particular context. When an attention mechanism uses a single sequence as both input and context, it is often referred to as self-attention. Although attention mechanisms were initially designed to facilitate the capture of long-term dependencies in recurrent neural networks (RNNs), recent work (Vaswani et al., (2017)) shows that fully self-attention networks can achieve state-of-the-art performance in machine translation tasks on their own. (Velickovic et al., (2018)) extended self-attention on graphs by enabling each node to focus on its neighbors, thus achieving state-of-the-art results in semi-supervised node classification tasks on static graphs.

[0045] Recent works have proposed methods for learning node representations on dynamic graphs. To capture evolutionary patterns, these methods primarily utilize two types of techniques: Recurrent Neural Networks (RNNs) [Goyalet et al., 2020; Pareja et al., 2020] and attention mechanisms [Sankar et al., 2020; Xu et al., 2020]. RNN-based models take graph snapshots or a set of graph neural network (GNN) weights as input to each time step, allowing their hidden states to be optimized to summarize and learn historical graph changes. On the other hand, attention-based methods model temporal information by weighting and aggregating structural information from each graph snapshot at different time steps. However, training both types of models can be time-consuming, especially when modeling graphs with long time sequences. Specifically, RNN-based models require processing each graph snapshot sequentially, while attention-based models compute weight coefficients for the entire graph sequence. Furthermore, both types of models pose significant challenges to hardware memory requirements. These challenges prevent existing dynamic graph representation learning methods from being applied to domains with large dynamic graphs having many time steps.

[0046] In contrast, embodiments of this disclosure provide a novel neural architecture for efficiently learning node representations on dynamic graphs. Specifically, embodiments can employ self-attention for structural neighbors and temporal dynamics. Embodiments can employ graph structure learning layers to efficiently capture structural proximity information, followed by temporal convolutional layers to efficiently capture the temporal evolution of graph sequences. For example, embodiments can allow an analytics computer to generate node representations by considering that the neighbors of a node follow a self-attention strategy, and then that the historical representations of the node follow a temporal convolutional strategy. Unlike static graph embedding methods that focus entirely on maintaining structural proximity, embodiments can learn dynamic node representations that reflect the temporal evolution of the graph structure on historical snapshots of varying numbers of changes. Embodiments are able to accurately capture both structural characteristics and temporal evolution patterns. In contrast to methods based on temporal smoothness, embodiments can learn attention weights that capture temporal dependencies at a fine-grained node-level granularity.

[0047] Compared to previous methods, this embodiment achieves better processing efficiency when capturing the temporal evolution of graph sequences. First, each lightweight convolutional kernel may focus on only a single dimension of the input features within a temporal kernel window. This is acceptable because cross-dimensional feature interactions have already been captured from the structural layers. Therefore, this embodiment avoids unnecessary computations, thereby improving model optimization. Furthermore, this embodiment can share weights (e.g., kernels) across certain feature dimensions, thereby reducing the number of parameters. This both regularizes the model and reduces computational costs. These advantages make the embodiments of the present invention powerful and efficient, especially when modeling dynamic graphs with long time sequences.

[0048] A. System Overview

[0049] Figure 1 A block diagram of a system 100 comprising multiple components according to some embodiments is shown. System 100 includes an analysis computer 102, a graph data database 104, a model database 106, and a request client 108. The analysis computer 102 can communicate operationally with the graph data database 104, the model database 106, and the request client 108 (e.g., a remote computer).

[0050] For the sake of simplicity, in Figure 1 A certain number of components are shown. However, it should be understood that embodiments of this disclosure may include more than one of each component. Additionally, some embodiments of this disclosure may include more than one of each component. Figure 1 All components shown are either fewer or more components.

[0051] Figure 1 Messages between devices in System 100 can be sent using secure communication protocols, such as, but not limited to, File Transfer Protocol (FTP); Hypertext Transfer Protocol (HTTP); Secure Hypertext Transfer Protocol (HTTPS), SSL, ISO (e.g., ISO 8583), etc. The communication network can include any one and / or a combination of the following: direct interconnection; the Internet; a local area network (LAN); a metropolitan area network (MAN); Operational Mission as a Node on the Internet (OMNI); a secure custom connection; a wide area network (WAN); a wireless network (e.g., employing protocols such as, but not limited to, Wireless Application Protocol (WAP), I-mode, etc.); etc. The communication network can use any suitable communication protocol to generate one or more secure communication channels. In some instances, the communication channel can include a secure communication channel, which can be established in any known manner, such as by using mutual authentication and session keys, and by establishing a Secure Sockets Layer (SSL) session.

[0052] Graph data database 104 can securely store graph data. Graph data database 104 can store graph data (e.g., topology graph data). In some embodiments, graph data database 104 can store multiple graph snapshots of a dynamic graph.

[0053] Model database 106 can securely store models. For example, analysis computer 102 can create models (e.g., machine learning models) and store said models in model database 106. In some embodiments, graph data database 104 and model database 106 can be conventional, fault-tolerant, relational, scalable, and secure databases, such as those available from Oracle™, Sybase™, etc.

[0054] Analysis computer 102 is capable of performing dynamic graph representation learning via self-attention networks and lightweight convolutions as described herein. Analysis computer 102 is capable of retrieving graph data from graph data database 104. In some embodiments, analysis computer 102 is capable of retrieving graph snapshots from graph data database 104.

[0055] The analysis computer 102 is capable of extracting multiple first datasets from multiple graph snapshots using a graph structure learning module. Then, the analysis computer 102 is capable of extracting at least a second dataset from the multiple first datasets across the multiple graph snapshots using a temporal convolution module. The extraction of multiple first and second datasets is described in further detail herein. The analysis computer 102 is also capable of performing graph context prediction using at least the second dataset.

[0056] The requesting client 108 may include any suitable device external to the analysis computer 102. In some embodiments, the requesting client 108 may receive outputs and / or decisions made by the analysis computer 102. In other embodiments, the requesting client 108 may send a request (e.g., a prediction request) to the analysis computer 102. The request may include request data about the model. The requesting client 108 may request the analysis computer 102 to run the model to, for example, predict whether two nodes in graph data will be connected via an edge in a future graph snapshot. After receiving a request including the request data, the analysis computer 102 may determine the output data. For example, the analysis computer 102 may input the request data into the model to determine the output data output by the model. The analysis computer 102 may then provide the output data to the requesting client 108.

[0057] For example, in some embodiments, the analysis computer 102 may receive a prediction request from a requesting client 108. The prediction request may include, for example, a request regarding whether a first author, represented by a first node in the prediction collaboration graph data, will connect to a second author, represented by a second node, at a future point in time (e.g., to perform research with the second author).

[0058] Then, the analysis computer 102 can determine the prediction based at least on performing graph context prediction using at least a second dataset. For example, the analysis computer 102 can use a model created as described herein to predict whether the first and second authors will collaborate on a research paper at a given time step. For example, the analysis computer 102 can determine that there is a 90% probability that the two authors will collaborate on a research paper within the next year.

[0059] After determining the prediction, the analysis computer 102 can perform any appropriate action based on the prediction. For example, the action could include sending a prediction response message, which includes at least the prediction, to the requesting client 108. For instance, the analysis computer 102 could send a message providing a prediction that two authors are likely to collaborate within the next year. In another instance, the analysis computer could send an advisory notification indicating that a transaction is likely to occur, or that a currently attempted transaction is unlikely to occur and is therefore likely fraudulent.

[0060] B. Analyzing computers

[0061] Figure 2 A block diagram of an analysis computer 200 according to an embodiment is shown. The exemplary analysis computer 200 may include a processor 204. The processor 204 may be coupled to a memory 202, a network interface 206, an input element 210, an output element 212, and a computer-readable medium 208. The computer-readable medium 208 may include a graph structure learning module 208A, a temporal convolution module 208B, and a context prediction module 208C.

[0062] Memory 202 can be used to store data and code. Memory 202 can be coupled internally or externally to processor 204 (e.g., a cloud-based data storage device) and can include any combination of volatile and / or non-volatile memory such as RAM, DRAM, ROM, flash memory, or any other suitable memory device. For example, memory 202 can store graph data, vectors, datasets, etc.

[0063] Computer-readable medium 208 may include code executable by processor 204 to perform a method comprising: extracting a plurality of first datasets from a plurality of graph snapshots by an analysis computer using a graph structure learning module; extracting at least a second dataset from the plurality of first datasets by the analysis computer across the plurality of graph snapshots using a temporal convolution module; and performing graph context prediction by the analysis computer using the at least second dataset.

[0064] Graph structure learning module 208A may include code or software executable by processor 204 to perform graph structure learning, such as structure self-attention. Graph structure learning module 208A, in conjunction with processor 204, can focus on specific nodes (e.g., nodes). The graph structure learning module 208A, combined with processor 204, can focus on neighboring nodes by determining attention weights that vary with the input nodes (e.g., in an attention neural network). In some embodiments, the graph structure learning module 208A, combined with processor 204, can use equation (1) to determine the intermediate vector representation of each node in each of a plurality of graph snapshots, which will be described in further detail below. The graph structure learning module 208A, combined with processor 204, can determine the intermediate vector representation of each graph snapshot independently of other graph snapshots.

[0065] For example, the graph structure learning module 208A, in conjunction with processor 204, can receive a first snapshot of graph data (e.g., a dynamic graph). The graph data can be communication data, including specific users (e.g., represented as nodes) and communication between users (e.g., represented as edges). The graph structure learning module 208A, in conjunction with processor 204, can first determine which nodes are connected to the first node (e.g., the first user in the communication network). Nodes connected to the first user (via edges) can be neighboring nodes. When determining the embedding of the first node, the neighboring nodes of the first node can be used. In this way, when determining the vector representation of the first node, attention can be paid to the neighboring nodes of the first node, thereby capturing structural patterns in the graph data.

[0066] The temporal convolution module 208B may include code or software executable by the processor 204 to perform temporal convolution. The temporal convolution module 208B, in conjunction with the processor 204, can capture temporal evolution patterns in graph data across multiple graph snapshots. The input to the temporal convolution module 208B may include intermediate vector representations determined by the structural convolution module 208A in conjunction with the processor 204. For example, the temporal convolution module 208B, in conjunction with the processor 204, may accept at least a vector representation from a first node of each graph snapshot as input. The vector representation of the first node may constitute an encoding of the local structure surrounding the first node. In some embodiments, the temporal convolution module 208B, in conjunction with the processor 204, may extract at least a second dataset from multiple first datasets across multiple graph snapshots using equation (2), as described in further detail below.

[0067] For example, the graph structure learning module 208A, in conjunction with processor 204, can determine the intermediate vector representation of the first node. Multiple intermediate vector representations can include the intermediate vector representation of the first node at each graph snapshot. Then, the temporal convolution module 208B, in conjunction with processor 204, can receive multiple intermediate vector representations of the first node. The temporal convolution module 208B, in conjunction with processor 204, can utilize the multiple intermediate vector representations to convolve over the historical representation of the first node (e.g., using lightweight convolution), thereby imprinting information from the intermediate vector representations of the first node onto each other (e.g., within a specific time window). Therefore, the temporal convolution module 208B, in conjunction with processor 204, can determine one or more final node representations of the first node of the graph data. The final node representation can be a vector representing the change of the intermediate vector representations over time (e.g., within a specific time window). Therefore, the final node representation can encode data about the structure of the graph and how that structure changes over time.

[0068] For example, the final node representation of a first node can represent the communication habits of the first user and how they evolve over time. A first node may communicate with a specific group of nodes for a period of time, and then drift to communicate with different groups of nodes. The final node representations of the first node can be configured to indicate or reflect changes in the first user's communication.

[0069] In some embodiments, the analytical computer may use at least a second dataset to create any suitable type of model, such as a machine learning model (e.g., support vector machine (SVM), artificial neural network, decision tree, Bayesian network, genetic algorithm, etc.). In some embodiments, the model may include a mathematical description of a system or process to assist in computation and prediction (e.g., fraud model, anomaly detection model, etc.).

[0070] For example, the analysis computer 200 can create a model, which may be a statistical model, that can be used to predict unknown information based on known information. For instance, the analysis computer 200 may include a set of instructions for generating regression lines (supervised learning) from training data, or a set of instructions for grouping data into clusters of different categories based on similarity, connectivity, and / or distance between data points (unsupervised learning). The regression lines or data clusters can then be used as a model for predicting unknown information based on known information.

[0071] Once the analysis computer has built a model based on at least a second dataset, the model can be used by the context prediction module 208C in conjunction with the processor 204 to generate predictive outputs based on a request. The context prediction module 208C may include code or software that can be executed by the processor 204 to perform context predictions. For example, the received request may be a request for a prediction associated with the presented data. For instance, the request may be a request to classify a transaction as fraudulent or non-fraudulent, or a request for user suggestions.

[0072] Graph context prediction module 208C, in conjunction with processor 204, can perform any suitable prediction based on the context of the graph data. For example, analysis computer 200 can determine predictions related to the graph data. In some embodiments, the predictions may be related to the context of the graph associated with the graph data. Analysis computer 200 may, for example, perform graph context prediction to determine whether a resource provider and a user will transact at some point next week. As an illustrative example, a second dataset determined by temporal convolution module 208B in conjunction with processor 204 can be used as input to a machine learning model, such as a regression or classification model, to make predictions, such as whether two nodes will be linked or the category to which a node will belong. In some embodiments, the second dataset can be used to train a neural network. For example, the second dataset may correspond to graph data including resource providers and users connected via interactive connections. The neural network can be trained in any suitable manner using the second dataset including vectors. In some embodiments, the neural network can be trained to classify input vectors as, for example, fraud or non-fraud. As another example, the neural network can be trained to predict whether two nodes will transact via an edge connection (e.g., a specific resource provider and user) in a future graph snapshot, the time associated with such a snapshot, and / or whether the edge will represent an approved or rejected transaction.

[0073] Network interface 206 may include an interface that allows analysis computer 200 to communicate with external computers. Network interface 206 enables analysis computer 200 to perform data communication with another device (e.g., a requesting client). Some examples of network interface 206 may include a modem, a physical network interface (e.g., an Ethernet card or other network interface card (NIC)), a virtual network interface, a communication port, a PCMCIA slot and card, etc. Wireless protocols enabled by network interface 206 may include Wi-Fi. TMData transmitted via network interface 206 may be in the form of signals, which may be electrical signals, electromagnetic signals, optical signals, or any other signals that can be received by an external communication interface (collectively, "electronic signals" or "electronic messages"). These electronic messages, which may include data or instructions, may be provided between network interface 206 and other devices via a communication path or channel. As described above, any suitable communication path or channel may be used, such as wires or cables, optical fibers, telephone lines, cellular links, radio frequency (RF) links, WAN or LAN networks, the Internet, or any other suitable medium.

[0074] II. Related Work

[0075] Examples may relate to representation learning techniques for static graphs, dynamic graphs, self-attention mechanisms, etc.

[0076] Early research on unsupervised graph representation learning utilized the spectral properties of various matrix representations of graphs (e.g., Laplacian) to perform dimensionality reduction (Belkin and Niyogi, 2001; Tenenbaum et al., 2000). To improve the scalability of large graphs, recent work on graph embeddings, inspired by the success of natural language processing, has established the effectiveness of random walk methods. For example, Deepwalk (Perozzi et al., 2014) learns node embeddings by maximizing the co-occurrence probability of nodes appearing within a window during a random walk. Node2vec (Grover and Leskovec, 2016) extends the flexibility of said models between homology and structural equivalence. In recent years, several graph neural network architectures based on convolutional generalizations have achieved great success, the vast majority of which are designed for supervised or semi-supervised learning (Niepert et al., 2016; Defferrard et al., 2016; Kipf and Welling, 2017; Sankar et al., 2017; Velickovic et al., 2018). Furthermore, Hamilton et al. (2017) extended graph convolution methods with trainable neighbor aggregation functions to propose a general framework applicable to unsupervised representation learning. However, these methods are not designed to model temporal evolution behavior in dynamic graphs.

[0077] Most techniques employ temporal smoothness regularization to ensure embedding stability across consecutive time steps (Zhu et al., 2016; Li et al., 2017). Zhou et al. (2018) further improved upon this by using ternary closures (Kossinets and Watts, 2006) as guidance. Neural methods have recently been explored in the knowledge graph domain by Trivedi et al. (2017), who employed recursive neural structures for temporal reasoning. However, their model is limited to tracking link evolution but neglects local neighbors when computing node representations. Goyal et al. (2017) learned incremental node embeddings through initialization from previous time steps; however, this may be insufficient for modeling changes over historical time. Unlike previous approaches, this implementation learns adaptive temporal evolution patterns at the node-level granularity using a self-attention architecture.

[0078] Dynamic graphs can generally be classified into discrete graphs and continuous graphs based on their representation. Discrete graphs use an ordered sequence of graph snapshots, where each snapshot represents aggregate dynamic information over a fixed time interval. Continuous graphs, on the other hand, retain detailed temporal information and are often more complex to model than discrete graphs. This work focuses on the discrete graph setup and learning node representations from a sequence of graph snapshots.

[0079] For discrete dynamic graph learning, many existing methods utilize recursive models to capture temporal dynamics into the hidden states. Some works use separate GNNs to model individual graph snapshots and RNNs to learn the temporal dynamics [Seo et al., 2018; Manessi et al., 2020]; others integrate GNNs and RNNs together into a single layer, aiming to learn both spatial and temporal information simultaneously [Pareja et al., 2020; Chen et al., 2018]. However, recursive structures introduce order dependencies during training, which present scalability issues when modeling long input sequences. Sankar et al. [Sankar et al., 2020] used self-attention mechanisms along both the spatial and temporal dimensions of the dynamic graph, demonstrating better performance than GNNs utilizing RNN methods. However, both RNN units and attention mechanisms can become inefficient when modeling dynamic graphs with long input sequences.

[0080] Existing work on continuous dynamic graphs includes RNN-based methods, temporal random walk-based methods, and time-point process-based methods. RNN-based methods perform representation updates via a recursive model at fine-grained timestamps [Kumar et al., 2019], while the other two categories incorporate temporal dependencies through temporal random walks and parameterized time-point processes [Nguyen et al., 2018; Trivedi et al., 2019]. However, these methods are not suitable for dynamic graphs without detailed event timestamps.

[0081] Recent advances in many Natural Language Processing (NLP) tasks have demonstrated the advantages of self-attention in achieving state-of-the-art performance (Vaswani et al., 2017; Lin et al., 2017; Tan et al., 2018; Shen et al., 2018; Shaw et al., 2018). In embodiments of this disclosure, self-attention can be employed to compute dynamic node representations by focusing on the neighbors of dynamic nodes as well as their previous historical representations. Some embodiments may include using self-attention on neighbors and may be related to Graph Attention Networks (GATs) (Velickovic et al., 2018), which employ neighbor attention for semi-supervised node classification in static graphs.

[0082] III. Dynamic Graph Light Convolutional Network Architecture

[0083] In some embodiments, the analysis computer can be configured to determine the embedding of graph data. For example, the analysis computer can determine a final node representation, which may be the final embedding. The graph representation can then be used for graph context prediction. To determine the graph representation, the analysis computer can retrieve graph data from a graph data database. In some embodiments, after retrieving the graph data, the analysis computer can determine multiple graph snapshots from the graph data. In other embodiments, the graph data may be stored as multiple graph snapshots in a graph data database, in which case the analysis computer can retrieve multiple graph snapshots from the graph data database.

[0084] The analysis computer can then use a graph structure learning module to extract multiple first datasets from multiple graph snapshots. These multiple first datasets may include, for example, intermediate vector representations of each node in each snapshot of the multiple graph snapshots. The intermediate vector representations may be vectors representing the nodes of the graph snapshots. For example, the intermediate vector representations may be in a vector space that represents the characteristics of the graph data. For example, if two nodes in a graph snapshot are similar (e.g., sharing multiple attributes), then the vectors representing the two nodes may be similar in the vector space.

[0085] As an illustrative example, graph data can include interaction data (e.g., transaction data, etc.). Graph data can be a dynamic graph comprising multiple graph snapshots. Each graph snapshot can include any suitable number of nodes and edges. Nodes in the graph data can represent resource providers and users. Edges can connect resource provider nodes to user nodes when resource providers and users have executed transactions. The analytics computer can determine a first dataset from each graph snapshot. For example, for each node, the analytics computer can determine a vector (e.g., an intermediate vector representation) based on the node's neighboring nodes (e.g., local structure). The intermediate vector representation can be determined using a self-attention neural network, where the analytics computer determines how much attention (e.g., weights) to give to neighboring nodes based on the influence of the node's neighboring nodes on the node.

[0086] For example, during the self-attention process, the analysis computer can determine an intermediate vector representation of the first user node. The analysis computer can determine values ​​representing the attention that can be placed on the links between the first user node and each resource provider node connecting the first user node. For example, the first user node may be connected via an edge to three resource provider nodes, including a first resource provider located in San Francisco providing grocery resources, a second resource provider located in San Francisco providing electronic device resources, and a third resource provider located in New York providing digital book resources. The analysis computer can focus on nodes to determine an intermediate vector representation of the first user node. For example, the first user node may be associated with a location in San Francisco and as part of an electronic community group. The analysis computer can use a self-attention neural network to determine the values, where the inputs may include the first user node and neighboring nodes, as described further in detail herein. The output of the neural network may include vectors containing values ​​representing how closely the first user node is related to each input node. For example, in some embodiments, the first user node may be most closely related to itself because it shares all its own characteristics. Then, the first user node can be associated with the second resource provider (San Francisco, Electronics), the first resource provider (San Francisco, Grocery), and the third resource provider (New York, Digital Books) in descending order of similarity, since the first user node is associated with San Francisco and Electronics.

[0087] The analysis computer can then use a temporal convolution module across multiple graph snapshots to extract at least a second dataset from multiple first datasets. The second dataset may include, for example, multiple final node representations (also called final vector representations of nodes) of a graph comprising multiple graph snapshots. These multiple final node representations may be vectors that further represent the changes in node structure over time (e.g., within a time window defined by the kernel size). For example, the final node representations may reside in a vector space that represents the characteristics of the graph data. For example, if the vectors of intermediate vector representations are similar over time, they may be represented by final node representations that are close to each other in the final vector space.

[0088] For example, if two nodes representing resource providers depict similar characteristics over time (e.g., both resource providers transact with many users in the summer but then do not perform many transactions in the winter), then the final node representations of these two resource providers can be close to each other (e.g., the vectors have similar magnitudes and orientations). For example, the first user node described above can be associated with an intermediate vector representation that describes the local structure around the first user node (e.g., including weights describing the relationship between the first user node and each neighboring node). Between a first graph snapshot and a second graph snapshot, the local structure around the first user node can change. A temporal convolution process can determine how the intermediate vector representation of the first user node changes throughout the graph snapshot. In this way, a temporal pattern can be determined and encoded into a set of final node representations that can represent the local structure of the first user node over time (e.g., within a predefined time window).

[0089] After extracting a second dataset (e.g., a set of final node representations), the analysis computer can utilize at least the second dataset to perform graph context prediction. As an illustrative example, the second dataset can be used as input to a machine learning model, such as a regression or classification model, to make predictions, such as whether two nodes will be linked or the category to which a node will belong. In some embodiments, graph context prediction can include determining whether a first node will interact with a second node in the future. For example, the analysis computer can use the final node representations (also called the final vector representations of nodes) to train any suitable machine learning model. The analysis computer can train, for example, a feedforward neural network capable of determining whether two nodes will interact via edge connections in a future graph snapshot.

[0090] Illustratively, the analysis computer can determine whether a first node representing a resource provider will transact with a second node representing a user (e.g., a consumer) within the next week, next month, next two months, etc. The analysis computer can also perform actions such as sending a notification message to the resource provider regarding the predicted transaction.

[0091] A. Problem Definition

[0092] Discrete-time dynamic graphs can include a series of observed snapshots. ,in It can be multiple time steps. Each snapshot It can be a weighted undirected graph, which includes a shared set of nodes. Link (e.g., edge) set and weight Thus depicting time Graph structure. Graph snapshot. The corresponding weighted adjacency matrix can be obtained from This disclosure indicates that, unlike some previous work that assumes a dynamic graph grows only over time, embodiments of this disclosure can allow both the addition and removal of links (e.g., edges). Embodiments can allow analytical computers to learn each node. In time steps implicit representation , so that the representation Maintain nodes The surrounding local structure is modeled, and the evolution of this local structure over time is also modeled. (Implicit representation) It can be represented as a final node.

[0093] Figure 3 An example of graph embedding according to an embodiment is shown. The analysis computer can learn the implicit vector space representation of nodes, such that two structurally similar nodes in graph data can be similarly embedded in the vector space. Figure 3 Includes a snapshot of Figure 302, which includes nodes and the edges connecting the nodes. Figure 3 It also includes a representation of vector space 304 of Figure 302. Although two dimensions are shown in vector space 304, it should be understood that vector space 304 can be multi-dimensional.

[0094] In some embodiments, an embedding can be a mapping from a discrete or categorical variable to a continuously numbered vector. In the context of a neural network, an embedding can be a low-dimensional, learned continuous vector representation of a discrete variable. Neural network embeddings can be useful because they can reduce the dimensionality of categorical variables and meaningfully represent categories in a transformation space. In some embodiments, a neural network can be used to determine the vectors that can represent nodes.

[0095] The analysis computer can determine the vector representation of each node in Figure 302. Vector space 304 can show the position of each vector corresponding to each node in the vector space. For example, node number 13 in Figure 302 can be embedded as the vector [1.1, -1.0] in vector space 304.

[0096] For example, Figure 302 could be a communication network representing (e.g., via edges) users (e.g., nodes) communicating with each other. Nodes 8 and 2 could represent users, for example, those with similar communication habits. The user represented by node 2 could communicate with other users (e.g., via email, telephone, text, etc.), as indicated by the edges of the other nodes in Figure 302. The user represented by node 8 could communicate with many of the same users that node 2 communicates with. Therefore, nodes 2 and 8 could have similar characteristics.

[0097] The analysis computer can determine the embeddings of the nodes in Figure 302. The analysis computer can determine the vector representation of each node in Figure 302. For example, the analysis computer can determine that the vector for node 2 in vector space 304 is [0.75, -0.81], and the vector for node 8 is [0.80, -0.82]. Because nodes 2 and 8 have similar properties, the analysis computer can determine similar vectors for nodes 2 and 8.

[0098] B. Model Overview

[0099] This section describes the architecture of the embodiment. The embodiment can efficiently generate representative node embeddings to track the temporal evolution of a dynamic graph. The graph structure learning module can capture structural information from each graph snapshot. The time series learning module can efficiently fuse structural information learned from historical time steps. Both modules can be used in unsupervised methods.

[0100] In some embodiments, the graph structure learning block may be followed by a temporal convolutional block, as described below. Figure 8 As shown, each block can contain multiple stacked layers of a corresponding type. The graph structure learning block can use self-attention to extract features from local neighbors (e.g., around a node) via self-attention aggregation to compute an intermediate vector representation for each graph snapshot. These representations can be fed as input to a temporal convolutional block, which can convolve over multiple time steps for each feature of each vector, thereby capturing temporal variations in the vector. The output of the temporal convolutional process can include a final node representation (also known as the final vector representation of the node) that can be used to train a machine learning model and perform graph context prediction.

[0101] C. Graph Structure Learning

[0102] Figure 4 A flowchart illustrating a method for graph structure learning via structural self-attention according to some embodiments is shown. Figure 4 The structured self-attention method can accept three inputs. , , And. The attention function can be described as applying the query... and a set of key-value pairs (e.g., respectively) and The query is mapped to the output, where the query, key, value, and output can all be vectors, or in some embodiments, matrices. The output can be computed as a weighted sum of values, where the weight assigned to each value is calculated by a compatibility function between the query and the corresponding key.

[0103] The graph structure learning process can learn graph snapshots by aggregating information from the nearest neighbors of each node. The structural characteristics of v. For example, the input to a graph structure learning layer could be a graph snapshot. ,in It can be a dynamic graph (e.g., graph data); and a set of input node representations. ,in This can be the dimension of the input embedding. Graph structure learning layers can output dimensions... A new set of node representations For example, a graph structure learning layer can output intermediate vector representations of nodes.

[0104] Graph structure learning layers can focus on nodes. In time The neighbors are calculated by computed attention weights as a function of the input node embedding. In some embodiments, the structural attention layer can be a weighted variable of the GAT applied to the graph snapshot (Velickovic et al., 2018): (1) In equation (1) above, It can be a node in a graph snapshot. A group of close neighbors, This can be a shared weight transformation applied to each node in a graph snapshot. According to... Figure 4 The analysis computer can apply different linear transformations to the query in steps 402, 404, and 406, respectively. ,key Sum Linear transformations can be applied to queries. ,key Sum Any suitable linear transformation. In equation (1), the linear transformation can be, for example, by a shared weight transformation. application. It can be the weight vector of the attention function implemented as a parameterized feedforward layer.

[0105] In step 408, the analysis computer can query the linear transformation. s and keys The concatenated matrix or vector can be formed. In some embodiments, in step 410, an additional linear transformation may be applied to the concatenated matrix. For example, in equation (1), It can be a cascading operation, which can cascade queries with linear transformations. s and keys .

[0106] It can be a snapshot of the current image. Middle link The weights are a set of learned coefficients obtained by softmax on the neighbors of each node (e.g., in step 412). Can indicate nodes For the nodes in the current graph snapshot The importance or contribution of the analysis computer. In some embodiments, the analysis computer may utilize a sparse matrix to implement masked self-attention on neighboring nodes.

[0107] In step 414, the analysis computer can determine the value of the linear transformation. (From step 406) and the output of step 412 perform a Matmul process (e.g., matrix multiplication). For example, the analysis computer can multiply the learned coefficients, the shared weight transformation, and the corresponding input node representations of neighboring nodes (e.g., ), to determine the node The value of each of the next set of neighboring nodes. The computer can analyze this and determine the sum of these values, which can indicate the relationship between each neighboring node and the node. The weights of the influence. Then, the analysis computer can apply the activation function to the sum value. For example, in equation (1), It can be a non-linear activation function. For example, in an artificial neural network, given an input or a set of inputs, the activation function of a node can define the output of that node. For example, the output of the activation function can include values ​​ranging from 0 to 1.

[0108] For example, regarding the self-attention mechanism of translating sentences from one language to another, queries It can be a translatable input sentence. (Key) This can hide the encoder state. For example, the key. It can be related to the input query. The words related to the sentence (in vector format). Then, the values... It can be made of keys and give each key The value determined by the attention score. In some embodiments, the query... It can include specific nodes from a graph snapshot. Key This can include queries The neighboring nodes of a node (e.g., nodes connected via an edge). Value It can be a node to be queried AND key Attention score for connections between adjacent nodes.

[0109] As another example, query vectors, key vectors, and value vectors can be created. These vectors can be created by multiplying the embeddings by, for example, three matrices trained during the training process. In some embodiments, computational attention can be performed by first obtaining the query and each key and calculating the similarity between them to obtain weights. The analysis computer can utilize any suitable similarity function, such as dot product, concatenation, detectors, etc. The analysis computer can then normalize these weights using the softmax function and can weight these weights by combining the corresponding values ​​to obtain the final attention.

[0110] In some embodiments, the analytical computer can additionally employ multi-head attention (Vaswani et al., 2017) to collectively focus on different subspaces at each input, resulting in a leap in model capacity. Examples can use multiple attention heads in graph structure learning layers, followed by cascading, as summarized below:

[0111] In the above equation, It could be the number of heads to focus on. It can be the output of structured multi-head attention. Structured attention can be applied to a single snapshot.

[0112] Multi-head attention processes compute a weighted sum of multiple attentions, rather than a single attention ignoring the value. To learn diverse representations, multi-head attention can apply different linear transformations to the value, key, and query for each head of the attention process. A single attention head can apply a unique linear transformation to its input query, key, and value. An attention score can then be computed between each query and key, and the values ​​are then weighted and summed using those attention scores. The outputs of the attention process can then be cascaded for each head of the attention process performed.

[0113] about Figure 4 Further details can be found in [Ashish Vaswani et al., see...] The advancement of the neural information processing system 30 Exhibition: 2017 Annual Conference on Advances in Neural Information Processing Systems 30:Annual Conference on Neural Information Processing Systems 2017) [December 4-9, 2017, Long Beach, California, USA, pp. 6000-6010, 2017. ], The cited references are incorporated herein by reference for all purposes.

[0114] D. Temporal convolution

[0115] The node representation computed by the structuring blocks can be input into a temporal convolutional layer, which can perform calculations for each node across a series of time steps with different time windows (e.g., on different series of graph snapshots). One or more temporal convolutions are computed independently. In some embodiments, a temporal convolutional layer may characterize a node at a given time point and the degree of correlation between that node and itself at other time points (e.g., within a time window).

[0116] The temporal convolution module 208B can be more generally referred to as a time-series learning module, designed to capture the temporal evolution of dynamic graphs. This module can utilize lightweight convolutions [Wu et al., 2019], which summarize the learned structural information from each historical graph snapshot into a unified, representative embedding. A major advantage of applying lightweight convolutions is efficiency. Lightweight convolutions are a form of depthwise convolution that aggregates information only from a temporal perspective, thereby avoiding unnecessary higher-order feature interactions already well performed by the graph structure learning module. Furthermore, lightweight convolutions share weights across certain channels, further reducing the number of parameters, which lowers computational costs and regularizes the model.

[0117] For each node The input to the temporal convolutional layer can be the output from the graph structure learning module. For example, for each node... The input can be from a set of intermediate vector representations. The value of each specific dimension, where This can be the number of time steps (e.g., graph snapshots), and It can represent a specific dimension of the input vector. These indicate that these values ​​were obtained before the temporal convolution occurred.

[0118] The output of a temporal convolutional layer can be each node A new set of vector representations at each time step (e.g., the final node representation) (e.g., having dimensions) of ,in These indicate values ​​that occur after the temporal convolution. The input embedding representation is packed together in all graph snapshots and can be obtained from a matrix. express. The output embedding representation is packed together in all graph snapshots, each consisting of a matrix. express.

[0119] The goal of temporal convolutional layers can be to capture the temporal changes of a graph structure across multiple time steps. Nodes In time steps Input vector representation at the location It can form the surrounding The encoding of the current local structure. It can be represented adjacent to its time (e.g., , (etc.) convolution, thus allowing Local temporal neighbors Yes, it has an impact. Therefore, temporal convolution facilitates the learning of dependencies between various representations of nodes at different time steps.

[0120] I. Depthwise Convolution

[0121] Figure 5 A flowchart illustrating a depthwise temporal convolution method according to some embodiments is shown. As discussed above, the input can be a node representation computed by a structuring block (e.g., an intermediate vector representation of each node). Typically, each node will be individually processed and convolved using its own different snapshot vector versions. Figure 5 The diagram illustrates the process of temporal convolution for a single node embedding. This process can be performed for each node embedding.

[0122] In step S510, the data to be convolved may be received by, for example, the temporal convolution module 208B of the analysis computer 200. The data may include multiple different temporal snapshots, each snapshot comprising multiple node representations determined by a structure block. As discussed above, these may be intermediate vector representations of each node.

[0123] In step S512, information about a single specific node embedding can be retrieved from a dataset with multiple node embeddings. The data for a single node embedding may include node embeddings spanning different time snapshots (e.g., t1, t2, ..., t...). k Various versions of ) (e.g., intermediate vector representations of nodes). At each time snapshot, the node embedding can be described by a set of feature dimension values. Figure 5 The example in the diagram illustrates three feature dimensions: F1, F2, and F3. The value for each feature dimension at each time snapshot is represented by a box in the diagram. Therefore, the first node is described by a first set of feature values ​​for each feature dimension across a set of timestamps. These feature values ​​can be referred to as the first feature values ​​because the convolution process will provide a new set of second feature values ​​for each feature dimension across the set timestamps.

[0124] In step S514, feature values ​​of each of the multiple feature dimensions can be separated and isolated. For example, a set of timestamp-specific feature values ​​of the first feature dimension F1 can be retrieved (e.g., at t1, t2, ..., t...). kThe value of F1 can be used to retrieve a set of timestamp-specific feature values ​​of the second feature dimension F2 (e.g., at t1, t2, ..., t...). k The value of F2 can be used to retrieve a set of timestamp-specific feature values ​​of the third feature dimension F3 (e.g., at t1, t2, ..., t...). k (The value of F3 at that time).

[0125] In step S516, temporal convolution can be performed individually for each of the multiple feature dimensions (and also for each node individually). Temporal convolution can be performed using separate feature dimension values ​​and corresponding convolution kernels from multiple convolutional kernels. As shown, multiple convolutional kernels can exist, and each feature dimension can be associated with a different corresponding convolutional kernel from the multiple convolutional kernels. Feature dimension F1 can be convolved using kernel K1, feature dimension F2 can be convolved using kernel K2, and feature dimension F3 can be convolved using kernel K3.

[0126] Each kernel can have a specific predefined length (or number of values). In this example, each kernel has three values ​​(e.g., a window or length of three). For example, the first kernel K1 has values ​​w1, w2, and w3, the second kernel K2 has values ​​w4, w5, and w6, and the third kernel K3 has values ​​w7, w8, and w9. However, embodiments allow kernels to have any suitable length or number of values ​​(e.g., 2, 3, 4, 5, 6, 7, 8, 9, or 10 values). Kernel values ​​can be normalized trainable weight parameters that can be trained during a training process (e.g., a machine learning process), as described in more detail below.

[0127] A kernel value can reflect the impact of certain feature dimension values ​​at a previous snapshot on those feature dimensions at the current snapshot, and therefore can be a tool for focusing on certain values ​​of feature dimensions from some previous snapshots. Thus, the length of the kernel can determine how many recent snapshots should be considered when transforming the current feature dimension of the current snapshot.

[0128] To perform a depthwise convolution, a kernel can be applied to feature values ​​corresponding to the feature dimension. The kernel can be applied multiple times, each time to a different subset of the feature values, where each subset is consecutive (e.g., belonging to consecutive timestamps). For example, a series of dot product calculations can be performed using kernel weight parameter values ​​and feature dimension values ​​(e.g., the first feature value of the feature dimension). Each dot product calculation can utilize a subset of the feature values. Using the first feature dimension F1 as an example, the dot product can be computed using the kernel K1 and a first subset of three consecutive feature values ​​of feature dimension F1 (e.g., the F1 values ​​of the first three consecutive timestamps t1, t2, and t3). This produces a result as a single scalar value. This result can be used as a temporal convolution feature value (also called a second feature value or final feature value) at a specific timestamp, which in some embodiments may be the last (or rightmost) of consecutive input timestamps (e.g., t3). A second dot product can be computed using the kernel K1 and a second subset of three consecutive feature values ​​of feature dimension F1 (e.g., the F1 values ​​of the second consecutive timestamp t2, the third consecutive timestamp t3, and the fourth consecutive timestamp t4). This produces another scalar value result. In some embodiments, this second result can be used as the temporal convolution value (also referred to as the second feature value or final feature value) for the next timestamp (e.g., t4). A third dot product can be computed using a third subset of the three consecutive feature values ​​of kernel K1 and feature dimension F1 (e.g., the F1 values ​​of the third consecutive timestamp t3, the fourth consecutive timestamp t4, and the fifth consecutive timestamp t5). This produces a third result as a third scalar value, which in some embodiments can be used as the temporal convolution value (also referred to as the second feature value or final feature value) for a subsequent snapshot (e.g., t5).

[0129] The dot product calculation can continue in this manner until the final time snapshot t. k The end of the eigenvalues ​​in the first feature dimension F1. As a visual representation of this process, in Figure 5 In this concatenation, kernel K1 can be overlaid on feature dimension F1. The dot product can be obtained between kernel K1 and the overlaid subset of feature values ​​in feature dimension F1. Kernel K1 can then slide a value to the right of feature dimension F1 and obtain another dot product using that subset of feature values. Kernel K1 can continue sliding to the right until the end of feature dimension F1. When concatenated, the result provides a new set of feature values ​​across a set of timestamps in feature dimension F1. These new values ​​can be referred to as the second or final feature value of feature dimension F1. In summary, the second feature value across timestamps comprises a temporal convolutional version of feature dimension F1 (e.g., for the first node).

[0130] In some embodiments, the feature dimension F1 can be padded with one or more null values ​​at the beginning (e.g., before time t1). This ensures that the temporal convolutional version of feature dimension F1 has the same length or number of values ​​as the original feature dimension F1. For example, if the dot product result is used as the convolutional value of the last input snapshot of that dot product, the first overlay of kernel K1 on feature dimension F1 produces the convolutional value of the third temporal snapshot t3. To produce the convolutional values ​​of the first and second temporal snapshots, the kernel slides to the left into a region without feature dimension values. Therefore, null values ​​(e.g., zeros) can be padded to the left so that the kernel can still be used to obtain the dot product. This produces the convolutional values ​​of the first and second temporal snapshots, thereby maintaining the same number of values ​​as the total convolutional feature dimension F1.

[0131] like Figure 5 As shown, this kernel dot product process for depthwise convolution can be performed for each feature dimension and its corresponding kernel.

[0132] In step S518, the convolutional feature dimension data can be recombinated to recreate different timestamp-specific versions of the node embedding, but now the node embedding is temporally convolutional. Each different feature dimension value can be combined according to its associated timestamp (also called a temporal snapshot or time step). For example, the new feature value of feature dimension F1 at the first timestamp t1 (also called the second feature value or final feature value), the new feature value of feature dimension F2 at the first timestamp t1, and the new feature value of feature dimension F3 at the first timestamp t1 can be combined to create a temporally convolutional embedding (also called a vector representation) of the first node for the first timestamp t1. Thus, an output vector representing the change of the node's local structure over time (e.g., over the same number of time steps as the kernel length) is created. This can be referred to as the final vector representation of the node at that timestamp (e.g., the first timestamp t1). The final vector representations can be combined for each timestamp to create a set of final vector representations of the first node, each vector representation corresponding to a different timestamp. Thus, a final vector representation is generated for the first node.

[0133] This process can be performed on a per-node embedding basis. Mathematically, the total depthwise convolution process utilizing each kernel at each node can be described by the following formula:

[0134] Once completed, node embedding information can include both structural and temporal information. For example, an academic co-authoring network can include multiple authors who periodically change their collaborative behavior. Node embeddings can include structural information from snapshots at each time step. The structural information can incorporate author interactions and author characteristics based on the author's behavior at that time (e.g., behaviors the author has collaborated on). Temporal information can indicate the evolutionary pattern of author behavior. For example, if the temporal convolution uses a kernel of length 3, the embedding of the author at a specific time step can be transformed based on the convolution of the previous two time step snapshot versions of the author embedding, thereby tracking the evolutionary pattern of behavior.

[0135] Node embedding information, including both structural and temporal information, is useful for predicting future events, such as whether two authors will collaborate in the future (e.g., whether an edge will connect its two nodes). The prediction process is discussed in more detail below.

[0136] II. Lightweight Convolution

[0137] According to some embodiments, additional and alternative convolutional methods can be utilized. For example, lightweight convolution is a specific type of depthwise convolution in which some kernel weights can be shared across specific feature dimensions. Figure 5 In traditional deep learning convolutions, a different kernel is used for each feature dimension. In contrast, lightweight convolutions can utilize a specific kernel across multiple feature dimensions. Reusing kernels across multiple feature dimensions reduces data processing overhead because it reduces the number of kernel weight parameters to be determined through the machine learning process.

[0138] Figure 6 A flowchart illustrating a lightweight temporal convolution method according to some embodiments is shown.

[0139] In step S610, the dataset to be convolved can be received, and this step can be combined with... Figure 5 The steps in step S510 are the same or similar. The data may include multiple snapshots at different times, wherein each snapshot includes multiple node representations determined by the structure block.

[0140] In step S612, information about a single specific node embedding can be retrieved from a dataset with multiple node embeddings. This step can be combined with... Figure 5 The steps in step S512 are the same or similar. In this example, the node embedding has nine feature dimensions: F1, F2, F3, F4, F5, F6, F7, F8, and F9.

[0141] In step S614, the values ​​of each feature dimension can be separated, and this step can be combined with... Figure 5Step S514 is similar. Instead of isolating each feature dimension, subgroups of feature dimensions can be formed. For example, the first subgroup may include feature dimensions F1, F2, and F3, the second subgroup may include feature dimensions F4, F5, and F6, and the third subgroup may include feature dimensions F7, F8, and F9.

[0142] In step S616, temporal convolution can be performed separately for each feature dimension based on the separate feature dimension values ​​and the corresponding kernel. This step can be combined with... Figure 5 Step S516 is similar. Similar to... Figure 5 Step S516 in the algorithm can convolve each feature dimension individually. (Compared to...) Figure 5 Unlike step S516, a specific kernel can be used for more than one feature dimension. For example, a first kernel K1 can be used for each of the first subgroups (e.g., feature dimensions F1, F2, and F3), a second kernel K2 can be used for each of the first subgroups (e.g., feature dimensions F4, F5, and F6), and a third kernel K3 can be used for each of the third feature subgroups (e.g., feature dimensions F7, F8, and F9).

[0143] In this example, the kernel is again shown as having a length or value of three. However, the embodiment allows the kernel to have any suitable length or number of values. The kernel value can be a normalized trainable weight parameter that can be trained during the training process (e.g., a machine learning process). The kernel value can be determined by focusing on the feature dimension of different intermediate vector representations of the same node from adjacent time snapshots. Thus, the kernel parameter value can indicate the relevance of previous snapshot values ​​of the feature dimension. The kernel length determines the number of previous snapshots considered. The kernel length can be considered a hyperparameter and can be chosen experimentally. For example, a larger kernel can capture longer-term temporal relationships and therefore provide more accurate results when the number of graph snapshots is large. However, a larger kernel also increases the computational complexity of the model. The following discusses experiments testing an optimal kernel size that captures sufficient temporal information without excessive computational complexity.

[0144] In step S618, the convolutional feature dimension data can be recombined to recreate the node embeddings, which are now temporally convolutional. This step can be combined with... Figure 5 The steps in step S518 are the same or similar. Each different feature dimension value can be combined based on its associated time snapshot. For example, the values ​​can be combined in the first time snapshot t. l The values ​​of feature dimensions F1, F2, and F3 from the first subgroup, in the first snapshot t l The values ​​of feature dimensions F4, F5, and F6 from the second subgroup, and the values ​​from the first snapshot t. lThe values ​​of feature dimensions F7, F8, and F9 from the third subgroup are combined to target the first time snapshot t. l Create the temporal convolutional embedding for the first node.

[0145] This process can be performed on a per-node embedding basis. Mathematically, the lightweight convolution process can be described as a modification of the depthwise convolution formula: (2) The softmax function is used to normalize the weight parameters and can take the following form:

[0146] Once completed, the embedded information of the temporal convolution nodes can be used to predict future events, as discussed in more detail below.

[0147] III. Additional Modules

[0148] In some embodiments, time-series learning may include additional processing functions used in conjunction with convolutions (e.g., deep or lightweight). For example, gated linear units, feedforward layers, residual connections, softmax normalization operations, and / or any other suitable tools may be used to improve the temporal convolution process.

[0149] Figure 7 A flowchart illustrating a time-series learning method with additional modular processes according to some embodiments is shown. As discussed above, the input can be a node representation computed by structural blocks. Typically, each node will be individually processed and convolved using its own different snapshot vector versions. Before being convolved (as by...), instruct), It can include a single node In time steps t All feature dimension values ​​at the location (e.g., provided by the structure block). It can represent the values ​​embedded in the same node, but in comparison... One earlier step ( t-1 (At) similar location. Can be compared Two steps earlier.

[0150] In some embodiments, in addition to convolution, gated linear units (GLUs) [Dauphin et al., 2017] can also be used to enhance the predictive power of the model. GLUs can advantageously filter out uninformative dimensions and time steps. For example, in some embodiments, in step S710, the input values ​​can first be... The feed is sent to a GLU, which can take the following form:

[0151] In the above equation, , These are learnable parameters. It is an sigmoid function, and It is the Hadamard product.

[0152] In step S712, the output of the GLU from step S710 can be used for the convolution process. For example, the feature dimension values ​​F1, F2, and F3 can be aggregated separately across time steps, and then the corresponding kernels can be used (e.g., as mentioned above regarding...). Figure 5 and 6 The feature dimension values ​​are processed separately (as discussed).

[0153] As discussed above, the embodiments can utilize depthwise convolution. In mathematical terms, depthwise convolution can involve transforming the input data (e.g., the output from step S710) using a weight matrix. The input data can be represented as a matrix with dimensions defined by the number of snapshots and the number of feature dimensions of the nodes in each snapshot:

[0154] The weight matrix can be represented as a matrix with dimensions defined by the length of the kernel and the number of feature dimensions of the nodes in each snapshot (e.g., due to the use of a different kernel for each feature dimension):

[0155] in K This is the convolution kernel length (e.g., the number of different kernel parameter values). For time steps... t and output dimensions c This can produce a depthwise convolution output matrix with the same dimensions as the input matrix:

[0156] In general, a depthwise convolution performed on data received from a GLU process can be represented as the dot product of the input data matrix and the weight matrix:

[0157] Implementation examples can be obtained by... The input matrix is ​​padded with K-1 rows of all-zero vectors before the first row to include the padded input matrix. Unlike traditional depthwise convolutions that position the convolution kernel in the middle of the target index, this embodiment can utilize a right-aligned kernel. This encodes the temporal order in a way that prevents the absorption of future information for current prediction. Therefore, a right-aligned kernel can absorb historical information into the current time step and avoids reflecting relatively future data back into the current time step. The padded input matrix can be represented as:

[0158] As discussed above, the implementation can utilize lightweight convolutions. Lightweight convolutions [Wu et al., 2019] are a specific type of depthwise convolution that shares weights across a specific channel. This further reduces the spatial complexity of the module. The output of a lightweight convolution can be represented as:

[0159] In general, a lightweight convolution performed on data received from a GLU process can be represented as:

[0160] in H L This represents the number of convolution kernels, which is a factor that reduces the number of parameters. In contrast, for the original lightweight convolution proposed in [Wu et al., 2019], some embodiments may omit softmax normalization in order to preserve the original weights. Additionally, embodiments may omit adding positional encoding, since positional information can be encoded within the convolutional layers [Islam et al., 2020].

[0161] In some embodiments, in addition to convolution, residual connections can also be used to enhance the predictive power of the model. For example, in some embodiments, in step S714, the output of the convolution from step S712 can be recombined into a single vector, thereby effectively feeding information from previous time steps forward into the current time step where convolution is being performed. The convolution values ​​can then be input into the residual connections, which can take the following form:

[0162] in The nodes are at different time steps The final output matrix of the embedding representation at position, where It is the ReLU activation function. The weight matrix can be represented as:

[0163] In some embodiments, the temporal convolutional embedding representation can be fed back to the beginning of the temporal convolution module and processed again. Any suitable number of temporal convolutional processes (e.g., 1, 2, 3, etc.) can be performed on the same embedding data. This can effectively incorporate more time steps into the convolution. For example, if the first convolution uses a kernel of length 3, then two previous time steps are used to modify the current time step through the convolution. If each time step is convolved a second time, then the two previous time steps currently used to convolve the current time step have now been modified by earlier time steps (e.g., four and five earlier time steps), and those earlier time steps can now influence the current time step.

[0164] The neural architecture according to the embodiment can use the graph structure learning layer and temporal convolutional layer defined above as modules. Figure 8 A flowchart illustrating the DyCLN architecture according to some embodiments is shown. Figure 8 As shown, the embodiment can have three modules from top to bottom: (1) a structural attention block (e.g., including 810-834), (2) a temporal convolution block (e.g., including 840, 852, 862, and 872), and (3) a graph context prediction (e.g., including 854, 864, and 874). The analysis computer can accept this. The set of graph snapshots is taken as input, and output hidden node representations (e.g., intermediate vector representations) can be generated at each time step.

[0165] The graph structure learning block module can include multiple stacked self-attention layers to extract features from nodes at different distances. Each layer can be applied independently at different snapshots with shared parameters, such as... Figure 8 As shown, this is to capture the local neighboring structure around a node at each time step (e.g., a graph snapshot). However, since the graph can change over time, the values ​​input to the layer can potentially differ across different graph snapshots. The vector representation output by the structured attention block can be represented as... It can be fed as input to the temporal attention block.

[0166] For example, Figure 8Three graph snapshots are shown, including a first snapshot 810, a second snapshot 820, and a third snapshot 830. The first snapshot 810 may be a portion of the dynamic graph at time 1. In some embodiments, time 1 may be a time range (e.g., 8 / 11 / 2021 to 8 / 17 / 2021). The second snapshot 820 may be a portion of the dynamic graph at time 2. The third snapshot 830 may be a portion of the dynamic graph at time T. The dynamic graph may include graph data, which may be, for example, communication graph data. For example, each node in the first snapshot 810 may represent a user's email address. Edges connecting nodes in the first snapshot 810 may indicate communication between users with two email addresses. In the first snapshot 810, node V may be connected to nodes 2 and 3 via edges. This may correspond to receiving emails from and / or sending emails from email addresses associated with nodes 2 and 3.

[0167] As the dynamic graph changes over time, as indicated in the subsequent snapshots, its structure can evolve. For example, a new edge can be created when two email addresses that were not previously communicating begin to communicate with each other, and a new node can be created when a new email address is created. Furthermore, nodes and edges can be removed when an email address is deactivated or when two email addresses stop communicating.

[0168] Each node in each graph snapshot can be associated with one or more characteristics. For example, a node indicating a user's email address can have characteristics such as local portion, domain, character length, sub-address, etc. For example, the characteristics of node 2 can be shown by characteristic 812 and can be different from those of node V. Similarly, node V in the third graph snapshot 830 can have neighboring nodes 3 and 4, which can be considered when determining the intermediate vector representation of time T.

[0169] Dashed arrows (e.g., arrow 813) can indicate which nodes (e.g., neighboring nodes) can be considered when performing a self-attention process on a given node. For example, node V in the first graph snapshot 810 may have neighboring nodes 2 and 3, which can be considered when determining the intermediate vector representation of node V.

[0170] The analysis computer can use the graph structure learning module as described herein to extract multiple first datasets from multiple graph snapshots. The multiple first datasets may include intermediate vector representations 814, 824, and 834 for each node of each snapshot in the multiple graph snapshots (e.g., first graph snapshot 810, second graph snapshot 820, and third graph snapshot 830). Each dataset in the multiple first datasets may include multiple vectors. In some embodiments, the intermediate vector representation 814 may include any suitable number of vectors. In some embodiments, each node corresponding to a graph snapshot may have one vector.

[0171] For example, the analysis computer can determine the first intermediate vector representation 814 of the first snapshot 810 (represented as...). The first intermediate vector representation 814 can be determined by embedding the nodes of the first graph snapshot 810 using a self-attention neural network. For example, the analysis computer can analyze node V of the first graph snapshot 810. The analysis computer can use the above equation (1) to determine the vector representing node V and its neighboring nodes 2 and 3. In some embodiments, the vector may have fewer dimensions than node V. For example, node V and its neighboring nodes can be input into an embedded self-attention neural network to determine the output (e.g., the intermediate vector representation) representing the structure of node V and its surrounding neighboring nodes 2 and 3.

[0172] The analysis computer can individually determine the intermediate vector representation corresponding to each graph snapshot. The analysis computer can determine the intermediate vector representation for any suitable number of graph snapshots. For example, the analysis computer can determine the intermediate vector representation based on each graph snapshot that has been recorded and / or measured and subsequently stored in a graph data database. In some embodiments, the analysis computer may have predetermined the intermediate vector representations, in which case the analysis computer can retrieve the intermediate vector representations from the database.

[0173] In some embodiments, after extracting a first intermediate vector representation 814 from the first snapshot 810, the analysis computer may apply positional embeddings to the intermediate vector representation to equip it with a sense of order. For example, modules can be equipped with a sense of order through positional embeddings (Gehring et al., 2017). This can embed the absolute temporal location of each snapshot. The location embedding can then be combined with the output of the structured attention block to obtain the nodes. The input representation spanning multiple time steps: The input indicates that it is fed into the temporal lightweight convolution module 840.

[0174] Next, step 840 will be discussed, in which data is input into the temporal lightweight convolution module. According to some embodiments, the temporal lightweight convolution module can perform the functions described above regarding... Figure 7 Some or all of the processes described. For example, in some embodiments, Figure 7 This can represent a temporally lightweight convolutional module. This block can also follow a similar structure with multiple stacked temporally lightweight convolutional layers. Temporally Lightweight Convolutional Module 840.

[0175] For example, in step 840, the analysis computer can use a temporal convolutional learning module to extract at least a plurality of second datasets from a plurality of first datasets across multiple graph snapshots. The plurality of second datasets may include, for example, final node representations of multiple graph snapshots. The plurality of second datasets may include the same number of graph snapshots as the plurality of first datasets. The final node representations may include any suitable number of vector representations of nodes. In some embodiments, the final node representations may include a plurality of vectors equal to the number of nodes.

[0176] For example, the analysis computer can input a first dataset (e.g., intermediate vector representations) determined from a previously executed structured self-attention neural network into a second convolutional neural network to determine a second dataset of final vector representations. The first dataset may include intermediate vector representations determined from each previous and current graph snapshot. For example, at time T=2, the first dataset may include intermediate vector representations from a first graph snapshot 810 and a second graph snapshot 820. For example, a first intermediate vector representation generated by node V in the first graph snapshot 810 may be input into the neural network together with a second intermediate vector representation generated by node V in the second graph snapshot 820.

[0177] For node V, the input could be, for example... ,in It can be the intermediate vector representation of node V at graph snapshot 1 (e.g., 810), and where This could be an intermediate vector representation of node V at graph snapshot 2 (e.g., 820). Although a single node is discussed, it should be understood that the analysis computer can determine the intermediate vector representation of every node in each graph snapshot. This input representation of node V can form the encoding of the local structure surrounding node V. The value can be the query input for the convolution process and can be used to convolve on the historical representation of node V, thereby tracking The evolution of its value over time.

[0178] For example, between the first snapshot 810 and the second snapshot 820, node V, representing an email address in the communication network, can begin communicating with a new email address represented by node 4. Since the analysis computer determines the intermediate vector representation of the local structure of node V, changes in the local structure over time can be analyzed.

[0179] The temporal lightweight convolutional module 840 can determine weights, indicating the degree of relevance between a portion of a given input and the rest of the input, via a training process (e.g., neural network learning). For example, the analysis computer can determine weights indicating the degree of relevance between a first intermediate vector representation of a first node and several other intermediate vector representations of the first node corresponding to subsequent time snapshots. These weights can then be used in a convolutional kernel to convolve the intermediate vector representations and produce the final vector representation.

[0180] For example, a first node representing a first email address can correspond to three defined intermediate vector representations. Each intermediate vector representation indicates the local structure of the graph data surrounding the first node. For example, a first intermediate vector representation could indicate the structure surrounding the first node during a first week (e.g., based on email interactions that occurred during the first week). A second intermediate vector representation could indicate the structure surrounding the first node during a second week (e.g., based on email interactions that occurred during the second week). A third intermediate vector representation could indicate the structure surrounding the first node during a third week (e.g., based on email interactions that occurred during the third week).

[0181] The analysis computer can determine the weights indicating the similarity of a portion of the input (e.g., the first intermediate vector representation) by focusing on the rest of the input (e.g., second and third intermediate vector representations). For example, week one might have a similar local structure to week two because users might have continued email conversations from week one. Week one might have a different local structure than week three because email conversations from week one might have ended. New email conversations might have started in week two and might continue into week three, so week two might have a similar local structure to both week one and week three, even if the structures of week one and week three are different. Therefore, the analysis computer can determine that week two has a higher weight value relative to week three than week one.

[0182] In this way, the analysis computer can determine the degree of correlation between email behavior in the first and second weeks and that in the third week. For example, the analysis computer can determine that, when considering the third week, the behavior in the first week has a weight value of 0.1, and the behavior in the second week has a weight value of 0.3. A relative weight of 0.6 can also be assigned to the third week, which can indicate the degree of independence of the behavior in the third week from that in the previous two weeks.

[0183] These week-based weights are presented conceptually. As discussed above, in practice, different weights can be determined for each feature dimension of a week, rather than a single weight for each node of that week. For example, a first set of three weights for the first, second, and third weeks can be determined for the first feature dimension (e.g., email length), a second set of three weights for the first, second, and third weeks can be determined for the second feature dimension (e.g., the email time of the day), and a third set of three weights for the first, second, and third weeks can be determined for the third feature dimension (e.g., email subject). These feature-dimension-specific weights can be used as kernel values ​​for different feature-dimension kernels during convolution. Any suitable training process, such as machine learning via a neural network, can be used to determine these kernel weight parameters.

[0184] As an additional example, the first node representing a resource provider can correspond to five defined intermediate vector representations. Each intermediate vector representation indicates the local structure of the graph data surrounding the first node. For example, the first intermediate vector representation could indicate the structure surrounding the first node during the summer (e.g., the time of the graph snapshot is in the summer). The second, third, fourth, and fifth intermediate vector representations could indicate the structure surrounding the first node during the fall, winter, spring, and subsequent summer.

[0185] The analysis computer can determine weights indicating the similarity between an input value (e.g., a first intermediate vector representation) and the rest of the input (e.g., second, third, fourth, and fifth intermediate vector representations). In this example, due to the similar local structure around the first node during the summer, the analysis computer can determine a larger weight between the first and fifth intermediate vector representations. For example, the resource provider represented by the first node may transact with a similar number and group of users during the summer, while the local structure may decrease (or change in any suitable way) during the fall, winter, and spring.

[0186] In this way, the computer analysis can determine the correlation between trading behavior in the first summer, autumn, winter, and spring and the second summer. For example, the analysis can determine that, when considering the second summer, the behavior in the first summer has a weight of 0.2, the behavior in autumn has a weight of 0.15, the behavior in winter has a weight of 0.1, and the behavior in spring has a weight of 0.15. A relative weight of 0.4 can also be assigned to the second summer, which can indicate the degree of independence of the behavior in the second summer from the first four seasons.

[0187] These season-based weights are presented conceptually. As discussed above, in practice, different weights can be determined for each feature dimension of the season, rather than a single weight for the node of the week. For example, a first set of five weights for the first summer, fall, winter, spring, and second summer can be determined for the first feature dimension (e.g., transaction amount), a second set of five weights for the first summer, fall, winter, spring, and second summer can be determined for the second feature dimension (e.g., transaction location), and a third set of five weights for the first summer, fall, winter, spring, and second summer can be determined for the third feature dimension (e.g., type of item purchased). These feature-dimension specific weights can be used as kernel values ​​for different feature-dimension kernels during convolution. Any suitable training process, such as machine learning via neural networks, can be used to determine these kernel weight parameters.

[0188] The analysis computer can determine the final node representation for each node at each time step in step 840. For example, the analysis computer can determine the first set of final node representations 852 for the first time step based on the weights determined by the convolutional neural network. The second set of final nodes in the second time step represents 862 (e.g., The third set of final nodes for the most recent time step represents 872 (e.g., ).

[0189] The final node representation can be determined, for example, by performing a convolution operation on the intermediate vector representation using a dimensional feature-specific kernel with kernel weight parameters. The final node representations for each time step can then be assembled to create a second dataset.

[0190] For example, to determine the final value of the first feature dimension of the first node at time step 3, the kernel of that feature dimension can be applied to the intermediate values ​​of that feature dimension from time steps 1, 2, and 3 (e.g., if the kernel has a length of 3). This can include computing the dot product of the three intermediate values ​​using the three kernel weights. For example, the first feature dimension is the length of an email in characters. For the first, second, and third time steps, the intermediate values ​​of this feature dimension are 400 characters, 200 characters, and 300 characters, respectively, and the kernel weights are 0.1, 0.2, and 0.7. The dot product then produces a final value of 290 characters. This final value is used as the final feature dimension for the third time step (e.g., replacing the intermediate value of 300 characters). This convolution process can be performed for the feature dimension of each node at each time step. Therefore, the intermediate node representation can be transformed by combining it with a set of previous versions of the same node representation on a feature-weighted basis.

[0191] Conceptually, performing this convolution to transform intermediate node representations into final node representations can be viewed as similar to performing a task that utilizes values ​​from the most recent time step (e.g., values ​​from two previous time steps when the kernel is length 3) to influence the current time step. Intermediate node representations can be based solely on activities and interactions that occurred within that time frame (e.g., the week, season, etc.). By convolving to provide the final node representation, past activities and interactions from previous time frames are considered and incorporated into the current time frame, even if these past activities and interactions are given less weight (e.g., depending on the kernel weight values). The intermediate node representation is somewhat similar to previous intermediate node representations, or moves towards previous intermediate node representations in the vector space. This effectively moves the vectors back towards the previous version, or can be viewed as reducing or slowing down the movement of vectors towards new positions over time. The magnitude of the transformation and movement towards previous versions of the intermediate node representation (e.g., past relevance) is given by the kernel weight values. Therefore, the final node representation can be created based on longer time frames including multiple snapshots with different local structures, and different snapshots can have different amounts of influence based on the kernel weight values.

[0192] Therefore, the final node representation It can be a vector representing the change of the node's local structure over time, with the time dimension based on the length of each time step and the length of the convolution kernel. For example, the final node representation corresponding to node V could include a vector indicating the addition of communication with node 4 at second graph snapshot 820 and the removal of node 2 at third graph snapshot 830.

[0193] In some embodiments, the analysis computer can determine multiple final node representations for multiple snapshots. Each final node representation for each snapshot can correspond to a node in the graph data. These vectors can then be used in any suitable local graph context prediction process. For example, in some embodiments, the analysis computer can use the final node representations to train neural networks, SVMs, etc. The analysis computer can train machine learning models known to those skilled in the art.

[0194] Next, we will discuss graph context prediction. In some embodiments, to ensure that the learned representation captures both structural and temporal information, the implementation may define an objective function that preserves the local structure around a node across multiple time steps.

[0195] Implementation examples may use nodes In time steps Dynamic representation at (e.g., To predict the time of occurrence At the node The occurrence of nodes of local neighbors. For example, in some embodiments, the analysis computer may use a binary cross-entropy loss function at each time step to facilitate nodes that co-occur in a fixed-length random walk having similar vector representations. For example, this is given by the following equation:

[0196] In the above equation, It can be an S-shaped function. It can be in time The snapshot of the graph at that time and the node A set of nodes that occur during a fixed-length random walk. It can be a snapshot. The negative sampling distribution, and It can be a negative sampling ratio. The negative sampling ratio can be an adjustable hyperparameter to balance positive and negative samples.

[0197] In steps 854, 864, and 874, the analysis computer can determine predictions about one or more nodes at a future time (e.g., in a future graph snapshot). This can be done using classification and / or regression models. For example, the analysis computer can base its predictions on the representation of the final node. The trained model determines whether two nodes will be connected to each other via an edge. Steps 854, 864, and 874 can be collectively represented as the final node representation from each step. Combine them into a second dataset and use the second dataset for prediction (e.g., using classification and / or regression models).

[0198] The model can include any suitable machine learning model. The analytical computer can perform any suitable prediction based on the context of the graph data. For example, the analytical computer can use a trained neural network trained on the final node representations to perform graph context prediction. As an illustrative example, a second dataset can be used as input to a machine learning model, such as a regression model or a classification model, to make predictions, such as whether two nodes will be linked or the category to which the nodes will belong.

[0199] For example, the second dataset could correspond to graph data that includes nodes representing email addresses. The graph data could include three graph snapshots, each containing email interaction data over a week. The final node representation of the first node (e.g., for the first email address) could represent the evolution of the graph structure in the most recent time step. For example, the final node representation of the first node at the third time step could represent the evolution in the previous two time steps. This could represent the evolution of a user's initiated, pending, and concluded email conversations via the first email address, as described above.

[0200] The analysis computer can then make predictions about the first email address. For example, the analysis computer can determine whether the first email address will communicate with (e.g., connect to) the second email address in the fourth week (e.g., the fourth snapshot). In this example, the analysis computer can predict that the first email address will connect to the second email address in the fourth snapshot because the connection between the first and second email addresses in previous snapshots was used for an ongoing email conversation, and the probability that the conversation will be completed before the fourth snapshot is low.

[0201] In some embodiments, the final vector representations of two nodes can be used to compute a prediction of whether two nodes (e.g., email addresses, authors, etc.) will interact (e.g., be edge-connected) in future time-graph snapshots. For example, the analysis computer can compute a Hadamard product using two vectors: a first final vector representation of the first node and a second final vector representation of the second node (e.g., a vector corresponding to the most recent snapshot). The Hadamard product can be used as a vector representing a potential link between the two nodes. The analysis computer can then feed this potential link vector into a logistic regression classifier to compute the probability that the link exists. The parameters of this logistic regression classifier can also be trained based on training data.

[0202] The analysis computer can then perform additional processing, such as, but not limited to, performing prediction-based actions. These actions may include sending the prediction message to another device, determining whether the probability value associated with the prediction (determined by a machine learning model) exceeds a predetermined probability threshold, and / or any other appropriate processing of the prediction. In one instance, the analysis computer may send an advisory notification indicating that a transaction is likely to occur, or that a currently attempted transaction is unlikely to occur and is therefore likely fraudulent.

[0203] Figure 9 A flowchart illustrating a dynamic graph representation learning process according to some embodiments is shown. This will be described in the context of analyzing communication graph data using analytical computers. Figure 9 The method is illustrated in the figure. However, it should be understood that the invention can be applied to other situations (e.g., analyzing other types of graph data). In some embodiments, each of a plurality of graph snapshots may include a plurality of nodes. Each of a plurality of graph snapshots may include graph data associated with a timestamp.

[0204] In step 902, the analysis computer can use a graph structure learning module to extract multiple first datasets from multiple graph snapshots. The multiple first datasets may include intermediate vector representations of each node in each of the multiple graph snapshots. In some embodiments, extracting the multiple first datasets may further include determining an intermediate vector representation for each node for each of the multiple graph snapshots, based on learned coefficients and intermediate vector representations corresponding to neighboring nodes.

[0205] In step 904, the analysis computer can extract multiple second datasets from multiple first datasets using a temporal convolution module across multiple graph snapshots. The multiple second datasets may include the final vector representation of each node in each of the multiple graph snapshots. In some embodiments, extracting the multiple second datasets may further include determining the final vector representation of each node based on convolutions of intermediate vector representations corresponding to the same node at different snapshots. The different snapshots may be a sequence of snapshots immediately preceding the current snapshot. In some embodiments, the intermediate and final vector representations of each node at each snapshot may be embeddings of each node in a vector space representing the characteristics of multiple nodes.

[0206] In step 906, the analysis computer can utilize at least multiple second datasets to perform graph context prediction. For example, the analysis computer can use at least multiple second datasets to train a machine learning model. The analysis computer can then use the machine learning model to determine predictions, such as whether two nodes will be connected by an edge in a future graph snapshot.

[0207] In step 906, the analysis computer may perform additional processing, such as, but not limited to, performing prediction-based actions. These actions may include sending the prediction message to another device, determining whether the probability value associated with the prediction (determined by a machine learning model) exceeds a predetermined probability threshold, and / or any other appropriate processing of the prediction. In one instance, the analysis computer may send an advisory notification indicating that a transaction is likely to occur, or that a currently attempted transaction is unlikely to occur and is therefore likely fraudulent.

[0208] Embodiments of the present invention can advantageously generate node embedding representations that include both local structural information and temporal evolution information. Furthermore, these results can be achieved through efficient and scalable processes. For example, temporal convolution can have linear complexity relative to the number of snapshots of the input graph (e.g., with...). t (Proportional). This provides a significant improvement over other time analysis methods, such as time self-attention (e.g., where each time step attends to every other time step and uses the entire graph dynamic history), which have quadratic complexity relative to the number of input graph snapshots (e.g., with). t 2(Proportional). Methods with linear complexity can process longer sequences of graph snapshots more efficiently than methods with quadratic complexity (e.g., with less processing power, memory, and processing time).

[0209] Embodiments of the present invention can further improve efficiency by incorporating specific convolution techniques. For example, depthwise convolution can reduce the complexity of feature dimensions from... F 2 (As generated by modeling via Graph Attention Network (GAT)) reduced to F Furthermore, lightweight convolutions can further reduce spatial complexity by sharing kernel parameters across multiple feature dimensions.

[0210] The table below (Table 1) compares the spatial and temporal complexity of the Dynamic Graph Light Convolutional Network (DGLC) according to embodiments of the present invention with that of DySAT and DybAERNN, which are RNN-based and attention-based alternative models for dynamic graph modeling, respectively. Table 1:

[0211] Spatial complexity analysis: According to some implementation examples, DGLC The overall spatial complexity is ,in It is the number of nodes in a single graph snapshot. That is the corresponding number of edges. It is the feature dimension. It is the number of time steps. This refers to the number of convolutional kernels. Table 1 describes a comparison of the spatial complexity among the selected models, and the selected models are described in more detail below. It should be noted that in graphs with long dynamic evolution histories (which is often the case in many practical settings), DynAERNN is... Dominated, and DySAT is by Dominant. In practice, when in and When the memory space is large, it is a limiting factor for both DynAERNN and DySAT, which will be discussed in more detail below.

[0212] Time complexity analysis: Similarly, DGLC The implementation example has been achieved. The overall time complexity, where the kernel size When smaller, the dominant term is The time complexity of DySAT can be expressed as: This includes enabling large-scale modeling. Inefficient when creating dynamic graphics As an RNN-based model, DynAERNN has sequential operation dependencies, which makes parallel processing infeasible and results in a significantly slower actual training time compared to both attention-based and convolution-based methods. The relative complexity is discussed in more detail below.

[0213] IV. Experiment

[0214] In this section, the effectiveness of DGLC is evaluated on link prediction tasks across six real-world datasets, compared to five state-of-the-art baselines. The following experiments aim to answer the following research questions: (RQ1) How does DGLC perform link prediction tasks compared to other state-of-the-art methods? (RQ2) Is the training more efficient when compared with representative RNN-based and attention-based dynamic graph representation learning methods? How effective and robust are each component of (RQ3)DGLC in modeling dynamic graph evolution?

[0215] A. Dataset

[0216] Experiments were conducted using four different real-world dynamic graph datasets, comprising three communication networks and one rating network. Table 3 summarizes the detailed statistics for these datasets. Specifically, Enron and Radoslaw It includes email interactions between employees, where nodes represent employees and links represent exchanged emails; UCI This includes message interaction between users in online communities; and ML-10M This is a binary network describing the movies tagged by different users over time. More details about the dataset are provided below.

[0217] B. Experimental Setup

[0218] Five state-of-the-art graph learning algorithms were selected for evaluation, including two static graph learning methods. These algorithms represent a diverse set of techniques commonly used in graph representation learning. Specifically, node2vec [Grover and Leskovec, 2016], GraphSAGE [Hamilton et al., 2017], DynGEM [Goyal et al., 2018], DynAERNN [Goya et al., 2020], and DySAT [Sankar et al., 2020] were chosen. Further details about the baseline methods are provided below.

[0219] Using PyTorch [Paszke et al.] ， To be implemented in 2019 DGLCFor the two Enron datasets, the experimental procedure was carried out by... A structured attention layer consists of several attention heads, each of which computes independently. Features, in total Each feature is used in all other datasets. One and The two structural attention layers of each attention head are evaluated, and each attention head calculates... Features, in total One and The experiment also performed a grid search to determine the optimal convolutional kernel size and number of kernels at each layer of the time series learning module. Adam [Kingma and Ba, 2015] was used as the optimizer with weight decay as regularization to train the module in all experiments. DGLC Da has Batch size Each model underwent several rounds of testing. For each model, the experimental procedure used three different random seeds to perform training and evaluation, and the average results and corresponding standard deviations are reported. About DGLC Further details regarding the hyperparameter settings for other baselines are provided below.

[0220] C. Link prediction experiment (RQ 1)

[0221] In this section, the experimental procedure is described, and the results and insights observed on the future link prediction task are reported.

[0222] Task Description. Experimental Procedure: Future link prediction was chosen as the baseline for evaluation. DGLC The task of effectiveness is widely used in the evaluation of dynamic graph representation learning [Sanker et al.]. ，

[2020] . In particular, the experimental procedure used snapshot sequences. Come to train DGLC And other baselines. The task is to use targeted... The node representation learned from the latest snapshot To predict the existence of links For each dataset, the experimental procedure was conducted using up to [number missing] [units missing]. The snapshot of the graph is used to train individual models at each time step The evaluation model is used for each graph snapshot. ,and This indicates the number of initial time steps (graph snapshots) used for evaluation; for diversity purposes, this number varies across different datasets. The experimental procedures are presented in Table 3 for the different datasets used. .

[0223] Experimental setup. Each dataset was split into a sequence of discrete graph snapshots, where each snapshot corresponds to a fixed time interval containing a sufficient number of links. In each experiment, the previous... Model training is performed using snapshots. The learned node representations are then obtained. Then, for any two nodes and The experimental process will The probability is defined as:

[0224] in It is a scoring function that takes two node embeddings as input. In the experiments, logistic regression was used as the classifier. Specifically, based on... The classifier is trained using equally sampled linked and unlinked node pairs. For the linked set... The experimental procedure was randomly selected. For training, Used for verification, and For testing purposes.

[0225] Evaluation metrics. Given that link prediction can be viewed as a binary classification problem, the experimental procedure was chosen... Receiver operating characteristics Area under the curve The Area Under the Receiver Operating Characteristic Curve (AUC) metric is used to measure the performance of different models that follow the same practices in existing work on dynamic graph representation learning [Sankar et al., 2020; Kumar et al., 2020]. Experimental procedures are evaluated using both macro-AUC scores and micro-AUC scores. When the experimental procedure at each... When evaluating models on graph snapshots, for each model, the experimental procedure calculates the final metric score of the model by averaging the AUC scores obtained across all graph snapshots on which the model is evaluated. Specifically, macro AUC is calculated by treating performance from all time steps equally, while micro AUC considers the individual contributions of each time step based on the number of evaluation links.

[0226] Results and Discussion. The experimental procedures are shown in Table 2 for macroscopic AUC results and Table 4 for microscopic AUC results. Observations include: DGLC Achieve superior performance on most datasets. Specifically, compared to other baselines, DGLC average have The macroscopic AUC improvement. This indicates that, compared to other methods, DGLCIt can better capture both the dynamics of the structure graph and the dynamics of the time graph. Additionally, DGLC They also tend to maintain smaller differences, which suggests DGLC It is more robust to random weight initialization. This can be attributed to the regularization capabilities provided by lightweight convolutions and the fine-grained control obtained from GLU.

[0227] Some dynamic baselines, such as DynGEM and DynAERNN, exhibit poor performance on certain datasets. The results of the hyperparameter search and analysis, described in more detail below, further indicate that the performance of these methods may be sensitive to hyperparameter values, particularly for... Sensitive. Smaller This may slow down the embedding optimization process and reduce model performance.

[0228] Static methods can exhibit different performance depending on whether aggregated or non-aggregated graphs are used on different datasets. For example, a model trained on an aggregated graph may perform differently. Enron-II The model trained on the aggregated graph performs better, while the model trained on the non-aggregated graph performs better. Radoslaw The performance was better. This also indicates that using temporal information to guide the learning of dynamic graph representations is a top priority.

[0229] Below, in Table 2: Linked predictions of macroscopic AUC results. Two versions of the static method evaluation are presented: with and without information aggregation. GraphSAGE results show the best-performing aggregator: For GCN, is the average value, † is the average pooling, and ‡ is the max pooling.

[0230]

[0231] D. Efficiency Comparison (RQ 2)

[0232] In this section, the experimental procedures empirically demonstrate the effectiveness of the embodiments. DGLC The efficiency advantages are highlighted. Specifically, the experimental procedure compares the DGLC model with DySAT and DynAERNN at different time steps regarding the average training time per epoch. DySAT was chosen because it not only performs better than other dynamic baselines but also offers better scalability compared to RNN-based models due to its temporal self-attention advantage. To fully evaluate the scalability of these two models on long-term dynamic graphs, the experimental procedure uses the Yahoo employee message dataset. YHM Furthermore, the dynamic graph sequence was sampled in one thousand steps. Details of the experimental setup are provided below.

[0233] Efficiency comparison Figure 11 As shown in the diagram. The results are as expected, since training time is linearly proportional to the number of time steps, and DySAT describes a quadratic complexity pattern with increasing time steps due to its self-attention mechanism. More importantly, DySAT quickly fills most of the GPU memory as the number of time steps increases, thus achieving the desired results as discussed in Section 4.4. Memory requirements make it virtually impossible to scale to longer sequences or multi-level setups. In contrast, DGLC It requires significantly less memory even at the largest time step and scales linearly to the number of time steps. The empirical results validate the theoretical complexity analysis, thus demonstrating that DGLC is more efficient, i.e., DGLC is powerful in modeling dynamic graph datasets with long-term series.

[0234] Figure 10 and Figure 11 exist DGLC Compared with other dynamic baseline methods DGLC, A comparison of the average training time per epoch between DySAT and DynAERNN. For additional efficiency studies, such as... Figure 5 As shown, DynAERNN scalability deteriorates over time, causing it to exhibit an exponential growth pattern immediately, even for datasets with a finite number of time steps. The results confirm this. DGLC The efficiency advantage of convolution time modeling compared to RNN-based methods.

[0235] E. Ablation study (RQ 3)

[0236] The experimental process involves ablation studies to investigate D. GLC How different components can affect a model's temporal dynamics modeling capabilities? Specifically, the experiment selects four components in the time series learning module: 1) GLU; 2) feedforward layer; 3) residual connection; and 4) weighted softmax normalization in lightweight convolution, and observes how enabling and disabling different components affects model performance. The experiment selects two datasets ( Enron-I and Radoslaw This was used to cover dynamic graphs with different time step lengths. Detailed experimental setup and results are further provided below. Observations are summarized below: GLU and residual connections are crucial in temporal dynamics modeling. Experiments showed that models with both GLU and residual connection components consistently outperformed other variants on both datasets. This suggests that dimension-wise gating and integrating the original time-series signal can help capture temporal evolution patterns in dynamic graphs more effectively.

[0237] DGLC It is robust to enabling or disabling weighted softmax normalization in lightweight convolutions. The original lightweight convolution [Wu et al., 2019] adds weighted softmax normalization after depthwise convolution; the results show that models with or without this normalization operator perform comparably on both datasets and provide clues for further simplification of lightweight convolutions.

[0238] Compared to several variants such as GLU-only or FC-only models, applying only lightweight convolutions without using other components demonstrates competitive performance. This not only indicates the robustness of lightweight convolutions but also shows that combining different components can be more powerful than using any single technique.

[0239] F. Conclusion

[0240] Embodiments of the present invention provide DGLC A novel GNN framework is proposed that effectively and efficiently learns node representations on discrete dynamic graphs. Specifically, the embodiment provides a graph structure learning module, including multiple stacked layers of graph attention blocks for learning structural information of each graph snapshot, and a time-series learning module that combines GLU, lightweight convolutions, and residual connections to capture evolutionary patterns with temporal information. Experimental results show that it achieves optimal training efficiency on real-world datasets. DGLC It offers significant performance gains compared to state-of-the-art baselines.

[0241] V. Algorithm

[0242] Embodiments of the present invention can be implemented using the following algorithm. The algorithm input can be: all graph snapshots G={ 1 , 22 ,…, T T} , t = ( t t , E t ) ， t V ， 1 ≤ t ≤ T ; Ls The number of learning layers in the graph structure. L t The number of learning layers in time is [number]. The algorithm output can be: at each time step of capturing the evolution pattern of the dynamic graph. t This applies to all v Node embeddings learned in V The algorithm can take the following form: 1: for t = 1. . .do

[0243] 2: for l s = 1 ...L s do

[0244] 3: Graph Structure Learning Layer (GSLL)

[0245] 4: = GSLL ( , ), =

[0246] 5: end for

[0247] 6: =

[0248] 7: for l t = 1 ...L t do

[0249] 8: = GLU ( )

[0250] 9: Graph Time Learning Layer (STLL)

[0251] 10: , . . . , = STLL ( , . . . , )

[0252] 11: end for

[0253] 12: UpdateGradient()

[0254] 13: end for

[0255] 14: ForwardPass()

[0256] 15: For all v ∈ V, return

[0257] VI. Hyperparameter Settings

[0258] This section discusses DGLC And details of other baseline hyperparameter settings. As shown above, in DGLC The loss function used in this study attempts to promote similar representations of nearby nodes at different times [Hamilton et al., 2017]. Nearby nodes are selected from random walks, and for each node, 10 walks with a context window size of 10 and a length of 40 are sampled. For each time step, 10 negative samples are used with a negative sampling distribution based on the node degree with a smoothing parameter of 0.75. For the remaining time steps... Enron For datasets other than those used in the previous example, two structural layers were employed, with 16 and 8 attention heads, each calculating 16 features, for a total of 256 and 128 features respectively. Enron A single structural layer is used, with 16 attention heads, each calculating 8 features, for a total of 128 features. A weight decay parameter of 5 × 10⁻⁶ is used. -4 The Adam optimizer was used for training, while the loss rate of the structure learning module was 0. . 1. The model was trained for 200 epochs with a batch size of 256. For the time series learning module, two lightweight convolutional layers were used. Validation set performance was assessed using a grid search from {10}. -4 , 10 -3 , 5 × 10 -3 , 10 -2 Adjust the learning rate, from {1 , 0 . 1 , 0 . 01} Adjust the negative sampling rate, from {3} , 5 , 7} Adjust the kernel size, and from {4} , 8 , 16 , 32} Adjust the number of convolutional kernels.

[0259] All baseline hyperparameters were tuned according to their recommendations. For node2vec, following the paper's recommendations, a 10-walk approach with a context window size of 10 and a length of 80 was used, starting from {0... . 25 , 0 . 50 , 1 , 2 , 4) Adjust the input-output and return hyperparameters. p , q And from {1 , 10} Adjust the training epochs. For GraphSAGE, following the original paper's setup, a two-layer model with sample sizes of 25 and 10 was used, and the number of training epochs was adjusted based on the validation results from {10}. , 50 , Choose the best-performing aggregator and training round from 100.

[0260] DynAERNN was adapted following the guidelines suggested in this article. (From {0) . 1 , 1 , 2 , 5 , 8} Adjust scaling and normalization hyperparameters β , from {10 -4 , 10 -6}Adjustment ν 1 and from {10 -3 , 10 -6}Adjustment ν 2. Similarly, adjust DynGEM from {10 -5 , 10 -6 Adjusting scaling and normalization hyperparameters α , from {0 . 01 , 0 . 05 , 0 . 1 , 1 , 2 , 5 , 8} Adjustment β , from {10 -4 , 10 -6}Adjustment ν 1 and from {10 -3 , 10 -6}Adjustment ν 2. For DySAT, the same as... DGLCThe same structure learning module, optimizer, and loss function settings were used because they provided optimal performance. Sixteen temporal attention heads with a temporal loss of 0.5 were used as recommended, with negative sampling rates adjusted from {1, 0.1, 0.01} and the number of temporal layers adjusted from {1, 2}. For all methods, the node embeddings to be learned were 128.

[0261] VII. Dataset

[0262] This section provides additional dataset details. To obtain a dynamic graph as a sequence of graph snapshots, the entire dataset is divided into informative snapshots at fixed time intervals based on consecutive timestamps provided in the original data, while ensuring that each snapshot contains sufficient interactions / links between nodes. The weight of a link is determined by the number of interactions between corresponding nodes in a given snapshot.

[0263] Table 3 below: Dataset statistics: Number of nodes (| V |); Number of edges (| E |); Number of time steps ( T ); Initial time steps of the assessment ( K ).

[0264]

[0265] Graph snapshots are taken at fixed time intervals, ensuring that each snapshot includes a sufficient number of links. For Enron and UCI Two time-segmentation strategies are applied to obtain graph snapshot sequences of different granularities, enabling better rendering in diverse scenarios. DGLC Comparisons were made with other baselines. In the experiments, one-hot encoding of node IDs was used as the node feature for these datasets. However, DGLC It is also designed to support datasets that include node attributes. The scripts used to process the dataset, as well as all the data processed, will be exposed.

[0266] Enron. The original Enron dataset is available at https: / / www.cs.cmu.edu / ~. / enron / , and the interactions among Enron employees primarily focus on her. Two versions of animated graphs are obtained from Enron. (Includes...) Enron-1 using a snapshot of a graph Months are used as time intervals, and have Enron-2 snapshots Days are used as time intervals.

[0267] UCI. The original UCI dataset is available at http: / / networkrepository.com / opsahl_ucsocial.php. This dataset tracks message interactions between users in the University of California, Irvine's online community. Similar to Enron, two versions of the dynamic graph are obtained from UCI. (Contains...) UCI-1 use of individual image snapshots Days are obtained as time intervals and have UCI-2 snapshots Days are used as time intervals.

[0268] Radioslaw. The original Radoslaw dataset is available at http: / / networkrepository.com / ia-radoslaw-email.php. This dataset contains internal email communications between employees of a manufacturing company. A snapshot of the image is obtained by using The time interval for creating a day is specified.

[0269] ML-10M. The original ML-10M dataset is available at http: / / networkrepository.com / ia-movielens-user2tags-10m.php. This dataset tracks the tagging behavior of MovieLen users; the link represents the tags applied to movies by the user, and the nodes correspond to the user and the movie. Image snapshots are obtained by using Created at monthly intervals.

[0270] YHM. The original YHM dataset is available at http: / / networkrepository.com / ia-yahoo-messages.php. This dataset tracks messages sent between Yahoo employees. Due to the large size of the original dataset, which leads to resource exhaustion issues for most methods, node sampling techniques are employed to extract messages with the highest density. Nodes, and then utilize To create fine-grained time steps by time intervals A snapshot of the image.

[0271] VIII. Link Prediction Experiment

[0272] A. Experimental Setup

[0273] For static graph representation learning methods, to ensure a fair comparison, two strategies are used to transform dynamic graphs to make training and inference feasible. One strategy is to train models using only the most recent graph snapshots so that they can learn the latest graph information. The other strategy constructs an aggregated supergraph for training while setting link weights to cumulative weights independent of the timing of link occurrences. This allows the model to access the entire history of graph snapshots and obtain a summary of all graph sequence information.

[0274] node2vec [Grover and Leskovec, 2016]: A static graph embedding algorithm that learns node representations through neighbor context prediction and biased random walk sampling.

[0275] GraphSAGE [Hamilton et al., 2017]: A spatial static graph representation learning algorithm that learns node embeddings through node sampling and neighbor aggregation.

[0276] DynGEM [Goyal et al., 2018]: A dynamic graph learning algorithm that learns node embeddings progressively through a deep autoencoder.

[0277] DynAERNN [Goyal et al., 2020]: A dynamic graph learning algorithm that utilizes both dense autoencoders and recurrent units to capture temporal graph dynamics.

[0278] DySAT [Sankar et al., 2020]: A dynamic graph learning algorithm that learns node representations by decoupling structural self-attention mechanisms and temporal self-attention mechanisms on graph snapshots.

[0279] B. Experimental Results

[0280] Table 4 below presents the microAUC results of the link prediction experiment described in the experimental section above.

[0281] IX. Complexity Analysis

[0282] A. Spatial complexity

[0283] exist DGLC In the graph structure learning module, the space complexity of the graph attention layer is... O ( F 2 + NTF + ET ),in N It represents the number of nodes in a single graph snapshot.E It is the corresponding number of edges, and F It is the feature dimension. For a lightweight convolutional layer in a time series learning module, the space complexity is O(n log n). O ( NTF + HK + F 2 ).therefore, DGLC The overall spatial complexity is O ( F 2 + NTF + ET + HK On the other hand, DySAT adopts and has O ( F 2 + NTF + ET )of DGLC The same structural space complexity. Utilizing the time-dependent self-attention layer. O ( F 2 + NTF + NT 2 DySAT generates O ( F 2 + NTF + ET + NT 2 The total space complexity is ). For DynAERNN, since it utilizes a fully connected encoder to capture the low-dimensional temporal representations of node neighbors, the total space complexity is . O ( TF 2 + NTF + F 2 ).

[0284] B. Time complexity

[0285] The time complexity of a single graph attention layer in the graph structure learning module is O ( NF 2 + EF It should be noted that structure learning is time-independent and therefore can be parallelized. The time complexity of a single layer in the time series learning module is... O ( TKF ),in T It is the number of time steps, and K That's the kernel size. When GLU and fully connected layers are added, the time complexity becomes... O ( TFK + TF 2 Since time computation is independent between nodes, it can also be parallelized to further improve time complexity. When both modules have only a single layer, the time complexity of DGLC for all nodes in a non-parallelized graph sequence is... O ( NTF 2 + ETF + NTFK ), where K When it is small, the dominant term is NTF 2 .

[0286] As described above, two state-of-the-art models were chosen: DynAERNN and DySAT, which serve as baselines in this experiment. These can be viewed as dynamic graph representation learning methods representing two main categories: RNN-based models and attention-based models. For DySAT, the time complexity per layer of temporal self-attention is... O ( T 2 F This is because DySAT requires each time step to pay attention to every other time step in the sequence. When using the same graph attention layer as the structure learning module, the total time complexity of DySAT with one structure attention layer and one time attention layer for all nodes in the graph sequence without parallelization is... O ( NTF 2 + ETF + NT 2 F It should be noted that DySAT includes in its total time complexity... T 2 The term, which makes the modeling have a large T The efficiency of dynamic graphs is low.

[0287] Each time layer of DynAERNN includes a fully connected layer as an input encoder and an LSTM unit as a recursive unit, which processes... T The snapshot has time complexity. O ( ETF + TF 2 However, due to the sequential dependencies of recursive layers, they cannot be processed in parallel, resulting in a significantly slower actual training time compared to attention-based methods. Applying a convolution-based solution... DGLC Treatment of history and TMesolinear time complexity has no order dependency, making it powerful for modeling dynamic graphs with long time series. A detailed comparison can be found in Table 1.

[0288] X. Efficiency Comparison

[0289] A. Experimental Setup

[0290] This section provides experimental details for the efficiency studies. To ensure fairness, while maintaining all common settings (i.e., batch size), the experimental procedures were optimized for... DGLC Both models use the same architecture learning module setup as DySAT and the same number of time layers. Both models are implemented via PyTorch, and the experimental procedure calculates the average training time per 10 epochs across 10 epochs for every 100 time steps from 100 to 800 on the YHM dataset, running on an Nvidia Tesla V100 with 64 CPU cores.

[0291] The experimental process also includes... DGLC Additional efficiency studies were conducted compared with DynAERNN to empirically demonstrate... DGLC Compared to RNN-based dynamic graph learning methods, this offers an efficiency advantage. Similar to previous research, the experimental process will [determine / improve] at different time steps. DGLC The average training time per epoch was compared with DynAERNN, where both utilized the complete sequence of dynamic graph snapshots. The experimental procedure used the original DynAERNN implementation based on TensorFlow, and both models were run on an Nvidia Tesla P100 with 48 CPU cores. UCI-I The average training time per epoch is calculated for each time step from 2 to 13 on the dataset.

[0292] B. Experimental Results

[0293] Figure 4 and Figure 5 exist DGLC The average training time per epoch is compared with other dynamic baseline methods, DySAT and DynAERNN. For both graphs, the x-axis represents the time step, and the y-axis represents the average time spent per epoch when training the corresponding graph snapshot at the corresponding time step. The experimental procedure is also indicated by blue. DGLC And orange is used to indicate the baseline model. For additional efficiency studies, such as... Figure 5 As shown, DynAERNN scalability deteriorates over time, causing it to exhibit an exponential growth pattern immediately, even for datasets with a finite number of time steps. The results confirm this. DGLCThe efficiency advantage of convolution time modeling compared to RNN-based methods.

[0294] Table 4 below shows the micro-AUC results of the link prediction experiment. Two versions of the static method evaluation are presented: with and without information aggregation. GraphSAGE results show the best-performing aggregator: For GCN, This is the average value. For average pooling, and Max pooling is used. The best results for each dataset are highlighted in bold.

[0295]

[0296] XI. Ablation Research

[0297] As described in Section 5.5, the four components selected for analysis in the ablation study were: 1) weighted softmax normalization in a lightweight convolutional operator; 2) GLU; 3) feedforward layers with ReLU activation; and 4) residual connections. The experimental procedure for constructing 2 4 = A thorough search was conducted on all possible combinations of 16 model variants, and their performance was compared in Table 5 (macro AUC) and Table 6 (micro AUC), where The symbol indicates the existence of the corresponding component, and The symbol indicates that the corresponding component does not exist. The experiment selected two datasets ( Enron-I and Radoslaw These can be considered as dynamic graph representations with different time step lengths. Similar to the link prediction experiment (Section 5.3), the experimental procedure uses three different random seeds for training. DGLC The experiment consisted of 200 rounds with a batch size of 512. The experiment was conducted using an Nvidia Tesla P100 with 48 CPU cores.

[0298] In Table 5 below: Ablation studies of DGLC time module component combinations were evaluated using macroscopic AUC with standard deviation on the Enron-I and Radoslaw datasets. It should be noted that the standard deviation is averaged across time steps for each setting.

[0299]

[0300] In Table 6 below: Ablation studies of DGLC time module component combinations were evaluated using microscopic AUC with standard deviation on the Enron-I and Radoslaw datasets. It should be noted that the standard deviation is averaged across time steps for each setting.

[0301]

[0302] Any software component or function described in this application may be implemented as software code executed by a processor using any suitable computer language such as Java, C, C++, C#, Objective-C, Swift, or a scripting language such as Perl or Python, employing techniques such as conventional or object-oriented methods. This software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable media include random access memory (RAM), read-only memory (ROM), magnetic media (e.g., hard disk drives or floppy disks), or optical media (e.g., optical discs (CDs) or digital versatile optical discs (DVDs)), flash memory, and so on. The computer-readable medium may be any combination of such storage or transmission means.

[0303] Such programs can also be encoded and transmitted using carrier signals suitable for transmission over wired, optical, and / or wireless networks conforming to various protocols, including the Internet. Therefore, a computer-readable medium according to an embodiment of the invention can be created using data signals encoded with such programs. Computer-readable media encoded with program code can be packaged with a compatible device or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable medium can reside on or within a single computer product (e.g., a hard disk drive, CD, or an entire computer system) and can exist on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing a user with any of the results mentioned herein.

[0304] The above description is illustrative and not restrictive. Many variations of the invention will become apparent to those skilled in the art after reading this disclosure. Therefore, the scope of the invention should not be determined by reference to the foregoing description, but rather by reference to the pending claims together with their full scope or equivalents.

[0305] Without departing from the scope of the invention, one or more features from any embodiment may be combined with one or more features from any other embodiment.

[0306] As used herein, unless explicitly indicated otherwise, the terms “a,” “an,” or “the” are intended to mean “at least one.”

[0307] XII. References

[0308] [Bruna et al., 2014] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. ICLR China, 2014.

[0309] [Chen et al., 2018] Jinyin Chen, Xuanheng Xu, Yangyang Wu and Haibin Zheng. GC-LSTM: graph convolution embeddedLSTM for dynamic link prediction. CoRR ,abs / 1812.04206, 2018.

[0310] [Dauphin et al., 2017] Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. ICLR Chinese, Volume 70, pp. 933–941, 2017.

[0311] [Deferrard et al., 2016] Michael L. Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. NeurIPS Chinese, pp. 3837–3845, 2016.

[0312] [Goyal et al., 2018] Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dynamic graphs. CoRR,abs / 1805.11273, 2018.

[0313] [Goyal et al., 2020] Palash Goyal, Sujit Rokka Chetri, and Arquimedes Canedo. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge base system , 187, 2020.

[0314] [Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. KDD Chinese, pp. 855–864, 2016.

[0315] [Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. NeurIPS Chinese, pp. 1024–1034, 2017.

[0316] [He et al., 2016] Kiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR Chinese, pp. 770-778, 2016.

[0317] [Islam et al., 2020] Md. Amirul Islam, Sen Jia and Neil

[0318] DB Bruce. How much position information do convolutional neural networks encode? ICLR China, 2020.

[0319] [Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR China, 2015.

[0320] [Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR China, 2017.

[0321] [Kumar et al., 2019] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. KDD Chinese, pp. 1269–1278, 2019.

[0322] [Levie et al., 2019] Ron Levie, Federico Monti, Xavier Bresson, and Michael M. Bronstein. Cayleynets: Graph convolutional neural networks with complex rational spectral filters. IEEE Easy Signal Processing Volume 67, Issue 1, pp. 97–109, 2019.

[0323] [Manessi et al., 2020] Franco Manessi, Alessandro Rozza, and Mario Manzo. Dynamic graph convolutional networks. Pattern recognition Volume 97, 2020.

[0324] [Nguyen et al., 2018] Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network embeddings. WWW Chinese, pp. 969–976, 2018.

[0325] [Pareja et al., 2020] Aldo Pareja, Giacomo Domenici, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, and Charles E. Leiserson. Evolvegcn: Evolving graphconvolutional networks for dynamic graphs. AAI in Pages 5363–5370, 2020.

[0326] [Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Ko¨pf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative, high-performance deep learning library. CoRR ,abs / 1912.01703,2019.

[0327] [Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: online learning of social representations. In KDD Chinese, pp. 701–710, 2014.

[0328] [Sankar et al., 2020] Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and HaoYang. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. WSDM Chinese, pp. 519–527, 2020.

[0329] [Seo et al., 2018] Youngjoo Seo, Michael Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks. ICONIP middle ， Volume 11301, pp. 362–373, 2018.

[0330] [Trivedi et al., 2019] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. ICLR China, 2019.

[0331] [Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio`, and Yoshua Bengio. Graph attention networks. In ICLR China, 2018.

[0332] [Wu et al., 2019] Felix Wu, Angela Fan, Alexei Baevski, Yann N. Dauphin, and Michael Auli. Pay less attention with lightweight and dynamic convolutions. ICLR China, 2019.

[0333] [Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? ICLR China, 2019.

[0334] [Xu et al., 2020] Da Xu, Chuanwei Ruan, Evren Ko¨rpeoglu, Sushant Kumar, and Kannan Achan. Inductive representation learning on temporal graphs. ICLR China, 2020.

[0335] [Ying et al., 2018] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. KDD Chinese, pp. 974–983, 2018.

[0336] [Zitnik et al., 2018] Marinka Zitnik, Monica Agrawal, and Jure Leskovec. Modeling polypharmacy side effects with graphconvolutional networks. Bioinformatics Volume 34, Issue 13: Pages i457–i466, 2018.

Claims

1. A method comprising: The analysis computer uses a graph structure learning module to extract multiple first datasets from multiple graph snapshots, wherein each of the multiple graph snapshots includes multiple nodes representing entities in a transaction network and multiple edges representing data exchange for transaction interactions between the entities, each of the multiple nodes is connected to its neighboring nodes through one or more of the multiple edges, and wherein the multiple first datasets include an intermediate vector representation of each node in each of the multiple graph snapshots, each of the intermediate vector representations including a first plurality of feature values ​​corresponding to multiple feature dimensions; The analysis computer uses a temporal convolution module across the multiple first datasets to extract multiple second datasets, including: Determine a plurality of convolutional kernels, each of which corresponds to at least one of the plurality of feature dimensions; and Each of the intermediate vector representations is executed-time convolved using the plurality of convolutional kernels to produce a final vector representation for each node of each of the plurality of graph snapshots, each of the final vector representations including a second plurality of feature values ​​corresponding to the plurality of feature dimensions, wherein the plurality of second datasets include the final vector representation, and wherein the intermediate vector representations and the final vector representations are embeddings of each node in a vector space representing the characteristics of the plurality of nodes; The analysis computer performs graph context prediction based on the plurality of second datasets and thereby determines the probability value of whether two nodes among the plurality of nodes will be connected by a future edge in a future graph snapshot, wherein the future edge represents a future transaction between two entities associated with the two nodes; and The analysis computer performs an action based on the graph context prediction, wherein the action includes at least one of the following: Determine whether the probability value exceeds a predetermined probability threshold; and Send an advisory notice indicating that a future transaction is likely to occur, or that a current transaction being attempted between the two entities is unlikely to occur and is therefore likely to be fraudulent.

2. The method of claim 1, wherein each of the plurality of graph snapshots includes graph data associated with a timestamp.

3. The method of claim 2, wherein each of the plurality of nodes is temporally convolved separately, and each feature dimension of each node is temporally convolved separately.

4. The method of claim 3, wherein performing time-varying convolution comprises: For each feature dimension of each node, the corresponding convolution kernel from the plurality of convolution kernels is applied to a subset of the first feature values ​​of the feature dimension, the subset of the first feature values ​​corresponding to a subset of consecutive timestamps.

5. The method of claim 4, wherein the corresponding convolutional kernel is applied to provide a result, and the result is used as a second feature value of the feature dimension at the last timestamp in the subset of the consecutive timestamps.

6. The method of claim 4, wherein each convolutional kernel has a predefined length, and wherein the number of first feature values ​​in the subset of the first feature values ​​is equal to the predefined length of the convolutional kernel.

7. The method of claim 1, wherein the temporal convolution module utilizes depthwise convolution or lightweight convolution.

8. The method of claim 1, wherein extracting the plurality of first datasets further comprises: For each of the plurality of graph snapshots, the intermediate vector representation of each node is determined based on the learned coefficients and the intermediate vector representations corresponding to the neighboring nodes.

9. An analysis computer, comprising: processor; as well as A computer-readable medium coupled to the processor, the computer-readable medium including code executable by the processor to perform a method comprising: The graph structure learning module extracts multiple first datasets from multiple graph snapshots. Each of the multiple graph snapshots includes multiple nodes representing entities in a transaction network and multiple edges representing data exchange for transaction interactions between the entities. The multiple first datasets include intermediate vector representations of each node in each of the multiple graph snapshots, each intermediate vector representation including first multiple values ​​corresponding to multiple feature dimensions. Multiple second datasets are extracted from the multiple first datasets using a temporal convolution module, including: Multiple convolutional kernels are determined based on the intermediate vector representation, each of the multiple convolutional kernels corresponding to at least one of the multiple feature dimensions; Each of the intermediate vector representations is convolved at execution time using the plurality of convolutional kernels; and The final vector representation of each node in each of the plurality of graph snapshots is determined based on the temporal convolution, each of the final vector representations including a second plurality of values ​​corresponding to the plurality of feature dimensions, wherein the plurality of second datasets include the final vector representation; Graph context prediction is performed based on the plurality of second datasets to determine the probability that two nodes among the plurality of nodes will be connected by a future edge in a future graph snapshot, where the future edge represents a future transaction between two entities associated with the two nodes; and An action is performed based on the graph context prediction, wherein the action includes at least one of the following: Determine whether the probability value exceeds a predetermined probability threshold; and Send an advisory notice indicating that a future transaction is likely to occur, or that a current transaction being attempted between the two entities is unlikely to occur and is therefore likely to be fraudulent.

10. The analysis computer according to claim 9, further comprising: The graph structure learning module coupled to the processor; as well as The temporal convolution module coupled to the processor.

11. The analysis computer according to claim 9, wherein the method further comprises: Receive prediction requests from the requesting client; The prediction is determined at least based on graph context prediction performed on the plurality of second datasets; as well as Send a prediction response, including the prediction, to the requesting client.

12. The analysis computer according to claim 9, further comprising: At least the aforementioned secondary datasets are used to train the machine learning model.

13. The analytical computer of claim 12, wherein the graph context prediction is performed using the plurality of second datasets and the machine learning model.

14. The analysis computer of claim 12, wherein the machine learning model is an SVM or a neural network.

Images