Accelerator, computer system and method

Abstract

The present disclosure provides an accelerator, a computer system and a method. The accelerator comprises: a key graph memory configured to store a key graph; a node classification circuit configured to: extract the key graph from the key graph memory; receive a current graph for performing a temporal graph neural network calculation using the key graph; and identify one or more nodes of the current graph based on a comparison between the key graph and the current graph; and a node reconstruction circuit configured to: perform a spatial calculation on the one or more nodes identified by the node classification circuit to obtain an updated node; generate an updated key graph based on the key graph and the updated node; and store the updated key graph in the key graph memory for processing a next graph. The embodiments of the present disclosure improve the performance of the temporal GNN calculation, optimize the energy efficiency of the temporal GNN calculation, and save the storage space occupancy of the temporal GNN in the cache / memory space of the processor.

Description

Technical Field

[0001] This disclosure generally relates to accelerating temporal graph neural networks (temporal GNNs). More specifically, this disclosure relates to methods and systems for accelerating the performance and energy efficiency of temporal graph neural networks through hardware-software co-design. Background Technology

[0002] While traditional deep learning models excel at pattern recognition and data mining by capturing hidden patterns in Euclidean data (e.g., images, text, videos), graph neural networks (GNNs) have demonstrated the ability to extend machine learning capabilities to non-Euclidean domains represented as graphs with complex relationships and interdependencies between objects. Research shows that GNNs can outperform current state-of-the-art performance in applications ranging from molecular reasoning to community detection.

[0003] Temporal GNNs are a novel type of GNN that have been widely applied in various practical applications involving spatiotemporal data processing, such as traffic flow prediction, weather forecasting, skeleton-based action recognition, and video understanding. Temporal GNNs extend static graph structures with temporal connections and apply traditional GNNs to extended graphs. This disclosure describes a novel method for improving the performance and energy efficiency of temporal GNNs. Summary of the Invention

[0004] Various embodiments of this specification may include systems, methods, and non-transitory computer-readable media for accelerating time-based GNNs.

[0005] According to one aspect, a hardware accelerator for accelerating temporal graph neural network (GNN) computation is provided. The hardware accelerator may include: a key-graph memory configured to store key graphs; a node classification circuit configured to: retrieve key graphs from the key-graph memory; receive a current graph for performing temporal GNN computation using the key graphs; and identify one or more nodes in the current graph based on a comparison between the key graphs and the current graph; and a node reconstruction circuit configured to: perform spatial computation on the one or more nodes identified by the node classification circuit to obtain updated nodes; generate an updated key graph based on the key graphs and the updated nodes; and store the updated key graphs in the key-graph memory for processing the next graph.

[0006] In some embodiments, in order to identify one or more nodes in the current graph, the node classification circuit is configured to: for each node in the current graph, identify the corresponding node in the key graph; determine the distance between a first feature vector of the node in the current graph and a second feature vector of the corresponding node in the key graph; and if the distance is greater than a threshold, select the node in the current graph.

[0007] In some embodiments, the distance is a Hamming distance.

[0008] In some embodiments, in order to determine the distance between the first feature vector of a node in the current graph and the second feature vector of the corresponding node in the key graph, the node classification circuit is further configured to: determine a unit of bits to be compared between the first feature vector and the second feature vector based on the data types in the first feature vector and the second feature vector; for each bit in the first feature vector, compare the exponent bit and one or more fraction bits in each bit with the corresponding bit in the second feature vector to obtain multiple matching bits; and determine the distance between the first feature vector and the second feature vector based on the multiple matching bits.

[0009] In some embodiments, the node classification circuit is further configured to: in response to the key graph received from the key graph memory being empty, send the received current graph to the node reconstruction circuit; wherein the node reconstruction circuit is further configured to: perform spatial computation on each node in the current graph to obtain a new key graph, wherein the spatial computation includes: GNN computation; and send the new key graph to the key graph memory for storage.

[0010] In some embodiments, in order to perform spatial computation on one or more identified nodes, the node reconstruction circuit is further configured to: obtain the feature vector of one of the identified one or more nodes and the adjacency matrix of the current graph; identify one or more neighboring nodes based on the adjacency matrix; and recursively aggregate and transform the feature vectors of one or more neighboring nodes and the feature vector of the node to obtain an updated feature vector of the node.

[0011] In some embodiments, the hardware accelerator may further include timing computation circuitry configured to perform timing computations based on the key graph and the updated key graph.

[0012] In some embodiments, time computation includes: using a convolutional neural network (CNN) to determine the temporal features between the key graph and the updated key graph.

[0013] In some embodiments, time computation includes: using a Long Short-Term Memory (LSTM) neural network to determine the temporal characteristics between the key graph and the updated key graph.

[0014] In some embodiments, in order to generate an updated key graph based on the key graph and the updated nodes, the node reconstruction circuit is configured to: identify one or more first nodes in the key graph corresponding to one or more updated nodes; and generate an updated key graph by replacing the feature vectors of one or more first nodes in the key graph with the feature vectors of one or more updated nodes.

[0015] According to other embodiments, a system is provided, the system comprising: one or more processors; one or more computer-readable storage devices coupled to the one or more processors and having instructions stored thereon, the one or more processors executing the instructions to perform the methods of any of the above embodiments.

[0016] According to other embodiments, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium being configured with instructions that one or more processors can execute to cause one or more processors to perform the methods of any of the above embodiments.

[0017] According to other embodiments, a computer system for accelerating Temporal Graphical Neural Network (TNN) computation is provided. The computer system includes: a first memory configured to store a key graph; a second memory configured to store a current graph for temporal GNN computation; a receiving circuit configured to receive a key graph from the first memory and a current graph from the second memory; an identification circuit configured to identify one or more nodes in the current graph based on a comparison between the key graph and the current graph; a computation circuit configured to perform spatial computation on the identified one or more nodes to obtain updated nodes; and an updating circuit configured to generate an updated key graph based on the key graph and the updated nodes, wherein the first memory stores the updated key graph for temporal GNN computation.

[0018] The embodiments disclosed in this specification have various technical advantages. As briefly mentioned in the background section, temporal GNNs typically process data collected from multiple consecutive timesteps, such as video of traffic flow data. Because versions collected from adjacent timesteps may not change significantly, this data can be highly redundant. Storing redundant data can lead to additional storage overhead when performing temporal GNN computations, and performing computations on this redundant data can result in poor performance and energy consumption spikes. In some embodiments described herein, this data redundancy is utilized to effectively reduce the amount of data to be processed in a temporal GNN. For example, suppose the data collected from multiple timesteps is represented as a series of graphs (i.e., data structures), where one graph can be identified as the key graph, and the others can be identified as secondary graphs. Spatial computation can be performed on all nodes in the key graph using a conventional GNN to obtain an updated key graph. For the secondary graphs, spatial computation can be performed only on a subset of nodes in the secondary graph (to avoid redundant computation on nodes similar to those in the key graph) to obtain the corresponding updated nodes. Other nodes in the secondary graph can skip the spatial computation using a conventional GNN. These updated nodes can be merged into the key graph to obtain an updated key graph. The updated key graph can then be used as a new key graph to process the next incoming input graph. In some embodiments, a subset of nodes is determined by comparing the key graph and auxiliary graphs. For example, a node can be selected into the subset if the distance between a node in the auxiliary graph and its corresponding node in the key graph is greater than a threshold. Subsequently, different versions of the key graph (e.g., two key graphs from two adjacent time steps) can be temporally computed to explore temporal features in the data. These temporal features can then be used to predict future time steps (e.g., predicting key graphs for future time steps). Compared to existing temporal GNN designs where all nodes in each graph require both spatial and temporal computation, the above embodiments significantly reduce the amount of data to be processed across all auxiliary graphs, thereby improving the performance of temporal GNN computations and optimizing their energy efficiency. In some embodiments, nodes skipped in the auxiliary graphs can be prevented from being sent to the processor for caching (e.g., in on-chip memory or cache), thus saving storage space occupied by the temporal GNN in the processor's cache / memory space.

[0019] These and other features of the systems, methods, and non-transitory computer-readable media of this disclosure, as well as the operation and function of the related elements of the structure, and the combination and manufacturing economy of the components, will become more apparent upon consideration of the following description and the appended claims. The appended claims refer to the accompanying drawings, which form part of this specification, in which similar reference numerals denote corresponding parts. However, it should be understood that the drawings are for illustration and description only and are not intended to be a definition of limitation of this disclosure. Attached Figure Description

[0020] Figure 1 A schematic diagram of an exemplary hardware environment for implementing the embodiments and features of this disclosure is shown.

[0021] Figure 2 A schematic diagram of a hardware device for implementing a temporal graph neural network (temporal GNN) accelerator according to an embodiment of the present disclosure is shown.

[0022] Figure 3 A schematic framework of a temporal GNN according to an embodiment of the present disclosure is shown.

[0023] Figure 4 An exemplary workflow for accelerating time-dependent GNN computation using deduplication is shown according to embodiments of this disclosure.

[0024] Figure 5 An internal structural diagram of a time-based GNN accelerator according to an embodiment of the present disclosure is shown.

[0025] Figure 6 An exemplary method for accelerating time-varying GNN preprocessing according to embodiments of this disclosure is shown.

[0026] Figure 7 A block diagram of a computer system apparatus for accelerating a time GNN according to an embodiment of the present disclosure is shown. Detailed Implementation

[0027] This specification is provided to enable any person skilled in the art to make and use the described embodiments, and to provide this specification in the context and environment of a particular application. Various modifications to the embodiments of this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but is granted the widest scope consistent with the principles and features disclosed herein.

[0028] Graph Neural Networks (GNNs) are gaining popularity across various fields, including social networks, knowledge graphs, recommender systems, and even life sciences. At a higher level, GNNs involve working with graph structures... Calculation of graph structure Represents a (undirected or directed) graph, where Represents a vertex. The graph represents an edge. In some embodiments, each node in the graph can be associated with multiple features. Depending on the use case, the graph can have different practical meanings. For example, GNNs can be applied to mine user features on social media networks and / or learn relationships between users. As another example, nanoscale molecules have an inherent graph-like structure with ions or atoms as nodes and bonds between ions or atoms as edges. GNNs can be applied in two cases: learning existing molecular structures and discovering new chemical structures.

[0029] Temporal GNNs are an extension of GNNs, adding an extra temporal dimension for use cases that process graph representations of data that evolve over time, such as traffic flow prediction, video understanding, and skeleton-based action recognition. Social networks are a good illustration of dynamic graphs: when a user joins a platform, new vertices are created. When a user follows another user, edges are created. When a user changes their profile, the vertices are updated. Typically, temporal GNNs can involve graph structures... The calculation, in which Represents a vertex. Let T represent the edge and T represent the time dimension. Existing temporal GNNs perform traditional GNN computations (e.g., feature mining) on ​​each graph representation of data collected from each time step. GNN computation involves recursively aggregating and transforming feature vectors of nodes in the graph, making it both computationally and storage-intensive. Furthermore, in practical applications, the graphs are often very large (e.g., graphs representing millions of users and their interactions on a social network), making existing temporal GNNs unsuitable for time-sensitive use cases, such as real-time or near-real-time predictions. To address this issue, this disclosure describes a novel approach to improve the performance of temporal GNNs by reducing redundant computations.

[0030] Figure 1 A schematic diagram of an exemplary hardware environment for implementing the embodiments and features of this disclosure is shown. Figure 1 The hardware environment described includes a computing device 140 for illustrative purposes. Depending on the implementation, the computing device 140 may include fewer, more, or alternative components.

[0031] like Figure 1As shown, computing device 140 includes a storage / memory component 210 connected to scheduler cluster 270 and accelerator cluster 280. Scheduler cluster 270 may include multiple schedulers 220, and accelerator cluster 280 may include multiple accelerators 230. In some embodiments, accelerator 230 may refer to a dedicated processing unit designed to accelerate the processing speed of a neural network model at different stages (e.g., input data preprocessing, convolution operations, pooling operations, etc.). Accelerators may be implemented as graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs), etc., to implement logic for accelerating neural network operations. Scheduler 220 may refer to a processing unit used to determine the scheduling schedule of accelerators 230 and distribute instructions and / or data to be executed to each accelerator 230. In some embodiments, the scheduler 220 may be implemented as a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other suitable forms.

[0032] In contrast, traditional CPU architectures allocate most of their resources to control and storage units, leaving computation units with insufficient resources. While CPUs are highly efficient in logic control, they are not very efficient in large-scale parallel computing. Therefore, various hardware accelerators have been developed to improve computational processing speed for different functions and domains. The hardware accelerators presented in this specification include processing units dedicated to accelerating GNN computational performance. These hardware accelerators are data-driven parallel computing architectures designed to handle large numbers of operations (e.g., graph partitioning, row / column reordering, hardware-aware-granularity-aware matrix partitioning, convolution, pooling, other suitable operations, or any combination thereof). Throughout GNN processing, the data and intermediate results of these operations can be closely related to each other and are frequently used. Without accelerators, existing CPU frameworks, with their small storage capacity within the CPU cores, result in numerous and frequent memory accesses to external (e.g., external CPU) memory. These memory accesses are costly and lead to low processing efficiency. Using accelerators designed to speed up data processing for GNNs can significantly improve processing efficiency and computational performance for at least the following reasons: (1) the input data (graph) can be partitioned into multiple submatrices to cluster similar nodes (nodes with similar eigenvectors), (2) the rows and columns of each submatric can be reordered to cluster data with similar sparsity, and (3) each submatric can be further partitioned into smaller units called tiles, based on the data processing granularity of the underlying processor used to perform GNN computations (e.g., convolution, aggregation, transformation, pooling, etc.). Because the tiles are carefully sized to fit the underlying processor, on-chip memory in each processor can be used for GNN computations, and frequent memory accesses to off-chip memory can be avoided.

[0033] In some embodiments, memory 210 may store various neural network models (e.g., nodes of these models and weights or parameters of these nodes) and input data of these models (e.g., input graphs of GNNs, such as nodes, feature vectors of nodes, edges, etc.). Accelerator 230 in this specification may perform preprocessing on the input data of the models to accelerate subsequent neural network computation. For example, scheduler 220 may send the address of the input graph in memory 210 to accelerator 230 in the form of an instruction. Subsequently, the accelerator may (e.g., at a predetermined time point) directly locate and retrieve the input data from memory 210 and temporarily store the input data in its on-chip memory for preprocessing. The output of the preprocessing may include multiple tiles of data with different sparsities. In some embodiments, these tiles may be assigned to multiple low-level processors for accelerating computation. Different low-level processors may be optimized to perform neural network computations on datasets with different sparsities. Assigning tiles to low-level processors may include assigning each tile to a low-level processor optimized for processing datasets with a specific sparsity in each tile. The outputs of the low-level processors may be aggregated to generate a final computation result. In some embodiments, these underlying processors may be implemented as part of accelerator 230 or separately from accelerator 230. If the underlying processors are implemented as part of the accelerator, scheduler 220 may send the addresses of parameters corresponding to the neural network model in memory 210 to accelerator 230 in the form of instructions. Subsequently, accelerator 230 may directly locate these parameters (e.g., weights) in memory 210 and temporarily store these parameters in its on-chip memory for the underlying processors to perform computations based on the aforementioned tiles.

[0034] Figure 2 A schematic diagram of a hardware device for implementing a hardware accelerator according to an embodiment of the present disclosure is shown. Figure 2 The hardware devices shown in Figure 1 The internal structure of the scheduler 220 and accelerator 230, and the data / instruction flow between the scheduler 220, accelerator 230 and memory 210.

[0035] like Figure 2As shown, the scheduler 220 may include multiple processors 222 and a cache 221 shared by the multiple processors 222. Each processor 222 may include an instruction fetching unit (IFU) 223, an instruction decoding unit (IDU) 224, an instruction transmitting unit (ITU) 225, and an instruction execution unit (IEU) 226.

[0036] In some embodiments, the instruction fetch unit (IFU) 223 may fetch instructions or data to be executed from memory 210 into register file 229. After obtaining the instructions or data, the scheduler 220 enters the instruction decoding stage. The instruction decoding unit (IDU) 224 decodes the obtained instructions according to a predetermined instruction format to determine operand fetch information, wherein the operands are required to execute the obtained instructions. In some embodiments, the operand fetch information may include immediate values, pointers or addresses to registers or other software / hardware that provide the operands.

[0037] In some embodiments, the instruction sending unit (ITU) 225 can be configured to perform instruction scheduling and management between the instruction decoding unit (IDU) 224 and the instruction execution unit (IEU) 226. The instruction sending unit (ITU) 225 can efficiently distribute instructions to different instruction execution units (IEUs) 226 for parallel processing.

[0038] In some embodiments, after the instruction sending unit (ITU) 225 assigns an instruction to an instruction execution unit (IEU) 226, the instruction execution unit (IEU) 226 can execute the instruction. However, if the instruction execution unit (IEU) 226 determines that the instruction should be executed by the accelerator 230, it can forward the instruction to the corresponding accelerator 230 for execution. For example, if the instruction points to GNN computation based on the input graph, the instruction execution unit (IEU) 226 can send the instruction to the accelerator 230 for execution via the bus 231 for the accelerator 230.

[0039] In some embodiments, accelerator 230 may include a plurality of cores 236 ( Figure 2 Four cores are shown, but those skilled in the art will understand that the accelerator 230 may also include other numbers of cores 236, command processor 237, direct storage access (DMA) interface 235, and bus channel 231.

[0040] Bus channel 231 may include a channel through which instructions / data enter and exit accelerator 230. Direct Memory Access (DMA) interface 235 may refer to a function provided by some computer bus architecture that enables devices to directly read and / or write data from memory 210. This contrasts with a method where all inter-device data transfers are handled through scheduler 220. Figure 2 The architecture shown significantly improves data access efficiency. For example, the core of accelerator 230 can directly access memory 210 and read parameters (e.g., weights of each node) and / or input data of the neural network model.

[0041] Command processor 237 can be configured to distribute instructions sent by scheduler 220 to accelerator 230 via instruction execution unit (IEU) 226 to core 236 for execution. After instructions to be executed enter accelerator 230 from bus channel 231, the instructions can be cached in command processor 237, and command processor 237 can select core 236 and distribute instructions to core 236 for execution. In addition, command processor 237 can also handle synchronization operations between cores 236.

[0042] In some embodiments, instructions allocated by command processor 237 may include preprocessing the input graph for accelerating GNN computation. This instruction may be sent to graph preprocessing core 238 to perform preprocessing. In some embodiments, the input graph may be directly allocated and retrieved from memory 210 via direct memory access (DMA) interface 235. In some embodiments, the input graph may be represented as an adjacency matrix. Each node in the input graph may correspond to a row and column in the adjacency matrix, and the features of each node may be represented as an eigenvector in the adjacency matrix.

[0043] Figure 3 An exemplary framework 300 for a time-graph neural network (GNN) according to an embodiment of the present disclosure is shown.

[0044] Figure 3 The framework shown depicts the workflow of a temporal GNN. Depending on the implementation and use case, a temporal GNN can have more, fewer, or alternative layers or components.

[0045] The temporal GNN in framework 300 can be trained to make predictions 340 based on input data 310. Depending on the actual use case, the input data 310 and the predictions 340 can have various practical meanings. For example, the input data 310 can include video recordings of traffic flow. Video can be collected from one or more cameras monitoring traffic at one or more intersections. In this context, the predictions 340 can include predictions of future traffic conditions based on spatial features 320 and temporal features 330 learned from the input data 310.

[0046] In some embodiments, input data 310 may include multiple input datasets collected across multiple time steps. Each input dataset may be represented as a graph with vertices (representing objects) and edges (representing relationships between objects). In the context of traffic flow prediction, each input dataset may include a "snapshot" of traffic conditions at one or more intersections at one time step. Assuming the current time is t, traffic data collected from the previous n time steps can be represented as n graphs, denoted as... Figure 3 X in t-n ... X t-1 X t .

[0047] Each of the n graphs can include multiple spatial features 320 among its vertices and edges. In some embodiments, these spatial features 320 can be explored using a Generative Neural Network (GNN). For example, an input graph can include multiple vertices with initial feature vectors. The initial feature vectors of the vertices can include the feature values ​​of the vertices. For example, in a social network setting, each user can be represented as a node, and the user's features (e.g., profile information, current state, recent activity, etc.) can be represented as feature vectors. After performing GNN computation on the input graph, an updated graph can be generated, and the updated graph can include multiple nodes with updated feature vectors. The updated feature vectors can be embedded with the features of the neighboring nodes. The GNN computation can follow a neighborhood aggregation scheme, where the feature vectors of a vertex are computed by recursively aggregating and transforming the feature vectors of the vertex's neighboring nodes.

[0048] After learning the spatial features 320 of each of the multiple graphs, one or more updated graphs can be generated.

[0049] Temporal features 330 can be explored by performing temporal computations on each updated graph. In some embodiments, temporal computations can be performed using a convolutional neural network (CNN) or a long short-term memory (LSTM) neural network. For example, in updating graph X... t-n ... X t-1 X t (For example, after updating the feature vectors of nodes via a GNN) a time computation can be performed on the updated graph to learn the evolutionary trends of nodes and / or edges in the updated graph. A CNN or LSTM can be used to take the graph as input and output a prediction 340 for the next time step, denoted as X. t+1 .

[0050] Figure 4 An exemplary workflow 400 is shown that utilizes deduplication to accelerate time-consuming GNN computation according to an embodiment of this disclosure. Workflow 400 is for illustrative purposes only. Workflow 400 can... Figure 1 In the hardware environment shown, through Figure 2 The hardware devices shown are used to achieve this, in order to improve... Figure 3 The diagram illustrates the performance and energy efficiency of temporal GNN computations. Depending on the implementation, workflow 400 may include fewer, more, or alternative steps. Some steps can be split or combined, and some steps can be executed in a different order or in parallel. Workflow 400 demonstrates how deduplication can improve the efficiency of spatial computations in temporal GNNs.

[0051] Temporal GNNs can be used to explore spatial and temporal features of an object across multiple snapshots (e.g., object features / states) at different time steps. These snapshots can be represented in a graph data structure. For example, each object could refer to a vehicle or intersection in the context of traffic control, a user or organization in the context of a social network, or an ion or atom in the context of nanoscale molecules used to learn existing molecular structures and discover new chemical structures. For example, an object could refer to one or more geographic locations, and the features / state of an object at one time step could include traffic images captured from one or more geographic locations at that time step.

[0052] In some embodiments, a graph collected from one time step can be selected as the key graph, and other graphs can be considered as derivatives of the key graph, or secondary graphs. All vertices in the key graph can be used for spatial computation, but only a subset of vertices in the secondary graph may need to be processed. As mentioned above, since multiple graphs can be collected from multiple consecutive time steps, the changes between graphs at two adjacent time steps can be limited to a small number of vertices and / or edges. That is, the graph can include a large amount of repetitive data, which can be skipped in the computation to accelerate spatial computation. For example, after the key graph has undergone complete spatial computation, the next (secondary) graph, which shares one or more vertices with the key graph, may only need to perform spatial computation on the updated vertices. This significantly reduces the computational cost of performing spatial computation on the secondary graph and the amount of data that the processor needs to cache / process.

[0053] In some embodiments, key graphs can be determined in various ways. For example, a graph collected from the earliest time step can be identified as a key graph. As another example, a key graph can be selected from multiple received graphs by the following steps: for each graph in the multiple graphs, an overall graph distance is determined based on the graph distance between that graph and every other graph; the graph with the smallest overall graph distance is identified as the key graph. Various techniques can be used to determine graph distances, such as edit distance / graph isomorphism, feature extraction, and iterative methods.

[0054] refer to Figure 4 Workflow 400 in the example. In step 410, the first step may include receiving a representation of X. t The input graph data is given by X, where t refers to the timestamp t. Next, in step 420, the classification unit can be used to classify the input graph data X. t Classify the graph as either a key graph or an auxiliary graph. If the input graph data X... t If the key graph is 430, then in step 432, the input graph data X can be processed. t All nodes in the graph perform a complete set of spatial computations. In step 434, the spatial computations can generate the input graph data X. t The representation is an updated version of the key space GNN. For the input graph data X... t After performing spatial computation, the key spatial GNN can be output as spatial graph data 450. In some embodiments, the key spatial GNN can be stored in a buffer as an updated version of the key graph 430 for the next round of computation.

[0055] If the input graph data X t If the auxiliary graph is 440, then in step 442, each node in the auxiliary graph 440 can be compared with a corresponding node in the key graph 430 (e.g., an updated key graph from a previous time step). In some embodiments, the comparison in step 442 may include determining the distance between each node in the auxiliary graph 440 and its corresponding node in the key graph 430. This distance may refer to the feature vector distance determined, for example, by the Hamming distance between the feature vectors of the two nodes. In some embodiments, the comparison in step 442 may include identifying one or more nodes in the auxiliary graph 440 that are not present in the key graph 430.

[0056] In some embodiments, if the distance between two nodes is less than a threshold, then in step 445, nodes in the auxiliary graph 440 can skip spatial computation. If the distance between two nodes is greater than the threshold, then nodes in the auxiliary graph 440 can be identified as "altered," and therefore spatial computation is performed in step 446. This allows for improved efficiency in spatial computation by skipping duplicate or unchanged nodes in the auxiliary graph 440. In some embodiments, the threshold can determine a trade-off between improved accuracy and efficiency in spatial computation. For example, a higher threshold might result in fewer nodes in the auxiliary graph 440 being identified as "altered" and processed (e.g., having spatial features extracted), potentially leading to lower accuracy in the output graph but faster processing speed. Therefore, a machine learning algorithm can be used to determine the threshold to find the optimal trade-off.

[0057] In some embodiments, spatial computation may also be referred to as GNN computation, which may include: obtaining the feature vectors of nodes (e.g., nodes from auxiliary graph 440) and the adjacency matrix of the graph to which the nodes belong (e.g., auxiliary graph 440); determining the neighboring nodes of the nodes in the graph based on the adjacency matrix; recursively aggregating and transforming the feature vectors of the neighboring nodes and the feature vectors of the nodes; and obtaining the updated feature vectors of the nodes.

[0058] After the "changed" nodes in auxiliary graph 440 are computed by the GNN, their feature vectors can be updated. In some embodiments, then, in step 448, these "changed" nodes (with updated feature vectors) and other "unchanged" nodes (with original feature vectors, also known as skipped nodes) can be merged into an output auxiliary graph. To generate updated feature vectors for all nodes in auxiliary graph 440, the "unchanged" nodes in auxiliary graph 440 can directly use the updated feature vectors of the corresponding nodes in key graph 430 without GNN computation. In this way, all nodes in auxiliary graph 440 can be merged into spatial graph output 450. For example, spatial graph output 450 (e.g., an updated auxiliary graph) can be obtained by inserting updated nodes into key graph 430. This process may include: identifying one or more first nodes in key graph 430 corresponding to one or more updated nodes; and generating spatial graph data output 450 by replacing the feature vectors of one or more first nodes with the feature vectors of one or more updated nodes.

[0059] The output 450 of this spatial map can be considered as a response to the input map data X. tThe output of the spatial computation is performed. In some embodiments, before performing spatial computation on the "changed" nodes in step 446, the feature vectors of the "unchanged" nodes can be updated based on the corresponding nodes in the key graph 430. By doing so, if a "changed" node in the auxiliary graph 440 has multiple "unchanged" nodes as neighbors, the feature update of the "changed" node can be based on the updated feature vectors of its neighboring nodes. Here, "unchanged" does not necessarily mean "the same". In some cases, the distance between two feature vectors within a threshold can indicate that two corresponding nodes are "unchanged". Therefore, using the updated features of the "unchanged" nodes can improve the accuracy of the spatial computation.

[0060] When generating multiple input graph data (e.g., X) t-1 and X t After the spatial graph outputs 450, time calculations can be performed to explore temporal features. In some embodiments, time calculations may include: based on the first updated graph (e.g., based on X at time step t-1). t-1 The spatial graph output is 450) and the second update graph (e.g., based on X at time step t). t The system generates a spatial graph output 450, trains a temporal neural network, and generates a predicted graph representing the state of one or more objects at the next time step based on the temporal neural network. In some embodiments, the temporal neural network may be a convolutional neural network (CNN) or a long short-term memory (LSTM) neural network. In some embodiments, the spatial graph data output 450 at two consecutive time steps may be referred to as two updated key graphs that can undergo temporal computation. A rolling buffer may store the two most recently updated key graphs used to perform temporal computation. When a new version of the key graph is computed through spatial computation, the new version of the key graph replaces the old version in the rolling buffer.

[0061] In some embodiments, the time calculation in step 460 can be based on deduplication acceleration. For example, not based on X. t-1 and X t The temporal neural network can be trained using two completely updated key maps, or it can be based on the first updated key map (e.g., X). t-1 Spatial map data output) and "changed" nodes (e.g., X) in the second updated key map. t Training is performed on the "changed" nodes.

[0062] In some embodiments, after spatial calculations are performed on each auxiliary graph 440, the key graph 430 may be updated. For example, assume Xt-n Selected as key graph 430. Based on key graph 430, auxiliary graph X... t-n+1 After performing spatial calculations, an updated auxiliary graph can be generated. In processing Figure X t-n+2 Previously, key graph 430 could be updated to .

[0063] Spatial GNN computation involves an iterative process. Workflow 400 illustrates the steps of one round in the iterative process. The spatial graph data output 450 (updated key graph) can be cached for the next round of temporal GNN computation on newly received auxiliary graphs.

[0064] Figure 5 A diagram 500 showing the internal structure of a time-GNN accelerator according to an embodiment of the present disclosure is provided. Figure 5 The temporal GNN accelerator 500 in the text is for illustrative purposes only and, depending on the implementation, may include fewer, more, or alternative components / data communication channels. For example, Figure 5 The memory stack 520 can be implemented as on-chip memory (internal to the temporal GNN accelerator 500) or off-chip memory (external to the temporal GNN accelerator 500). The temporal GNN accelerator 500 illustrates data exchange between two hardware layers. The memory stack 520 is implemented using any type of transient and non-transitory computer memory. The processing circuitry 530 is configured to perform spatial computations using a GNN and temporal computations using a CNN or LSTM.

[0065] like Figure 3 and Figure 4 As described, the input to a temporal GNN can include a series of input data collected from a series of time steps. The input data at one time step can include features of multiple objects represented as a graph at that time step. The output of the temporal GNN can include predictions (e.g., predicted features of objects at the next time step). In some embodiments, a series of graphs can be sequentially fed into a memory heap 520. One graph in the series can be selected as the key graph. Nodes in the key graph can undergo a complete set of spatial computations to obtain an updated key graph. Other graphs (also called auxiliary graphs) can be compared with the key graph. "Changed" nodes in the auxiliary graphs can undergo spatial computations, while "unchanged" (also called repeated) nodes can skip computationally intensive spatial computations.

[0066] In some embodiments, Figure 5The memory stack 520 can acquire a series of input data represented in the graph from another storage medium (e.g., persistent memory) or directly from an input device (e.g., a camera). The memory stack 520 may include a key graph buffer 524 (e.g., a first memory) and a current graph buffer 522 (e.g., a second memory). The current graph buffer 522 can be configured to store newly received input graph data, and the key graph buffer 524 can be configured to store the most recently updated key graph. For example, when a first graph collected from the earliest time step is received via the memory stack 520, the first graph can be selected as the key graph and stored in the key graph buffer 524. Subsequently, the first graph can be sent to the processing circuitry 530 to perform a complete set of spatial calculations to generate an updated key graph. This updated key graph can be sent back to the memory stack 520 and stored in the key graph buffer 524 for use in the next round of processing.

[0067] When the memory stack 520 receives the second graph, it can be stored in the current graph buffer 522. Then, both the second graph in the current graph buffer 522 and the updated key graph in the key graph buffer 524 can be sent to the processing circuit 530 for processing. In some embodiments, the second graph and the updated key graph can first be sent to the node classification circuit 532 to determine which node or part of the node in the second graph needs to be used for spatial computation. The node classification circuit 532 can be implemented by hardware circuitry called a node classification circuit. For example, if the Hamming distance between a node in the second graph and its corresponding node in the updated key graph is greater than a threshold, the node in the second graph can be selected for spatial computation. As a data structure in the field of computer technology, each node in the graph can be represented as a feature vector. The feature vector can include one or more values ​​of various data types. These values ​​can be stored internally in the computer as a sequence of bits. For example, the value of a 32-bit floating-point number can include the first bit as the sign bit, the next 8 bits as the exponent bit, and the remaining 23 bits as the fraction bit. As another example, the value of a 64-bit floating-point number may include the first bit as the sign bit, the next 11 bits as the exponent, and the remaining 52 bits as the fractional bits. Comparing a node in the second graph with its corresponding node in the updated key graph may include: determining a unit of bits to be compared between the first feature vector of a node and the second feature vector of other nodes, based on the data types within the first feature vector of the node and the second feature vector of other nodes; for each unit of bits within the first feature vector, comparing the exponent and one or more fractional bits within each unit with the corresponding bits in the second feature vector to obtain multiple matching bits; and determining the distance between the first and second feature vectors based on the multiple matching bits.

[0068] In some embodiments, one or more nodes identified in the auxiliary graph as exceeding a threshold (e.g., having a distance greater than the threshold from the corresponding node in the updated key graph) can be sent to node reconstruction circuit 534 to perform spatial computation. Node reconstruction circuit 534 can be implemented by hardware circuitry called a node reconstruction circuit. Other nodes in the second graph can skip the spatial computation and can be updated by directly copying the feature vectors of their corresponding nodes in the updated key graph. Subsequently, the nodes exceeding the threshold and the skipped nodes can be merged to generate a new key graph. This new key graph can be sent back to the memory heap and stored in a key graph buffer for the next round of processing.

[0069] The next round of processing can begin by reading a third graph from multiple graphs and replacing the second graph in the current graph buffer 522. One or more nodes in the third graph can be identified based on a comparison between the third graph and a new key graph stored in the key graph buffer. Comparisons can be made based on the Hamming distance between corresponding nodes and / or whether a node in the third graph is a new node (e.g., not present in the new key graph). Then, one or more nodes can be updated via GNN computation (e.g., spatial computation) to obtain one or more updated feature vectors. These updated feature vectors and the new key graph can be merged to construct an updated third graph. In some embodiments, the merging step can occur before or after performing GNN computation on the identified nodes. The newly generated updated third graph can be sent to the key graph buffer for storage and become the most recently updated key graph for the next round of computation.

[0070] In some embodiments, at least two latest versions of the key graph can be stored in the key graph buffer. Timing calculations can be performed to explore the temporal features within the stored key graphs. This can be achieved through a processing circuit 530 called a timing calculation circuit. Figure 5 Hardware circuitry (not shown) performs timing calculations. In some embodiments, timing calculations may include learning temporal features and predicting future time steps (e.g., predicting a keymap) using a trained convolutional neural network (CNN) or a long short-term memory (LSTM) neural network. In some embodiments, the keymap buffer may be a FIFO memory that stores two most recently updated keymaps. When a new updated keymap is generated, the relatively older versions of the two keymaps in the keymap buffer can be replaced by the newly updated keymap.

[0071] In some embodiments, the circuit described above can be implemented in various hardware forms, such as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other suitable forms.

[0072] Figure 6 An exemplary method 600 for accelerating time-consuming GNN computation using deduplication, according to embodiments of this disclosure, is illustrated. It can be used... Figure 1 Method 600 is implemented in the hardware environment shown. It can be achieved through... Figures 2 to 5 The apparatus, device, or system shown performs method 600. Depending on the implementation, method 600 may include additional, fewer, or alternative steps performed in various sequences or in parallel.

[0073] Box 610 includes receiving the current graph collected from the current time step.

[0074] Box 620 includes determining whether the current graph is a key graph or an auxiliary graph. In some embodiments, determining whether the current graph is a key graph or an auxiliary graph includes: if the current graph is the first graph received, then determining that the current graph is a key graph.

[0075] Box 630 includes performing spatial computations on the nodes in the key graph in response to the current graph being a key graph, to obtain an updated key graph.

[0076] Box 640 includes, in response to the current graph being an auxiliary graph: identifying one or more nodes in the auxiliary graph based on a comparison between the key graph and the auxiliary graph; performing spatial computation on the identified one or more nodes to obtain updated nodes; and generating an updated key graph based on the key graph and the one or more updated nodes. In some embodiments, identifying one or more nodes in the auxiliary graph includes: for each node in the auxiliary graph, identifying a corresponding node in the key graph; determining a distance between a first feature vector of the node in the auxiliary graph and a second feature vector of the corresponding node in the key graph; and selecting a node in the auxiliary graph if the distance is greater than a threshold.

[0077] Box 650 includes performing temporal computations based on the key graph and the updated key graph to predict graphs for future time steps. In some embodiments, the temporal computation includes utilizing a convolutional neural network (CNN) or a long short-term memory (LSTM) neural network to determine temporal features between the key graph and the updated key graph.

[0078] Figure 7 A block diagram of a computer system 700 for accelerating a timed GNN according to an embodiment of the present disclosure is shown. The components of the computer system 700 presented below are intended to be illustrative. System 700 may include additional, fewer, or alternative components. Computer system 700 may be Figures 1 to 2 The illustrated hardware device embodiment can be implemented Figures 3 to 6 The method or workflow shown.

[0079] Computer system 700 may include various circuits. For example, computer system 700 is implemented by one or more processors and one or more non-transitory computer-readable storage media (e.g., one or more memories), the one or more non-transitory computer-readable storage media being coupled to one or more processors and configured with instructions, the one or more processors being executable to cause a system or device (e.g., a processor) to perform the embodiments described above. Computer system 700 may include various units / modules corresponding to instructions (e.g., software instructions).

[0080] In some embodiments, the computer system 700 may include a first memory 710, a second memory 720, a receiving circuit 730, an identification circuit 740, a calculation circuit 750, and an updating circuit 760. In some embodiments, the first memory 710 may be configured to store the most recently updated key graph. The second memory 720 may be configured to store the current graph used for spatial GNN calculations. The first memory 710 and the second memory 720 may be implemented within the same computer memory at different addresses or may be implemented as two separate memories.

[0081] In some embodiments, the receiving circuit 730 may be configured to receive a key graph from a first memory 710 and a current graph from a second memory 720. The identification circuit 740 may be configured to identify one or more nodes in the current graph based on a comparison between the key graph and the current graph. The calculation circuit 750 may be configured to perform spatial computation on the identified one or more nodes to obtain updated nodes. The updating circuit 760 may be configured to generate an updated key graph based on the key graph and the updated nodes, with the first memory storing the updated key graph for temporal GNN computation. In some embodiments, the circuitry described above may be implemented within the same processor or by multiple processors. The circuitry and memory may be implemented within the same hardware accelerator or as different hardware devices.

[0082] In some embodiments, the computer system 700 may also include computational circuitry configured to perform time calculations based on a key graph and an updated key graph (i.e., two consecutive versions of the key graph) to learn time characteristics / trends between different key graphs and to predict key graphs for future time steps.

[0083] The performance of some of the operations described in this disclosure can be distributed among processors, residing not only in a single device but also deployed across multiple devices. In some exemplary embodiments, the processor or processor-implemented circuitry can reside on a single die or on different dies. In other exemplary embodiments, the processor or processor-implemented engine can be distributed across multiple geographical locations.

[0084] Each process, method, and algorithm described in the preceding sections can be embedded in a code module and can be fully or partially automated by the code module, which can be executed by one or more computer systems or computer processors, including computer hardware. The processes and algorithms can be implemented, partially or entirely, in dedicated circuitry.

[0085] When the functions of this disclosure are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (in whole or in part) or aspects contributing to the prior art of this disclosure can be embodied in the form of a software product. The software product can be stored in a storage medium including multiple instructions to cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods of the embodiments of this disclosure. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, another medium capable of storing program code, or any combination thereof.

[0086] Specific embodiments also provide a system comprising: a processor; and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps of any method of the disclosed embodiments above. Specific embodiments also provide a non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause one or more processors to perform operations corresponding to steps of any method of the disclosed embodiments above.

[0087] The various features and processes described above can be used independently of each other or in combination in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some implementations. The methods and processes described herein are not limited to any particular sequence, and the associated blocks or states may be executed in other suitable sequences. For example, the blocks or states may be executed in a different order than that specifically disclosed, or multiple blocks or states may be combined in a single block or state. Example blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the example embodiments of this disclosure. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the exemplary embodiments of this disclosure.

[0088] The various operations of the exemplary methods described herein can be performed at least in part by algorithms. Algorithms may include program code or instructions stored in memory (e.g., the aforementioned non-transitory computer-readable storage medium). These algorithms may include machine learning algorithms. In some embodiments, a machine learning algorithm may not explicitly program the computer to perform the function, but it may learn from training data to build a predictive model for performing that function.

[0089] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily or permanently configured (e.g., by software) to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute the engine of a processor implementation that runs to perform one or more of the operations or functions described herein.

[0090] Similarly, using a specific processor or a processor as exemplary hardware, the methods described herein can be implemented at least partially by a processor. For example, at least some operations of the methods can be performed by one or more processors or an engine implemented by the processor. Furthermore, one or more processors can also operate to support the performance of related operations in a “cloud computing” environment or as “software as a service” (SaaS). For example, at least some operations can be performed by a cluster of computers (as an exemplary machine including processors), and these operations can be accessed via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., Application Program Interfaces, APIs).

[0091] The performance of certain operations can be distributed across processors, and can reside not only in a single machine but also deployed across multiple machines. In some exemplary embodiments, the processor or the engine implemented by the processor can reside in a single geographic location (e.g., in a home environment, an office environment, or a server farm).

[0092] In other exemplary embodiments, the processor or processor-implemented engine may be distributed across multiple geographical locations.

[0093] In this specification, multiple instances can implement components, operations, or structures described as single instances. Although a single operation of one or more methods is shown and described as a separate operation, one or more single operations can be executed simultaneously, and the order in which they are executed is not required. Structures and functions presented as separate components in the example configuration can be implemented as composite structures or components. Similarly, structures and functions presented as single components can be implemented as separate components. These, and other variations, modifications, additions, and improvements fall within the scope of this document.

[0094] Although an overview of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes can be made to these embodiments without departing from the broader scope of embodiments of this disclosure. These embodiments of the subject matter are referred to individually or collectively by the term "this disclosure" only for convenience and not intended to voluntarily limit the scope of this disclosure to any single disclosure or concept. In fact, if multiple disclosures or concepts exist, then multiple disclosures or concepts are disclosed.

[0095] The embodiments shown herein have been described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, allowing for structural and logical substitutions and changes without departing from the scope of this disclosure. Therefore, the detailed description section is not used in a limiting sense, and the scope of the various embodiments is defined only by the appended claims and all their equivalents.

[0096] Any process description, element, or block in the flowcharts described herein and / or in the accompanying drawings should be understood to potentially represent modules, code segments, or code portions, including one or more executable instructions for implementing a specific logical function or step in the process. Alternative implementations are included within the scope of the embodiments described herein, wherein, as those skilled in the art will understand, elements or functions may be removed from or executed out of order (including substantially simultaneously or in reverse order) depending on the functionality involved.

[0097] As used herein, “or” is inclusive, not exclusive, unless explicitly stated otherwise or indicated by the context. Therefore, here, “A, B, or C” means “A, B, C, A and B, A and C, B and C, or A, B, and C”, unless otherwise explicitly stated or indicated by the context. Furthermore, “and” is consequential, unless otherwise explicitly stated or indicated by the context. Therefore, here, “A and B” means “A and B, jointly or separately”, unless otherwise explicitly stated or indicated by the context. Furthermore, multiple instances may be provided for a resource, operation, or structure described herein as a single instance. Moreover, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are described in the context of a particular illustrative configuration. Other allocations of functionality are contemplated, and they may fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as separate resources in the example configuration may be implemented as composite structures or resources. Similarly, structures and functions presented as single resources may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of this disclosure as defined by the appended claims. Therefore, the instructions and drawings should be considered illustrative rather than restrictive.

[0098] The terms “comprising” or “including” are used to indicate a feature subsequently stated, but do not exclude the addition of other features. Conditional languages, such as “may,” “may,” or “can,” among others, unless expressly stated otherwise or otherwise understood in the context in which they are used, are generally intended to convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, these conditional languages ​​are not generally intended to imply that one or more embodiments require features, elements, and / or steps in any way, or that one or more embodiments necessarily include logic for determining whether such features, elements, and / or steps are included in or will be performed in any particular embodiment, with or without user input or prompting.

Claims

1. An accelerator for accelerating computation of time-graph neural networks, comprising: The key graph memory is configured to store key graphs. The node classification circuit is configured as follows: Extract the key graph from the key graph memory; Receive the current graph for performing temporal graph neural network computations using the key graph; and Based on a comparison between the key graph and the current graph, one or more nodes in the current graph are identified. To identify these nodes, the node classification circuit is configured to: for each node in the current graph, identify the corresponding node in the key graph; determine the distance between a first feature vector of that node in the current graph and a second feature vector of the corresponding node in the key graph; if the distance is greater than a threshold, select that node in the current graph; and The node reconfiguration circuit is configured as follows: Spatial calculations are performed on the one or more nodes identified by the node classification circuit to obtain updated nodes; Based on the key graph and the updated node, an updated key graph is generated; and The updated key graph is stored in the key graph memory to process the next graph.

2. The accelerator according to claim 1, wherein, The distance mentioned is the Hamming distance.

3. The accelerator according to claim 1, wherein, To determine the distance between the first feature vector of a node in the current graph and the second feature vector of the corresponding node in the key graph, the node classification circuit is configured as follows: Based on the data types within the first feature vector and the second feature vector, determine the bit units to be compared between the first feature vector and the second feature vector; For each bit unit within the first feature vector, the exponent bit and one or more decimal bits within each bit unit are compared with the corresponding bit in the second feature vector to obtain multiple matching bits; and Based on the multiple matching bits, the distance between the first feature vector and the second feature vector is determined.

4. The accelerator according to claim 1, wherein, The node classification circuit is also configured to: In response to the key graph received from the key graph memory being empty, the received current graph is sent to the node reconstruction circuit; The node reconfiguration circuit is further configured as follows: Spatial computation is performed on each node in the current graph to obtain a new key graph, wherein the spatial computation includes graph neural network computation; and The new key graph is sent to the key graph memory for storage.

5. The accelerator according to claim 1, wherein, In order to perform spatial computation on the identified one or more nodes, the node reconstruction circuit is further configured to: Obtain the feature vector of one node from the identified one or more nodes and the adjacency matrix of the current graph; Identify one or more neighboring nodes based on the adjacency matrix; The feature vectors of one or more neighboring nodes and the feature vector of the node are recursively aggregated and transformed to obtain the updated feature vector of the node.

6. The accelerator according to claim 1, wherein, Also includes: A timing calculation circuit is configured to perform timing calculations based on the key graph and the updated key graph.

7. The accelerator according to claim 6, wherein, The time calculation includes: A convolutional neural network is used to determine the temporal features between the key map and the updated key map.

8. The accelerator according to claim 6, wherein, The time calculation includes: The temporal characteristics between the key graph and the updated key graph are determined using a long short-term memory neural network.

9. The accelerator according to claim 1, wherein, In order to generate an updated key graph based on the key graph and the updated node, the node reconstruction circuit is configured as follows: Identify one or more first nodes in the key graph that correspond to the one or more update nodes; as well as The updated key graph is generated by replacing the feature vectors of one or more first nodes in the key graph with the feature vectors of one or more updated nodes.

10. A computer system for accelerating computation of time-graph neural networks, comprising: The first memory is configured to store the key graph; The second memory is configured to store the current graph used for time-graph neural network computation; A receiving circuit is configured to receive the key map from the first memory and the current map from the second memory; An identification circuit is configured to identify one or more nodes in the current graph based on a comparison between the key graph and the current graph, wherein, in order to identify one or more nodes in the current graph, the identification circuit is configured to: for each node in the current graph, identify the corresponding node in the key graph; determine the distance between a first feature vector of the node in the current graph and a second feature vector of the corresponding node in the key graph; and if the distance is greater than a threshold, select the node in the current graph. The computing circuitry is configured to perform spatial calculations on the identified one or more nodes to obtain updated nodes; and An update circuit is configured to generate an updated key graph based on the key graph and the update node, wherein the first memory stores the updated key graph for computation by the time-graph neural network.

11. The computer system according to claim 10, wherein, To determine the distance between the first feature vector of the node in the current graph and the second feature vector of the corresponding node in the key graph, the recognition circuit is configured as follows: Based on the data types within the first feature vector and the second feature vector, determine the bit units to be compared between the first feature vector and the second feature vector; For each bit unit within the first feature vector, the exponent bit and one or more decimal bits within each bit unit are compared with the corresponding bit in the second feature vector to obtain multiple matching bits; and Based on the multiple matching bits, the distance between the first feature vector and the second feature vector is determined.

12. The computer system according to claim 10, wherein, The identification circuit is further configured to: In response to the key graph received from the first memory being empty, the current graph received from the second memory is sent to the first memory as a new key graph.

13. The computer system according to claim 10, wherein, The identification circuit is further configured to: In response to the key graph received from the first memory being empty, all nodes in the current graph are identified; and The computing circuit is also configured to: Spatial calculations are performed on all nodes in the current graph to obtain the updated key graph.

14. The computer system according to claim 10, wherein, In order to perform spatial calculations on the identified one or more nodes, the computing circuit is further configured to: Obtain the feature vector of one node from the identified one or more nodes and the adjacency matrix of the current graph; Identify one or more neighboring nodes based on the adjacency matrix; The feature vectors of one or more neighboring nodes and the feature vector of the node are recursively aggregated and transformed to obtain the updated feature vector of the node.

15. The computer system according to claim 10, wherein, Also includes: The second computing circuit is configured to perform time calculations based on the key graph and the updated key graph.

16. A computer implementation method for accelerating time-graph neural network computation, comprising: Receive the current graph collected from the current time step; Determine whether the current graph is a key graph or an auxiliary graph; In response to the fact that the current graph is a key graph, spatial computation is performed on the nodes in the key graph to obtain an updated key graph; In response to the fact that the current graph is an auxiliary graph, execute: Based on the comparison between the key graph and the auxiliary graph, one or more nodes in the auxiliary graph are identified. In order to identify one or more nodes in the auxiliary graph, for each node in the auxiliary graph, the corresponding node in the key graph is identified. The distance between the first feature vector of the node in the auxiliary graph and the second feature vector of the corresponding node in the key graph is determined. If the distance is greater than a threshold, the node in the auxiliary graph is selected. Perform spatial computation on the identified one or more nodes to obtain updated nodes; and Based on the key graph and the one or more update nodes, generate an updated key graph; and Based on the key graph and the updated key graph, time calculations are performed to predict the graph for future time steps.

17. The method according to claim 16, wherein, Determining whether the current graph is a key graph or an auxiliary graph includes: If the current graph is the first graph received, then the current graph is determined to be a key graph.

18. The method according to claim 16, wherein, The time calculation includes: Using convolutional neural networks or long short-term memory neural networks, the temporal features between the key graph and the updated key graph are determined.

Images