Programmable access engine architecture for graph neural networks and graph applications

Abstract

This specification describes methods and systems for accessing attribute data during graph neural network (GNN) processing. An example system includes: multiple cores, each of the multiple cores includes a key-value extractor and a filter, and can be programmed using a software interface to support multiple data formats of GNN attribute data, wherein the key-value extractor can be programmed using the software interface to perform key-value extraction associated with accessing GNN attribute data, and the filter of at least one of the multiple cores can be programmed using the software interface to sample a node identifier associated with accessing GNN attribute data; and a first memory coupled to the multiple cores in communication, wherein the first memory is configured to store data shared by the multiple cores.

Description

Technical Field

[0001] This specification relates to the field of graph neural networks (GNNs), and more particularly to accelerating data access to GNNs. Background Art

[0002] Graph neural networks ("GNNs") can be effective models for modeling and processing unstructured data. Recently, GNNs are increasingly used in applications such as recommendation systems, risk control systems, etc. Graph data can be unstructured. Therefore, accessing graph data can result in random memory access. Summary of the invention

[0003] An embodiment of the present specification provides a system for accessing GNN attribute data when performing graph neural network (GNN) processing, including: multiple cores, each of the multiple cores includes a key-value extractor and a filter, and is programmable using a software interface to support multiple data formats of the GNN attribute data, wherein: the key-value extractor can be programmed using the software interface according to one of the multiple data formats of the GNN attribute data for performing key-value extraction associated with accessing the GNN attribute data, and the filter of at least one of the multiple cores can be programmed using the software interface according to one of the multiple data formats of the GNN attribute data for sampling a node identifier associated with accessing the GNN attribute data; and a first memory, communicatively coupled to the multiple cores, wherein the first memory is configured to store data shared by the multiple cores.

[0004] In some embodiments, the software interface includes at least one or more of the following configurable parameters for programming the key-value extractor: a memory address of the first memory that stores the key for the key-value extraction; a memory address of the second memory that stores the value for the key-value extraction; the number of key-value pairs to be extracted; the length of the data to be extracted; a target memory address in the first memory for writing the result; or a flag indicating whether reordering is enabled.

[0005] In some embodiments, the software interface includes at least one of the following configurable parameters for programming the filter: a sampling algorithm; or a regular expression for filtering sampled results generated by the sampling algorithm.

[0006] In some embodiments, the key-value extractor of at least one of the multiple cores can also be programmed through a software interface to: determine a memory address of the first memory storing a value corresponding to the key; extract the value corresponding to the key from the memory address; and write the extracted value to the first memory.

[0007] In some embodiments, the multiple cores include a first core; the key-value extractor of the first core can be programmed through a software interface to: determine an address in the first memory from which edge data of the root node is to be extracted, wherein the edge data includes identifiers of neighboring nodes of the root node, and extract the edge data of the root node from the determined address; and the filter of the first core can be programmed through a software interface to: use a sampling algorithm to sample from identifiers of neighboring nodes in the edge data to determine one or more sampled node identifiers, and write the one or more sampled node identifiers to the first memory.

[0008] In some embodiments, the edge data of the root node includes a weight value for each edge of the root node; and the filter of the first core can also be programmed through a software interface to: sample according to the weight value of each edge of the root node to determine one or more sampled node identifiers.

[0009] In some embodiments, the multiple cores include a second core; and the key-value extractor of the second core can be programmed through a software interface to: determine the address of an edge list pointer in the first memory from which the root node is to be extracted, extract the edge list pointer of the root node from the determined address, wherein the edge list pointer points to an edge list of the root node stored in the second memory, and each edge list connects the root node to a neighboring node, extract one or more identifiers of the neighboring nodes of the root node based on the edge list of the root node from the second memory, and write the one or more identifiers of the neighboring nodes of the root node into the first memory for access by the first core.

[0010] In some embodiments, the multiple cores include a third core; and the key-value extractor of the third core can be programmed through a software interface to: receive a root node identifier corresponding to the root node, determine an edge list pointer based on the root node identifier, and write the edge list pointer to the first memory for access by the second core.

[0011] In some embodiments, the multiple cores include a third core and a fourth core; the key-value extractor of the third core can be programmed through a software interface to: receive a root node identifier corresponding to the root node, extract a pointer corresponding to the root node based on the root node identifier, wherein the pointer corresponding to the root node identifier points to the structural data of the root node, and the structural data of the root node includes an edge list pointer, and write the pointer corresponding to the root node into the first memory for access by the fourth core; and the key-value extractor of the fourth core can be programmed through a software interface to: determine an address in the first memory from which the pointer corresponding to the root node is to be extracted; extract the pointer corresponding to the root node from the memory buffer from the determined address, determine the edge list pointer based on the pointer corresponding to the root node, and write the edge list pointer to the first memory for access by the second core.

[0012] In some embodiments, the multiple cores include a fifth core and a sixth core; the key-value extractor of the fifth core can be programmed through a software interface to: determine an address in the first memory from which a sampled node identifier is to be extracted, extract the sampled node identifier from the first memory from the determined address, extract a sampled pointer pointing to attribute data of the sampled node identifier stored in the second memory, and write the sampled pointer to the first memory for access by the sixth core; and the key-value extractor of the sixth core can be programmed through a software interface to: determine an address in the first memory from which a sampled pointer is to be extracted, extract the sampled pointer from the first memory from the determined address, extract attribute data from the second memory using the sampled pointer, and send the attribute data to a dedicated processor for GNN processing.

[0013] In some embodiments, each of the plurality of cores further comprises a reordering scoreboard, wherein the reordering scoreboard is programmable through a software interface to: receive data from a key-value extractor or filter in the same core; and sort the received data.

[0014] In some embodiments, each of the multiple cores is implemented on a field programmable gate array (FPGA).

[0015] According to the method and system of the present specification, the GNN memory access system can be implemented as a GNN accelerator. The GNN accelerator may include multiple cores that can be configured through one or more software programmable interfaces. Through one or more software programmable interfaces, the GNN accelerator can be configured or programmed to provide great flexibility in processing GNN data access. Since the cores of the GNN memory access system are programmable, they can be flexibly programmed to match a variety of different data formats (e.g., data structures or layouts), including compressed sparse row (CSR) format, coordinate (COO) format, linked list format, array format, dynamic graph format, key-value format, weight value format about the edge, encoding attribute format, indirect node-identifier format, arbitrary hop number format, skip extraction weight or attribute format, other formats or any combination thereof. Compared with existing GNN accelerators with internal pipeline hard-coding / wiring for processing specific GNN data formats, the GNN memory access system described herein is programmable to be compatible with known mainstream GNN data formats. Therefore, the functionality of the GNN memory access system is improved, and the usability of the system is expanded to process and support GNNs represented in different data formats. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of an example diagram according to some embodiments of the present specification.

[0017] Figure 2is a schematic diagram of an example system using a GNN accelerator architecture according to some embodiments of the present specification.

[0018] Figure 3 is a schematic diagram of an example system for accelerating GNN performance according to some embodiments of the present specification.

[0019] Figure 4 is a schematic diagram of an example GNN access engine according to some embodiments of the present specification.

[0020] Figure 5 is a schematic diagram of an example improved GNN memory access system according to some embodiments of the present specification.

[0021] Figure 6 is a schematic diagram of an example improved GNN memory access system for performing attribute data access according to some embodiments of the present specification.

[0022] Figure 7 is a schematic diagram of an example improved GNN memory access system for performing attribute data access according to some embodiments of the present specification.

[0023] Figure 8 is a flowchart of an example method for accessing GNN attribute data when performing GNN processing according to some embodiments of the present specification. DETAILED DESCRIPTION

[0024] The details of the embodiments of the present specification are described below in conjunction with the accompanying drawings. The accompanying drawings show some embodiments of the present specification, and the present specification can be implemented in various ways and is not limited by the embodiments described herein. On the contrary, these embodiments are provided to more thoroughly understand the present specification and to fully convey the scope of the present specification to those skilled in the art.

[0025] In many embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than the steps described in this specification. In addition, the single step described in this specification can be divided into multiple steps for description in other embodiments, and the multiple steps described in this specification can be combined into a single step for description in other embodiments.

[0026] Data can be structured or unstructured. For structured data, information can be arranged according to a preset data model or scheme. For unstructured data, information can be arranged without using a preset data model or predefined manner. For example, a text file (e.g., an email, a report, etc.) may include information that does not have a predefined structure (e.g., individual letters or words). Therefore, unstructured data can include irregularities and ambiguities that make it difficult to understand using traditional programs or data structures.

[0027] One way to represent unstructured data is by using a graph. A graph is a data structure that includes two components — nodes (or vertices) and edges. For example, a graph G can be defined as a set of nodes V and a set of edges E between the set of nodes. The nodes in the graph can have a set of features or attributes (e.g., user profiles). For example, a node can have up to f features or attributes. Therefore, for a graph with n nodes, the dimension of the node attribute matrix can be n×f. A node can be defined as a neighbor of another node if it shares edges with other nodes. Graphs can be highly flexible data structures because graphs may not require predefined rules to determine how many nodes or edges must be in a graph. Since graphs can provide a lot of flexibility, it is one of the data structures widely used to store or represent unstructured data (e.g., text files).

[0028] Figure 1 is a schematic diagram of an example diagram according to some embodiments of this specification. Figure 1 As shown, the graph includes nodes n111, n112, n113, n114, n115, and n116. In addition, the graph includes edges e121, e122, e123, e124, e125, e126, and e127. Each node has one or more adjacent nodes. For example, nodes n112 and n113 are adjacent to node n111 because node n112 shares edge e121 with node n111, and node n113 shares edge e122 with node n111.

[0029] When storing graphs in memory, nodes, edges, and attributes can be stored in many different data formats (also called data structures). For example, graphs can be stored in compressed sparse row (CSR) format, coordinate (COO) format, linked list format, array format, dynamic graph format, key-value format, weight value format about edges, encoded attribute format, indirect node identifier format, arbitrary hop number format, skip extraction weight or attribute format, etc. Different data formats may require different data access methods.

[0030] Using the array format, node identifiers of the graph can be stored in an array, where each node identifier provides an address or pointer to the location of the attribute data of the corresponding node. The attributes of all nodes can be stored together and can be accessed by reading the address or pointer stored in the corresponding node identifier. By separating the attribute data from the corresponding node, the data structure can provide faster traversal access to the graph.

[0031] Using the CSR format, the nodes and edges of a graph can be stored in separate arrays, where the indices of these arrays correspond to the node identifiers and edge identifiers. The edge array can be sorted by the source of each edge and include edge information or the node identifier at the other end of the edge. The node array can store offsets into the edge array, providing the offset to the first edge of each node. For example, Figure 1 As shown, the edge array of the graph may be "n112, n113, n111, n113, n114, n111, n112, n115, n112, n115, n116, n113, n114, n114". The first two entries "n112" and "n113" represent two edges originating from the first node n111. The next three entries "n111", "n113" and "n114" represent three edges originating from the second node n112. The same logic may be applied to the rest of the edge array. The node array may include only the offsets into the edge array, such as "1,3,6,9,12,14". The CSR format is considered suitable for storing sparse graphs by packing the entries in a compressed array, thereby allowing fast traversal of the nodes and edges of the graph.

[0032] Using the COO format, the edges of a graph can be stored as a list of tuples, where each edge tuple can include a source node identifier, a destination node identifier, appropriate attribute information of the edge, or any combination thereof. Figure 1 As shown, edge e121 can be represented as [n111, n112]. In some embodiments, if the edge is weighted, an additional entry in the tuple can be created to record the weight value of each edge. Figure 1 As shown, if the weight value of edge 121 is 3, the tuple is represented as [n111, n112, 3]. In some embodiments, the graph data stored in the COO format is not sorted. For example, the tuples of all edges can be stored in any order. The COO format can be suitable for graphs that are frequently updated. For example, if a new node or a new edge is added to the graph, the COO format only needs to add one or more tuples for the newly added data.

[0033] Using a linked list format, the nodes and / or edges of a graph can be stored in a linked list. For example, the graph nodes can be stored in a first linked list, and the edges can be stored in a second linked list. The first linked list and the second linked list can be cross-referenced (e.g., by using a cross-list pointer). Compared to an array-based data format, a linked list-based data format can provide better flexibility and higher random access speeds, but may suffer from lower sequential access speeds.

[0034] Using dynamic graph formats, graphs can be extended using data structures such as linked lists.

[0035] Utilize key-value search and key-value reverse search format, the nodes and / or edges of the graph can be stored as key-value pairs, so that key-value search can be efficiently performed on the graph. For example, for a given node identifier, the key-value search format can perform an efficient search for the attribute data, edges or adjacent nodes of the node identifier. In addition, the reverse search can also be performed in an efficient manner. For example, for a given attribute data, edges or adjacent nodes, the key-value reverse search format can perform an efficient search for the corresponding node identifier.

[0036] With the weight value on edge format, each edge in the graph can carry a weight value (e.g., similar to the COO format). The weight values ​​on the edges can be used, for example, in the GNN sampling process.

[0037] Using the encoded attribute format, the memory location of the attribute data of a node may be obtained by performing additional calculations based on the node identifier of the node. The calculation may refer to decoding the encoded memory location of the attribute data.

[0038] Using the arbitrary hop count format, nodes that are indirectly connected to a given node can be efficiently accessed. For example, Figure 1 As shown, node n116 is connected to node n111 via at least three edges (e.g., edges e121, e124, and e127). Although node n116 is three edges away from node n111, given node n111, node n116 can still be accessed in an efficient manner.

[0039] With the skip-extraction weight or attribute format, the attribute data and the edge weight values ​​do not have to be extracted together. For example, for a given node identifier, the edge weight values ​​can be efficiently extracted without extracting the attribute data corresponding to the node identifier or the neighboring nodes of the node identifier.

[0040] Graph Neural Network (GNN) is a neural network that can operate directly on graphs. GNNs are more suitable for operating on graphs than traditional neural networks (e.g., convolutional neural networks) because GNNs can be better equipped to adapt to graphs of arbitrary size or complex topology. GNNs can perform inference on data described in graph format. GNNs can perform node-level, edge-level, or graph-level prediction tasks.

[0041] GNN processing can involve GNN training and GNN inference, both of which can involve GNN computation. A typical GNN computation on a node (or vertex) can involve aggregating features of its neighbors (direct neighbors or each neighbor's neighbors), and then computing a new activation of the node to determine a feature representation (e.g., a feature vector) of the node. Therefore, GNN processing for a small number of nodes typically requires input features of a significantly large number of nodes. Using all neighbors for message aggregation is too costly because the nodes required for the input features can easily cover a large portion of the graph, especially for real-world graphs of enormous size (e.g., hundreds of millions of nodes with billions of edges).

[0042] To execute a GNN, the system may retrieve graph data from memory and send the data to one or more processors for processing. Figure 2 2 is a schematic diagram of an example system using a GNN accelerator architecture according to some embodiments of the present specification. As shown in FIG. 2, system 200 includes one or more processors 210, a GNN module 220, a memory 230, and one or more dedicated processors 240. In some embodiments, one or more processors 210 include one or more CPUs (central processing units). In some embodiments, one or more dedicated processors 240 may include one or more graph processing units (GPUs) or dedicated neural network processing units (e.g., dedicated graph neural network processing units). In some embodiments, memory 230 may include synchronous dynamic random access memory (SDRAM), such as double data rate (DDR) SDRAM.

[0043] like Figure 2 As shown, the GNN module 220 can receive instructions and information about the GNN from one or more processors 210, and extract data related to the GNN from the memory 230. After receiving the data from the memory 230, the GNN module 220 can process the data and send the processed data to one or more dedicated processors 240 for processing.

[0044] In some embodiments, Figure 2As shown, the GNN module 220 may include a graph structure processor 221, a GNN sampler 222, a GNN attribute processor 223, and an address mapper 224. The graph structure processor 221 may receive instructions and information about the GNN from one or more processors 210, and extract information about more and more nodes and their edges from the memory 230. The graph structure processor 221 may then send the extracted information to the GNN sampler 222.

[0045] The GNN sampler 222 is configured to select one or more sampled nodes for GNN processing based on the edge information of one or more nodes. In some embodiments, the GNN sampler 222 may select one or more sampled nodes based on positive sampling or negative sampling. For example, based on positive sampling, one or more sampled nodes may be selected from nodes adjacent to one or more nodes. It should be understood that any algorithm other than positive sampling and negative sampling may be used to select sampled nodes.

[0046] After selecting the sampled nodes, the GNN sampler 222 may send the selection information of the sampled nodes to the GNN attribute processor 223. Based on the information of the sampled nodes, the GNN attribute processor 223 is configured to extract the information of the sampled nodes from the memory 230. In some embodiments, the information of the sampled nodes may include one or more features of each sampled node. The GNN sampler 222 may also be configured to send the extracted information of the sampled nodes and the information of one or more nodes and their edges to the dedicated processor 240. The dedicated processor 240 may perform GNN processing based on the information received from the GNN attribute processor 223.

[0047] In some embodiments, the graph structure processor 221 and the GNN attribute processor 223 may use the address mapper 224 to extract information from the memory 230. The address mapper may be configured to provide hardware address information in the memory 230 based on the node identifier and the edge. For example, the identifier n111 (e.g., Figure 1 111) to identify the node as part of the input GNN. If the graph structure processor 221 wants to extract information of the node n111 (e.g., attribute data of the node n111), the graph structure processor 221 can provide the identifier n111 to the address mapper 224, and the address mapper 224 can determine the physical address of the storage 230 storing the information of the node n111 (e.g., attribute data of the node n111). In some embodiments, the address mapper 224 can also determine the physical address of the storage 230 storing the edge about the node n111 (e.g., Figure 1 One or more physical addresses of information of edges e121 and e122).

[0048] Figure 2 The illustrated system 200 may be used to accelerate GNN memory accesses for many different systems while accelerating GNN performance. Figure 3 3 is a schematic diagram of an example system for accelerating GNN performance according to some embodiments of the present specification. As shown in FIG3 , system 300 includes an access engine 310 . Figure 3 The access engine 310 shown in FIG. 3 may be similar to Figure 2 The access engine 310 may be configured to retrieve information from a memory (e.g., Figure 3 DDR shown in ) retrieves the data required to perform the GNN calculation. For example, the access engine 310 can retrieve the node identifier, the edge identifier, and the attribute data corresponding to the node identifier. The data retrieved by the access engine 310 can be provided to the execution engine (e.g., the execution engine GEMM 340 or the execution engine VPU 350) or the processor for GNN related calculation. For example, Figure 3 As shown, system 300 includes two execution engines, execution engine GEMM 340 and execution engine VPU 350. Execution engine GEMM 340 can be configured for general matrix multiplication (GEMM) algorithm and execution engine VPU 350 can be configured for vector processing unit (VPU). Both types of engines can perform specific GNN related calculations in an accelerated manner.

[0049] Although the system 300 may include an acceleration engine and a processor to speed up GNN-related computations, it is the access engine 310 that may become a bottleneck in the overall performance of the system 300 because data retrieval performed by the access engine may be slower than data processing performed by the execution engine. Figure 4 is a schematic diagram of an example GNN access engine according to some embodiments of this specification. It can be understood that Figure 4 The access engine 400 shown in FIG. 4 may be similar to Figure 3 The access engine shown in . Figure 4 As shown, the access engine 400 may include a GetNeighbor module 410 , a GetSample module 420 , a GetAttribute module 430 , and a GetEncode module 440 .

[0050] In some embodiments, GetNeighbor module 410 is configured to access or identify neighboring nodes for an input node identifier. Figure 2 The graph structure processor 221 shown in FIG. 1 , the GetNeighbor module 410 can receive instructions and information about the GNN and obtain the information from the DDR (e.g., corresponding to Figure 2The GetNeighbor module 410 may then send the extracted information to the GetSample module 420 (e.g., corresponding to Figure 2 GNN sampler 222).

[0051] In some embodiments, the GetSample module 420 is configured to receive information about one or more nodes from the GetNeighbor module 410 and perform node sampling on the one or more nodes for GNN processing. Figure 2 In the GNN sampler 222 shown, the GetSample module 420 can be configured to select one or more sampled nodes for GNN processing based on the edge information of one or more nodes. In some embodiments, the GNN sampler 222 can select one or more sampled nodes based on positive sampling and / or negative sampling. After selecting the sampled nodes, the GetSample module 420 can send the selection information of the sampled nodes to the GetAttribute module 430.

[0052] In some embodiments, the GetAttribute module 430 may be configured to receive information of a selected or sampled node from the GetSample module 420 and retrieve the information from a memory (eg, Figure 4 DDR or Figure 2 230) to extract attribute information about the sampled nodes. For example, similar to the GNN attribute processor 223, the GetAttribute module 430 can be configured to extract attribute data of the sampled nodes from the memory 230 based on the received sampled nodes (e.g., sampled node identifiers). In some embodiments, the attribute data of the sampled nodes may include one or more features of each sampled node.

[0053] like Figure 2 , Figure 3 ,and Figure 4 As shown, when a GNN-based system performs a memory access (e.g., via Figure 3 ), GNN-based systems can predefine their hardcoded / wired internal pipelines (e.g. Figure 4 Modules 410-440) perform multiple steps, including determining neighbors, sampling nodes, and extracting attributes about the sampled nodes. The hard-coded / wired internal pipeline of the access engine in the GNN-based system means that the GNN-based system may only support one specific (or a few similar) data structures. For example, Figure 4The access engine shown in can support the Compressed Sparse Row (CSR) format, where the nodes and edges of the graph can be stored in separate arrays, where the indexes of these arrays correspond to the node identifiers and edge identifiers. However, for many other GNN data formats, such as the Coordinate List (COO) format that stores a list of (row, column, value) tuples (e.g., each tuple corresponds to a non-zero value and contains the row / column index of the non-zero value), Figure 1 The hardcoded / wired internal plumbing in the access engine as described in may not be able to correctly parse the data format and access the data.

[0054] Other problems arise when a GNN-based system receives GNNs or graphs in multiple types of formats. For example, commonly used graph formats may include data formats based on CSR format, COO format, linked list format, array format, dynamic graph format, key-value format, weight value format about edges, encoded attribute format, indirect node-identifier format, arbitrary hop number format, skip extraction weight or attribute format, etc. Some GNN-based systems also receive GNNs or graphs in a combination of different types of formats. In order to support all types of these formats or data structures, a new design is needed on the access engine to summarize the process when extracting data for GNN processing.

[0055] Embodiments of the present specification provide novel systems and methods for accessing data across a wide range of data formats for use in GNN processing. Figure 5 is a schematic diagram of an improved GNN memory access system according to some embodiments of the present specification. Figure 5 As shown, the GNN memory access system 500 includes multiple cores 510, a bus 520, and one or more memory buffers 530. In some embodiments, multiple cores 510 can access a memory that hosts data for use in a GNN. For example, the memory storage device can be a DDR (e.g., Figure 4 DDR shown in ). Bus 520 can be communicatively coupled to core 510 and memory buffer 530. In some embodiments, bus 520 can be a crossbar switch that provides interconnection between multiple cores 510 and memory buffer 530. In some embodiments, memory buffer 530 can be multiple groups of buffers. In some embodiments, GNN memory access system 500 can replace the above-mentioned access engine. For example, GNN memory access system 500 can replace Figure 3 . In some embodiments, the GNN memory access system 500 can replace Figure 2 GNN module 220 or Figure 4, the GetNeighbor module 410, the GetSample module 420, and the GetAttribute module 430 shown in . In some embodiments, the GNN memory access system 500 may also include a memory management system 540 for managing memory access (e.g., memory access in a memory storage device).

[0056] In some embodiments, each of the plurality of cores 510 may include a key-value extractor 511, a filter 512, or a reordering scoreboard 513. In some embodiments, the plurality of cores 510, including the key-value extractor 511, the filter 512, and the reordering scoreboard 513 may be programmable hardware (e.g., a field programmable gate array (FPGA)). For example, the plurality of cores 510 may be programmed by software, a software interface, or by a software user through a software interface. The key-value extractor 511 may be configured or programmed to perform a key-value extraction function. For example, the key-value extractor 511 may receive an identifier (key) and extract data corresponding to the identifier (value). The filter 512 may be configured or programmed to perform a filtering function. For example, for an input having multiple identifiers, the filter 512 may select a subset from the multiple identifiers based on one or more filtering rules (e.g., a sampling algorithm). In some embodiments, the reordering scoreboard 513 is a unit that may be configured to perform sorting or reordering of data received from the key-value extractor 511 or the filter 512.

[0057] The software interface can be implemented in various ways. In some embodiments, a software interface can be used to program multiple cores 510. In some embodiments, each of the multiple cores 510 can be programmed by a corresponding software interface. In some embodiments, the multiple cores can be divided into groups, and each group can be programmed by a corresponding software interface.

[0058] In some embodiments, at least one of the cores 510 represented as the first core may be configured or programmed to replace Figure 4 4. For example, the first core may be configured to access or identify neighboring nodes for an input node identifier. Thus, similar to Figure 4In the GetNeighbor module 410 in the first core, the first core performs a key-value extraction function, where the input node identifier is a key and the neighboring node or neighboring node identifier is a value (e.g., extracting the value based on the key). Therefore, the first core can use a key-value extractor 511 to perform the key-value extraction function. The key-value extractor 511 can receive the input node identifier (key) and extract the neighboring node identifier (value) corresponding to the input node identifier. In some embodiments, the key-value extractor 511 can be configured or programmed to extract the input node identifier from the buffer 530 and write the neighboring node identifier back to the buffer 530 so that the neighboring node identifier can be accessed 510 by other cores.

[0059] In some embodiments, at least one of the cores 510 represented as the second core may be configured or programmed to replace Figure 4 For example, the second core may be configured to receive information about the selected or sampled node or node identifier from the first core and retrieve the information from a memory (e.g., Figure 4 ) extracts attribute data about the sampled nodes. Therefore, similar to Figure 4 In the GetAttribute module 430 in the second core, the second core performs a key-value extraction function, where the sampled node identifier is a key for extracting attribute data as a value. Therefore, the second core can use a key-value extractor 511 to perform the key-value extraction function. The key-value extractor 511 can receive the sampled node identifier (key) and extract the attribute data (value) corresponding to the sampled node identifier.

[0060] In some embodiments, at least one of the cores 510, represented as a third core, may be programmed to replace Figure 4 For example, the third core is configured / programmed to receive adjacent node identifiers and perform node sampling according to the adjacent node identifiers to generate sampled node identifiers. Figure 4 In GetSample module 420 in the graph, the third core performs a filtering function that selects a subset of node identifiers based on neighboring node identifiers. Therefore, the third core can use filter 512 to perform a filtering function and select a subset of node identifiers in the graph as sampled node identifiers based on neighboring node identifiers.

[0061] In some embodiments, to achieve Figure 4 To accommodate the functions of various modules in different data formats, multiple cores 510 may need to be configured or programmed for each module. Figure 6 is a schematic diagram of an example improved GNN memory access system for performing attribute data access according to some embodiments of the present specification. Figure 6 As shown, the GNN memory access system 600 includes multiple cores 611-617. In some embodiments, Figure 6 The GNN memory access system 600 shown in FIG. 6 is similar to Figure 5 The GNN memory access system 500 shown in FIG. 5 , and the plurality of cores 611-617 are similar to Figure 5 The GNN memory access system 600 also includes a memory buffer 630 (e.g., similar to Figure 5 530) and a bus 620 (e.g., similar to Figure 5 In some embodiments, the GNN memory access system 600 also includes a memory management (e.g., similar to Figure 5 Memory management 540 as shown in FIG. Figure 6 As shown, multiple cores 611-617 are configured or programmed (e.g., via a software interface) to perform various steps when accessing attribute data. For example, various steps may include Figure 6 Steps 1-16 are shown using arrows. The GNN memory access system 600 can perform attribute data access on many different types of data formats, including CSR format, linked list format, array format, dynamic graph format, indirect node-identifier format, arbitrary hop number format, or a combination thereof. For example, based on a specific data format, multiple cores can be programmed to adapt to the data format. In some embodiments, when the access system 600 needs to access attribute data stored in a new data format, multiple cores can be reprogrammed to adapt to the new data format.

[0062] like Figure 6 As shown, in step 1, core 611 is configured or programmed to receive a root node identifier. In some embodiments, the root node identifier is received from a host (such as Figure 2 In step 2, the key-value extractor of core 611 may be configured or programmed to extract a pointer (value) corresponding to the root node based on the root node identifier (key). The pointer points to the structure data of the root node, which may include an edge list pointer. In some embodiments, the pointer corresponding to the root node is extracted from a memory storage device. For example, Figure 6As shown, core 611 can access DDR as part of the memory storage device. In some embodiments, the root node identifier already includes information about the pointer corresponding to the root node. Therefore, steps 1-2 may not be required to perform key-value extraction to convert the root node identifier into a pointer. After the pointer corresponding to the root node is extracted, the pointer is written to the memory buffer 630 for access by the next core (e.g., core 612). Since the memory buffer 630 is coupled to multiple cores 611-617 via bus 620, each of the multiple cores 611-617 can access the memory buffer 630 and extract data therefrom.

[0063] In step 3, core 612 is configured or programmed to receive and extract a pointer corresponding to the root node from memory buffer 630. In step 4, the key-value extractor of core 612 may be configured or programmed to extract an edge list pointer (value) corresponding to the root node based on the received pointer corresponding to the root node. The edge list pointer points to an edge pointer list of the root node. In some embodiments, the edge list pointer may be extracted from a memory storage device. For example, Figure 6 As shown, core 612 may access DDR as part of the memory storage device. After retrieving the edge list pointer, core 613 is programmed or configured to write the edge list pointer to memory buffer 630.

[0064] In step 5, core 613 is configured or programmed to receive and extract the edge list pointer from memory buffer 630. In step 6, the key-value extractor of core 613 may be configured or programmed to extract edge data about one or more edges based on the edge list pointer and write the edge data to memory buffer 630. In some embodiments, edge data is extracted from a memory storage device. For example, Figure 6 As shown, core 613 can access DDR as part of the memory storage device. The edge data extracted in step 6 may include edges connecting the root node and its adjacent nodes. In some embodiments, the edge data includes edge identifiers or node identifiers of adjacent nodes. In some embodiments, one or more edge identifiers or node identifiers are stored continuously in an array, and the memory address of the array is pointed to by an edge list pointer. In some embodiments, in addition to the memory address, the edge list pointer may also include the number of edges or adjacent nodes of the root node. Therefore, when the key-value extractor in 613 extracts edge data, the key-value extractor can locate the edge data based on the memory address and extract an array of the size of the number of edges.

[0065] In step 7, core 614 is configured or programmed to receive and extract edge data from memory buffer 630. In some embodiments, edge data includes node identifiers of adjacent nodes. In some embodiments, node identifiers are stored in linked list format or dynamic graph format. Therefore, one or more additional steps may be required to extract all node identifiers of adjacent nodes. For example, in step 8, the next pointer in the linked list may be provided to the key-value extractor of core 613, and the key-value extractor of core 613 may be configured or programmed to extract edge data about one or more edges based on the next pointer. This step may be repeated until each pointer in the linked list is processed. In step 9, the filter of core 614 is configured or programmed to sample from the node identifiers of adjacent nodes and select one or more sampled node identifiers. In some embodiments, the node identifiers of adjacent nodes may be sampled according to one or more sampling algorithms, and the sampling algorithm may be programmed into the filter through a software interface. Core 613 is also configured or programmed to write the sampled node identifier into storage buffer 630.

[0066] In step 10, core 615 is configured or programmed to receive and extract the sampled node identifier from memory buffer 630. In step 11, the key-value extractor of core 615 can be configured or programmed to extract the sampled pointer to the structure data of the sampled node identifier and write the sampled pointer to memory buffer 630. In some embodiments, the sampled pointer can be extracted from a memory storage device. For example, Figure 6 As shown, core 615 may access DDR as part of the memory storage device. In some embodiments, the structure data of the sampled node identifier may include a sampled attribute pointer pointing to the attribute data of the sampled node identifier.

[0067] In step 12, core 616 is configured or programmed to receive and extract a sampled pointer to the structure data of the sampled node identifier from memory buffer 630. In step 13, the key-value extractor may be configured or programmed to use the sampled pointer to extract a sampled attribute pointer to the attribute data of the sampled node identifier. In some embodiments, the sampled attribute pointer may be extracted from a memory storage device. For example, Figure 6As shown, core 616 can access DDR as part of the memory storage device. Core 616 is also configured or programmed to write the sampled attribute pointer to memory buffer 630. In some embodiments, the sampled node identifier already includes information about the sampled pointer. As a result, steps 12-13 may not be required to perform key-value extraction to convert the sampled node identifier to a sampled pointer. In some embodiments, steps 12-13 may be suitable for a data format that includes a node identifier that is indirectly associated with a pointer (e.g., an indirect node identifier format).

[0068] In step 14, core 617 is configured or programmed to receive and extract sampled attribute pointers from memory buffer 630. In some embodiments, there may be more than one root node. Therefore, one or more additional steps may be required to extract all sampled attribute pointers for each root node identifier. For example, in step 15, a loop operation may be initiated to perform some or all of steps 1-14 for another root node identifier. One of the cores (e.g., core 617) may be configured or programmed to initiate a loop operation. In some embodiments, it may be necessary to access adjacent nodes of the sampled node. For example, in an arbitrary hop count format, it may be necessary to access indirectly connected nodes that are multiple edges away from the root node. Therefore, one or more additional steps may be required to extract the attribute pointers of these nodes. For example, in step 15, a loop operation may be initiated to perform some or all of steps 1-14 on the sampled node. One of the cores (e.g., core 615) may be configured or programmed to initiate a loop operation based on the sampled node identifier.

[0069] In step 16, the key-value extractor of core 617 may be configured or programmed to extract attribute data of the sampled node identifier using the sampled attribute pointer. In some embodiments, the attribute data of the sampled node identifier may be extracted from a memory storage device. For example, Figure 6 As shown, core 611 can access DDR as part of the memory storage device. In some embodiments, core 617 can also be configured or programmed to send the extracted attribute data to other units for GNN processing (e.g., Figure 2 The dedicated processor 240 shown in Figure 3 Execution Engine shown in ).

[0070] like Figure 6 As shown, the GNN memory access system 600 includes multiple cores, which can be configured or programmed to implement Figure 2At the same time, since the cores of the GNN memory access system 600 can be programmed through one or more software programmable interfaces, they can be flexibly programmed to match a variety of different data structures, including CSR format, linked list format, array format, dynamic graph format, indirect node identifier format, arbitrary hop number format or any combination thereof. In addition, if the data format changes from an old data format to a new data format, multiple cores can be programmed again to adapt to the new data format. Therefore, the functionality of the GNN memory access system has been greatly improved, and the usability of the system has been expanded to handle many more input data formats.

[0071] In some embodiments, to achieve Figure 4 The functions of various modules in for various GNN data formats (such as COO format), Figure 5 The multi-core 510 shown in FIG. 5 may need to be configured or programmed differently. Figure 7 is a schematic diagram of an example improved GNN memory access system for performing attribute data access according to some embodiments of the present specification. Figure 7 As shown, the GNN memory access system 700 includes multiple cores 711-717. In some embodiments, according to the GNN data format for storing graph data, some cores (e.g., cores 716-717) can be configured or programmed to implement appropriate data access methods to access data stored in the data format. In some embodiments, Figure 7 The GNN memory access system 700 shown in FIG. Figure 5 The GNN memory access system 500 shown in FIG. 5 , and the plurality of cores 711-717 are similar to Figure 5 The GNN memory access system 700 also includes a memory buffer 730 (e.g., similar to Figure 5 530) and a bus 720 (e.g., similar to Figure 5 In some embodiments, the GNN memory access system 700 also includes a memory management (e.g., similar to Figure 5 7 ). As shown in FIG. 7 , multiple cores 711 and 717 are configured or programmed to perform various steps when accessing attribute data. For example, various steps may include using Figure 7 Steps 1-11 are shown by arrows in . The GNN memory access system 700 can perform attribute data access on a variety of different types of data formats (including COO format, array format, key-value format, edge weight value format, encoded attribute format, or a combination thereof).

[0072] like Figure 7 As shown, in step 1, core 711 is configured or programmed to receive a root node identifier. In some embodiments, the root node identifier is received from a host (such as Figure 2 In step 2, the key-value extractor of core 711 may be configured or programmed to extract an edge list pointer (value) corresponding to the root node based on the root node identifier (key). The edge list pointer points to the edge pointer list of the root node. In some embodiments, the edge list pointer may be extracted from a memory storage device. For example, Figure 7 As shown, core 711 can access DDR as part of the memory storage device. After retrieving the edge list pointer, core 711 is programmed or configured to write the edge list pointer to memory buffer 730. Since memory buffer 730 is communicatively coupled to multiple cores 711-715 via bus 720, each of multiple cores 711-715 can access memory buffer 730 and extract data therefrom.

[0073] In step 3, core 712 is configured or programmed to receive and extract the edge list pointer from memory buffer 630. In step 4, the key-value extractor of core 712 is programmed or configured to extract edge data about one or more edges based on the edge list pointer. In some embodiments, the edge data can be extracted from a memory storage device. For example, Figure 7 As shown, the core 712 can access the DDR as a part of the memory storage device. The edge data extracted in step 4 may include the edge connecting the root node and its adjacent nodes. In some embodiments, the edge data includes an edge identifier and / or a node identifier of an adjacent node. In some embodiments, the edge data includes a weight value, and the key-value extractor of the core 712 can be programmed or configured to extract the weight value of each edge together with the edge data. In some embodiments, the reordering scoreboard of the core 712 can be configured or programmed to sort the edge data based on the weight value of each edge. The sorting weight value of the edge can promote subsequent node sampling steps. For example, the adjacent nodes connected to the edge with a weight value greater than a threshold value can be sampled as positive samples. In some embodiments, the edge data can be sorted so that the edge with a lower weight value is stored before the edge with a higher weight value. In some embodiments, the core 712 is further programmed or configured to store the edge data in the memory buffer 730.

[0074] In step 5, core 713 is configured or programmed to receive and extract edge data from memory buffer 730. In some embodiments, edge data includes node identifiers of adjacent nodes. In some embodiments, edge data includes weight values ​​of edges. In step 6, the filter of core 713 is configured or programmed to sample from the node identifiers of adjacent nodes and select one or more sampled node identifiers. In some embodiments, sampled node identifiers are selected according to the weight values ​​of edges. For example, edge information can be sorted from the previous steps (e.g., step 4). As a result, the filter of core 713 can be configured or programmed to select sampled node identifiers from the first part of edge data (e.g., edges with smaller weight values). Core 713 is also configured or programmed to write sampled node identifiers into memory buffer 730.

[0075] In step 7, core 714 is configured or programmed to receive and extract the sampled node identifier from memory buffer 730. In some embodiments, the graph is stored in an encoded attribute format. Therefore, additional steps may be required to obtain the attribute data of the sampled node from the sampled node identifier. For example, in step 8, the key-value extractor of core 714 can be configured or programmed to extract the code of the sampled node identifier and write the encoded sampled node identifier to memory buffer 730. In some embodiments, the code of the sampled node identifier can be extracted from the memory storage device. For example, Figure 7 As shown, core 714 may access DDR as part of the memory storage device. In step 9, the key-value extractor of core 715 is configured or programmed to receive and extract the encoded sampled node identifier from memory buffer 730. In step 10, the key-value extractor of core 715 may be configured or programmed to use the encoded sampled node identifier to extract attribute data of the sampled node identifier. In some embodiments, the attribute data may be extracted from the memory storage device. For example, Figure 7 As shown, core 715 can access DDR as part of the memory storage device. In some embodiments, the graph is not stored in an encoded attribute format. As a result, steps 8-9 can be skipped, and core 714 is configured or programmed to use the sampled node identifier to extract the attribute data of the sampled node identifier. In some embodiments, core 714 or core 715 can also be configured or programmed to send the extracted attribute data to other units for processing (e.g., Figure 2 The dedicated processor 240 shown in Figure 3 Execution Engine shown in ).

[0076] like Figure 7 As shown, the GNN memory access system 700 includes multiple cores, which can be configured or programmed to implement Figure 2 At the same time, since the cores of the GNN memory access system 700 are programmable through one or more software programmable interfaces, they can be flexibly programmed to match a variety of different data structures, including COO format, array format, key-value format, edge weight value format, encoding attribute format or any combination thereof. Figure 6 and 7 As shown, the hardware settings of systems 600 and 700 can be the same or similar. At the same time, based on different ways of programming or configuring each component of the system (e.g., multiple cores), the GNN memory access system can be configured to support a wide range of graph data structures. Therefore, the functionality of the GNN memory access system is greatly improved and the usability of the system is expanded to handle many more input data formats.

[0077] It should be understood that the Figure 6 and 7 The steps in the multiple cores shown in are exemplary. Depending on the exact GNN data structure, more cores or fewer cores can be programmed, and the programming on each core can be adjusted accordingly to best match the needs when accessing a given GNN data structure. Figure 5 , Figure 6 and Figure 7 One of the advantages of the GNN memory access system shown in is the flexibility provided in the programmable hardware, so that potentially various GNN data structures can be handled by the GNN memory access system through hardware programming.

[0078] like Figure 6 and Figure 7 As shown, each core in the GNN memory access system can be programmed to meet specific needs when performing attribute data access in each type of data structure. The software interface for programming these cores can provide many programmable or configurable parameters for programming. For example, for the key-value extractor of each core, the parameters may include the memory address of the memory buffer storing the key of the key-value extraction, the memory address of the memory storage device storing the value of the key-value extraction, the number of key-value pairs to be extracted, the length of the data to be extracted, the target memory address of the memory buffer for writing the results of the key-value extraction, or a flag indicating whether reordering is enabled. For the filter of each core, the parameters may include a sampling algorithm or a regular expression for filtering the sampled results generated by the sampling algorithm.

[0079] Embodiments of the present specification also provide a method for accessing GNN attribute data during GNN processing. Figure 8is a flowchart of an example method for accessing GNN attribute data during GNN processing according to some embodiments of the present specification. It can be understood that Figure 8 The method 800 shown in FIG. 8 may be performed by Figure 5 The system 500 shown in FIG.

[0080] In step 810, in order to program the plurality of cores using the software interface, the key-value extractor of each of the plurality of cores is programmed using the software interface to execute the key-value extractor associated with accessing the GNN attribute data. Figure 6 and Figure 7 As shown, the key-value extractor of each core in the GNN memory access systems 600 and 700 is programmed to perform one or more steps when accessing GNN attribute data.

[0081] In some embodiments, method 800 also includes a step of determining a data format corresponding to the GNN processing. The data format may include, for example, a linked list format, an array format, a dynamic graph format, a key-value format, a weight value format about an edge, an encoded attribute format, an indirect node identifier format, an arbitrary hop number format, a skip extraction weight or attribute format, other formats, or a combination thereof. Multiple cores may be programmed according to the determined data format. For example, if the data format is determined to be a CSR format, then the data format may be programmed in the same format as Figure 6 In another example, if the data format is determined to be a COO format, the multiple cores may be programmed in a manner similar to the multiple cores in the GNN memory access system 600 shown in FIG. 7 . In some embodiments, if the data format is changed from an old data format to a new data format, the multiple cores may be programmed again to adapt to the new data format.

[0082] The data format can be determined by various means. For example, it can be manually specified or input or automatically detected. In some embodiments, the GNN accelerator implementing method 800 can receive a root node for GNN processing. The GNN accelerator can determine the memory address of the external memory storing the attribute data of the root node. By reading the header of the attribute data (such as the metadata portion of the attribute data), the GNN accelerator can learn the data format of the attribute data of the root node (and other graph nodes). The data format can be used to calculate the data size of the attribute data of each graph node. For example, if the data format is CSR, the data size includes the size of three one-dimensional arrays; and if the data format is COO, the data size includes the size of a 3-tuple. After learning the data size, the GNN accelerator can extract data of the data size from the memory address of the external memory to obtain the attribute data of the root node. The data may include an edge identifier of an edge connecting the root node and its neighboring nodes, a node identifier of a neighboring node, an edge weight, or any combination thereof. The GNN accelerator can perform sampling based on the extracted data to obtain one or more sampled nodes for GNN processing of the root node. In some embodiments, sampling can be based on the weights of the edges connecting the root node and the neighboring nodes. For example, the edges can be sorted / reordered according to the weights. The neighboring nodes corresponding to the edges with weights greater than a threshold can be obtained as sampled nodes. Subsequently, the GNN accelerator can determine the memory address of the attribute data of the sampled node in a memory (e.g., an external memory). By reading data of the data size from the memory address, the GNN accelerator can obtain the attribute data of the sampled node and send the extracted attribute data to a GNN processor (such as a GPU, TPU, NPU, etc.) for GNN processing of the root node.

[0083] In some embodiments, the above functions can be implemented by programming the key-value extractor, filter, and reordering board of each of the multiple cores in the GNN accelerator. Programming the key-value extractor using the software interface may include programming at least one of the configurable parameters. For example, the configurable parameters may include a memory address of a memory buffer storing a key for key-value extraction, a memory address of a memory storage device storing a value for key-value extraction, the number of key-value pairs to be extracted, the length of data to be extracted, a target memory address of a memory buffer for writing key-value extraction results, or a flag indicating whether reordering is enabled.

[0084] In some embodiments, the key-value extractor may be programmed to determine a memory address of the first memory storing a value corresponding to the key, extract the value corresponding to the key from the memory address, and write the extracted value to the first memory. Figure 6As shown in steps 3 and 4 of , the key-value extractor of core 612 is programmed to extract a pointer corresponding to the root node from memory buffer 630 based on the memory address determined in memory buffer 630, wherein the extracted pointer corresponding to the root node is the key in the key-value extraction. The key-value extractor of core 612 then uses the key to extract an edge list pointer corresponding to the root node and writes the edge list pointer to the memory buffer. In some embodiments, additional configurable parameters may be programmed for each key-value extractor. For example, Figure 6 As shown in steps 3 and 4 in , the length of the pointer corresponding to the root node can be determined and programmed as a configurable parameter of the multiple key-value pairs to be extracted. In addition, the length of the edge list pointer can be determined and programmed as a configurable parameter of the length of the data to be extracted.

[0085] In some embodiments, to program the key-value extractor of each of the plurality of cores, the key-value extractor of the first core is programmed through the software interface to determine an address in the memory buffer from which edge data of the root node is to be extracted, and extract the edge data of the root node from the determined address. Figure 6 Step 7 or Figure 7 As shown in step 5 of , the key-value extractor of core 614 or core 713 can be programmed to extract edge data from memory buffer 630 using the determined address in memory buffer 630 .

[0086] In some embodiments, in order to program the key-value extractor of each of the plurality of cores, the key-value extractor of the second core is programmed through a software interface to determine an address in a memory buffer from which an edge list pointer of a root node is to be extracted, extract the edge list pointer of the root node from the determined address, extract one or more identifiers of neighboring nodes of the root node based on the edge list pointer, and write the one or more identifiers of neighboring nodes of the root node into the memory buffer for access by the first core. For example, Figure 6 As shown in steps 5 and 6 of , the key-value extractor of core 613 can be programmed to extract the edge list pointer from memory buffer 630 using the determined address in memory buffer 630, and extract the edge data including the identifier of the neighboring node from the memory storage device based on the edge list pointer. In some embodiments, additional configurable parameters can be programmed. For example, the length of the edge data can be programmed as a configurable parameter of the length of data to be obtained.

[0087] In some embodiments, in order to program the key-value extractor of each of the plurality of cores, the key-value extractor of the third core is programmed through a software interface to receive a root node identifier corresponding to the root node, extract a pointer corresponding to the root node according to the root node identifier, and write the pointer corresponding to the root node into a memory buffer for access by other cores. Figure 6 Steps 1 and 2 in Figure 7 As shown in steps 1 and 2 in , the key-value extractor of core 611 or core 711 can be programmed to receive a root node identifier from the host and extract a pointer corresponding to the root node from the memory storage device based on the root node identifier. The pointer can be written to the memory buffer 630 or 730.

[0088] In some embodiments, to program the key-value extractor of each of the plurality of cores, the key-value extractor of the fourth core is programmed through the software interface to determine an address in a memory buffer from which a pointer corresponding to a root node is to be extracted, extract the pointer corresponding to the root node from the memory buffer based on the determined address, determine an edge list pointer according to the pointer corresponding to the root node, and write the edge list pointer to the memory buffer for access by the second core. For example, Figure 6 As shown in steps 3 and 4 in , the key-value extractor of core 612 is programmed to extract a pointer corresponding to the root node from the memory buffer 630 based on the determined address, extract an edge list pointer corresponding to the root node based on the pointer received from the memory storage device, and write the edge list pointer to the memory buffer 630 for access by core 613.

[0089] In some embodiments, to program the key-value extractor of each of the plurality of cores, the key-value extractor of the fifth core is programmed through the software interface to determine an address in the memory buffer from which to extract the sampled node identifier, extract the sampled node identifier from the first memory using the determined address, extract a sampled pointer pointing to the attribute data of the sampled node identifier stored in the memory storage device, and write the sampled pointer to the first memory for access by other cores. In some embodiments, many of these steps can be programmed into more than one core. For example, Figure 6As shown in steps 10-14 in , the key-value extractor of core 615 can be programmed to extract the sampled node identifier from memory buffer 630, extract the sampled pointer pointing to the structure data of the sampled node identifier, and write the sampled pointer to memory buffer 630 for access by core 616. In addition, the key-value extractor of core 616 can be programmed to extract the sampled pointer pointing to the structure data of the sampled node identifier from memory buffer 630, extract the sampled pointer from the memory storage device, and write the sampled attribute pointer to memory buffer 630.

[0090] In some embodiments, to program the key-value extractor of each of the plurality of cores, the key-value extractor of the sixth core is programmed through a software interface to determine an address in a memory buffer from which a sampled pointer is to be extracted, extract the sampled pointer from the memory buffer based on the determined address, extract attribute data from the memory storage device using the sampled pointer, and send the attribute data to a dedicated processor for GNN processing. For example, Figure 6 As shown in steps 15 and 16 of , the key-value extractor of core 617 is programmed to extract attribute data of the sampled node identifier using the sampled attribute pointer from the memory storage device. The key-value extractor of core 617 can also be programmed to send the extracted attribute data to other units for GNN processing.

[0091] In some embodiments, additional steps may be programmed into the multiple cores as part of the steps of programming the key-value extractor of each of the multiple cores. Figure 6 or Figure 7 Any of the steps shown in can be programmed into the key-value extractors of multiple cores.

[0092] Reference back Figure 8 In step 820, in order to program the plurality of cores using the software interface, a filter of at least one of the plurality of cores is programmed using the software interface to sample a node identifier associated with accessing GNN attribute data. For example, Figure 6 and Figure 7 As shown in , the filter of core 614 or core 713 can be programmed to sample and select one or more sampled node identifiers from the node identifiers of neighboring nodes. Core 614 or core 713 (e.g., a key-value extractor or filter of core 614 or core 713) can be further programmed to write the sampled node identifiers to memory buffer 630.

[0093] In some embodiments, programming the filter using the software interface may include programming at least one of the configurable parameters. For example, the configurable parameters may include a sampling algorithm or a regular expression for filtering sampled results generated by the sampling algorithm. In some embodiments, additional steps may be programmed into the multiple cores as part of the step of programming the filter for each of the multiple cores.

[0094] for Figure 8 For each step described in method 800, a single step can be programmed into multiple cores or multiple steps can be programmed into one core, based on the exact type of data structure during GNN processing.

[0095] Each process, method and algorithm described in the foregoing sections can be embodied in a code module executed by one or more computer systems or computer processors including computer hardware, and fully or partially automated by these code modules. The processes and algorithms can be implemented in part or in whole in a dedicated circuit.

[0096] When the functions disclosed herein are implemented in the form of software functional units and sold or used as independent products, They can be stored in a non-volatile computer-readable storage medium executable by a processor. The specific technical solutions disclosed herein (all or part) or aspects that contribute to current technology can be embodied in the form of software products. Software products can be stored in storage media, including multiple instructions that enable a computing device (which can be a personal computer, server, network device, etc.) to perform all or some steps of the method of an embodiment of the present application. Storage media can include flash drives, portable hard drives, ROM, RAM, disks, optical disks, another medium operable to store program code, or any combination thereof.

[0097] A specific embodiment also provides a system, which includes a processor and a non-transitory computer-readable storage medium storing instructions, the instructions being executable by the processor so that the system performs operations corresponding to the steps in any method of the above disclosed embodiments. A specific embodiment also provides a non-transitory computer-readable storage medium, which is configured with instructions executable by one or more processors, the instructions causing one or more processors to perform operations corresponding to the steps in any method of the above disclosed embodiments.

[0098] The embodiments disclosed herein may be implemented by a cloud platform, a server or a server group (hereinafter collectively referred to as a "service system") that interacts with a client. The client may be a terminal device or a client registered by a user on the platform, wherein the terminal device may be a mobile terminal, a personal computer (PC), and any device that may be installed with a platform application.

[0099] The various features and processes described above can be used independently of each other or can be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. In addition, certain method or process blocks can be omitted in some embodiments. The methods and processes described herein are also not limited to any particular order, and the blocks or states associated therewith can be performed in other appropriate orders. For example, the blocks or states described can be performed in an order other than the specifically disclosed order, or multiple blocks or states can be combined in a single block or state. The example blocks or states can be performed serially, in parallel, or in some other way. Blocks or states can be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein can be configured differently from the lock description. For example, compared to the disclosed example embodiments, elements can be added to, removed, or rearranged.

[0100] The various operations of the example methods described herein may be performed at least in part by an algorithm. The algorithm may be contained in a program code or instructions stored in a memory (e.g., the non-transitory computer-readable storage medium described above). Such an algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly program the computer to perform a function, but may learn from training data to make a predictive model that performs the function.

[0101] The various operations of the example methods described herein may be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute a processor-implemented engine that operates to perform one or more operations or functions described herein.

[0102] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some operations of a method may be performed by one or more processors or an engine implemented by a processor. In addition, one or more processors may also be operable to support the performance of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some operations may be performed by a group of computers (as an example of a machine including a processor), wherein these operations may be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interface (API)).

[0103] The performance of certain operations can be distributed among processors, not only residing in a single machine, but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine can be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented engine can be distributed across multiple geographic locations.

[0104] Throughout this specification, multiple instances can implement components, operations or structures described as single instances. Although the individual operations of one or more methods are illustrated and described as individual operations, one or more of the independent operations can be performed simultaneously, and it is not necessary to perform these operations in the order shown. The structure and function presented as a separate component in the example configuration can be implemented as a combined structure or component. Similarly, the structure and function presented as a single component can be implemented as a separate component. These and other changes, modifications, additions and improvements fall within the scope of this paper theme.

[0105] Although an overview of the subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present disclosure. Such embodiments of the subject matter may be referred to herein, individually or collectively, as the term "invention", only for convenience, and are not intended to voluntarily limit the scope of the present application to any single disclosure or concept if, in fact, more than one is disclosed.

[0106] The embodiments shown herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, the detailed description should not be construed as limiting, and the scope of the various embodiments is limited only by the appended claims and the full scope of equivalents to which these claims are assigned.

[0107] Any process description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or code segment, which includes one or more executable instructions for implementing specific logical functions or steps in the process. Alternative implementations are included within the scope of the embodiments described herein, in which elements or functions may be deleted, performed from the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved, as will be understood by those skilled in the art.

[0108] As used herein, "or" is inclusive and not exclusive, unless otherwise expressly stated or the context indicates otherwise. Therefore, "A, B, or C" herein means "A, B, A and B, A and C, B and C, or A, B and C", unless otherwise expressly stated or the context indicates otherwise. In addition, "and" is both a union and several, unless otherwise expressly stated or the context indicates otherwise. Therefore, "A and B" herein means "A and B, jointly or separately", unless otherwise expressly stated or the context indicates otherwise. In addition, multiple instances can be provided for resources, operations, or structures described herein as a single instance. In addition, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are shown in the context of a specific illustrative configuration. Other allocations of functionality are contemplated and may fall within the scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configuration may be implemented as combined structures or resources. Similarly, structures and functionality presented as single resources may be implemented as separate resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of the present disclosure represented by the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

[0109] The terms "comprises" or "includes" are used to indicate the presence of subsequently claimed features, but they do not exclude the addition of other features. Conditional language, such as "may," "might," "might," or "would," unless specifically stated otherwise or otherwise understood in the context of use, is generally intended to convey that some embodiments include, while other embodiments do not include, certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that features, elements, and / or steps are in any way necessary for one or more embodiments, or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether such features, elements, and / or steps are included or will be performed in any particular embodiment.

Claims

1. A system for accessing GNN attribute data when performing GNN processing, comprising: A plurality of cores, each of the plurality of cores comprising a key-value extractor and a filter and being programmable using a software interface to support a plurality of data formats of the GNN attribute data, wherein: A key-value extractor can be programmed using the software interface according to one of a plurality of data formats for performing key-value extraction associated with the GNN attribute data, and A filter of at least one of the plurality of cores is programmable using the software interface according to one of a plurality of data formats for sampling a node identifier associated with accessing the GNN attribute data; and A first memory is communicatively coupled to the plurality of cores, wherein the first memory is configured to store data shared by the plurality of cores.

2. The system of claim 1 , wherein the software interface includes at least one or more of the following configurable parameters for programming the key-value extractor: A memory address of a key for key-value extraction stored in the first memory; a memory address in the second memory storing the value of the key-value extraction; The number of key-value pairs to be extracted; The length of the data to be extracted; a target memory address in the first memory for writing the result; or Flag indicating whether reordering is enabled.

3. The system according to claim 1, wherein: The software interface includes at least one of the following configurable parameters for programming the filter: Sampling algorithm; or A regular expression used to filter the sampled results generated by the sampling algorithm.

4. The system of claim 1, wherein: The plurality of cores includes at least one core; and The key-value extractor of the at least one core is programmable via the software interface to: determining a memory address of a second memory storing a value corresponding to the key; extracting a value corresponding to the key from a memory address of the second memory; as well as The extracted value is written into the first memory.

5. The system of claim 1, wherein: The plurality of cores includes a first core; The key-value extractor of the first core can be programmed through the software interface to: determining an address in the first memory from which edge data of a root node is to be extracted, wherein the edge data comprises identifiers of neighboring nodes of the root node, and Extracting edge data of the root node from the determined address; and The filter of the first core can be programmed through the software interface as: using a sampling algorithm to sample from identifiers of neighboring nodes of the edge data to determine one or more sampled node identifiers, and One or more sampled node identifiers are written into the first memory.

6. The system of claim 5, wherein: The edge data of the root node includes a weight value of each edge of the root node; and The filter of the first core can be programmed through the software interface as: Sampling is performed according to the weight value of each edge about the root node to determine one or more sampled node identifiers.

7. The system of claim 5, wherein: The plurality of cores include a second core; and The key-value extractor of the second core can be programmed through the software interface to: determining the address of the edge list pointer in the first memory from which the root node is to be extracted, extracting an edge list pointer of the root node from the determined address, wherein the edge list pointer points to an edge list of the root node stored in the second memory, and each edge list connects the root node to an adjacent node, extracting one or more identifiers of neighboring nodes of the root node based on the edge list of the root node from the second memory, and One or more identifiers of neighboring nodes of the root node are written into the first memory for access by the first core.

8. The system of claim 7, wherein: The plurality of cores include a third core; and The key-value extractor of the third core can be programmed through the software interface to: receiving a root node identifier corresponding to the root node, determining the edge list pointer according to the root node identifier, and The edge list pointer is written into the first memory for access by the second core.

9. The system of claim 7, wherein: The plurality of cores include a third core and a fourth core; The key-value extractor of the third core can be programmed through the software interface to: receiving a root node identifier corresponding to the root node, extracting a pointer corresponding to the root node according to the root node identifier, wherein the pointer corresponding to the root node identifier points to structural data of the root node, and the structural data of the root node includes an edge list pointer, and Writing a pointer corresponding to the root node into the first memory for access by the fourth core; and The key-value extractor of the fourth core can be programmed through the software interface to: determining an address in the first memory from which to extract a pointer corresponding to the root node; Extracting a pointer corresponding to the root node from the first memory from the determined address; Determine an edge list pointer according to a pointer corresponding to the root node; and The edge list pointer is written into the first memory for access by the second core.

10. The system of claim 5, wherein: The plurality of cores include a fifth core and a sixth core; The key-value extractor of the fifth core can be programmed through the software interface to: determining an address in the first memory from which to extract a sampled node identifier; extracting a sampled node identifier from the first memory from the determined address; extracting a sampled pointer to attribute data of a sampled node identifier stored in a second memory; as well as writing the sampled pointer to the first memory for access by the sixth core; and The key-value extractor of the sixth core can be programmed through the software interface to: determining an address in the first memory from which a sampled pointer is to be retrieved; extracting a sampled pointer from the first memory from the determined address; extracting attribute data from the second memory using the sampled pointer; as well as Send attribute data to a dedicated processor for GNN processing.

11. The system of claim 1 , wherein each of the plurality of cores further comprises a reorder scoreboard, wherein the reorder scoreboard is programmable through the software interface to: receiving data from a key-value extractor or filter in the same core; and Sort the received data.

12. The system of claim 1, wherein the plurality of data formats include a compressed sparse row (CSR) format and a coordinate list (COO) format.

13. A method for accessing GNN attribute data when performing GNN processing, comprising: Receiving, by a GNN accelerator, a root node identifier of a root node for performing GNN processing, wherein the GNN accelerator comprises a plurality of cores programmable through a software interface, the plurality of cores being a plurality of cores according to any one of claims 1-12; Determining, by the GNN accelerator, a data format from a plurality of data formats storing attribute data of the root node, wherein the plurality of data formats are supported by the GNN accelerator; determining, by one or more of the plurality of cores, a memory address of the external memory based on a root node identifier and a data size based on a data format; extracting, by the one or more cores, data of the data size from a memory address of the external memory, wherein the data includes identifiers of neighboring nodes of the root node; Sampling one or more neighboring nodes by the one or more cores; extracting, by the one or more cores, attribute data corresponding to the one or more sampled nodes based on identifiers and data sizes of the one or more sampled nodes; as well as The attribute data is sent by the one or more cores to a GNN processor for GNN processing.

14. The method according to claim 13, wherein determining, by the GNN accelerator, a data format storing attribute data of the root node comprises: Read the header of the attribute data of the root node; as well as determining the data format based on the header; The step of extracting data of the data size from the memory address of the external memory by the one or more cores further comprises: Extracting data of the data size from a memory address of the external memory by a key-value extractor of one or more cores in the GNN accelerator, wherein the key-value extractors of the cores are programmed based on the data format.

Images