Methods, systems, and storage media for accelerating graph neural network attribute access

Abstract

The present disclosure provides a method and system for accelerating attribute data access of graph neural network processing. An example method includes: receiving a root node identifier corresponding to a node in a graph for graph neural network processing; determining one or more candidate node identifiers according to the root node identifier, attribute data corresponding to the one or more candidate node identifiers being stored in a memory in turn; and sampling one or more graph node identifiers from the one or more candidate node identifiers at least to perform the graph neural network processing. The present disclosure improves the access efficiency of the memory.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to memory structures, and more particularly to a memory structure for optimizing memory access efficiency in graph neural networks (GNNs). BACKGROUND

[0002] Traditional deep learning models are good at pattern recognition and data mining by capturing hidden patterns of Euclidean data (such as images, texts, videos, etc.), but it has been proven that graph neural networks (GNNs) can extend the capabilities of machine learning to non-Euclidean fields characterized by graph data with complex relationships and interdependence between objects. Studies have shown that graph neural networks can outperform the current state-of-the-art in applications from molecular inference to community detection.

[0003] Graph neural networks are a very effective model for modeling and processing unstructured data. In recent years, graph neural networks have been increasingly used in recommendation systems, risk control systems, and other applications. Since graph data is unstructured, accessing graph data can result in random memory access. SUMMARY

[0004] One object of the present disclosure is to realize an access-friendly memory structure for graph neural network sampling.

[0005] According to one aspect of the present disclosure, a computer-implemented method is provided, comprising: receiving a root node identifier corresponding to a node in a graph for graph neural network processing; determining one or more candidate node identifiers from the root node identifier, attribute data corresponding to the one or more candidate node identifiers being stored in a memory in sequence; and sampling one or more graph node identifiers from the one or more candidate node identifiers for graph neural network processing.

[0006] In some embodiments, determining one or more candidate node identifiers from the root node identifier comprises: determining a plurality of neighboring node identifiers in the graph from the root node identifier; obtaining physical addresses of the plurality of neighboring node identifiers in the memory; identifying one or more candidate physical addresses from the physical addresses, the one or more candidate physical addresses being contiguous in the memory; and determining the one or more candidate node identifiers from the one or more candidate physical addresses.

[0007] In some embodiments, identifying one or more candidate physical addresses from the physical addresses comprises: identifying the one or more candidate physical addresses, wherein attribute data corresponding to the one or more candidate physical addresses is obtained from the one or more candidate physical addresses in one read operation of the memory.

[0008] In some embodiments, the method further comprises: determining a number of sampled node identifiers for which attribute data is stored in a bank of the memory; determining whether the number of sampled node identifiers exceeds a selection threshold for the bank for which attribute data is stored in the bank; and in response to determining that the number of sampled node identifiers in the bank exceeds the selection threshold for the bank, setting an invalid flag to prevent sampling more than the selection threshold of node identifiers from the bank of the memory.

[0009] In some embodiments, the determining a number of sampled node identifiers for which attribute data is stored in a bank of the memory comprises: determining the number of sampled node identifiers for which attribute data is stored in the bank by reading a count register of the bank; the method further comprising: updating the count register of the bank in response to the node identifiers of the bank being sampled.

[0010] In some embodiments, the determining whether the number of sampled node identifiers exceeds a selection threshold for the bank for which attribute data is stored in the bank comprises: for a plurality of banks including the bank, determining a plurality of count registers, each of the plurality of count registers corresponding to one of the plurality of banks; determining a minimum value from the plurality of count registers; and determining whether a difference between the number of sampled node identifiers and the minimum value exceeds the selection threshold for the bank.

[0011] In some embodiments, the sampling one or more graph node identifiers comprises: sampling one or more positive node identifiers that are within a preset distance from the root node; and sampling one or more negative node identifiers that are outside the preset distance from the root node.

[0012] In some embodiments, the preset distance is 1.

[0013] In some embodiments, the memory comprises a double data rate synchronous dynamic random access memory.

[0014] According to another aspect of the present disclosure, a system for accelerating graph neural network attribute access is provided, comprising: a memory configured to store attribute data of nodes in a graph; and a graph neural network accelerator configured to: receive a root node identifier corresponding to a node in a graph for graph neural network processing; determine one or more candidate node identifiers according to the root node identifier, wherein attribute data corresponding to the one or more candidate node identifiers are stored in the memory in sequence; and sample one or more graph node identifiers from at least the one or more candidate node identifiers for graph neural network processing.

[0015] According to another aspect of the present disclosure, one or more non-transitory computer-readable storage media storing instructions executable by one or more processors are provided, wherein execution of the instructions causes the one or more processors to: receive a root node identifier corresponding to a node in a graph for graph neural network processing; determine one or more candidate node identifiers according to the root node identifier, wherein attribute data corresponding to the one or more candidate node identifiers are stored in the memory in sequence; and sample one or more graph node identifiers from at least the one or more candidate node identifiers for graph neural network processing.

[0016] According to another aspect of the present disclosure, a graph neural network acceleration apparatus is provided, comprising: an obtaining module configured to receive a root node identifier corresponding to a node in a graph for graph neural network processing; a determining module configured to determine one or more candidate node identifiers according to the root node identifier, wherein attribute data corresponding to the one or more candidate node identifiers are stored in the memory in sequence; and a sampling module configured to sample one or more graph node identifiers from at least the one or more candidate node identifiers for graph neural network processing.

[0017] According to another aspect of the present disclosure, a graph neural network acceleration apparatus is provided, comprising: an obtaining module configured to obtain a root node identifier corresponding to a node in a graph for graph neural network processing; a sampling module configured to perform node sampling for a graph neural network according to a candidate node identifier group determined based on the root node identifier; a first determining module configured to determine a number of sampled node identifiers, attribute data of the sampled node identifiers being stored in a storage repository of the memory; a second determining module configured to determine whether the number of the sampled node identifiers exceeds a selection threshold of the storage repository, attribute data of the sampled node identifiers being stored in the storage repository; and a setting module configured to, in response to determining that the number of the sampled node identifiers in the storage repository exceeds the selection threshold of the storage repository, set an invalid flag to avoid sampling more than the selection threshold of node identifiers from the storage repository of the memory.

[0018] Embodiments of the present disclosure provide systems and methods that improve memory access for graph neural network sampling. According to the systems and methods, candidate node identifiers can be determined with the purpose of increasing sequential access to memory (e.g., DDR) for graph neural network sampling. For many types of memory that allow a single read operation to read multiple attribute data, candidate node identifiers that are consecutive to each other allow attribute data corresponding to multiple candidate node identifiers to be read from the memory in a single read operation, thus improving access efficiency to the memory overall in graph neural network sampling. Furthermore, a bank counter can be used to track the amount of data to be accessed in each bank of the memory. With the support of the bank counter, graph neural network sampling can be balanced to reduce or minimize bank conflicts on the memory, thus further improving access efficiency of the memory. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings are used to illustrate the methods, systems, and apparatuses for accelerating graph neural network attribute access of embodiments of the present disclosure. The present disclosure can be best understood with reference to the description and drawings.

[0020] Figure 1 A schematic diagram showing an example graph is provided according to embodiments of the present disclosure.

[0021] Figure 2 A schematic diagram showing an example system using a graph neural network accelerator structure is provided according to embodiments of the present disclosure.

[0022] Figure 3 A schematic diagram showing an example improved system using a graph neural network accelerator structure is provided according to embodiments of the present disclosure.

[0023] Figure 4 A schematic diagram showing an example address processor is provided according to embodiments of the present disclosure.

[0024] Figure 5 A schematic diagram showing an example graph neural network sampler is provided according to embodiments of the present disclosure.

[0025] Figure 6 A schematic diagram showing an example bank balancing processor is provided according to embodiments of the present disclosure.

[0026] Figure 7 A flowchart showing an example method of performing graph neural network based memory access is provided according to embodiments of the present disclosure.

[0027] Figure 8 A flowchart showing an example method of performing graph neural network based memory access using a bank counter is provided according to embodiments of the present disclosure.

[0028] Figure 9 A block diagram of an exemplary graph neural network acceleration device that improves memory access is shown, in accordance with an embodiment of the present disclosure.

[0029] Figure 10 A block diagram of an exemplary graph neural network acceleration device that improves memory access using a storage bank counter is shown, in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Details of embodiments of the present disclosure are described with reference to the accompanying drawings. The accompanying drawings show some embodiments of the present disclosure and the present disclosure can be implemented in various ways and is not limited by the embodiments described herein. Rather, these embodiments are provided so that the present disclosure can be understood more thoroughly, and the scope of the present disclosure is fully conveyed to those skilled in the art. The various features shown in the drawings are not drawn to scale.

[0031] In many embodiments, the steps of the respective methods are not necessarily performed in the order shown and described in the present disclosure. In some other embodiments, the methods can include more or fewer steps than those described in the present disclosure. In addition, a single step described in the present disclosure can be divided into multiple step descriptions in other embodiments, and multiple steps described in the present disclosure can be combined into a single step described in other embodiments.

[0032] Data can be structured or unstructured. For structured data, information can be arranged according to a pre-set data model or template. For unstructured data, information can be arranged without using a pre-set data model or predefined manner. For example, a text file (e.g., an email, a report, etc.) can include information (e.g., individual letters or words) that does not have a predefined structure. Thus, unstructured data can have irregularities and ambiguities that make it difficult to understand using conventional procedures or data structures.

[0033] One way to represent unstructured data is to use a graph. A graph contains two components: a data structure of nodes (or vertices) and edges. For example, a graph G can be defined as a set of nodes V and a set of edges E connecting the set of nodes. A node in a graph can have a set of features or attributes (e.g., a user profile in a graph representing a social network). If a node is connected to another node by an edge, the node can be defined as a neighboring node of the other node. A graph can be a highly flexible data structure because a graph can not require predefined rules to determine how many nodes are in a graph or how nodes are connected by edges. Because a graph can provide a great deal of flexibility, it is one of the data structures widely used to store or represent unstructured data (e.g., a text file).

[0034] Figure 1A schematic diagram showing an example graph according to an embodiment of the present disclosure. As shown, graph 100 includes nodes n111, n112, n113, n114, n115, and n116. In addition, graph 100 includes edges e121, e122, e123, e124, e125, e126, and e127. Each node has one or more neighboring nodes. For example, nodes n112 and n113 are adjacent to node n111 because nodes n112 and n113 share edges e121 and e122, respectively, with node n111. Figure 1

[0035] When storing a graph in computer memory, the nodes, edges, and properties can be stored in many different data structures. One way to store a graph is to separate the property data from the corresponding nodes. For example, node identifiers can be stored in an array, each node identifier providing an address or pointer to the location of the property data for the corresponding node. The properties for all nodes can be stored together, and can be accessed by reading the address or pointer stored in the corresponding node identifier. By separating the property data from the corresponding nodes, the data structure can provide faster traversal access on the graph.

[0036] A graph neural network is a neural network that can directly operate on a graph. Because a graph neural network can better adapt to an arbitrary size of a graph or a complex topology of a graph, a graph neural network is more suitable for operating on a graph than a traditional neural network (e.g., a convolutional neural network). A graph neural network can perform inference on data described in a format of a graph. A graph neural network can perform node-level, edge-level, or graph-level prediction tasks.

[0037] Graph neural network processing involves both graph neural network training and graph neural network inference, both of which involve graph neural network computation. A typical graph neural network computation for a node (or vertex) involves aggregating features of its neighbor nodes (direct neighbor nodes or neighbor nodes of each neighbor node) and then computing new activation data for the node to determine a feature representation (e.g., a feature vector) for the node. Thus, graph neural network processing for a small number of nodes often requires input features for a large number of nodes. Because the required nodes for input features can easily cover a large portion of a graph, especially for a real-world graph of a huge scale (e.g., with hundreds of millions of nodes and tens of billions of edges), the cost of using all neighbor nodes for message aggregation is too high.

[0038] ​To make graph neural networks more suitable for these real-world applications, node sampling is often used to reduce the number of nodes involved in message / feature aggregation. For example, positive and negative sampling can be used to determine the optimization objective and the resulting variance in graph neural network processing. For a given root node whose feature representation is being computed, positive sampling can sample graph nodes that are connected to the root node (directly or indirectly) via edges (e.g., connected to the root node and within a preset distance from the root node); negative sampling can sample graph nodes that are not connected to the root node via edges (e.g., beyond a preset distance from the root node). Positive and negative sampling nodes can be used to train feature representations of root nodes with different training objectives.

[0039] To execute a graph neural network, the system can retrieve graph data from memory and send the data to one or more processors for processing. Figure 2 This diagram illustrates an exemplary system using a graph neural network accelerator architecture according to an embodiment of the present disclosure. Figure 2 As shown. System 200 includes one or more processors 210, a graph neural network accelerator 220, memory 230, and one or more dedicated processors 240. In some embodiments, the one or more processors 210 include one or more central processing units (CPUs). In some embodiments, the one or more dedicated processors 240 may include one or more CPUs, one or more graphics processing units (GPUs), one or more tensor processing units (TPUs), one or more neural processing units (NPUs), one or more dedicated graph neural network processing units, etc. In some embodiments, memory 230 may include synchronous dynamic random access memory (SDRAM), such as double data rate (DDR) synchronous dynamic random access memory.

[0040] like Figure 2 As shown, the graph neural network accelerator 220 can receive instructions and information related to the graph neural network from one or more processors 210, and retrieve data related to the graph neural network from memory 230. After receiving data from memory 230, the graph neural network accelerator 220 can process the data and send the processed data to one or more dedicated processors 240 for further processing.

[0041] In some embodiments, such as Figure 2As shown, the graph neural network accelerator 220 can include a graph structure processor 221, a graph neural network sampler 222, a graph neural network attribute processor 223, and an address mapper 224. The graph structure processor 221 can be configured to receive instructions and information related to a graph neural network from the one or more processors 210 and fetch information of one or more root nodes and their edges from the memory 230. The graph structure processor 221 can then send the fetched information to the graph neural network sampler 222.

[0042] The graph neural network sampler 222 can be configured to select one or more sampled nodes for graph neural network processing according to the edge information of the one or more root nodes. In some embodiments, the graph neural network sampler 222 can select the one or more sampled nodes according to positive sampling or negative sampling. For example, based on positive sampling, the one or more sampled nodes can be selected from the nodes connected to (e.g., adjacent to) the one or more root nodes by edges. Based on negative sampling, the one or more sampled nodes can be selected from the nodes not directly connected to the one or more root nodes by edges (e.g., not adjacent to or close to the one or more root nodes). In some embodiments, positive sampling can be selected from the neighbor nodes of the root nodes, which are connected to the root nodes and within a preset distance of the root nodes. The connection can be a direct (one edge from a source node to a destination node) or indirect (multiple edges from a source node to a destination node) connection. The “preset distance” can be configured according to the implementation. For example, if the preset distance is 1, it means that only the directly connected neighbor nodes are selected for positive sampling. If the preset distance is infinity, it means that the nodes are not directly connected or indirectly connected. Negative sampling can be selected from the nodes outside the preset distance of the root nodes. It can be understood that any algorithm other than positive sampling and negative sampling can be used to select the sampled nodes.

[0043] After the sampled nodes are selected, the graph neural network sampler 222 can send the selection information of the sampled nodes to the graph neural network attribute processor 223. Based on the information of the sampled nodes, the graph neural network attribute processor 223 can be configured to fetch the information of the sampled nodes from the memory 230. In some embodiments, the information of the sampled nodes can include one or more features or attributes (also referred to as attribute data) of each sampled node. The graph neural network attribute processor 223 can also be configured to send the fetched information of the sampled nodes and the information of the one or more root nodes and their edges to the special-purpose processor 240. The special-purpose processor 240 can perform graph neural network processing based on the information received from the graph neural network attribute processor 223.

[0044] In some embodiments, the graph structure processor 221 and the graph neural network attribute processor 223 may use an address mapper 224 to retrieve information from memory 230. The address mapper 224 may be configured to provide hardware address information in memory 230 based on node and edge information. For example, the identifier n111 (e.g., ...) may be used. Figure 1 The graph structure processor 221 identifies the root node (node ​​n111) as part of the graph neural network input. If the graph structure processor 221 intends to obtain information about node n111 (e.g., attribute data of node n111), the graph structure processor 221 can provide the identifier n111 to the address mapper 224, and the address mapper 224 can determine the physical address where the information about node n111 (e.g., attribute data of node n111) is stored in memory 230. In some embodiments, the address mapper 224 can also determine the edges of node n111 (e.g., ... Figure 1 The information of edges e121 and e122 is stored in memory 230 at one or more physical addresses.

[0045] like Figure 2 As shown, graph neural network processing involves acquiring graph information, which may include unstructured data. This unstructured data may be stored discontinuously in memory 230. Furthermore, graph neural network processing may involve acquiring information about sampled nodes, and this information may also be stored discontinuously in memory 230. Therefore, when the graph neural network accelerator 220 accesses memory 230 to retrieve data for the graph neural network, data access can result in random access. For example, a graph may include nodes and edges that do not have a pre-defined structure, and the number of nodes and edges can be changed and updated on the graph. Therefore, in order to extract a specific dataset on the graph, such as nodes and their neighboring nodes and corresponding edges, the graph neural network accelerator 220 may need to access many different locations in memory 230 that are geographically dispersed.

[0046] Other problems may arise when memory 230 uses banks to store data. For example, memory 230 may be DDR, which stores data in banks. When memory stores information in rows, columns, or memory pages, rows, columns, and memory pages can be divided into multiple banks, with each bank storing multiple rows, columns, or memory pages. For various types of memory that store information in banks, bank conflicts may occur when the same bank is accessed consecutively at random addresses. For example, if different rows of the same bank are accessed consecutively, these two accesses may not be performed simultaneously. In fact, for many types of memory (e.g., DDR), there may be multiple latency cycles between each memory access to the same bank. For example, if the next memory access is on a different row in the same bank, that next memory access may need to wait 12 cycles to complete. Therefore, bank conflicts can cause significant delays in the system. It should be understood that bank conflicts are used as an example only. Different types of memory may also encounter problems with conflicts in other sub-units of memory. For example, memory may also experience latency for each memory access to the same memory page. Therefore, consecutive memory accesses to the same memory page can also lead to latency and conflicts.

[0047] To address the aforementioned problems, embodiments of this disclosure provide novel systems and methods for accessing memory for unstructured data. Figure 3 This diagram illustrates an exemplary improved system using a graph neural network accelerator architecture according to embodiments of the present disclosure. Figure 3 As shown, system 300 may include graph neural network accelerator 320 and memory 330 (e.g., similar to...). Figure 2 (Memory 230). In some embodiments, the system 300 may further include one or more processors (e.g., similar to memory 230). Figure 2 One or more processors 210) and one or more dedicated processors (e.g., similar to Figure 2 One or more dedicated processors 240). It is understood that, Figure 3 Only the hardware components related to attribute data acquisition and storage are shown, and depending on the implementation, system 300 may include more, fewer, or alternative components for implementing other appropriate functions.

[0048] In some embodiments, the graph neural network accelerator 320 may include a graph structure processor 321 (e.g., similar to...). Figure 2 Graph structure processor 221), graph neural network sampler 322 (e.g., similar to...) Figure 2 Graph neural network sampler 222), graph neural network attribute processor 323 (similar to Figure 2The graph neural network attribute processor 223 and address processor 325 are described. In some embodiments, address processor 325 may include candidate identifier generator 326 and address mapper 324 (e.g., similar to...). Figure 2 Address mapper 224).

[0049] In some embodiments, address processor 325 is configured to generate candidate node identifiers that facilitate memory (e.g., DDR) communication. Furthermore, address processor 325 may be configured to generate repository counters to track the number of accesses in each repository (e.g., the DDR repository). The information stored in the repository counters effectively helps reduce repository conflicts in memory 330.

[0050] In some embodiments, address processor 325 is configured to send generated candidate node identifiers or repository counters to graph neural network sampler 322. Graph neural network sampler 322 is configured to use information from the candidate node identifiers or repository counters to assist in performing node sampling, such as positive or negative sampling.

[0051] Figure 4 A schematic diagram of an exemplary address processor provided according to an embodiment of the present disclosure is shown. Figure 4 As shown, the address processor 425 can be similar to Figure 3 The address processor 425 is shown. In some embodiments, the address processor 425 may include a candidate identifier generator 426 (e.g., similar to...). Figure 3 The candidate identifier generator 326 is shown. Memory 430 can be similar to... Figure 3 The memory 330 shown. It should be understood that, Figure 4 Only the hardware components related to address processing are shown, and depending on the implementation, the address processor 425 may include more, fewer, or alternative components for implementing other appropriate functions.

[0052] In some embodiments, the address processor 425 may include an address mapper 424 (e.g., similar to...). Figure 3 The address mapper 324 and the address buffer 428 are similar. Figure 2 Address mapper 224, address mapper 424 can be configured to provide hardware address information in memory 430 based on node identifiers or edges. For example, as Figure 4 As shown, address mapper 424 can receive information about nodes (e.g., node identifiers), edges (e.g., edge pointers), and attributes (e.g., attribute pointers). Using a node with identifier n111 (e.g., ... Figure 1As an example, address mapper 424 can determine a physical address in memory 430 that stores information (e.g., attribute data) of node n111. In some embodiments, address mapper 224 can also determine one or more physical addresses in memory 430 that stores information of edges of node n111.

[0053] Address buffer 428 can be configured to store physical addresses to be sent to memory 430. It can be appreciated that address mapper 424, address buffer 428, or both can access memory 430.

[0054] Candidate identifier generator 426 can be configured to generate candidate node identifiers from the physical addresses. A candidate node identifier can be a node identifier that is sampled when processing a graph neural network. In some embodiments, if there is an address buffer (e.g., address buffer 428) in address processor 425, candidate identifier generator 426 can be configured to generate candidate node identifiers from the physical addresses stored in the address buffer. In some embodiments, the candidate node identifiers are generated based on one or more objectives. For example, one objective of generating candidate node identifiers is to increase sequential access within a storage bank (e.g., to increase sequential access within each storage bank). When increasing sequential access within a storage bank, access efficiency of the memory in the system can be improved. For example, when fetching information of a node from the memory, a single read to the memory can fetch an entire row of data containing the information of the node. Such a single read operation within a storage bank can provide the best memory access efficiency. As mentioned above, non-sequential reads within a storage bank can result in multiple read operations that have to be processed in series, which will cause latency and significantly impair the access performance of the memory. In addition, another objective of generating candidate node identifiers is to balance access between storage banks (e.g., to distribute access between storage banks in an even manner) to reduce storage bank conflicts, which can also improve the access efficiency of the memory.

[0055] In some embodiments, the candidate identifier generator 426 can include an address processing unit 441 and an address demapper 442. In some embodiments, to increase sequential access of the storage, the address processing unit 441 is configured to receive a physical address (e.g., from the address buffer 428) and generate a candidate physical address according to the received physical address. In some embodiments, the received physical address can include a physical address storing attribute data of nodes adjacent to one or more root nodes. In some embodiments, the purpose of generating the candidate physical address is to increase sequential access. For example, the address processing unit 441 can select candidate physical addresses that are continuous to each other or close to each other (e.g., from the same row or column within the storage). The physical addresses to be selected from can be obtained from a plurality of node identifiers adjacent to the root node identifier in the graph. Thus, the address processing unit 441 can provide candidate physical addresses storing data of candidate node identifiers, and the data of the candidate node identifiers are stored in a continuous manner. In some embodiments, some of the candidate physical addresses selected by the address processing unit 441 are continuous to the received physical address corresponding to the root node. In some embodiments, the continuous addresses can be obtained using fixed-size nodes in the memory 430. For example, when the nodes are stored in the memory 430, the nodes can have a fixed size of 128 bytes. Thus, from the received physical address, the address processing unit 441 can select candidate physical addresses in increments of 128 bytes, thereby increasing or maximizing sequential access. In some embodiments, the candidate physical addresses can include identifiers of rows or columns of the storage and the candidate physical addresses.

[0056] In some embodiments, to increase sequential access, the address processor 425 can be configured to generate candidate physical addresses based on the size of data obtained from the memory (e.g., the memory 430) in one read operation. For example, a single read operation of the memory 430 can obtain 1024 bytes of data. Assuming that the information of one node (e.g., attribute data of the node) occupies 128 bytes of the memory, the single read operation can obtain data of 8 nodes. Since all the nodes in the 8 nodes can be obtained together, each of the 8 nodes can become a candidate node, and the information of the 8 nodes can be used without taking additional read operations on the memory. Thus, the addresses of the candidate nodes can be determined by the address processing unit 441 as candidate physical addresses. If these multiple candidate nodes are subsequently selected to join a sampling node group for graph neural network processing, only one read operation can be taken to obtain the attributes of these multiple candidate nodes from the memory, thus improving the access efficiency of the memory.

[0057] The address demapper 442 can be configured to receive candidate physical addresses from the address processing unit 441. In some embodiments, the candidate physical addresses can include an identifier of a storage bank and an identifier of a row or column of the candidate physical address. The address demapper 442 can be configured to generate candidate node identifiers from the received candidate physical addresses. For example, the address demapper 442 can use a table to convert the candidate physical addresses to candidate node identifiers, information of which (e.g., attribute data of nodes corresponding to the candidate node identifiers) is stored in the candidate physical addresses. The address demapper 442 can be configured to send the candidate node identifiers for graph neural network sampling (e.g., Figure 3 of the graph neural network sampler 322).

[0058] In some embodiments, to reduce storage bank conflicts, the address processing unit 441 can be configured to generate storage bank counters. Each storage bank counter can refer to a variable corresponding to a storage bank and indicate a number of candidate nodes sampled from the storage bank. In some embodiments, each candidate physical address has a storage bank identifier indicating in which storage bank the candidate physical address is stored. Using the storage bank identifier, the address processing unit 441 can increment the storage bank counter corresponding to the storage bank identified by the storage bank identifier. In some embodiments, the address demapper 442 can be configured to receive the storage bank counters from the address processing unit 441 and send the storage bank counters for graph neural network sampling (e.g., Figure 3 of the graph neural network sampler 322). In some embodiments, the storage bank counters can be sent directly by the address processing unit 441 for graph neural network sampling. For example, the address demapper 442 can not be necessary to receive the storage bank counters so that the storage bank counters reach the graph neural network sampler.

[0059] Figure 5 A schematic diagram of an exemplary graph neural network sampler is shown according to embodiments of the present disclosure. The graph neural network sampler 522 can be similar to the graph neural network sampler 322 shown in Figure 3 In some embodiments, the graph neural network sampler 522 can include a positive sampler 545 and / or a negative sampler 546. In some embodiments, the graph neural network sampler 522 can include a candidate identifier buffer 543 or a storage bank balancing processor 544. It can be understood that, Figure 5 Only hardware components related to sampling in graph neural network processing are shown, and the graph neural network sampler 522 can include more, less, or alternative components for implementing other appropriate functions according to implementation.

[0060] In some embodiments, the candidate identifier buffer 543 can be configured to receive candidate node identifiers (e.g., from the address demapper 442) and send the candidate node identifiers for graph neural network sampling (e.g., Figure 3Address processor 325, Figure 4 Address processor 425 or Figure 4 (Address demapper 442). The candidate identifier buffer 543 can be configured to store the received candidate node identifiers and send the candidate node identifiers to the positive sampler 545 or the negative sampler 546.

[0061] In some embodiments, the repository balancing processor 544 can be configured to receive repository counters (e.g., Figure 3 Address processor 325, Figure 4 Address processor 425, Figure 4 Address demapper 442 or Figure 4 The address processing unit 441). The repository balancing processor 544 can be configured to update the value of the repository counter and track the repositories from which graph neural network sampling is enabled based on the repository counter. The repositories that can be used for sampling can be referred to as "selected repositories". In some embodiments, the value of the repository counter of the corresponding repository can be updated when sampling (implemented by positive sampler 545 or negative sampler 546) comes from candidate nodes of the corresponding repository. For example, after positive sampler 545 samples candidate nodes from the enabled repository, the repository counter of the enabled repository can be incremented by one to reflect the number of sampled nodes it provides. The aim is to keep the repository counters of the various repositories as close or balanced as possible. If a repository provides significantly more candidate nodes than other repositories, then that repository may be disabled for a period of time so that other repositories can catch up. Therefore, when a repository is removed from the selected repositories (e.g., marked as disabled by an invalidation flag), positive sampler 545 and negative sampler 546 may not sample nodes from that repository. In some embodiments, a repository can be marked as disabled when the difference between a repository counter (indicating the number of nodes sampled therein) and the minimum value of all repository counters is greater than a threshold. For example, this process may include: for multiple repositories including one repository, determining multiple count registers storing the repository counters; determining the minimum value from the multiple count registers; and determining whether the difference between the number of sampled node identifiers and the minimum value exceeds a selection threshold for that repository. This can reduce or avoid excessive access to a single repository.

[0062] In some embodiments, the repository counter can indicate the number of read operations on each repository. For example, if the positive sampler 545 samples candidate nodes from an enabled repository, but the candidate nodes do not cause additional read operations on memory (e.g., the attribute data of a newly selected candidate node can be obtained along with the attribute data of other selected candidate nodes in a single read operation), then the repository counter of the enabled repository may not increment because no additional read operations are required. The goal is to keep the repository counters of the repositories as close or balanced as possible. If one repository provides significantly more candidate nodes than other repositories, then that repository may be disabled for a period of time to allow other repositories to catch up.

[0063] In some embodiments, the positive sampler 545 is configured to receive candidate node identifiers and a selected repository (e.g., from a candidate identifier buffer 543 or a repository balancing processor 544). The positive sampler 545 may be configured to select one or more positive sampled node identifiers from the candidate node identifiers based on the selected repository. The one or more positive sampled node identifiers may be used with an address processor (e.g., ...). Figure 4 Address processor 425 or Figure 3 The root node identifiers received by the address processor 325 are adjacent or connected. In some embodiments, the positive sampler node identifiers may be within a "preset distance" from the root node. In some embodiments, one or more positive sampler node identifiers belong to a selected repository, and some candidate node identifiers that do not belong to the selected repository may not be selected by the positive sampler 545.

[0064] In some embodiments, negative sampler 546 is configured to receive candidate node identifiers and a selected repository (e.g., from candidate identifier buffer 543 or repository balancing processor 544). Negative sampler 545 may be configured to select one or more negative sample node identifiers from the candidate node identifiers based on the selected repository. The one or more negative sample node identifiers are then combined with an address processor (e.g., Figure 4 Address processor 425 or Figure 3 The root node identifiers received by the address processor 325 are not adjacent. In some embodiments, the negative sampled node identifiers may be outside a "preset distance" from the root node. In some embodiments, one or more negative sampled node identifiers belong to a selected repository, and some candidate node identifiers that do not belong to the selected repository may not be selected by the negative sampler 545.

[0065] In some embodiments, after selecting the positive sampling node identifiers, the positive sampler 545 can be configured to determine the number of positive sampling node identifiers in each repository. For example, if the positive sampler 545 selects 3 positive sampling node identifiers from repository 1 and 6 positive sampling node identifiers from repository 2, the positive sampler 545 can determine that the number of positive sampling node identifiers in repository 1 is 3 and the number of positive sampling node identifiers in repository 2 is 6. The positive sampler 545 can be configured to collect the number of positive sampling node identifiers in all repositories and send the collected numbers to the repository balancing processor 544.

[0066] In some embodiments, after selecting the negative sampling node identifiers, the negative sampler 546 can be configured to determine the number of negative sampling node identifiers in each repository. For example, if the negative sampler 546 selects 2 negative sampling node identifiers from repository 1 and 18 negative sampling node identifiers from repository 2, the negative sampler 546 can determine that the number of negative sampling node identifiers in repository 1 is 2 and the number of negative sampling node identifiers in repository 2 is 18. The negative sampler 546 can be configured to collect the number of negative sampling node identifiers in all repositories and send the collected numbers to the repository balancing processor 544.

[0067] In some embodiments, the repository balancing processor 544 can be further configured to receive the number of positive sampling node identifiers and the number of negative sampling node identifiers in each repository (e.g., from the positive sampler 545 and the negative sampler 546). The repository balancing processor 544 can be configured to generate or update the selected repositories according to the received repository counters, the number of positive sampling node identifiers and the number of negative sampling node identifiers in each repository. In some embodiments, the selected repositories are generated or selected according to the received repository counters and the number of positive sampling node identifiers and the number of negative sampling node identifiers in each repository to reduce or minimize repository conflicts.

[0068] One way to reduce or minimize repository collisions is to use a selection threshold for each repository. The selection threshold can be the maximum number of accesses in each repository. If the number of positive sampled node identifiers and the number of negative sampled node identifiers for a repository add up to more than the selection threshold for the repository, the repository balancing processor 544 can not select the repository as a selected repository. As such, the positive sampler 545 and the negative sampler 546 can not select positive sampled node identifiers and negative sampled node identifiers belonging to the repository. For example, for repository 1, the number of positive sampled node identifiers is 3 and the number of negative sampled node identifiers is 2. The total number of sampled node identifiers for repository 1 is 5. If the selection threshold for repository 1 is 6, the repository balancing processor 544 can still select repository 1 as a selected repository. As such, the positive sampler 545 and the negative sampler 546 can still select positive sampled node identifiers and negative sampled node identifiers belonging to repository 1. In another example, for repository 2, the number of positive sampled node identifiers is 6 and the number of negative sampled node identifiers is 18. The total number of sampled node identifiers (including positive sampled node identifiers and negative sampled node identifiers) for repository 2 is 24. If the selection threshold for repository 2 is 6, the repository balancing processor 544 can no longer select repository 2 as a selected repository. As such, the positive sampler 545 and the negative sampler 546 can not select positive sampled node identifiers and negative sampled node identifiers belonging to repository 2. In some embodiments, the selection threshold for each repository can be different from each other. In some embodiments, the selection threshold can be determined according to the minimum number of sampled node identifiers in the repositories.

[0069] In some embodiments, the repository balancing processor 544 is configured to update the selected repositories when the number of sampled node identifiers in any repository changes. For example, if the positive sampler 545 selects a positive sampled node identifier from repository 1, the positive sampler 545 can send the selection to repository 1 to the repository balancing processor 544. The repository balancing processor 544 can be configured to update a counter that tracks the number of sampled node identifiers for repository 1 and check the counter against the selection threshold for repository 1. If the counter exceeds the selection threshold for repository 1, the repository balancing processor 544 can remove repository 1 from the selected repositories so that the positive sampler 545 can not select a positive sampled node identifier from repository 1 for its next sampling selection.

[0070] Figure 6 An exemplary repository balancing processor is shown in accordance with an embodiment of the present disclosure. As shown, the repository balancing processor 644 can be similar to the repository balancing processor 544. Figure 6 Figure 5 ​The repository balancing processor 644 is shown. In some embodiments, the repository balancing processor 644 can track the number of sampled node identifiers in each of N repositories (i.e., repository 1 to repository N). It should be understood that... Figure 6 Only the hardware components related to graph neural network processing using the repository counter are shown, and depending on the implementation, the repository balancing processor 644 may include more, fewer, or alternative components for implementing other appropriate functions.

[0071] like Figure 6 As shown, the repository balancing processor 644 may include multiple repository count registers 620. Each of the multiple repository count registers 620 can track the number of sampled node identifiers in each repository. For example, as Figure 6 As shown, the Repository 1 count register 620 can track the number of sampled node identifiers in Repository 1. When sampling one or more node identifiers from Repository 1 (e.g., via positive sampler 545 or negative sampler 546), adder 1630 can be used to add the number of sampled node identifiers to the number stored in Repository 1 count register 620.

[0072] In some embodiments, such as Figure 6 As shown, the "minimum" module 640 compares the number of sampled node identifiers in all repositories to determine the minimum number. The comparison result can be sent from the minimum module 640 as a minimum counter to multiple comparison modules 650, each corresponding to one repository. For example, the number of sampled node identifiers in repositories 1 to N is between 8 and 34. Therefore, the minimum counter can be determined to be 8 by the minimum module 640.

[0073] In some embodiments, each comparison module 650 can be configured to determine whether a corresponding repository is a selected repository (e.g., a repository that can be sampled). The determination result can be output from the comparison module 650, which acts as a repository selector. In some embodiments, the determination result can be based on a selection threshold (e.g., similar to...). Figure 5 (The selection threshold used in the graph neural network sampler 522). In some embodiments, the determination result can be based on a minimum counter and a selection threshold. Taking repository 1 as an example, if the minimum counter is 8 and the number of sampling node identifiers in repository 1 is 15, then the comparison module 1650 can determine that the difference between the minimum counter and the number of sampling node identifiers is 7. If the selection threshold is 10, then because the determined difference is less than the selection threshold, the comparison module 1650 determines that repository 1 can be selected for sampling. If the difference between the minimum counter and the number of sampling node identifiers is determined to be greater than or equal to the threshold, then the corresponding repository can be left unsampled.

[0074] In some embodiments, Figure 3 , Figure 4 , Figure 5 and Figure 6 The systems, modules, and sub-units described herein can be implemented on field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). For example, Figure 3 The graph neural network accelerator 320 can be implemented and programmed on a field-programmable gate array.

[0075] This disclosure also provides methods for improving memory access in graph neural network processing. Figure 7 A flowchart illustrating an exemplary method for performing memory access based on a graph neural network, according to embodiments of the present disclosure, is shown. It will be understood that... Figure 7 In the method 700 shown, it can be derived from Figure 1 to Figure 6 The apparatus, equipment or system shown (e.g.) Figure 3 The method is executed by the graph neural network accelerator 320. Depending on the implementation, the method 700 may include additional, fewer, or alternative steps performed in various sequences or in parallel.

[0076] In step 710, a root node identifier corresponding to a node in the graph used for graph neural network processing is received. In some embodiments, multiple root node identifiers may be received. In some embodiments, the graph structure processor (e.g., graph structure processor 321) of the graph neural network processor (e.g., graph neural network accelerator 320) receives the root node identifier.

[0077] In step 720, one or more candidate node identifiers are determined based on the root node identifier. In some embodiments, information (e.g., attribute data) corresponding to the one or more candidate node identifiers is sequentially stored in memory. In some embodiments, the memory may be DDR (e.g., similar to...). Figure 3 memory 330 or Figure 4 (Memory 430).

[0078] In some embodiments, to determine one or more candidate node identifiers, one or more consecutive candidate physical addresses are determined. One or more candidate node identifiers can be determined based on one or more candidate physical addresses. For example, as... Figure 4As shown, the address processing unit 441 can generate candidate physical addresses by selecting consecutive or adjacent physical addresses. The physical addresses to be selected can be obtained from multiple adjacent node identifiers in the graph based on the root node identifier. After generating candidate physical addresses, the address demapping unit 442 can generate candidate node identifiers based on the candidate physical addresses received from the address processing unit 441. In some embodiments, some candidate physical addresses selected by the address processing unit 441 may be consecutive to the received physical addresses corresponding to the root node. In some embodiments, the candidate physical addresses may include identifiers of the repository and identifiers of the rows or columns of the candidate physical addresses.

[0079] In some embodiments, to increase sequential access, it can be based on reading from memory (e.g., in a single read operation) Figure 4 The size of the data obtained from the memory 430 in the memory is used to generate candidate physical addresses. For example, such as Figure 4 As shown, a single read operation on memory 430 can acquire 1024 bytes of data. Assuming that the information of one node (e.g., node attribute data) occupies 128 bytes of memory, a single read operation can acquire data from eight nodes. Since the data of all eight nodes can be acquired simultaneously, each of the eight nodes can become a candidate node, and the information of all eight nodes can be used without additional read operations on the memory. Therefore, the addresses of the candidate nodes can be determined as candidate physical addresses by the address processing unit 441. If multiple of these candidate nodes are subsequently selected to join the sampling node group for graph neural network processing, the attributes of these multiple candidate nodes can be acquired from memory in a single read operation, thus improving the efficiency of memory access.

[0080] Refer again Figure 7 In step 730, one or more candidate node identifiers are added to the candidate node identifier group. The candidate node identifier group can be used for sampling in graph neural network processing. For example, as... Figure 3 As shown, the graph neural network accelerator 320 can receive multiple root node identifiers. For each of the multiple root node identifiers, the address processor 325 can generate one or more candidate node identifiers, and one or more candidate node identifiers from each group can be added to the candidate node identifier group used by the graph neural network sampler 322. In some embodiments, the candidate node identifier group may include other node identifiers not generated or determined in step 720.

[0081] In step 740, one or more graph node identifiers are sampled from the candidate node identifier group for graph neural network processing. For example, a dedicated processor (e.g., Figure 2specialized processors 240) can perform graph neural network training or graph neural network processing using one or more graph node identifiers. Once one or more graph node identifiers are sampled, attribute data corresponding to the one or more graph node identifiers can be fetched from memory. For example, as shown in Figure 3

[0082] In some embodiments, a set of candidate node identifiers can be positively sampled. For example, as shown in Figure 5

[0083] In some embodiments, a set of candidate node identifiers can be negatively sampled. For example, as shown in Figure 5

[0084] Figure 8 An exemplary method of performing graph neural network based memory access using a repository counter is shown. It can be appreciated that the method 800 shown in Figure 8 Figure 1 to Figure 6 Figure 3 In some embodiments, the method 800 shown in Figure 8 Figure 7

[0085] ​​​​​​​In step 810, a root node identifier corresponding to a node in a graph for graph neural network processing is received. In some embodiments, similar to step 710 of FIG. 7 Figure 7 , a plurality of root node identifiers can be received. In some embodiments, a graph structure processor (e.g., graph structure processor 321) of a graph neural network processor (e.g., graph neural network accelerator 320) receives the root node identifiers.

[0086] In step 820, node sampling for the graph neural network is performed from a set of candidate node identifiers determined based on the root node identifier. In some embodiments, the set of candidate node identifiers can be sampled according to step 740 of FIG. 7 Figure 7 . For example, the set of identifiers can be sampled by performing positive sampling or negative sampling. In some embodiments, at least some of the set of candidate node identifiers can be determined from the root node identifier. For example, as shown in step 720 of FIG. 7 Figure 7 , one or more candidate node identifiers can be determined, where the attribute data corresponding to the one or more candidate node identifiers is stored sequentially in memory.

[0087] In step 830, a number of sampled node identifiers of a plurality of sampled nodes whose attribute data is stored in a storage bin of memory is determined. For example, as shown, the positive sampler 545 and the negative sampler 546 can determine the number of positive sampled node identifiers and the number of negative sampled node identifiers in each storage bin. The storage bin balancing processor 544 can receive the number of positive sampled node identifiers and the number of negative sampled node identifiers in each storage bin and generate the number of sampled node identifiers for each storage bin. Figure 5

[0088] In some embodiments, in step 830, a number of read operations on each storage bin is determined. For example, if the positive sampler 545 samples a candidate node from an enabled storage bin, but the candidate node does not cause an additional read operation on the memory (e.g., the attribute data of the newly selected candidate node can be fetched along with the attribute data of other selected candidate nodes in one read operation), then the storage bin counter of the enabled storage bin can not be incremented as an additional read operation is not needed. The goal is to get the storage bin counters of the individual storage bins as close or balanced as possible. If one storage bin is providing significantly more candidate nodes than the other storage bins, then this storage bin can be disabled for a period of time so that the other storage bins can catch up.

[0089] In some embodiments, a counter can track the number of sampled node identifiers in each storage bin. For example, as shown in FIG. 7 Figure 6 ​As shown, the count register 620 of the repository 1 can track the number of sampled node identifiers in the repository 1. When one or more node identifiers are sampled from the repository 1, the number of sampled node identifiers can be added to the number stored in the count register 620 of the repository 1 using the adder 1630. In some embodiments, a counter can track the number of read operations on each repository.

[0090] In step 840, a determination is made whether the number of sampled node identifiers of the sampled nodes whose attribute data is stored in the repository exceeds a selection threshold of the repository. For example, as shown in FIG. 8B, the selection threshold can be a maximum number of accesses in each repository, and the number of positive sampled node identifiers and the number of negative sampled node identifiers of the repository can be added to determine whether the sum exceeds the selection threshold of the repository. In another example, as shown in FIG. 8C, a minimum counter can be generated based on the number of sampled node identifiers in all repositories. The determination can be based on the minimum counter and the selection threshold. If it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the selection threshold, the corresponding repository can not be selected for sampling. It should be appreciated that different repositories can have different selection thresholds. In some embodiments, the number of read operations on a repository is compared to the selection threshold of the repository. Figure 5 Figure 6 As shown, the selection threshold can be a maximum number of accesses in each repository, and the number of positive sampled node identifiers and the number of negative sampled node identifiers of the repository can be added to determine whether the sum exceeds the selection threshold of the repository. In another example, as shown in FIG. 8C, a minimum counter can be generated based on the number of sampled node identifiers in all repositories. The determination can be based on the minimum counter and the selection threshold. If it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the selection threshold, the corresponding repository can not be selected for sampling. It should be appreciated that different repositories can have different selection thresholds. In some embodiments, the number of read operations on a repository is compared to the selection threshold of the repository.

[0091] In step 850, in response to determining that the number of sampled node identifiers in the repository exceeds the selection threshold of the repository, an invalid flag is set to avoid sampling more node identifiers from the corresponding repository of the memory than the selection threshold. For example, as shown in FIG. 8B, if the number of positive sampled node identifiers and the number of negative sampled node identifiers of the repository added together exceeds the selection threshold of the repository, the repository balancing processor 544 can not select the repository as a selected repository. As a result, the positive sampler 545 and the negative sampler 546 can not select the positive sampled node identifiers and the negative sampled node identifiers belonging to the repository. In another example, as shown in FIG. 8C, if it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the threshold, the corresponding repository can not be selected for sampling. In some embodiments, in response to the number of read operations on a repository exceeding the selection threshold of the repository, an invalid flag is set. Figure 5 Figure 6 As shown, the selection threshold can be a maximum number of accesses in each repository, and the number of positive sampled node identifiers and the number of negative sampled node identifiers of the repository can be added to determine whether the sum exceeds the selection threshold of the repository. In another example, as shown in FIG. 8C, a minimum counter can be generated based on the number of sampled node identifiers in all repositories. The determination can be based on the minimum counter and the selection threshold. If it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the selection threshold, the corresponding repository can not be selected for sampling. It should be appreciated that different repositories can have different selection thresholds. In some embodiments, the number of read operations on a repository is compared to the selection threshold of the repository.

[0092] ​​This disclosure provides systems and methods for improving memory access in graph neural network sampling. According to the system and method, candidate node identifiers can be determined based on the aim of increasing sequential access to memory (e.g., DDR) used for graph neural network sampling. For various types of memory that allow the extraction of multiple attribute data in a single read operation, it allows the reading of attribute data corresponding to multiple consecutive candidate node identifiers from memory in a single read operation, thus improving the overall memory access efficiency in graph neural network sampling. Furthermore, a repository counter can be used to track the amount of data to be accessed in each repository of memory. With the support of the repository counter, graph neural network sampling can be balanced to reduce or minimize repository conflicts on memory, thereby further improving memory access efficiency.

[0093] Figure 9 A block diagram of an exemplary graph neural network acceleration device with improved memory access provided according to embodiments of the present disclosure is shown. Figure 9 As shown, the graph neural network acceleration device 900 may include an acquisition module 910, a determination module 920, and a sampling module 930. According to embodiments, the graph neural network acceleration device 900 may include additional, fewer, or alternative components.

[0094] In some embodiments, the acquisition module 910 may be configured to acquire root node identifiers of nodes in a graph corresponding to graph neural network processing. In some embodiments, the acquisition module may be configured to acquire multiple root node identifiers. In some embodiments, the acquisition module 910 is configured to acquire root node identifiers via a graph structure processor (e.g., graph structure processor 321) of a graph neural network processor (e.g., graph neural network accelerator 320).

[0095] In some embodiments, the determining module 920 is configured to receive the acquired root node identifier from the acquiring module 910 and determine one or more candidate node identifiers based on the root node identifier. In some embodiments, information (e.g., attribute data) corresponding to the one or more candidate node identifiers is stored sequentially in memory. In some embodiments, the memory may be DDR (e.g., similar to...). Figure 3 memory 330 or Figure 4 (Memory 430).

[0096] In some embodiments, to determine one or more candidate node identifiers, the determining module 920 is further configured to determine one or more candidate physical addresses that are contiguous to each other. One or more candidate node identifiers can be determined based on one or more candidate physical addresses. For example, as... Figure 4As shown, the address processing unit 441 can generate candidate physical addresses by selecting physical addresses that are contiguous or close to each other. The physical addresses to be selected from can be obtained from a plurality of neighboring node identifiers in the graph according to the root node identifier. After generating the candidate physical addresses, the address demapper 442 can generate candidate node identifiers according to the candidate physical addresses received from the address processing unit 441. In some embodiments, some of the candidate physical addresses selected by the address processing unit 441 can be contiguous to the received physical address of the root node. In some embodiments, the candidate physical addresses can include an identifier of the storage repository and an identifier of a row or column of the candidate physical addresses.

[0097] In some embodiments, to increase sequential access, the determining module 920 is further configured to generate candidate physical addresses based on a size of data obtained from the memory (e.g., the memory 430) in one read operation. For example, as shown, a single read operation on the memory 430 can obtain 1024 bytes of data. Assuming that the information of one node (e.g., the attribute data of the node) occupies 128 bytes of the memory, the single read operation can obtain data of 8 nodes. Since the data of all nodes in the 8 nodes can be obtained together, each node in the 8 nodes can become a candidate node, and the information of the 8 nodes can be used without taking additional read operations on the memory. Thus, the addresses of the candidate nodes can be determined by the address processing unit 441 as the candidate physical addresses. If multiple of these candidate nodes are subsequently selected to join a sample node group for graph neural network processing, only one read operation can be taken to obtain the attributes of these multiple candidate nodes from the memory, thus improving the efficiency of memory access. Figure 4 Figure 4 In some embodiments, to increase sequential access, the determining module 920 is further configured to generate candidate physical addresses based on a size of data obtained from the memory (e.g., the memory 430) in one read operation. For example, as shown, a single read operation on the memory 430 can obtain 1024 bytes of data. Assuming that the information of one node (e.g., the attribute data of the node) occupies 128 bytes of the memory, the single read operation can obtain data of 8 nodes. Since the data of all nodes in the 8 nodes can be obtained together, each node in the 8 nodes can become a candidate node, and the information of the 8 nodes can be used without taking additional read operations on the memory. Thus, the addresses of the candidate nodes can be determined by the address processing unit 441 as the candidate physical addresses. If multiple of these candidate nodes are subsequently selected to join a sample node group for graph neural network processing, only one read operation can be taken to obtain the attributes of these multiple candidate nodes from the memory, thus improving the efficiency of memory access.

[0098] In some embodiments, the sampling module 930 is configured to receive one or more candidate node identifiers and sample one or more graph node identifiers from the one or more candidate node identifiers for graph neural network processing. For example, a special-purpose processor (e.g., the special-purpose processor 240 of Figure 2 ) can use the one or more graph node identifiers to perform graph neural network training or graph neural network processing. Once the one or more graph node identifiers are sampled, attribute data corresponding to the one or more graph node identifiers can be obtained from the memory. For example, as shown, the graph neural network attribute processor 323 can receive sample node identifiers from the graph neural network sampler 322 and obtain information of the sample nodes from the memory 330, the information of the sample nodes including one or more features or attributes of each sample node. Figure 3

[0099] ​​In some embodiments, the sampling module 930 can include a positive sampling module configured to perform positive sampling on the one or more candidate node identifiers. For example, as shown in Figure 5 FIG. 5, the positive sampler 545 can receive a set of candidate node identifiers and perform positive sampling on the set of candidate node identifiers to generate one or more positive sampled node identifiers. Based on the positive sampling, one or more positive sampled nodes corresponding to the one or more positive sampled node identifiers can be connected by edges and one or more root nodes corresponding to the one or more root node identifiers. For example, the one or more positive sampled nodes can be adjacent to the one or more root nodes. In some embodiments, the positive sampled node identifiers can be within a preset distance from the one or more root nodes.

[0100] In some embodiments, the sampling module 930 can include a negative sampling module configured to perform negative sampling on the one or more candidate node identifiers. For example, as shown in Figure 5 FIG. 6, the negative sampler 546 can receive a set of candidate node identifiers and perform negative sampling on the set of candidate node identifiers to generate one or more negative sampled node identifiers. Based on the negative sampling, one or more negative sampled nodes corresponding to the one or more negative sampled node identifiers can not be directly connected by edges and one or more root nodes corresponding to the one or more root node identifiers. For example, the one or more negative sampled nodes can not be adjacent to or close to the one or more root nodes. In some embodiments, the negative sampled node identifiers can be outside a preset distance from the one or more root nodes.

[0101] Figure 10 A block diagram of an exemplary graph neural network acceleration apparatus providing improved memory access using repository counters is shown, in accordance with an embodiment of the present disclosure. As shown in Figure 10 FIG. 10, the graph neural network acceleration apparatus 1000 includes an obtaining module 1010, a sampling module 1020, and a first determining module 1030, a second determining module 1040, and a setting module 1050. In some embodiments, the graph neural network acceleration apparatus 1000 can be implemented together with the graph neural network acceleration apparatus 900 shown in Figure 9 Depending on the implementation, the graph neural network acceleration apparatus 1000 can include additional, fewer, or alternative components.

[0102] In some embodiments, similar to the graph neural network acceleration apparatus 900 shown in Figure 9The illustrated obtaining module 910, the obtaining module 1010 can be configured to obtain a root node identifier corresponding to a node in a graph for graph neural network processing. In some embodiments, the obtaining module 1010 can be configured to obtain a plurality of root node identifiers. In some embodiments, the obtaining module 1010 is configured to obtain the root node identifier via a graph structure processor (e.g., the graph structure processor 321) of a graph neural network processor (e.g., the graph neural network accelerator 320).

[0103] In some embodiments, the sampling module 1020 is configured to perform node sampling on the graph neural network according to the group of candidate node identifiers determined based on the root node identifier. In some embodiments, at least some of the group of candidate node identifiers can be determined according to the root node identifier. For example, as illustrated in Figure 7 The illustrated step 740 samples the group of candidate node identifiers. For example, the group of identifiers can be sampled by performing positive sampling or negative sampling. In some embodiments, at least some of the group of candidate node identifiers can be determined according to the root node identifier. For example, as illustrated in Figure 7 As illustrated in step 720, one or more candidate node identifiers can be determined, where attribute data corresponding to the one or more candidate node identifiers is stored sequentially in the memory.

[0104] In some embodiments, the first determining module 1030 is configured to determine a number of sampled node identifiers whose attribute data is stored in a storage bin of the memory. For example, as illustrated in Figure 5 As illustrated, the positive sampler 545 and the negative sampler 546 can determine a number of positive sampled node identifiers and a number of negative sampled node identifiers in each bin. The bin balancing processor 544 can receive the number of positive sampled node identifiers and the number of negative sampled node identifiers in each bin and generate a number of sampled node identifiers for each bin.

[0105] In some embodiments, the first determining module 1030 is configured to determine a number of read operations on each bin. For example, as illustrated in Figure 5 As illustrated, if the positive sampler 545 samples a candidate node from an enabled bin, but the candidate node does not cause an additional read operation on the memory (e.g., attribute data of the newly selected candidate node can be obtained together with attribute data of other selected candidate nodes in one read operation), then the bin counter of the enabled bin can not be increased due to the lack of additional read operations. The goal is to have the bin counters of the individual bins as close or balanced as possible. If one bin provides significantly more candidate nodes than the other bins, then this bin can be disabled for a while so that the other bins catch up.

[0106] In some embodiments, a counter can track a number of sampled node identifiers in each bin. For example, as illustrated in Figure 6As shown, the count register 620 of the repository 1 can track the number of sampled node identifiers in the repository 1. When one or more node identifiers are sampled from the repository 1, the number of sampled node identifiers can be added to the number stored in the count register 620 of the repository 1 using the adder 1630. In some embodiments, a counter can track the number of read operations on each repository.

[0107] In some embodiments, the second determination module 1040 is configured to determine whether the number of sampled node identifiers whose attribute data is stored in a repository exceeds a selection threshold of the repository. For example, as shown in Figure 5 As shown, the selection threshold can be the maximum number of accesses in each repository, and the number of positive sampled node identifiers and the number of negative sampled node identifiers of a repository can be added to determine whether the sum exceeds the selection threshold of the repository. In another example, as shown in Figure 6 As shown, a minimum counter can be generated according to the number of sampled node identifiers in all repositories. The determination can be based on the minimum counter and the selection threshold. If it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the selection threshold, the corresponding repository can not be selected for sampling. It should be understood that different repositories can have different selection thresholds. In some embodiments, the number of read operations on a repository is compared with the selection threshold of the repository.

[0108] In some embodiments, the setting module 1050 is configured to set an invalid flag to avoid sampling more node identifiers from the repository of the memory than the selection threshold in response to determining that the number of sampled node identifiers in the repository exceeds the selection threshold of the repository. For example, as shown in Figure 5 As shown, if the number of positive sampled node identifiers and the number of negative sampled node identifiers of a repository add up to exceed the selection threshold of the repository, the repository balancing processor 544 can not select the repository as a selected repository. Therefore, the positive sampler 545 and the negative sampler 546 can not select the positive sampled node identifiers and the negative sampled node identifiers belonging to the repository. In another example, as shown in Figure 6 As shown, if it is determined that the difference between the minimum counter and the number of sampled node identifiers is greater than or equal to the threshold, the corresponding repository can not be selected for sampling. In some embodiments, in response to the number of read operations on a repository exceeding the selection threshold of the repository, an invalid flag is set.

[0109] Each of the processes, methods, and algorithms described in the preceding sections can be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms can be implemented partially or wholly in application-specific circuitry.

[0110] When the functions disclosed herein are implemented as software functional units and sold or used as independent products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product includes multiple instructions that may be stored in the storage medium to cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this disclosure. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, another medium operable for storing program code, or any combination thereof.

[0111] Specific embodiments also provide a system including a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to the steps in any of the methods of the above embodiments. Specific embodiments also provide a non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause one or more processors to perform operations corresponding to the steps in any of the methods of the above embodiments.

[0112] The embodiments disclosed herein can be implemented through a cloud platform, server, or group of servers (collectively referred to as the “server system”) that interacts with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.

[0113] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations fall within the scope of this disclosure. Furthermore, in some embodiments, certain method or process blocks may be omitted. The methods and processes described herein are not limited to any particular order, and associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than the specific disclosed order, or multiple blocks or states may be combined in a single block or state. Example blocks or states may be executed serially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or reset compared to the disclosed example embodiments.

[0114] The various operations of example methods described herein can be performed, at least partially, by an algorithm. The algorithm can be embodied in program code or instructions that are stored in memory (e.g., non-transitory computer-readable storage medium described above). Such an algorithm can include a machine learning algorithm. In some embodiments, a machine learning algorithm can not explicitly program a computer to perform a function, but can learn from training data to build a predictive model that performs the function.

[0115] The various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented engines that operate to perform one or more operations or functions described herein.

[0116] Similarly, the methods described herein can be at least partially processor- implemented, with a particular one or more processors being an example of hardware. For example, at least some of the operations of a method can be performed by one or more processors or processor-implemented engines. Moreover, the one or more processors can also operate to support performance of the relevant operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an application program interface (API)).

[0117] The performance of certain of the operations can be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented engines can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented engines can be distributed across a number of geographic locations.

[0118] In this disclosure, a plurality of instances can implement a component, operation, or structure described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations can be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations can be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component can be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0119] Although an overview of the subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of embodiments of this disclosure. These embodiments of this disclosure may be referred to individually or collectively by the term "this disclosure" merely for convenience and are not intended to automatically limit the scope of this disclosure to any single disclosure or concept, if in fact more than one disclosure or concept is disclosed.

[0120] The embodiments shown herein have been described in sufficient detail to enable those skilled in the art to practice the disclosed examples. Other embodiments may be used and derived therefrom, allowing for structural and logical substitutions and changes without departing from the scope of this disclosure. Therefore, the detailed description of the specific embodiments should not be construed as limiting, and the scope of the various embodiments is defined only by the appended claims and the full scope of their equivalents.

[0121] Any process description, element, or block described herein and / or in the flowcharts depicted in the accompanying drawings should be understood to potentially represent a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or step in the process. As will be understood by those skilled in the art, alternative implementations are included within the scope of the embodiments described herein, wherein elements or functions may be removed from those shown or discussed, performed out of order, including substantially simultaneously or in reverse order, depending on the functionality involved.

[0122] As used herein, “or” is inclusive, not exclusive, unless otherwise expressly indicated or indicated by the context. Therefore, here, “A, B, or C” means “A, B, A and B, A and C, B and C, or A, B, and C”, unless otherwise expressly indicated or indicated by the context. Furthermore, “and” is both consequential and individual, unless otherwise expressly indicated or indicated by the context. Therefore, here, “A and B” means “A and B, jointly or separately”, unless otherwise expressly indicated or indicated by the context. Furthermore, multiple instances may be provided for a resource, operation, or structure described herein as a single instance. Moreover, the boundaries between various resources, operations, engines, and data stores are arbitrary and specific operations are described within the context of a particular illustrative configuration. Other allocations of functionality are contemplated and may fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as separate resources in the example configuration may be implemented as combined structures or resources. Similarly, structures and functions presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of this disclosure as represented by the appended claims. Therefore, the specification and drawings are considered illustrative rather than restrictive.

[0123] The term "comprises" or "comprising" is used in the sense of "includes" or "including," but also in the sense of "consists of" or "consisting of," unless the context clearly indicates otherwise. The term "comprises" or "comprising" is used herein to indicate the presence of stated features, integers, steps, or components, but not to the exclusion of other features, integers, steps, or components. The term "comprises" or "comprising" is used in the sense of "includes" or "including," but also in the sense of "consists of" or "consisting of," unless the context clearly indicates otherwise. The term "comprises" or "comprising" is used herein to indicate the presence of stated features, integers, steps, or components, but not to the exclusion of other features, integers, steps, or components.

Claims

1. A method for accelerating attribute access in graph neural networks, comprising: Receive the root node identifier corresponding to the node in the graph used for graph neural network processing; One or more candidate node identifiers are determined based on the root node identifier, wherein the attribute data corresponding to the one or more candidate node identifiers are stored sequentially in the memory; At least one or more graph node identifiers are sampled from the one or more candidate node identifiers for graph neural network processing; Determining one or more candidate node identifiers based on the root node identifier includes: The identifiers of multiple adjacent nodes in the graph are determined based on the root node identifier; Obtain the physical addresses of the multiple adjacent node identifiers in the memory; One or more candidate physical addresses are identified based on the physical address, and the one or more candidate physical addresses are contiguous in the memory; The one or more candidate node identifiers are determined based on the one or more candidate physical addresses.

2. The method according to claim 1, wherein, Identifying one or more candidate physical addresses based on the physical address includes: Identify one or more candidate physical addresses, wherein, in a single read operation of the memory, attribute data corresponding to the one or more candidate physical addresses is obtained from the one or more candidate physical addresses.

3. The method according to claim 1, wherein, The method further includes: The number of sampling node identifiers is determined, and the attribute data of the sampling node identifiers is stored in the repository of the memory; Determine whether the number of sampling node identifiers exceeds the selection threshold of the repository; In response to the number of sampled node identifiers in the repository exceeding the selection threshold of the repository, an invalidation flag is set to avoid sampling more node identifiers than the selection threshold from the repository of the memory.

4. The method according to claim 3, wherein, The process of determining the number of sampling node identifiers, wherein the attribute data of the sampling node identifiers is stored in the repository of the memory, includes: The number of sampling node identifiers is determined by reading the counter register of the repository, and the attribute data of the sampling node identifiers is stored in the repository; The method further includes: In response to the node identifier of the sampled repository, update the counter register of the repository; Specifically, determining whether the number of sampling node identifiers exceeds the selection threshold of the repository, wherein the attribute data of the sampling node identifiers stored in the repository includes: For a plurality of repositories including the repositories, a plurality of counting registers are determined, each of the plurality of counting registers corresponding to one of the plurality of repositories; Determine the minimum value from the plurality of counter registers; and Determine whether the difference between the number of sampling node identifiers and the minimum value exceeds the selection threshold of the repository.

5. The method according to claim 1, wherein, Sampling one or more graph node identifiers includes: Sample one or more positive node identifiers, wherein the one or more positive node identifiers are within a preset distance from the root node; and Sample one or more negative node identifiers, which are located at a preset distance from the root node.

6. The method according to claim 1, wherein, The memory includes a double data rate synchronous dynamic random access memory.

7. A system for accelerating attribute access in graph neural networks, comprising: The memory is configured to store the attribute data of the nodes in the graph; as well as The graph neural network accelerator is configured as follows: Receive the root node identifier corresponding to the node in the graph used for graph neural network processing; One or more candidate node identifiers are determined based on the root node identifier, wherein the attribute data of the one or more candidate node identifiers are stored sequentially in memory; At least one or more graph node identifiers are sampled from the one or more candidate node identifiers for graph neural network processing; The graph neural network accelerator includes an address processor, which is configured to: The identifiers of multiple adjacent nodes in the graph are determined based on the root node identifier; Obtain the physical addresses of the multiple adjacent node identifiers in the memory; One or more candidate physical addresses are identified based on the physical address, and the one or more candidate physical addresses are contiguous in the memory; The one or more candidate node identifiers are determined based on the one or more candidate physical addresses; Determine one or more candidate physical addresses, wherein, in a single read operation of the memory, attribute data corresponding to the one or more candidate physical addresses is obtained from the one or more candidate physical addresses.

8. The system according to claim 7, wherein, The graph neural network accelerator also includes a graph neural network sampler, which is configured to: The number of sampling node identifiers is determined, and the attribute data of the sampling node identifiers is stored in a repository of the memory; Determine whether the number of sampling node identifiers exceeds the selection threshold of the repository; In response to the number of sampled node identifiers in the repository exceeding the selection threshold of the repository, an invalidation flag is set to avoid sampling more node identifiers than the selection threshold from the repository of the memory.

9. The system according to claim 8, wherein, The graph neural network sampler is also configured to: The number of sampling node identifiers is determined by reading the counter register of the repository, and the attribute data of the sampling node identifiers is stored in the repository; In response to the node identifier of the sampled repository, update the counter register of the repository.

10. The system according to claim 8, wherein, The graph neural network sampler is also configured to: For a plurality of repositories including the repositories, a plurality of counting registers are determined, each of the plurality of counting registers corresponding to one of the plurality of repositories; Determine the minimum value from the plurality of counter registers; as well as Determine whether the difference between the number of sampling node identifiers and the minimum value exceeds the selection threshold of the repository.

11. The system according to claim 8, wherein, The graph neural network sampler includes: A positive node sampler is configured to sample one or more positive node identifiers, said one or more positive node identifiers being within a preset distance from the root node; and A negative node sampler is configured to sample one or more negative node identifiers located at a preset distance from the root node.

12. A non-transitory computer-readable storage medium comprising instructions executable by one or more processors, wherein execution of the instructions causes the one or more processors to perform the method of any one of claims 1 to 6.

Images