An accelerator card data access method and accelerator card

Abstract

The application discloses an acceleration card data access method and an acceleration card, and relates to the technical field of artificial intelligence. Since an access interface between the acceleration card and a first file system is established, the interface is used for managing the mapping of page identification to physical address and pre-storing node heat data, so that the acceleration card can read the heat data through the interface and determine a high-heat node, and then load the high-heat node features from a storage device to a cache according to the first mapping and make the high-heat node features resident, and load target node features when a graph neural network is calculated. Thus, the acceleration card can autonomously complete node heat identification, high-heat node residency and feature data acquisition without the participation of a CPU, the number of interactions between the acceleration card and the CPU is reduced, the problem of low efficiency caused by the participation of the CPU is solved, and the data access efficiency and overall execution performance are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an accelerator card data access method and an accelerator card. Background Technology

[0002] With the widespread application of Graph Neural Networks (GNNs) in social networks, recommender systems, knowledge graphs, and other fields, the scale of graph data is exploding. GNN computation requires frequent access to large-scale node feature data. In related technologies, when using accelerator cards to accelerate GNN computation, the host-side Central Processing Unit (CPU) is required for scheduling, making it difficult to fully utilize the high parallel computing capabilities of the accelerator card, thus becoming a bottleneck for improving the computational efficiency of GNNs. Summary of the Invention

[0003] This invention provides a data access method for an accelerator card and an accelerator card, so as to at least solve the problem in the related art that the host-side central processing unit needs to participate in the scheduling when using an accelerator card to accelerate graph neural network calculations, resulting in low computing efficiency.

[0004] This invention provides a data access method for an accelerator card, applied to an accelerator card used to perform graph neural network calculations. The data access method includes: An access interface is established between the accelerator card and the first file system. The first file system is used to manage the correspondence between page identifiers and physical addresses of storage devices, and the first file system is pre-written with node heat data of the graph neural network. Access the first file system through the access interface, read the node popularity data, and determine high-popularity nodes based on the node popularity data; Obtain the first mapping between node identifiers and page addresses in the graph neural network; The first file system is accessed through the access interface, and the physical address of the graph neural network node is queried from the first file system according to the first mapping, so as to load the feature data of the high-hot node from the storage device to the accelerator card cache and keep it resident, and load the feature data of the target node from the storage device to the accelerator card cache when performing graph neural network calculation.

[0005] The present invention also provides an accelerator card, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described accelerator card data access methods.

[0006] This invention establishes an access interface between the accelerator card and a first file system. This first file system manages the mapping between page identifiers and the physical addresses of storage devices and pre-writes node popularity data for the graph neural network. The accelerator card can then access the first file system through this interface, read and determine high-hot nodes based on the node popularity data. Subsequently, based on the first mapping between the node identifiers and page addresses of the graph neural network, the accelerator card can access the first file system through this interface and query the physical addresses of the graph neural network nodes from the first file system according to the first mapping. This allows the feature data of the high-hot nodes to be loaded from the storage device into the accelerator card cache and persisted there. Furthermore, during graph neural network computation, the feature data of the target node is loaded from the storage device into the accelerator card cache. Therefore, the accelerator card can autonomously complete node popularity identification, high-hot node cache persistence, and target node feature data acquisition without CPU intervention. This reduces the number of interactions between the accelerator card and the CPU during graph neural network computation, solving the problem of low efficiency caused by CPU intervention in related technologies. This improves the data access efficiency and overall execution performance of the accelerator card during graph neural network computation. Attached Figure Description

[0007] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 An architecture diagram of an accelerator card data access system provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a method for accessing accelerator card data provided in an embodiment of the present invention; Figure 3 A first mapping schematic diagram provided for an embodiment of the present invention; Figure 4 This is a schematic diagram of a graph neural network access perception module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of an accelerator card caching and mapping management module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of an accelerator card cache direct-connect storage access module provided in an embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0010] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0011] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0012] Here, we will first explain some key terms used in the embodiments of the present invention.

[0013] Graph Neural Network (GNN): A neural network model capable of deep learning on graph-structured data. It achieves tasks such as node classification, link prediction, and graph classification by aggregating and transmitting neighbor information of nodes in the graph. In this embodiment of the invention, the computation process of GNN involves frequent access to node feature data and graph topology data.

[0014] Node: The basic building block of a graph. In the computation of a GNN, each node corresponds to a feature vector, and the connections between nodes are described by edges. Nodes typically have a unique node ID, used to distinguish and locate different nodes.

[0015] Node Feature Data: Attribute feature vectors associated with nodes, used as input and for computation in GNN models. The amount of node feature data typically grows linearly with the number of nodes and the feature dimension, and it is the main data object that needs to be frequently accessed during GNN computation.

[0016] Graph Structure Data: Data describing the connections between nodes (such as adjacency lists and adjacency matrices), used in the neighbor aggregation process of GNN computation. Compared to node feature data, topology structure data is smaller in size and accessed more frequently. In this embodiment of the invention, it can be pre-stored on an accelerator card to support independent topology-related calculations on the accelerator card side.

[0017] Page: The basic unit of data management in a storage device. In this embodiment of the invention, node feature data is organized and stored at the page level, and the accelerator card locates and accesses the data through page-level address mapping of the first file system.

[0018] The First File System (FLS) is a file system deployed on the Central Processing Unit (CPU) side. It has an interface directly accessible by the accelerator card and can manage the mapping relationship between page identifiers and storage device physical addresses. This file system organizes storage node characteristic data in units of pages, providing the accelerator card with CPU-independent data access capabilities.

[0019] Accelerator Card Cache: High-speed storage resources inside the accelerator card (such as the video memory of the graphics processing unit (GPU)) are used to temporarily store node feature data that needs to be accessed frequently during graph neural network computation, in order to reduce storage access latency.

[0020] Page Group: An input / output (I / O) operation unit consisting of multiple data pages that need to be swapped in or out simultaneously. By performing I / O operations on a page group basis, the number of small-granularity I / O requests can be reduced, improving storage bandwidth utilization.

[0021] During GNN computation, the accelerator card needs to frequently access large-scale node feature data. The amount of this node feature data often far exceeds the storage capacity of the accelerator card cache, so dynamic scheduling and migration of data between the accelerator card cache and storage devices are required.

[0022] In related technologies, accelerator cards typically rely on the CPU to access data in storage devices. Specifically, the CPU is responsible for loading node characteristic data from storage devices or host memory, and then copying the data to the accelerator card cache via a high-speed serial computer expansion bus (Peripheral Component Interconnect Express, PCIe) for computation. Under this architecture, the accelerator card lacks the ability to independently access storage devices and cannot perceive node access characteristics, thus hindering its ability to make autonomous cache management decisions. This deficiency is particularly prominent in GNN computing scenarios, mainly due to the following reasons: First, there are significant differences in the access frequency of different nodes during GNN computing. Some nodes (such as high-influence nodes in social networks) are repeatedly accessed in multiple batches, while other nodes are accessed less frequently. However, existing accelerator cards cannot perceive this difference, resulting in cache management strategies being unable to distinguish between hot and non-hot data. Second, the lack of a cache management mechanism oriented towards GNN access characteristics leads to the repeated loading and eviction of frequently accessed node data between different batches (i.e., cache thrashing), resulting in a waste of storage bandwidth. Third, the data access path requires continuous CPU participation. When the parallelism of the accelerator card is high, the CPU can easily become the system performance bottleneck, restricting the overall throughput of GNN computing.

[0023] To address the issue of low computational efficiency caused by the need for host-side central processing unit (CPU) intervention in scheduling when using accelerator cards for graph neural network (GNN) computation acceleration, this invention provides an accelerator card data access method and an accelerator card. An access interface is established between the accelerator card and a first file system. This first file system manages the mapping between page identifiers and physical addresses on storage devices and pre-writes node popularity data for the GNN. The accelerator card can access the first file system through this interface, read and determine high-hot nodes based on the node popularity data. Then, based on a first mapping between GNN node identifiers and page addresses, the first file system can be accessed through this interface. The physical addresses of GNN nodes are then queried from the first file system according to the first mapping. The feature data of high-hot nodes is loaded from the storage device into the accelerator card cache and kept resident. During GNN computation, the feature data of the target node is loaded from the storage device into the accelerator card cache. Therefore, the accelerator card can autonomously complete node heat identification, high-hot node caching and persistent caching, and target node feature data acquisition without CPU intervention. This reduces the number of interactions between the accelerator card and the CPU in graph neural network computation, solves the problem of low efficiency caused by the need for CPU intervention when the accelerator card performs graph neural network computation in related technologies, and improves the data access efficiency and overall execution performance of the accelerator card when performing graph neural network computation.

[0024] Figure 1This is an architecture diagram of an accelerator card data access system provided in an embodiment of the present invention.

[0025] The specific application environment architecture or specific hardware architecture on which the execution of the accelerator card data access method depends is described here.

[0026] like Figure 1 As shown, the system architecture on which the accelerator card data access method provided in this embodiment of the invention is based may include: a host, an accelerator card, and a storage device.

[0027] Among them, the accelerator card can be a graphics processing unit (GPU).

[0028] Accelerator cards can connect to the host via the PCIe bus to perform training or inference computation tasks for graph neural networks.

[0029] The accelerator card contains an internal cache (such as GPU memory) for temporary storage of node feature data that is frequently accessed during computation. Storage devices (such as Non-Volatile Memory Express (NVMe) solid-state drives) are used for persistent storage of large-scale node feature data.

[0030] The host side has a first file system deployed to manage the storage layout of data in the storage device.

[0031] The accelerator card side includes an application layer, a graph neural network access awareness module, an accelerator card cache and mapping management module, and an accelerator card cache direct-connect storage access module. Specifically, the application layer executes graph neural network computation tasks; the graph neural network access awareness module is responsible for data preprocessing and node popularity collection; the accelerator card cache and mapping management module is responsible for accelerator card cache management and page mapping management; and the accelerator card cache direct-connect storage access module is responsible for popularity metadata management and page group migration, interacting with the storage device through a direct-connect storage interface.

[0032] exist Figure 1 In the hardware architecture shown, when the accelerator card needs to access node feature data during graph neural network computation, it first checks whether the required data is already cached in its cache. If it is cached (cache hit), the data is read directly from the cache. If it is not cached (cache miss), the accelerator card initiates an I / O request to the storage device through the direct-access storage interface provided by the first file system, using the page identifier as an index. After the storage device returns the corresponding data page, the accelerator card loads the data page into its cache for computation. Throughout this process, the CPU does not participate in data location, I / O scheduling, or data transfer operations; these are all completed independently by the accelerator card.

[0033] The embodiments of the present invention provide a method for accessing data on an accelerator card. The method is described in detail below in conjunction with the execution flow of the accelerator card data access method.

[0034] Figure 2 This is a flowchart of an accelerator card data access method provided in an embodiment of the present invention.

[0035] like Figure 2 As shown, when applied to an accelerator card, the accelerator card data access method provided in this embodiment of the invention may include the following steps S201~S204.

[0036] In this embodiment of the invention, graph neural network computation includes graph neural network iterative training and graph neural network inference computation. For graph neural network iterative training, stage computation can refer to batch computation; for graph neural network inference computation, stage computation can refer to dividing the graph neural network inference task into multiple computation stages.

[0037] S201: Establish an access interface between the accelerator card and the first file system. The first file system is used to manage the correspondence between page identifiers and physical addresses of storage devices, and the first file system is pre-written with node heat data of the graph neural network.

[0038] Specifically, before performing graph neural network computation, a communication channel and data access interface need to be established between the accelerator card and the first file system. The first file system provides the accelerator card with file-level abstract management capabilities, used to manage the storage layout of node feature data in the storage device at the page level, and to maintain the correspondence between data page identifiers and storage device physical addresses. Through this access interface, the accelerator card can directly initiate page-level I / O requests to the storage device without going through CPU I / O scheduling and control. In other words, this access interface enables the accelerator card to independently access node feature data in the storage device, laying the foundation for the accelerator card to autonomously perform data reading and cache management. This step makes the data access path between the accelerator card and the storage device no longer dependent on the CPU, solving the technical problem in related technologies where accelerator card data access is limited by CPU participation.

[0039] In addition, before performing graph neural network computation, the access characteristics of all nodes in the graph neural network are analyzed to determine node popularity data, which is then written to the first file system. Specifically, the task of determining node popularity data can be performed by the CPU or an accelerator card. Node popularity data is used to characterize the likelihood of each node being repeatedly accessed during the graph neural network computation process.

[0040] In some optional embodiments of the present invention, the step of determining node heat data may include: using one or more of the following methods to calculate the access heat of nodes in the graph neural network, thereby obtaining node heat data; the node heat data is used to characterize the probability that a node is repeatedly accessed during the graph neural network calculation process.

[0041] Among these methods, statistical analysis based on node sampling results during training refers to statistically analyzing the sampling frequency of nodes in each batch during training, using the sampling frequency as a measure of node popularity. Analysis methods based on graph structure features estimate the frequency at which a node might be visited using graph structure information (such as node degree and number of neighbors). Methods for calculating node importance or centrality based on traditional graph algorithms calculate a node's importance score in the graph using algorithms such as PageRank, Betweenness Centrality, and Degree Centrality, using this score as a measure of node popularity. Node popularity estimation methods based on dynamic information from the training process combine the node's historical visit count, visit frequency, or dynamic trends during training to calculate node popularity. These multiple methods allow for flexible adaptation to different graph structure features and the needs of different graph neural network computation tasks, making node popularity assessment more accurate and comprehensive.

[0042] In this embodiment of the invention, node popularity data can be calculated all at once before the graph neural network computation begins, and the node popularity data remains unchanged during the computation process. The advantage of this design is that it avoids introducing additional popularity calculation and data synchronization overhead during the computation process, ensuring that the accelerator card can obtain popularity information by performing only read-only queries in the data access path, without the need for frequent popularity data synchronization with the CPU.

[0043] S202: Access the first file system through the access interface, read node popularity data, and determine high-hot nodes based on the node popularity data.

[0044] When the accelerator card performs graph neural network calculations, it first reads the node popularity data from the first file system, and then identifies nodes with high access frequency and those that may be repeatedly accessed in multiple calculation stages (such as nodes that are repeatedly accessed across batches) based on the node popularity data, and determines them as high-hot nodes.

[0045] The characteristic data of high-hot nodes will be prioritized for resident storage in the accelerator card cache to reduce the repeated loading of data from the same node during subsequent computations. This step enables the accelerator card to autonomously perceive the access characteristics of nodes and formulate cache management strategies accordingly, solving the technical problems in related technologies where the accelerator card cannot obtain node access characteristic information and cache management lacks specificity.

[0046] S203: Obtain the first mapping between node identifiers and page addresses in the graph neural network.

[0047] In this embodiment of the invention, before the graph neural network computation begins, the accelerator card obtains the mapping relationship (i.e., the first mapping) between node identifiers and page addresses.

[0048] The first mapping can be used to determine the corresponding page identifier and the offset within the page based on the node identifier; the first mapping remains unchanged during the execution of graph neural network calculations.

[0049] In other words, the first mapping is used to map the node identifier of each node to its corresponding data page identifier and offset within the page in the storage device. The node identifier is used to uniquely identify a node in the graph, and the page address is used to indicate the storage location of the node's feature data. The first mapping remains unchanged during the graph neural network computation, providing a unified and low-overhead basic data location method for all nodes. Through this step, the accelerator card can directly calculate the storage location of the node's feature data using the node identifier when performing graph neural network computation, without the need for CPU lookups and transformations, further reducing the accelerator card's dependence on the CPU.

[0050] In some optional implementations, the acquisition of the first mapping may include: storing node feature data and node identifiers separately; writing the node feature data to a first file system at the page level; recording the page identifier and page offset of each node's feature data during the writing process to form a correspondence table between node identifiers and page addresses. When the graph neural network computation begins, the accelerator card loads this correspondence table to obtain the first mapping.

[0051] In other words, in this embodiment of the invention, the first mapping is a fixed mapping relationship from node identifier to storage location. Based on the first mapping, given a node identifier, the data page identifier (i.e., which page the data belongs to) and the offset within the page (i.e., the specific starting position within the page) of the node's feature data can be calculated through the first mapping. The first mapping is established once before the graph neural network computation begins and remains fixed throughout the entire graph neural network computation process, without changing with batch variations. The first mapping serves all nodes (including high-hot nodes, dynamically cached nodes, and ordinary nodes), providing them with a unified basic data location method. During subsequent node feature access, when a node is not covered by the temporary mapping table (described in the following embodiments), the accelerator card will fall back to the first mapping to complete the location of the node to the page. By designing the first mapping as a structure that is fixed throughout the computation, the location computation has deterministic and low-overhead characteristics, while providing a stable fallback basis for subsequent dynamic mapping.

[0052] Figure 3 This is a first mapping diagram provided for an embodiment of the present invention.

[0053] like Figure 3 As shown, the first mapping maps node identifiers to specific pages and offsets within the storage device. For example, each page is 4KB in size, each node's feature data is 512B in size, and each page can store feature data for 8 nodes. The node identifier is used to determine its page identifier through integer division and its offset within the page through modulo operation.

[0054] It should be noted that S201 and S203 are two independent preparation steps in the embodiments of the present invention, and their execution order can be interchanged. That is, the first mapping can be obtained first and then the access interface can be established, or the access interface can be established first and then the first mapping can be obtained, or they can be performed simultaneously. The embodiments of the present invention do not limit this.

[0055] The accelerator card data access method provided in this embodiment of the invention may further include: extracting the graph structure information of the target graph neural network for graph neural network computation and storing it in the accelerator card before performing graph neural network computation.

[0056] Specifically, before performing graph neural network computation, the topological information (i.e., graph structure information) of the target graph structure is extracted and stored in the accelerator card. By storing the graph structure information directly on the accelerator card, topology-related computations (such as neighbor node aggregation) can be completed directly on the accelerator card, eliminating the need to repeatedly load graph structure information from the CPU or storage devices, thus avoiding the overhead of graph structure loading. This step, combined with the step of storing node feature data at the page level in the first file system, achieves separate management of topology structure and feature data. The former resides on the accelerator card to support topology computation, while the latter is stored at the page level in the first file system for the accelerator card to access on demand. This separate management method is a crucial foundation for the accelerator card to independently perform data location and cache management.

[0057] Figure 4 This is a schematic diagram of a graph neural network access perception module provided in an embodiment of the present invention.

[0058] like Figure 4 As shown, the graph neural network access perception module receives the unprocessed graph neural network stage dataset, performs sampling operations, and then processes the sampling results. This module includes a data preprocessing unit and a node heat collection unit. The data preprocessing unit extracts graph structure information and stores it in the accelerator card; the node heat collection unit calculates the heat value of each node and determines the static cache nodes. The node identifier and node feature data corresponding to the static cache nodes are loaded into the static cache area in the accelerator card cache.

[0059] S204: Access the first file system through the access interface, query the physical address of the graph neural network node from the first file system according to the first mapping, load the feature data of the high-hot node from the storage device to the accelerator card cache and keep it resident, and load the feature data of the target node from the storage device to the accelerator card cache when performing graph neural network calculation.

[0060] Specifically, after identifying high-hot nodes, the accelerator card determines the storage page containing the feature data of these nodes based on the first mapping (i.e., the correspondence between node identifiers and page addresses). Then, it reads the data from these pages through the direct-access storage interface provided by the first file system and loads the feature data of the high-hot nodes from the storage device into the accelerator card cache, making it resident. "Resident" means that the feature data of these high-hot nodes remains in the accelerator card cache throughout the graph neural network computation process and will not be evicted or replaced by subsequent data loading.

[0061] After preloading the high-hot nodes, the accelerator card begins to execute graph neural network computations. During the computation, for the feature data of the target nodes that need to be accessed in this stage of computation, the accelerator card also locates the page address where it is located according to the first mapping, reads the corresponding data page through the first file system, and loads the feature data of the target nodes from the storage device into the accelerator card cache for computation.

[0062] Through the above steps S201 to S204, this embodiment of the invention achieves autonomous access and cache management of node feature data on the accelerator card side. The accelerator card no longer relies on CPU I / O scheduling and instruction triggering, but directly accesses the storage device through the first file system and autonomously makes cache management decisions based on node popularity data. By keeping the feature data of high-hot nodes resident in the cache, these nodes are avoided from being repeatedly loaded and evicted between different computing batches, thereby significantly reducing redundant I / O overhead and cache thrashing.

[0063] Based on the above embodiments, in this embodiment of the invention, the accelerator card cache can be divided into multiple cache partitions, which are used to store the aforementioned resident high-hot node data and the target node data required for each stage of calculation.

[0064] In this embodiment of the invention, loading the feature data of the high-hot node from the storage device to the accelerator card cache and keeping it resident in S204 may include: loading the feature data of the high-hot node from the storage device to the first cache partition of the accelerator card cache and keeping it resident in S204.

[0065] In S204, loading the feature data of the target node from the storage device to the accelerator card cache during graph neural network computation may include: loading the feature data of the target node from the storage device to the second cache partition of the accelerator card cache during graph neural network computation.

[0066] Specifically, to avoid contention and eviction issues caused by different types of data competing for cache space in the accelerator card, this embodiment of the invention divides the accelerator card cache into two logical partitions: a first cache partition (i.e., a static cache area) and a second cache partition (i.e., a dynamic cache area). The first cache partition is specifically used to store the feature data of high-hot nodes. This data resides in this partition after loading and will not be evicted during the entire graph neural network computation process. The second cache partition stores the feature data of the target nodes that the current computation batch needs to access, and its content is dynamically updated as the computation batch changes. By dividing the cache into two functionally distinct partitions, high-hot node data can remain stably in the cache unaffected by dynamic data loading, while dynamic data can also obtain independent cache space to support flexible scheduling between batches. This separately managed cache architecture effectively addresses the cache thrashing problem in related technologies, where frequently accessed node data across batches is frequently evicted due to competition for cache space with other data.

[0067] In this embodiment of the invention, the size of the first cache partition and the size of the second cache partition can be determined based on the amount of data of high-hot nodes determined by node heat data and the amount of phased sampling data calculated by graph neural network.

[0068] In other words, the size of the first and second cache partitions can dynamically change depending on the graph neural network computation task. The size of the first cache partition is determined based on the total amount of data from high-hot nodes identified by node popularity data, ensuring that the cache space can accommodate all the feature data of high-hot nodes that need to be persistently accessed. The size of the second cache partition is determined based on the amount of sampling data in each stage of the graph neural network computation, that is, configured according to the total amount of node feature data actually needed to be accessed in each computation batch (or inference stage). By dynamically determining the partition size based on node popularity data and task characteristics, the cache space can be reasonably allocated under different task scenarios, avoiding waste or insufficiency of cache space.

[0069] It should be noted that, in order to avoid the data volume of high-hot nodes being too large, the number of high-hot nodes can be constrained by the topK method when determining the node popularity data.

[0070] In this embodiment of the invention, when performing graph neural network computation, the feature data of the target node is loaded from the storage device to the second cache partition of the accelerator card cache: before the start of the current stage of graph neural network computation, the set of nodes that actually need to be accessed in the current stage of computation is determined based on the sampling results of the current stage of computation; based on the node popularity data, nodes with access frequency higher than a first threshold are selected from the node set as the target nodes for the current stage of computation, and the feature data of the target nodes are loaded from the storage device to the second cache partition.

[0071] Specifically, before each computation batch (or each inference stage) begins, the accelerator card first acquires the sampling results for that batch (or stage) to determine the set of nodes that the computation actually needs to access. Then, combining node popularity data, the accelerator card filters out nodes from this set whose access frequency exceeds a first threshold, designating them as target nodes (i.e., dynamically cached nodes) for this batch. The first threshold can be set according to task requirements to define which nodes have high access frequency and require cache optimization within that batch. Subsequently, the accelerator card locates the feature data pages of these target nodes through a first mapping, reads the corresponding data pages through a first file system, and loads the feature data of the target nodes into a second cache partition. Thus, the accelerator card can accurately filter out nodes that need to be cached based on the actual needs of each batch, ensuring that cache resources are used on the most critical nodes requiring optimization, improving the utilization efficiency of cache resources. Simultaneously, because the content of the second cache partition is dynamically updated with batch changes, it ensures that the accelerator card can achieve flexible and accurate cache scheduling between different batches.

[0072] Figure 5 This is a schematic diagram of an accelerator card caching and mapping management module provided in an embodiment of the present invention.

[0073] like Figure 5 As shown, the accelerator card cache and mapping management module can receive the sampling results of the current stage of computation, manage the accelerator card cache, and maintain the node-page mapping relationship. The accelerator card cache includes a first cache partition and a second cache partition. The first cache partition corresponds to the static cache and is used for resident high-hot node feature data; the second cache partition corresponds to the dynamic cache and is used to store the node feature data required for the current stage of computation.

[0074] Furthermore, the second cache partition can be divided into two sub-cache partitions; when performing graph neural network computation, loading the feature data of the target node from the storage device to the second cache partition of the accelerator card cache can include: while performing the current stage computation based on one of the sub-cache partitions, storing the feature data of the target node for the next stage computation into the other sub-cache partition according to the sampling results of the next batch.

[0075] In other words, to further hide the data loading latency, the second cache partition can be further divided into two sub-cache partitions (sub-cache partition A and sub-cache partition B). When executing the current batch (i.e., the current computation stage), the accelerator card uses the data already loaded in one of the sub-cache partitions (e.g., sub-cache partition A) for computation. Simultaneously, based on the sampling results and node popularity data of the next batch, the accelerator card pre-fetches the feature data of these next batch target nodes into the other sub-cache partition (e.g., sub-cache partition B) through the first file system. After the current batch computation is completed, when the next batch computation begins, the accelerator card directly uses the pre-fetched data in sub-cache partition B for computation, while simultaneously starting to pre-fetch the data for the next batch in sub-cache partition A. In this way, parallel pipelined operation of data loading and computation execution is achieved, thereby hiding the data loading latency within the computation process and reducing the waiting time at the start of each batch.

[0076] Based on the above embodiments, the accelerator card data access method provided by the present invention may further include: at the beginning of the current stage of calculation, extracting the corresponding mapping information from the first mapping based on the target node selected in the current stage of calculation, and generating a temporary mapping table for the current stage of calculation; and accessing the feature data of the target node based on the temporary mapping table when performing graph neural network calculation.

[0077] like Figure 5 As shown, node-page mapping management maintains a first mapping and a temporary mapping table. The first mapping provides basic positioning from node identifier to page identifier and intra-page offset. For example, page identifier = node identifier / / 8, and intra-page offset = (node ​​identifier % 8) × 512. The temporary mapping table is used for fast positioning calculations in this stage.

[0078] In the above embodiment, the accelerator card filters out the target nodes (dynamic cache nodes) for this batch based on batch sampling results and node popularity data. Based on this, the accelerator card extracts the mapping entries (i.e., the correspondence between node identifiers and page addresses) corresponding to these target nodes from the first mapping, generating a temporary mapping table that is only valid during the current batch. The purpose of this temporary mapping table is to aggregate and map the feature data of the target nodes to a smaller number of data pages. Since the first mapping uniformly maps all nodes to storage pages, the data of different nodes may be scattered across a large number of discontinuous pages. The temporary mapping table, by filtering out the mapping information of the target nodes, makes the logical mapping relationship of these nodes more compact in the temporary mapping table, thereby forming a spatially concentrated data layout within this batch. When the accelerator card accesses the feature data of these target nodes through the temporary mapping table, the more concentrated data pages mapped reduce the number of data pages involved in I / O operations, improving the spatial locality of data access.

[0079] In this embodiment of the invention, when performing graph neural network calculation, accessing the feature data of the target node according to the temporary mapping table may include: when a node feature access request occurs during the execution of graph neural network calculation, prioritizing the location of the node to the page based on the temporary mapping table; if the temporary mapping table is not matched, locating the node to the page based on the first mapping.

[0080] Specifically, during the execution of graph neural network computation, when the accelerator card needs to access the feature data of a node, it first queries the temporary mapping table for this batch. If the node is in the temporary mapping table (i.e., the node is the target node in this batch), the node is located to the page according to the temporary mapping table, and the corresponding data page is accessed. If the node is not in the temporary mapping table (i.e., the node is not the target node in this batch, such as a non-target node within the neighbor range of the sampled nodes in this batch), it falls back to the global first mapping to locate the node to the page, and accesses the corresponding data page. Thus, for target nodes with high access frequency in this batch, the temporary mapping table achieves fast and centralized data location; while for non-target nodes with low access frequency in this batch, the first mapping achieves stable and low-overhead data location. Since the temporary mapping table only contains the mapping information of the target node, its query efficiency is high; the first mapping is a global mapping, and the page address is directly obtained through calculation without additional mapping maintenance overhead. Therefore, without increasing system complexity, most nodes can still use stable fixed mapping for location, while only a few key nodes are optimized using the temporary mapping table, balancing access performance and system stability.

[0081] In some optional embodiments of the present invention, graph neural network computation can correspond to the full graph inference task of graph neural networks. Therefore, the accelerator card data access method provided in the present invention may further include: dividing the full graph inference task into multiple inference stages before performing graph neural network computation. At the beginning of the computation of this stage, based on the target nodes selected in this stage, the corresponding mapping information is extracted from the first mapping to generate a temporary mapping table for this stage. This may include: at the beginning of this inference stage, based on the target nodes selected in this inference stage, extracting the corresponding mapping information from the first mapping to generate a temporary mapping table for this inference stage.

[0082] In other words, when applied to full-graph inference tasks using graph neural networks, the access pattern differs from that of mini-batch training based on neighbor sampling, as full-graph inference typically requires sequential forward computation of all nodes in the graph according to a predetermined order. In full-graph inference scenarios, the task can be pre-divided into multiple inference stages, each corresponding to a continuous inference sequence. At the beginning of each stage, the target nodes for that stage are selected based on the node access subset concentrated within that stage, and a temporary mapping table is generated based on the first mapping. This temporary mapping table is valid during the inference stage and is used to aggregate and map the feature data of frequently accessed nodes within that stage to fewer data pages. The temporary mapping table automatically expires after the inference stage ends. Through this staged temporary mapping management, dynamic mapping optimization can be performed based on staged access characteristics during full-graph inference, avoiding the problem of poor data access locality caused by using a uniform mapping method across the entire graph.

[0083] The accelerator card data access method provided in this embodiment of the invention may further include: clearing the temporary mapping table after the calculation in this stage is completed.

[0084] Specifically, the lifecycle of the temporary mapping table can be strictly limited to a single computation batch (or a single inference phase). After the computation of the current batch (or the current inference phase) is completed, the accelerator card immediately clears the temporary mapping table corresponding to that batch, releasing the mapping storage resources it occupies. At the start of the next batch, the accelerator card will regenerate a new temporary mapping table based on the target node of the next batch. By managing the lifecycle by batch, dynamic changes to the mapping table are always kept within the granularity of a single batch, avoiding long-term accumulation and large-scale frequent changes to mapping relationships.

[0085] In this embodiment of the invention, loading the feature data of the target node from the storage device to the accelerator card cache when performing graph neural network calculation may include: determining the set of data pages that need to be swapped in or out based on the first mapping and cache hit status; aggregating multiple pages that need to be swapped in or out at the same time into a page group; and performing a data migration operation on the storage device in units of page groups via the first file system.

[0086] Specifically, when the accelerator card needs to load node feature data from the storage device during graph neural network computation, it first determines the page identifier where the target node's feature data is located based on the first mapping (or temporary mapping table). Then, combined with the cache hit status of the accelerator card, it determines which pages are already in the cache (hits) and which pages are not in the cache (misses). For missing data pages, they need to be swapped in from the storage device; for data pages that are no longer needed in the cache, they need to be swapped out to the storage device to free up cache space. If multiple pages need to be swapped in or out simultaneously, these pages are aggregated into a page group, and storage I / O operations are performed on a page group basis. Specifically, the accelerator card packages the identifier information of all pages in the page group and initiates a one-time batch I / O request to the storage device through the first file system, allowing the storage device to complete the data transfer of multiple pages at once. Compared to the processing method of initiating I / O requests for each page separately, the page group aggregation method can significantly reduce the number of I / O operations, reduce the overhead of each I / O operation, and fully utilize the bandwidth capacity of the storage device.

[0087] In this embodiment of the invention, performing a data migration operation on a page group basis to load the feature data of the target node from the storage device to the accelerator card cache may include: initiating a data migration request for the page group to the storage device through the accelerator card direct storage access interface provided by the first file system, so as to perform a data migration operation between the accelerator card and the storage device.

[0088] Specifically, when performing page group-level data migration operations, the accelerator card can directly initiate I / O requests for the entire page group to the storage device through the accelerator card direct access interface provided by the First File System, without the CPU's involvement in I / O scheduling and control. This direct access interface is a dedicated interface provided by the First File System for the accelerator card. The accelerator card can directly send storage access commands through this interface, and the storage device directly responds to the accelerator card's requests and executes data transfer. During this process, the CPU only participates in the interface and mapping establishment during the initialization phase and does not participate in the operation at all during the data migration execution phase. Furthermore, during storage access, the accelerator card can also retrieve the access popularity information of the corresponding node from pre-stored node popularity data using the node identifier as an index. This allows for auxiliary prioritization of read / write priorities for data pages within the page group during I / O scheduling (e.g., prioritizing the swapping of data pages corresponding to nodes with higher access popularity), further optimizing the organization order of I / O operations.

[0089] Figure 6 This is a schematic diagram of an accelerator card cache direct-connect storage access module provided in an embodiment of the present invention.

[0090] like Figure 6As shown in this embodiment of the invention, the direct-access storage module based on the accelerator card cache includes a hot metadata management unit and a page group migration unit. The hot metadata management unit manages the node hotness table of the accelerator card-side storage. This hotness table, as training semantic metadata, together with file system metadata (including basic storage metadata such as page layout and page mapping), constitutes the metadata system accessible to the accelerator card. The page group migration unit aggregates multiple pages that need to be swapped in or out into a page group, and initiates a batch data migration operation to the storage device in units of page groups through the direct-access storage interface.

[0091] Based on the above embodiments, and in conjunction with specific application scenarios, the technical solutions of the embodiments of the present invention will be further described.

[0092] Scenario 1: Accelerating GNN training based on the OGBN-ARXIV dataset.

[0093] In this scenario, the OGBN-ARXIV dataset is used as the graph data. This dataset uses papers as nodes and citation relationships as edges. The node features are represented by high-dimensional vectors. During training, a mini-batch training method based on neighbor sampling is adopted.

[0094] First, data preprocessing is performed before training begins: graph structure information is extracted and stored in the accelerator card; node identifiers and node feature data are stored separately, and feature data is written to the first file system at the page level to establish the first mapping from node identifiers to page addresses; node popularity data is calculated based on the access characteristics of all graph nodes, and the node popularity data is written to the first file system as metadata.

[0095] Next, the accelerator card reads node popularity data from the first file system, identifies high-hot nodes (such as core paper nodes in citation relationships), preloads the feature data of these high-hot nodes into the first cache partition of the cache, and keeps them resident. Then, the second cache partition of the cache is divided into two sub-cache partitions for pipelined operations.

[0096] During training, at the start of each mini-batch, the accelerator card acquires the sampling results of that batch, combines them with node popularity data to filter out the target nodes for that batch, and generates a temporary mapping table. The accelerator card prioritizes accessing the feature data of the target nodes through the temporary mapping table; if a cache miss occurs, it falls back to the first mapping. For data pages that are not cached, the accelerator card aggregates them with other pages to be loaded in the same batch into a page group, and directly initiates a page group-level I / O request through the direct-access storage interface of the first file system. Simultaneously, two sub-cache partitions are used to achieve parallel execution of the prefetching of the next batch of data and the computation of the current batch. After the computation of the current batch is completed, the temporary mapping table is cleared.

[0097] Through the above process, in GNN training on the OGBN-ARXIV dataset, the accelerator card can significantly shorten the data preparation time for each batch, improve the cache reuse rate of feature data of frequently accessed nodes, reduce cache jitter, and thus improve the overall training throughput.

[0098] Scenario 2: Accelerating full-graph inference based on static large-scale graphs.

[0099] In this scenario, a GNN full graph inference task is performed using static large-scale graph data, and the graph structure, node set, and node features remain unchanged throughout the inference process.

[0100] First, before inference begins, node heat calculation is performed and the heat data is stored in the first file system. Graph structure information is extracted and fixedly stored in the acceleration card. The first mapping is established and the feature data of high-heat nodes is loaded into the first cache partition for permanent residence.

[0101] The full-graph inference task is pre-divided into multiple inference stages. At the beginning of each inference stage, the target nodes for that stage are selected based on a subset of nodes that need to be accessed centrally, and a temporary mapping table for that inference stage is generated. During inference, the accelerator card locates node data through the temporary mapping table (preferred) or the first mapping table. For data that is not found, I / O is performed through the direct-attached storage interface in units of page groups. After the inference of that stage is completed, the temporary mapping table is cleared, and the next inference stage begins.

[0102] Through the above process, the stability of the cache state is improved during the full graph reasoning process, the node data that is frequently accessed in stages is effectively aggregated, the uncertainty and latency fluctuation of storage access are reduced, and the overall execution efficiency of full graph reasoning is improved.

[0103] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0104] Embodiments of the present invention also provide an accelerator card, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the accelerator card data access method.

[0105] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the accelerator card data access method when running.

[0106] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0107] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the accelerator card data access method.

[0108] Embodiments of the present invention also provide another computer program product, including a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the accelerator card data access method.

[0109] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be performed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0111] The above provides a detailed description of the accelerator card data access method and the accelerator card itself. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are only intended to aid in understanding the method and core ideas of the invention. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the invention.

Claims

1. A method for accessing data on an accelerator card, characterized in that, Applied to an accelerator card, the accelerator card is used to perform graph neural network calculations, and the accelerator card data access method includes: An access interface is established between the accelerator card and the first file system, which is used to manage the correspondence between page identifiers and physical addresses of storage devices, and the first file system is pre-written with node heat data of the graph neural network. Access the first file system through the access interface, read the node popularity data, and determine high-popularity nodes based on the node popularity data; Obtain the first mapping between node identifiers and page addresses in the graph neural network; The first file system is accessed through the access interface, and the physical address of the graph neural network node is queried from the first file system according to the first mapping, so as to load the feature data of the high-hot node from the storage device to the accelerator card cache and keep it resident, and load the feature data of the target node from the storage device to the accelerator card cache when performing graph neural network calculation.

2. The accelerator card data access method according to claim 1, characterized in that, The steps for determining the node heat data include: One or more of the following methods are used to calculate the access popularity of nodes in a graph neural network, thereby obtaining the node popularity data: statistical analysis based on node sampling results during training, analysis method based on graph structure features, node importance or centrality calculation method based on traditional graph algorithms, and node popularity estimation method based on dynamic information during training. The node heat data is used to characterize the likelihood that a node will be repeatedly accessed during the graph neural network computation process.

3. The accelerator card data access method according to claim 1, characterized in that, Loading the feature data of the high-hot nodes from the storage device to the accelerator card cache and keeping it resident includes: The feature data of the high-hot node is loaded from the storage device into the first cache partition of the accelerator card cache and kept there permanently. Loading the feature data of the target node from the storage device to the accelerator card cache during graph neural network computation includes: When performing graph neural network computation, the feature data of the target node is loaded from the storage device to the second cache partition of the accelerator card cache.

4. The accelerator card data access method according to claim 3, characterized in that, The size of the first cache partition and the size of the second cache partition are determined based on the amount of data of the high-hot nodes determined by the node heat data and the amount of phased sampling data calculated by the graph neural network.

5. The accelerator card data access method according to claim 3, characterized in that, When performing graph neural network computation, loading the feature data of the target node from the storage device to the second cache partition of the accelerator card cache includes: Before the start of this stage of the graph neural network computation, the set of nodes that actually need to be accessed in this stage of computation is determined based on the sampling results of this stage of computation. Based on the node popularity data, nodes with access frequencies higher than a first threshold are selected from the node set as the target nodes for calculation in this stage, and the feature data of the target nodes are loaded from the storage device into the second cache partition.

6. The accelerator card data access method according to claim 3, characterized in that, The second cache partition is divided into two sub-cache partitions; When performing graph neural network computation, loading the feature data of the target node from the storage device to the second cache partition of the accelerator card cache includes: While performing the current stage calculation based on one of the sub-cache partitions, the feature data of the target node to be calculated in the next stage is stored in the other sub-cache partition according to the sampling results of the next batch.

7. The accelerator card data access method according to claim 1, characterized in that, The first mapping is used to determine the corresponding page identifier and the offset within the page based on the node identifier; The first mapping remains constant during the execution of the graph neural network computation.

8. The accelerator card data access method according to claim 7, characterized in that, Also includes: At the start of this stage of calculation, based on the target nodes selected in this stage of calculation, the corresponding mapping information is extracted from the first mapping to generate a temporary mapping table for this stage of calculation. When performing the graph neural network computation, the feature data of the target node is accessed according to the temporary mapping table.

9. The accelerator card data access method according to claim 8, characterized in that, When performing the graph neural network computation, accessing the feature data of the target node according to the temporary mapping table includes: When a node feature access request occurs during the execution of the graph neural network calculation, the node is located to the page based on the temporary mapping table first. If the temporary mapping table is not found, the node is located to the page according to the first mapping.

10. The accelerator card data access method according to claim 8, characterized in that, The graph neural network computes the full graph reasoning task corresponding to the graph neural network; Accelerator card data access methods also include: Before performing graph neural network computation, the full graph reasoning task is divided into multiple reasoning stages; At the start of this phase of calculation, based on the target nodes selected in this phase, the corresponding mapping information is extracted from the first mapping to generate a temporary mapping table for this phase of calculation, including: At the beginning of this inference phase, based on the target nodes selected in this inference phase, the corresponding mapping information is extracted from the first mapping to generate the temporary mapping table for this inference phase.

11. The accelerator card data access method according to claim 8, characterized in that, Also includes: After the calculations for this stage are completed, the temporary mapping table is cleared.

12. The accelerator card data access method according to claim 1, characterized in that, Loading the feature data of the target node from the storage device to the accelerator card cache during graph neural network computation includes: Based on the first mapping and cache hit status, determine the set of data pages that need to be swapped in or out; Multiple pages that need to be swapped in or out simultaneously are aggregated into a page group, and a data migration operation is performed on the page group to load the feature data of the target node from the storage device to the accelerator card cache.

13. The accelerator card data access method according to claim 12, characterized in that, Performing a data migration operation on a page-group basis, loading the feature data of the target node from the storage device to the accelerator card cache, includes: The accelerator card is used to initiate a data migration request for the page group to the storage device through the accelerator card direct access storage interface provided by the first file system, so as to perform the data migration operation between the accelerator card and the storage device.

14. The accelerator card data access method according to claim 1, characterized in that, Also includes: Before performing graph neural network computation, the graph structure information of the target graph neural network is extracted and stored in the acceleration card.

15. An accelerator card, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the accelerator card data access method as described in any one of claims 1 to 14 when executing the computer program.

Images