Training optimization method, CMM-DC equipment pool and host equipment

Abstract

The invention provides a training optimization method, a CMM-DC equipment pool and host equipment. The training optimization method is executed by a CMM-DC equipment pool, the CMM-DC equipment pool comprises multiple pieces of CMM-DC equipment, and the training optimization method comprises the steps that multiple pieces of sub-graph data partitioned by host equipment are received, and the host equipment is connected to the multiple pieces of CMM-DC equipment; receiving feature data corresponding to the sub-graph data; multiple sub-graph data and feature data are separately stored using multiple CMM-DC devices.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence acceleration and computer storage technology, and more specifically, to a training optimization method, a CMM-DC device pool, and a host device. Background Technology

[0002] Graph Neural Networks (GNNs) are a type of deep neural network that can effectively process and analyze graph-structured data, and are widely used for various graph-related tasks. As the size of real-world graph data increases, training large-scale GNNs within limited memory becomes difficult. Mini-batch GNN training based on distributed samples has emerged as a promising solution. Mini-batch GNN training mainly includes the following steps: data partitioning, sampling, feature extraction, data loading, aggregation, and model training.

[0003] Traditional mini-batch GNN training partitions large-scale graphs or graph data into subgraphs or subgraph data, and performs sampling, feature extraction, aggregation, and model training on the graphics processing unit (GPU). This provides more storage and computing resources for GNN training, but it still has the following shortcomings: when training large-scale or massive GNNs, it faces the limitation of GPU memory space; the GPU needs to access remote graph data and feature data from graph storage to create batches for local training, which leads to large-scale data movement; data I / O in subgraph sampling and feature retrieval consumes a lot of training time; and the GPU utilization is low, as aggregation functions involve many simple calculations that cannot effectively utilize the high computing performance of the GPU.

[0004] The optimized mini-batch GNN training performs sampling on the CPU and optimizes performance by preloading feature data. On one hand, sampling on the CPU reduces the GPU memory required to store some subgraphs; on the other hand, preloading features to the GPU improves GPU utilization. However, the optimized mini-batch GNN training still has the following drawbacks: although the GPU memory required for subgraphs sampled by the CPU is reduced, it is still limited by GPU memory when training large-scale GNNs. Frequent data movement still occurs when loading sampled graph data and feature data onto the GPU; and simple calculations during the aggregation process waste the GPU's high computational performance.

[0005] The above information is provided as background information only and does not imply that the above information constitutes prior art in this disclosure. Summary of the Invention

[0006] One of the purposes of this disclosure is to provide a training optimization method that can reduce the use of host memory and GPU memory to meet the memory requirements of GNN training on large-scale graphs.

[0007] One of the purposes of this disclosure is to provide a training optimization method that can reduce data preparation time and improve training performance.

[0008] One of the purposes of this disclosure is to provide a training optimization method that can improve the utilization of the central processing unit (CPU).

[0009] According to a first aspect of this disclosure, a training optimization method for a graph neural network is provided. The training optimization method is executed by a CMM-DC device pool, which includes multiple CMM-DC devices. The training optimization method includes: receiving multiple subgraph data partitioned by a host device, wherein the host device is connected to the multiple CMM-DC devices; receiving feature data corresponding to the multiple subgraph data; and storing the multiple subgraph data and feature data separately using the multiple CMM-DC devices.

[0010] Optionally, the training optimization method may further include: in response to a sampling request from the training manager in the host device, sampling the stored corresponding subgraph data to obtain sampling results; and in response to an aggregation request from the training manager, performing feature aggregation on the feature data of the sampling nodes in the sampling results to obtain aggregation results.

[0011] Optionally, the training optimization method may also include sending the sampling results and aggregation results as batch data to the memory of the host device.

[0012] Optionally, each CMM-DC device may store a portion of multiple subgraph data and a portion of feature data.

[0013] Optionally, a portion of the multiple CMM-DC devices may be used to store multiple subgraph data and another portion of the multiple CMM-DC devices may be used to store feature data.

[0014] According to a second aspect of this disclosure, a training optimization method for a graph neural network is provided. The training optimization method is executed by a host device and includes: partitioning graph data and corresponding feature data through a training manager of the host device; storing sampling results sampled by samplers in multiple CMM-DC devices in a CMM-DC device pool and aggregation results aggregated by aggregators in multiple CMM-DC devices in the memory of the host device, wherein the multiple CMM-DC devices are configured to separately store the graph data and feature data partitioned by the training manager.

[0015] Optionally, the sampling results and aggregation results can be stored as batch data in the memory of the host device. The training optimization method further includes loading the sampling results and aggregation results into the memory of the GPU device through a data loader in the CPU of the host device.

[0016] Optionally, the training optimization method may also include: triggering the GPU to perform training in response to a batch of data being loaded into the memory of the GPU device and the GPU being available.

[0017] According to a third aspect of this disclosure, a training optimization method for a graph neural network is provided. The training optimization method is executed by a GPU device. The training optimization method includes: in response to a training instruction from a training manager in a host device, performing module computation of a graph neural network model using sampling results and aggregation results loaded from the memory of the host device, wherein the sampling results and aggregation results are stored as batch data in the memory of the host device, the sampling results are sampled by samplers in multiple CMM-DC devices in a CMM-DC device pool, the aggregation results are aggregated by aggregators in the CMM-DC devices, and the multiple CMM-DC devices are configured to separately store graph data and feature data partitioned by the training manager.

[0018] According to a fourth aspect of this disclosure, a CMM-DC device pool is provided, comprising a plurality of CMM-DC devices configured to perform the above-described training optimization method.

[0019] According to a fifth aspect of this disclosure, a host device is provided, comprising: a processor and a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores instructions that, when executed by the processor, cause the processor to perform the training optimization method described above.

[0020] According to a sixth aspect of this disclosure, a GPU device is provided, comprising: a processor and a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform the training optimization method described above.

[0021] According to a seventh aspect of this disclosure, a computer system is provided, comprising: the aforementioned CMM-DC device pool, the aforementioned host device, and the aforementioned GPU device.

[0022] According to an eighth aspect of this disclosure, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to execute according to the training optimization method described above.

[0023] According to a ninth aspect of this disclosure, a training optimization method for a graph neural network is provided. The training optimization method includes: partitioning graph data and corresponding feature data; storing the partitioned graph data and feature data separately in multiple CMM-DC devices; sampling and feature aggregation of the partitioned graph data and feature data to obtain sampling results and aggregation results; loading the sampling results and aggregation results as batch data into the memory of a GPU device; and using the sampling results and aggregation results to perform module computation of the graph neural network model.

[0024] Optionally, the steps of sampling and feature aggregation on the partitioned graph data and feature data to obtain sampling results and aggregation results may include: in response to a sampling request from the training manager in the host device, all samplers in multiple CMM-DC devices sample the corresponding graph data to obtain sampling results; in response to an aggregation request from the training manager, all aggregators in multiple CMM-DC devices perform feature aggregation on the feature data of the sampled nodes in the sampling results to obtain aggregation results.

[0025] Optionally, the steps of loading the sampling results and aggregation results into the memory of the GPU device and using the sampling results and aggregation results to perform module computation of the graph neural network model may include: storing the sampling results sampled by all samplers and the aggregation results aggregated by all aggregators as batch data in the memory of the host device; loading the sampling results and aggregation results stored in the memory of the host device into the memory of the GPU device; and using the sampling results and aggregation results loaded into the memory of the GPU device to perform module computation of the graph neural network model.

[0026] The training optimization method according to embodiments of this disclosure performs subgraph sampling and feature aggregation in a CMM-DC device, requiring only the sampling results and aggregation results, which greatly reduces data movement between the CPU and host memory during feature retrieval. Attached Figure Description

[0027] The above and other objects and features of exemplary embodiments of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, which exemplarily illustrate the embodiments, wherein: Figure 1 This illustrates the traditional mini-batch GNN training process; Figure 2 The optimized mini-batch GNN training process is shown; Figure 3A This is a general architecture diagram based on embodiments of the present disclosure and Figure 3B The GNN training process of an embodiment of this disclosure is illustrated; Figure 4 This illustrates a pipeline for parallel GNN training according to embodiments of the present disclosure; Figure 5 This is a general flowchart illustrating a training optimization method according to an embodiment of the present disclosure; Figure 6 This illustrates the workflow of graph partitioning and graph storage according to embodiments of the present disclosure; Figure 7 This illustrates the workflow of the GNN training process according to an embodiment of the present disclosure, excluding partitioning; Figure 8 This is a flowchart illustrating a training optimization method according to a first embodiment of the present disclosure; Figure 9 This is a flowchart illustrating a training optimization method according to a second embodiment of the present disclosure; Figure 10 This is a flowchart illustrating a training optimization method according to a third embodiment of the present disclosure. Detailed Implementation

[0028] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, examples of which are illustrated in the drawings, wherein the same reference numerals always refer to the same parts. The embodiments will now be described with reference to the accompanying drawings in order to explain this disclosure.

[0029] The following detailed description is provided to aid in obtaining a full understanding of the methods, apparatus, and / or systems described herein. However, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein; equivalent substitutions or changes may be made, except for operations that must occur or be performed in a specific order. Furthermore, for clarity and conciseness, descriptions of content well-known in the art will be omitted or simplified.

[0030] Unless otherwise specified, the same reference numerals generally refer to the same elements (e.g., components, steps, and methods). Reference numerals described in previous embodiments that reappear in later embodiments may be omitted. Furthermore, technical features described in different or the same embodiments can be combined in any way, as long as the combined embodiment or technical solution is complete and can solve the technical problems of this application or achieve the technical effects described or not described in this disclosure but which can be determined based on the complete technical solution described above.

[0031] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner. A brief description of the terminology used in this disclosure is provided below.

[0032] The CXL (Compute Express Link) interface is an open standard for high-speed, low-latency, high-capacity interconnects between host processors and devices such as memory extenders, smart memory modules, and accelerators. In the system disclosed herein, the host device contains a central processing unit (CPU) that acts as the system's main control unit. This CPU connects to one or more CMM-DC devices via the CXL interface to transmit data and control commands. CXL memory expansion technology allows servers to expand memory capacity to tens of terabytes while increasing bandwidth to hundreds of gigabytes per second.

[0033] CMM-DC (CXL Memory Module – DRAM & Compute) devices integrate memory with accelerators or computing circuitry connected to that memory, reducing data migration between the host CPU and memory by utilizing memory for data computation. Because computation is performed closer to memory (i.e., the processor is closer to the memory, PNM), data transfer between the host CPU and memory is reduced, thus mitigating (or eliminating) bottlenecks in data transfer between the host CPU and memory when processing large-scale artificial intelligence (AI) models. Such bottlenecks can also be referred to as input / output (I / O) bottlenecks. CMM-DC (CMM-PNM) technology is a solution used in conjunction with high-capacity AI model processing. Tests have demonstrated that CMM-DC solutions based on the CXL interface offer more than double the performance in scenarios with high memory bandwidth applications, such as recommendation systems or in-memory databases. Training large AI models involves massive amounts of data and complex computational tasks, with the training process involving numerous matrix operations and parameter updates. Therefore, CMM-DC can improve data processing speed and shorten training time.

[0034] The host device may include a host CPU (or CPU device). The CPU, as the system's main control unit, is responsible for task scheduling, data management, and overall process control. The GPU or GPU device connects to the CPU or CPU device in the host device via a high-speed interface (such as PCIe). For example, the GPU device can be a separate device from the host device or it can be part of the host device. The host device can also connect one or more CMM-DC devices via the CXL interface to expand memory capacity and enable near-memory data preprocessing.

[0035] The training optimization method disclosed herein can achieve at least one of the following: reducing I / O between the GPU and graph storage devices, accelerating GPU training, achieving high GPU utilization, and meeting the memory requirements for training GNNs on large-scale graph data. This will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1This illustrates the traditional mini-batch GNN training process. Figure 2 The optimized mini-batch GNN training process is shown.

[0037] Reference Figure 1 In the traditional mini-batch GNN training process, the host CPU 120 splits the large-scale graph or graph data into subgraphs or subgraph data by data partitioning, the graph memory 110 stores the partitioned or partitioned subgraph data and feature data, and the GPU 130 completes sampling, feature acquisition, aggregation and model training, etc.

[0038] Reference Figure 2 During the optimized mini-batch GNN training process, sampling is performed on the host CPU 120, which reduces the memory space required for storing some subgraphs on the GPU 130; CPU utilization is improved by preloading features to the GPU.

[0039] The advantages and disadvantages of traditional mini-batch GNN training and optimized mini-batch GNN training are shown in Table 1 below.

[0040]

[0041] Figure 3A This is an overall architecture diagram based on an embodiment of the present disclosure. Figure 3B The GNN training process of at least one example embodiment of this disclosure is shown.

[0042] like Figure 3A As shown, the CMM-DC device pool 140 (i.e., intelligent GNN storage) may include multiple CMM-DC devices 141. The multiple CMM-DC devices 141 can interact with each other or with the host CPU 120 through the CXL switch 150. The multiple CMM-DC devices 141 can separately store the graph data and the corresponding feature data partitioned by the training manager 121 (a software module running in the CPU) in the host CPU 120. Here, "separate storage" means that each of the subgraph (or subgraph data) and feature (or feature data) is stored in a distributed manner.

[0043] Reference Figure 3BThe host CPU 120 sends a sampling instruction to the CMM-DC device 141, which samples the stored subgraph and notifies the host CPU 120 upon completion. The host CPU 120 then sends an aggregation instruction to the CMM-DC device 141, which performs feature aggregation and notifies the host CPU 120 upon completion. The sampling and aggregation results can be stored in the host memory. The host CPU 120 can preload the sampling and aggregation results, as batch data, into the memory of the GPU 130, and then the GPU 130 performs model computation. In some embodiments, the host may include (or be connected to) multiple GPUs 130 (e.g., GPU1, GPU2, GPU3, GPU4, etc.). For example, the host device may be connected to a GPU device that includes multiple GPUs 130. In some exemplary embodiments, batch data may be loaded into some and / or all of the memory of the multiple GPUs 130. In addition, although not shown, at least some of the results can be fed back to the host CPU 120 after the calculation is completed, and the host CPU 120 can continue to schedule the next round of sampling and aggregation tasks.

[0044] Reference Figure 3A A portion of multiple CMM-DC devices (which may include two or more CMM-DC devices) may be used to store graph data or subgraphs (i.e., partitioned subgraphs), and another portion of the multiple CMM-DC devices (which may include two or more CMM-DC devices) may be used to store feature data (i.e., partitioned feature data). The CMM-DC device used to store the subgraph may include a sampler 1411. The CMM-DC device used to store feature data may include an aggregator 1412. The sampler 1412 may be used to select a subset of nodes from the graph data as samples for training, and may perform sampling tasks according to specific rules or algorithms to accelerate model training and improve training performance. The sampling algorithms of the sampler may include node-wise sampling, layer-wise sampling, and subgraph-based sampling, etc. The task of the aggregator is to perform aggregation operations on node features in the graph data. The aggregation algorithms of the aggregator may include averaging, summing, finding the maximum / minimum value, etc. Separating subgraphs from features and storing them separately in a distributed manner facilitates efficient parallel computing.

[0045] In one example, each CMM-DC device can store a portion of graph data and a portion of feature data. In this case, each CMM-DC device can include a sampler 1411 and an aggregator 1412. The sampling algorithm of the sampler can include node-wise sampling, layer-wise sampling, and subgraph-based sampling. The aggregation algorithm of the aggregator can include averaging, summing, finding the maximum / minimum value, etc.

[0046] Reference Figure 3A After sampling, the neighborhood features of each node in the sampled subgraph are aggregated. After aggregation, the host CPU 120 loads the sampled subgraph and aggregated features into the memory cache of the GPU 130 for module computation. Finally, the module computation is performed on the GPU 130 side. The specific module computation may include forward propagation (computing the process from the input layer to the output layer), backpropagation (computing gradients based on the loss function and updating model parameters to minimize the error), and parameter update (adjusting network weights based on the gradients obtained from backpropagation). From this, the predicted value, error magnitude, gradient information, and the finally updated model parameters can be obtained. A portion of the predicted value, error magnitude, gradient information, and the finally updated model parameters can be sent back to the host CPU 120.

[0047] Reference Figure 3A The Training Manager 121 in the CPU of the host device (hereinafter referred to as the host CPU) can be used for training process management, mainly including data partitioning, scheduling, monitoring and optimization of the training process. The Training Manager can send partitioning instructions to the data partition in the CPU (i.e., data partitioning module 122). The data partition can split the graph data and feature data into multiple parts to facilitate subsequent small-batch processing. These data will then be stored in different CMM-DC devices.

[0048] Additionally, the memory (e.g., DRAM) in the host or host device can store metadata, sampled subgraph data, and aggregated feature data. This metadata may include storage location, historical data, index data, file records, etc., while the sampled subgraph data and aggregated features represent the sampling results from the sampler and the aggregation results from the aggregator, respectively.

[0049] The data loader 123 in the CPU can load batch data (i.e., sampling results and aggregation results) stored in the host memory into the GPU 130 (e.g., GPU memory), and then perform module computations on the GPU 130. In this disclosure, the training manager 121, data partitioning module 122, and data loader 123 contained in the host CPU are implemented by software running on the CPU.

[0050] With this design, the system can efficiently handle GNN training tasks on large-scale graph data, reduce I / O bottlenecks, and improve computational efficiency and GPU utilization. The main improvements of this disclosure are as follows: (1) Subgraphs and feature data are stored separately in different CMM-DC devices, thereby reducing the use of host memory and GPU memory to meet the memory requirements for GNN training on large-scale graphs; (2) Sampling and feature aggregation are performed in the CMM-DC device, reducing feature acquisition I / O, reducing data preparation time, and improving the utilization efficiency of CPU and GPU; (4) Sampling results and aggregation results are preloaded into GPU memory, improving GPU utilization.

[0051] As an example, to achieve high GPU utilization and accelerate training, data dependencies and resource contention can be minimized, and GNN training can be divided into four independent stages that can be parallelized: subgraph sampling, feature aggregation, data loading, and module computation.

[0052] Figure 4 This illustrates a pipeline for parallel GNN training according to an embodiment of the present disclosure.

[0053] Reference Figure 4 , Figure 4 The workflow of different components (CMM-DC device, CPU, and GPU) is shown in consecutive time periods (T0 to T4).

[0054] During time interval T0, the CMM-DC0 device samples a subgraph; during time interval T1, the CMM-DC0 device continues to sample a subgraph, while the CMM-DC1 device aggregates features; during time interval T2, the CMM-DC1 device continues to aggregate features, and the CPU0 loads the subgraph and feature data; during time interval T3, the CPU0 continues to load the subgraph and feature data, and the GPU0 performs module computation; during time interval T4, the GPU0 continues module computation. This parallel processing maximizes the system's performance and efficiency.

[0055] Compared with traditional mini-batch GNN training and optimized mini-batch GNN training, the advantages of the training scheme disclosed in this paper are shown in Table 2 below.

[0056]

[0057] As can be seen from Table 2 above, compared with traditional mini-batch training methods and optimized mini-batch training methods, the optimized training method disclosed in this paper is more efficient and supports larger-scale graph data processing. By reducing I / O operations, improving GPU utilization, and implementing parallel training, it overcomes the problems encountered in training GNNs on large-scale or even massive-scale graphs.

[0058] Figure 5 This is a general flowchart of a training optimization method according to embodiments of the present disclosure. Figure 6 The workflow of graph partitioning and graph storage according to embodiments of the present disclosure is illustrated. Figure 7 The workflow of a GNN training process according to an embodiment of the present disclosure is shown, excluding partitioning.

[0059] Reference Figure 5 The training optimization method for a graph neural network according to embodiments of the present disclosure may include steps S510, S520, S530 and S540.

[0060] In step S510, the graph data and the corresponding feature data are partitioned.

[0061] Graph data and the corresponding feature data can be pre-collected and prepared data. For example, in the field of GNNs, the original graph or graph data can be a social network graph, a traffic network graph, etc., and the features or feature data can represent information such as node attributes or edge weights.

[0062] As an example, the partitioning steps for graph data and feature data can be executed by the CPU of the host device. Graph partitioning algorithms can include hash partitioning, Metis-extend partitioning, and streaming partitioning. Hash partitioning allocates nodes to different partitions based on a hash function, ensuring that nodes with the same hash value reside in the same partition; Metis-extend partitioning optimizes graph partitioning by minimizing the number of cut edges, supporting large-scale graph data processing and improving computational efficiency and resource utilization; streaming partitioning is geared towards streaming data environments, processing graph data in real time and dynamically adjusting partitions. The partitioning steps can be executed by the data partitioning module 122 in the host CPU. The data partitioning module 122 can be implemented by software running on the CPU. Specifically, the data partitioning module 122 can use the graph partitioning algorithms described above to divide large-scale graph data and / or feature data, generating multiple sub-graph data and feature data. The sub-graph data and feature data can be allocated to multiple CMM-DC devices for storage.

[0063] For details, please refer to Figure 6After the original graph data and feature data are input into the data partitioning module 122, the training manager 121 sends a command to the data partitioning module 122 to start the partitioning operation. The partitioning process divides the graph data and its feature data into multiple parts according to the algorithms in the graph partitioning method library, so as to facilitate subsequent parallel processing.

[0064] In step S520, the partitioned graph data and feature data are stored separately in multiple CMM-DC devices.

[0065] As an example, the CPU's training manager on the host device can transfer the partitioned graph data and feature data to multiple CMM-DC devices in the aforementioned CMM-DC device pool, and these multiple CMM-DC devices can then perform distributed storage of the graph data and feature data respectively. The specific distributed storage method can be as described above. Furthermore, sub-graph data and their corresponding feature data can be dynamically allocated based on the actual conditions of each CMM-DC device, ensuring that the amount of data stored on each CMM-DC device is as balanced as possible, while minimizing the data transfer requirements between devices. This reduces the impact of storage limitations on individual CMM-DC devices (e.g., the storage limitations of SSDs within CMM-DC devices), while also considering their computing power, enabling efficient local execution of sampling and aggregation operations, thereby improving the overall system's concurrent processing capabilities and response speed.

[0066] Reference Figure 6 After partitioning is completed, the partitioned subgraphs and feature data are transferred to different CMM-DC devices 141 in the CMM-DC device pool 140 for storage. At the same time, the relevant metadata (including the storage location of each subgraph, historical data, etc.) is recorded in the host's DRAM for fast access.

[0067] In step S530, the partitioned graph data and feature data are sampled and feature aggregated to obtain sampling results and aggregation results. In each CMM-DC device, sampling and feature aggregation can be implemented by software sampling algorithms and feature aggregation algorithms running in dedicated computing circuitry (e.g., FPGA) integrated in the CMM-DC device, thereby reducing the amount of data sent back to the host memory, saving bandwidth, and improving data transmission efficiency.

[0068] The sampling of graph data can be offloaded to a portion of multiple CMM-DC devices 141, and the aggregation of features can also be offloaded to a portion of multiple CMM-DC devices 141, instead of sampling and aggregation being performed on the host CPU 120 or GPU 130. However, this is merely an example, and this disclosure is not limited thereto.

[0069] Sampling and / or aggregation operations can be performed according to the request of the training manager in the CPU. As an example, step S530, which involves sampling and feature aggregation of partitioned graph data and feature data to obtain sampling and aggregation results, may include: in response to a sampling request from the training manager 121 in the host CPU 120, all samplers 1411 in the plurality of CMM-DC devices 141 sample the corresponding graph data to obtain sampling results; in response to an aggregation request from the training manager 121, all aggregators 1412 in the plurality of CMM-DC devices 141 perform feature aggregation on the feature data of the sampled nodes in the sampling results to obtain aggregation results. It should be noted that it is not required that each CMM-DC device in the plurality of CMM-DC devices have both a sampler 1411 and an aggregator 1412. As an example, some of the CMM-DC devices 141 may have a sampler 1411, while another portion of the CMM-DC devices 141 may have an aggregator 1412. The sampler 1411 and aggregator 1412 here can be implemented using sampling software algorithms and aggregation software algorithms running on a field-programmable gate circuit (FPGA).

[0070] In one example, in response to a sampling request from the training manager 121 in the host CPU 120, all samplers 1411 in all CMM-DC devices for storing subgraphs in the multiple CMM-DC devices can sample the corresponding graph data (i.e., the subgraphs stored therein) to obtain sampling results; in response to an aggregation request from the training manager 121, all aggregators 1412 in all CMM-DC devices 141 for storing feature data in the multiple CMM-DC devices 141 can perform feature aggregation on the feature data of the sampled nodes in the sampling results to obtain aggregation results.

[0071] In one example, in response to a sampling request from the training manager 121 in the CPU of the host device, a sampler 1411 in each of the plurality of CMM-DC devices 141 may sample the corresponding graph data (i.e., the subgraphs stored therein) to obtain a sampling result; in response to an aggregation request from the training manager 121 in the CPU of the host device, an aggregator 1412 in each of the plurality of CMM-DC devices 141 may aggregate the feature data of the sampled nodes in the sampling result to obtain an aggregation result.

[0072] When training begins, the training manager 121 notifies all samplers 1411 in the CMM-DC device 141 to sample the graph data, and the sampling results are stored in the host memory. After sampling, the training manager 121 notifies all aggregators 1412 in the CMM-DC device 141 to perform feature aggregation on each sampled node, and the aggregated feature results are stored in the host memory.

[0073] In step S540, the sampling and aggregation results, which are the batch data, are loaded into the memory of the GPU device, and the module computation of the graph neural network model is performed using the sampling and aggregation results. The module computation of this disclosure is performed by the GPU device, which may include processing circuitry (e.g., a stream processor) and non-transitory computer-readable storage media (e.g., high-speed video memory). The high-speed video memory can store the model parameters, subgraph data, and feature data of the graph neural network, and the processing circuitry can execute the computational kernel program generated by the GNN framework to perform linear transformations, nonlinear activations, and other operations on the sampling and aggregation results, completing the forward and backward propagation of the GNN model. Since sampling and aggregation are already completed in the CMM-DC device, the GPU can directly perform efficient computation based on the preprocessed results, reducing the preprocessing burden on the host CPU.

[0074] Reference Figure 3A After aggregation, the training manager 121 can notify the data loader 123 to load the batch data (sampled subgraphs and aggregated features) into the GPU memory for training. At the same time, the training manager 121 will start new sampling and aggregation for the next training.

[0075] Specifically, the steps of loading the sampling results and aggregation results as batch data into the memory of the GPU device and using the sampling results and aggregation results to perform module computation of the graph neural network model may include: storing the sampling results sampled by all samplers 1411 and the aggregation results aggregated by all aggregators 1412 as batch data in the memory of the host device; loading the sampling results and aggregation results stored in the memory of the host device into the memory of the GPU device (e.g., DRAM); and using the sampling results and aggregation results loaded into the memory of the GPU device to perform module computation of the graph neural network model.

[0076] For details, please refer to Figure 7During the training of the GNN, the data interaction process between the various modules is as follows: First, the training manager 121 in the host CPU 120 sends a sampling instruction to the sampler 1411. The sampler 1411 in the CMM-DC device 141 samples the stored subgraph and informs the training manager 121 after sampling is complete. The training manager 121 sends an aggregation instruction to the aggregator 1412. The aggregator 1412 in the CMM-DC device 141 performs feature aggregation and informs the training manager 121 after aggregation is complete. The sampling and aggregation results are stored in the host memory. After the training manager 121 learns that the sampler 1411 and aggregator 1412 have completed their execution, it instructs the data loader 123 to preload the sampling and aggregation results as batch data into the memory of the GPU 130. Then, the GPU 130 begins to perform model calculations and, after the calculations are completed, feeds back at least part of the results to the host CPU 120. The training manager 121 can then schedule the next round of sampling and aggregation tasks to ensure the smooth progress of the entire training process. The following describes the relevant steps performed by each device involved in GNN training.

[0077] CMM-DC Equipment Pool The CMM-DC device pool 140 can perform at least one of distributed storage, sampling, and aggregation.

[0078] Figure 8 This is a flowchart illustrating a training optimization method according to a first embodiment of the present disclosure.

[0079] The training optimization method according to the first embodiment of this disclosure is executed by a CMM-DC device pool 140, which includes multiple CMM-DC devices 141, and the multiple CMM-DC devices 141 can be connected to each other via a CXL switch.

[0080] The training optimization method according to a first embodiment of this disclosure may include: separately storing subgraph data partitioned by the training manager in the host CPU 120 and corresponding feature data using multiple CMM-DC devices. As an example, the partitioned subgraph data and its feature data may be stored in the same CMM-DC device 141, a simple and easy storage method that reduces cross-device communication. As an example, the partitioned subgraph data and its feature data may be stored separately in different CMM-DC devices, a storage method that can expand resource utilization efficiency and improve parallel processing capabilities. Although not shown, the method performed by the CMM-DC device pool 140 of this disclosure may also include the step of acquiring subgraph data and corresponding feature data.

[0081] Reference Figure 8The training optimization method according to the first embodiment of this disclosure may further include steps S810 and S820.

[0082] In step S810, in response to the sampling request from the training manager, the stored corresponding subgraph data is sampled to obtain sampling results. The sampler 1411 can select a subset of nodes in the graph as samples according to specific rules used in multi-GPU mini-batch training, which can accelerate model training and improve training results. After receiving the sampling request from the training manager, the sampler 1411 can call the configured graph sampling algorithm to select nodes and store the sampled graph nodes in the host memory.

[0083] In step S820, in response to the aggregation request from the training manager 121, the CMM-DC device 141 performs feature aggregation on the feature data of the sampled nodes in the sampling results to obtain the aggregation result. The aggregator 1412 collects the features of all neighboring nodes and uses an aggregation function to generate an aggregation result for each sampled node. For example, after receiving the aggregation request from the training manager 121, the aggregator 1412 will call the configured aggregation algorithm to calculate the feature result.

[0084] The training optimization method according to the first embodiment of this disclosure may further include: sending sampling results and aggregation results as batch data to the memory of the host device.

[0085] In addition to updating the calculated features locally, the CMM-DC device 141 can also send the aggregated feature results to the host memory.

[0086] According to embodiments of this disclosure, a CMM-DC device pool 140 may include multiple CMM-DC devices. These multiple CMM-DC device pools can be interconnected via a CXL switch 150, and the multiple CMM-DC devices can separately store subgraph data partitioned by the training manager of the host device and corresponding feature data. Additionally, as described above, the CMM-DC device pool may also include samplers and aggregators. Each CMM-DC device may include a sampler 1411 and an aggregator 1412. However, this is merely an example; some CMM-DC devices may include samplers 1411, and some may include aggregators 1412. All samplers 1411 can sample the stored corresponding subgraph data in response to a sampling request from the training manager 121 in the host CPU 120 to obtain sampling results. All aggregators 1412 can perform feature aggregation on the feature data of the sampled nodes in the sampling results in response to an aggregation request from the training manager to obtain aggregation results.

[0087] host device The host device can perform at least one of partitioning and bulk data preloading.

[0088] Figure 9 This is a flowchart illustrating a training optimization method according to a second embodiment of the present disclosure.

[0089] Reference Figure 9 The training optimization method according to the second embodiment of the present disclosure may include steps S910 and S920.

[0090] In step S910, the training manager 121 of the host device partitions the graph data and the feature data corresponding to the graph data. The specific partitioning method can be as described above, and will not be repeated here.

[0091] In step S920, the sampling results sampled by the samplers 1411 in the multiple CMM-DC devices 141 in the CMM-DC device pool 140 and the aggregation results aggregated by the aggregators 1412 in the multiple CMM-DC devices 141 are stored in the memory of the host device, and the multiple CMM-DC devices 141 separately store the graph data and feature data partitioned by the training manager 121.

[0092] The training optimization method according to the second embodiment of this disclosure may further include: loading the sampling results and aggregation results, which are batch data, stored in the memory of the host device, into the memory of the GPU device via a data loader 123 in the host CPU 120. This step may be specifically performed by the data loader 123 in the host CPU 120.

[0093] The training optimization method according to the second embodiment of this disclosure may further include: triggering the GPU to perform training in response to a batch of data being loaded into the memory of the GPU device and the GPU 130 being available. This step may specifically be executed by the training manager in the CPU of the host device.

[0094] The host device according to embodiments of the present disclosure may include a processing unit (e.g., CPU) and a storage unit (e.g., DRAM), wherein the storage unit stores programs or instructions that, when executed by the processing unit, cause the processing unit to execute the training optimization method described above.

[0095] GPU devices GPU devices can be used to perform modular computations.

[0096] Figure 10 This is a flowchart illustrating a training optimization method according to a third embodiment of the present disclosure.

[0097] The training optimization method according to the third embodiment of this disclosure may include step S1010.

[0098] In step S1010, in response to training instructions from the training manager 121 in the host CPU 120, module computation of the graph neural network model is performed using the sampling results and aggregation results loaded from the host device's memory. The sampling results and aggregation results stored in the host device's memory as batch data are respectively the sampling results sampled by samplers 1411 in multiple CMM-DC devices 141 in the CMM-DC device pool 140 and the aggregation results aggregated by aggregators 1412 in the CMM-DC devices. The multiple CMM-DC devices 141 separately store the graph data and feature data (i.e., subgraph data and their feature data) partitioned by the training manager 121. The GPU can use the batch data preloaded into the GPU memory for module computation.

[0099] A GPU device according to an embodiment of the present disclosure may include: processing circuitry (e.g., a GPU) and a non-transitory computer-readable storage medium (e.g., GPU memory), the non-transitory computer-readable storage medium storing a program or instructions that, when executed by the processing circuitry, cause the processing circuitry to perform the training optimization method described above.

[0100] The above has been referred to Figures 1 to 10 Training optimization methods, etc., according to exemplary embodiments of the present disclosure are described. However, it should be understood that the devices, units, apparatuses, etc., shown in the drawings can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, these units and devices may correspond to dedicated integrated circuits, pure software code, or modules combining software and hardware. Furthermore, one or more functions implemented by these systems or devices may also be uniformly executed by components in physical entity devices (e.g., processors, clients, or servers).

[0101] Instructions or programs corresponding to the training optimization methods according to the above embodiments of this disclosure may be stored in a computer-readable storage medium. A computer system according to embodiments of this disclosure may include at least one of the above-described CMM-DC device pool, host device, and GPU device.

[0102] The instructions stored in the aforementioned computer-readable storage medium can be executed in environments deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the instructions can also be used to perform additional steps beyond those described above, or to perform more specific processing while executing the aforementioned steps. The details of these additional steps and further processing are already provided in the reference... Figures 1 to 10 As mentioned in the description of the relevant apparatus and methods, they will not be repeated here to avoid repetition.

[0103] It should be noted that the optimization method according to the exemplary embodiments of this disclosure may rely entirely on the operation of computer programs or instructions to achieve the corresponding functions. That is, each device corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., a lib library) to achieve the corresponding functions.

[0104] According to embodiments of this disclosure, subgraph sampling and feature aggregation are performed in a CMM-DC device, requiring only the results of sampling and aggregation, which greatly reduces data movement between the CPU and memory during feature retrieval.

[0105] According to embodiments of this disclosure, subgraph sampling and feature retrieval are offloaded to the CMM-DC device, the CPU is only responsible for training scheduling and preloading data, and the GPU only focuses on module computation.

[0106] According to embodiments of this disclosure, subgraph sampling and feature aggregation are performed in parallel on multiple CMM-DC devices, reducing data preparation time and improving training performance.

[0107] According to embodiments of this disclosure, subgraphs and features are partitioned and stored on multiple CMM-DC devices, reducing host and GPU memory usage and meeting the memory requirements for GNN training on large-scale graphs.

[0108] While specific terminology has been used to describe various embodiments of this disclosure, the specification and drawings are to be regarded as illustrative rather than restrictive in order to aid in understanding this disclosure. Various modifications and changes can be made by those skilled in the art (e.g., different features in different embodiments may be combined) without departing from the broader spirit and scope of this disclosure. Therefore, the scope of this disclosure is not limited by the specific embodiments and examples, but by the claims and their equivalents.

Claims

1. A training optimization method for graph neural networks, characterized in that, The training optimization method is executed by a CMM-DC device pool, which includes multiple CMM-DC devices. The training optimization method includes: Receives multiple sub-graph data partitioned by a host device, wherein the host device is connected to the multiple CMM-DC devices; Receive feature data corresponding to the plurality of subgraph data; The multiple subgraph data and the feature data are stored separately using multiple CMM-DC devices.

2. The training and optimization method for graph neural networks according to claim 1, characterized in that, The training optimization method further includes: In response to a sampling request from the training manager in the host device, the stored corresponding subgraph data is sampled to obtain sampling results; In response to the aggregation request from the training manager, feature aggregation is performed on the feature data of the sampling nodes in the sampling results to obtain the aggregation result.

3. The training and optimization method for graph neural networks according to claim 2, characterized in that, The training optimization method further includes sending the sampling results and the aggregation results as batch data to the memory of the host device.

4. The training and optimization method for graph neural networks according to claim 1, characterized in that, Each CMM-DC device stores a portion of the multiple subgraph data and a portion of the feature data.

5. The training and optimization method for graph neural networks according to claim 1, characterized in that, One portion of the plurality of CMM-DC devices is used to store the plurality of subgraph data, and another portion of the plurality of CMM-DC devices is used to store feature data.

6. A training optimization method for a graph neural network, characterized in that, The training optimization method is executed by the host device, and the training optimization method includes: The host device's training manager partitions the graph data and the feature data corresponding to the graph data. The sampling results sampled by the samplers in multiple CMM-DC devices in the CMM-DC device pool and the aggregation results aggregated by the aggregators in the multiple CMM-DC devices are stored in the memory of the host device. Multiple CMM-DC devices are configured to separately store graph data and feature data partitioned by the training manager.

7. The training and optimization method for graph neural networks according to claim 6, characterized in that, The sampling results and the aggregation results are stored as batch data in the memory of the host device, wherein the training optimization method further includes: The sampling results and the aggregation results are loaded into the memory of the GPU device by the data loader in the CPU of the host device.

8. The training and optimization method for graph neural networks according to claim 7, characterized in that, The training optimization method further includes: In response to the batch data being loaded into the memory of the GPU device and the GPU being available, the GPU is triggered to perform training.

9. A training optimization method for a graph neural network, characterized in that, The training optimization method is executed by a GPU device, and the training optimization method includes: In response to training instructions from the training manager on the host device, the module computation of the graph neural network model is performed using the sampling and aggregation results loaded from the host device's memory. The sampling results and aggregation results are stored as batch data in the memory of the host device. The sampling results are sampled by samplers from multiple CMM-DC devices in the CMM-DC device pool. The aggregation results are aggregated by the aggregator in the CMM-DC device, and The multiple CMM-DC devices are configured to store graph data and feature data partitioned by the training manager separately.

10. A CMM-DC device pool, characterized in that, The CMM-DC device pool includes multiple CMM-DC devices configured to perform the training optimization method according to any one of claims 1-5.

11. A host device, characterized in that, The host device includes: a processor and a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform the training optimization method according to any one of claims 6 to 8.

12. A GPU device, characterized in that, The GPU device includes: a processor and a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform the training optimization method according to claim 9.

13. A computer system, characterized in that, The computer system includes: the CMM-DC device pool according to claim 10, the host device according to claim 11, and the GPU device according to claim 12.

14. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the training optimization method for a graph neural network according to any one of claims 1-9.

Images