Large-scale graph embedding training method and system based on Optane DIMM

Abstract

The present invention provides a large-scale graph embedding training method and system based on Optane DIMM, comprising: an original graph processing step: processing the original graph to generate graph data that can be loaded into DRAM; a data preprocessing step: performing two-layer graph segmentation on the graph data based on features, splitting the complete graph into subgraphs, and storing them on disk, so that the subgraphs can be loaded into the GPU for partitioned training; a graph training step: storing the graph data used for training on different physical media based on the memory access characteristics of different media, dividing the algorithm based on the different characteristics of the data relied on during the training process, using the CPU and GPU to divide the training work, and balancing the overhead of CPU computing, GPU computing, and CPU-GPU communication. The present invention performs high-quality two-layer graph segmentation based on the characteristics of the graph, converting the large graph into subgraphs, and storing them on disk, so that the subgraphs can be loaded into the GPU for partitioned training.

Description

Technical Field

[0001] The present invention relates to the fields of computer storage, computer computing and deep learning technology, and in particular to a large-scale graph embedding training method and system based on Optane DIMM. Background Art

[0002] Graphs, such as social networks, word co-occurrence networks, and communication networks, are widely used in various real-world applications. By analyzing them, we can gain insights into social structures, languages, and different communication patterns, and thus Figure 1 It has always been a hot topic of research in academia.

[0003] The network representation using the adjacency matrix has computational efficiency issues. The adjacency matrix A uses a storage space of |V|×|V| to represent a graph. As the number of nodes increases, the space required for this representation grows exponentially. At the same time, the vast majority of the adjacency matrix is ​​0, and the sparsity of the data makes it difficult to apply fast and effective learning methods. Graph embedding learning refers to learning to obtain a low-dimensional vector representation of the nodes in the network. Formally speaking, the goal of graph embedding learning is to learn a real-valued vector for each node v∈V. Where κ<<|V| represents the dimension of the vector. The mathematical definitions of graphs and graph embeddings are as follows:

[0004] Graph: Graph G(V, E) is a vertex set V = {v1, ..., v n} and the set of edge sets E. e ij ∈E contains a source vertex v i and a target vertex v j For a weighted graph G, the adjacency matrix W contains the non-negative weights W associated with each edge. ij ≥ 0. If v i and v j If there is no connection, W ij Set to 0. For undirected weighted graphs, there is always

[0005] Graph Embedding: Given a graph G(V, E) and a predefined dimension of embedding d, a graph (node) embedding is a mapping f: The function f preserves some semantic features defined on the graph G. The graph embedding problem is to map the entire graph, subgraph or edge to a d-dimensional vector. The visual representation of graph embedding is as follows: Figure 1 shown.

[0006] Graph embedding technology is a technique in graph analysis and representation learning. Its purpose is to represent a graph as a low-dimensional vector while preserving its structure. As a simple and effective method for reducing dimensionality, graph embedding has been widely used in fields such as node classification, clustering, recommendation, link prediction, and network visualization.

[0007] Graph embedding algorithms often use random walks to measure similarity. Random walks are also the basis of a class of output-sensitive algorithms that use random walks to compute local structural information with a complexity linear in the size of the input graph. It is this connection with local structure that has led to random walks being used as a fundamental tool for extracting information from graphs. In addition to capturing community information, using random walks as the basis for algorithms has two other desirable properties. First, local exploration is easily parallelized. Several random walkers (on different threads, processes, or machines) can explore different parts of the same graph at the same time. Second, relying on information obtained from short random walks, small changes in the graph structure can be adapted without the need for global recalculation. We can iteratively update the learned model with new random walks, from regions that change sublinearly in time to the entire graph. Therefore, the present invention uses random walks as a basis to extract graph features.

[0008] In the era of big data, graph sizes continue to increase. For example, in social networks, the number of nodes has grown to billions, and the number of edges has also increased to tens of billions. Efficiently processing graphs of this size remains a challenge. For large graphs with hundreds of millions of vertices, it is difficult for a typical single-node server to guarantee sufficient DRAM capacity to support services. While existing swap partitioning technologies can create logical memory larger than physical memory for a machine, achieving terabyte-level capacity remains difficult and is limited by slow disk access speeds, resulting in low efficiency.

[0009] Another reference solution is to store graph data and embedding data on disk. However, since graph embedding algorithms often require byte addressing and disk access latency is high, the possibility of using disk to build a high-performance graph embedding system solution is low. Distributed solutions have limitations in efficiency, cost, and user-friendliness: First, graph data and embedding data need to be frequently accessed and transmitted between different machines. This will result in high network communication costs and latency. Second, purchasing a group of powerful machines is expensive, which is a burden for small companies and individual developers. Therefore, how to build an efficient large-scale graph embedding system on a single machine has become a current challenge and opportunity. Non-volatile memory is another direction for single machines to solve this problem.

[0010] The arrival of new hardware, Optane DIMMs, brings hope for efficient, large-scale embedding training on a single machine. Intel's Optane DIMMs are the first commercially available persistent memory, supporting byte-level access in the same order as DRAM. A single Optane DIMM can have a storage capacity of up to 512GB. For dual-socket machines, supported Optane DIMM sizes can reach up to 6TB (2 sockets * 6 channels * 512GB / DIMM).

[0011] In a Chinese patent document with publication number CN113343123A, a training method for a generative adversarial multi-relationship graph network model for detecting machine accounts is disclosed, the training method comprising: modeling a platform into a graph comprising nodes v and relationships r, wherein the number of the graphs is determined by the number of types of the relationships r; using a generator G to generate a false target node vt for the source node v; inputting the sampled node pairs (v, u) and (v, vt) into a connection relationship discriminator D respectively, and repeatedly training the connection relationship discriminator D; using the trained connection relationship discriminator D to infer the node pairs in the graph, determining the connection relationship of the node pairs, and then updating the structure of the graph; inputting the representation vector of the node into a classifier, back-propagating and updating the parameters of the model according to the loss function, performing multiple trainings, and obtaining a trained generative adversarial multi-relationship graph network model. Summary of the Invention

[0012] In view of the defects in the prior art, the purpose of the present invention is to provide a large-scale graph embedding training method and system based on Optane DIMM.

[0013] According to the present invention, a large-scale graph embedding training method based on Optane DIMM is provided, comprising:

[0014] Original image processing step: Process the original image to generate image data that can be loaded into DRAM;

[0015] Data preprocessing step: The graph data is segmented into two layers according to the features, the complete graph is split into subgraphs, and the subgraphs are stored on disk so that they can be loaded into the GPU for partitioned training;

[0016] Graph training steps: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. The algorithm is divided according to the different characteristics of the data relied on during training. The CPU and GPU are used to divide the training work and balance the overhead of CPU computing, GPU computing, and CPU-GPU communication.

[0017] Preferably, the original image processing step includes the following sub-steps:

[0018] S1: Initialize a hash table of size hash_table_size to store the mapping from vertex names to vertex IDs;

[0019] S2: Enter the loop, the number of loops is the size of the original graph list;

[0020] S3: Each loop reads an edge from the original graph, consisting of vertex v_name and target vertex u_name;

[0021] S4: Check if v_name has appeared in the hash table. If so, return the mapped v_id. If not, call the hash_table.AddVertex(name_v, count_num_vertices) method to insert a new vertex.

[0022] S5: If hash_table.AddVertex(name_v, count_num_vertices) is called, a new vertex is first created and added to the vertex set vertices, and then the counter count_num_vertices is incremented by one; if count_num_vertices exceeds the maximum capacity of the existing vertices, vertices will be automatically expanded;

[0023] S6: Loop until an empty space in the hash table is found, and insert the mapping relationship between vertex name and ID into the hash table;

[0024] S7: process u_name according to the steps in S4-S6;

[0025] S8: Write the mapped edges to the output file.

[0026] Preferably, the graph data is stored on a disk in the form of a file, and the data format is source_vertex_id, destination_vertex_id; for an undirected graph, source_vertex_id, destination_vertex_id; destination_vertex_id, source_vertex_id are stored simultaneously to represent two edges.

[0027] Preferably, the splitting of the graph data includes:

[0028] - Use edge splitting strategy: Divide the graph data by points, separate the edges, and divide the subgraph data into the number of GPUs. For devices with a single GPU, no edge splitting is performed;

[0029] - Use point splitting strategy: Divide the graph data by edges. The number of subgraphs is determined by the GPU memory size. When the subgraph can be fully loaded into the GPU, the point splitting strategy is not used.

[0030] Preferably, the CPU performs negative sampling and edge cutting training operations, the GPU performs positive sampling, positive sample training and negative sample training operations, and the CPU-GPU communication is performed using PCI load.

[0031] Preferably, the CPU tasks specifically include:

[0032] Data loading: Load the data required for training, including graph structure data and graph embedding data;

[0033] The graph structure data is loaded from the disk into the Optane DIMM, and a metadata structure called Graph is abstracted to point to the specific data in the Optane DIMM. The metadata is stored in DRAM.

[0034] The graph embedding data allocates memory and initializes it in DRAM, abstracts the metadata structure Embedding, points to the specific data, and stores it in DRAM;

[0035] Negative sampling: The system starts FIRST_PARTITION_NUM threads, each thread maintains NEG_SAMPLE_POOL_NUM block sampling pools, the sampling pool size is NEG_SAMPLE_POOL_SIZE, and different threads perform negative sampling in parallel;

[0036] Edge training: For each edge generated during edge segmentation, use the CPU for training; the CPU opens CROSS_EDGE_TRAIN_THREAD threads, each thread is responsible for training part of the edge

[0037] Task scheduling: The first layer of subgraphs are trained in parallel by different GPUs, with data isolation between different GPUs and no data communication overhead; the second layer of subgraphs are trained serially;

[0038] Graph Embedding Evaluation: We use the generated node embeddings to run different types of machine learning tasks, evaluate micro-f1 or macro-f1, and compare with other solutions.

[0039] Preferably, in the negative sampling process, the sampling strategy for each small sampling pool includes: counting the degree sum of the whole image points and the degree sum of the sub-image points, and determining the number of samples for each sub-image according to the ratio of the sub-image degree sum to the whole image degree sum, and satisfying the following conditions:

[0040] subgraph i- degree_num / graph_degree_num

[0041] =subgraph i- neg_sam_num / NEG_SAMPLE_POOL_SIZE.

[0042] Preferably, the GPU task includes:

[0043] Positive sampling: For each edge, positive sampling is performed by random walk. The input is the vertex vid and the output is a list of positively sampled edges.

[0044] Training: For each edge in the subgraph, use GPU for training.

[0045] Preferably, the PCI load task includes:

[0046] Graph structure data transmission: Graph structure data is copied from the Optane DIMM to the GPU for calculation via the PCIe protocol;

[0047] Negative sample transfer: Graph structure data is copied from host DRAM to the GPU for calculation via the PCIe protocol;

[0048] Embedded transmission: Graph structure data is copied from host DRAM to GPU for calculation via PCIe protocol;

[0049] Embedded transfer: The embedded data is copied from the GPU back to the host DRAM via the PCIe protocol.

[0050] According to the present invention, a large-scale graph embedding training system based on Optane DIMM is introduced, including:

[0051] Original image processing module: processes the original image and generates image data that can be loaded into DRAM;

[0052] Data preprocessing module: This module splits the graph data into two layers based on features, splits the complete graph into subgraphs, and stores them on disk so that the subgraphs can be loaded into the GPU for partitioned training.

[0053] Graph training module: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. The algorithm is divided according to the different characteristics of the data relied on during the training process. The CPU and GPU are used to divide the training work, balancing the overhead of CPU computing, GPU computing, and CPU-GPU communication.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The present invention performs high-quality two-layer graph segmentation based on the characteristics of the graph, converts the large graph into subgraphs, and stores them on disk, so that the subgraphs can be loaded into the GPU for partition training.

[0056] 2. According to the memory access characteristics of different media, the present invention stores the graph data used for training in different physical media respectively, stores frequently written data in DRAM, and stores read-only data in Optane DIMM.

[0057] 3. The present invention adopts a cutting algorithm based on the different characteristics of the data relied on in the algorithm process. The GPU processes positive sampling (random walk), positive sample training and negative sample training, and the CPU processes negative sampling and edge cutting training, thereby making full use of the system computing power.

[0058] 4. The asynchronous transmission in this invention balances computing and IO overhead, thereby building a high-performance end-to-end training complete execution flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0060] Figure 1 Schematic diagram of the image embedding process in the background technology;

[0061] Figure 2 Schematic diagram of the overall method of an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0063] According to the large-scale graph embedding training method based on Optane DIMM provided by the present invention, refer to Figure 1 ,include:

[0064] Original image processing step: Process the original image to generate image data that can be loaded into DRAM;

[0065] Data preprocessing step: The graph data is segmented into two layers according to the features, the complete graph is split into subgraphs, and the subgraphs are stored on disk so that they can be loaded into the GPU for partitioned training;

[0066] Graph training steps: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. The algorithm is divided according to the different characteristics of the data relied on during training. The CPU and GPU are used to divide the training work and balance the overhead of CPU computing, GPU computing, and CPU-GPU communication.

[0067] The number of edges in the original graph may be huge, exceeding the size of DRAM. To overcome this limitation, this system designs a streaming hash mapping solution that does not require loading the entire graph data into the memory. The original graph processing step includes the following sub-steps:

[0068] S1: Initialize a hash table of size hash_table_size to store the mapping from vertex names to vertex IDs;

[0069] S2: Enter the loop, the number of loops is the size of the original graph list;

[0070] S3: Each loop reads an edge from the original graph, consisting of vertex v_name and target vertex u_name;

[0071] S4: Check if v_name has appeared in the hash table. If so, return the mapped v_id. If not, call the hash_table.AddVertex(name_v, count_num_vertices) method to insert a new vertex.

[0072] S5: If hash_table.AddVertex(name_v, count_num_vertices) is called, a new vertex is first created and added to the vertex set vertices, and then the counter count_num_vertices is incremented by one; if count_num_vertices exceeds the maximum capacity of the existing vertices, vertices will be automatically expanded;

[0073] S6: Loop until an empty space in the hash table is found, and insert the mapping relationship between vertex name and ID into the hash table;

[0074] S7: process u_name according to the steps in S4-S6;

[0075] S8: Write the mapped edges to the output file.

[0076] Furthermore, the graph data is stored on the disk in the form of a file, and the data format is source_vertex_id, destination_vertex_id; for an undirected graph, source_vertex_id, destination_vertex_id; destination_vertex_id, source_vertex_id two edges are stored at the same time to represent it.

[0077] Graph partitioning divides a large complete graph into small subgraphs so that they can be loaded onto the GPU for computation. The graph data splitting strategy includes the following two layers:

[0078] The first layer uses an edge splitting strategy: the subgraph data is divided according to the number of GPUs. For devices with a single GPU, no edge splitting is performed. Edge splitting refers to dividing the data by points during the partitioning process, cutting the edges. The advantage is that the points in the subgraph are unique, which reduces data synchronization communication during training. The overhead is the generated cut edges, which require additional processing during training.

[0079] The second level uses a point-based splitting strategy: the number of subgraphs is determined by the GPU memory capacity. If a subgraph can be fully loaded onto the GPU, point-based splitting is not performed. Therefore, if the graph is too small to be fully loaded onto the GPU, the two-level graph splitting will not actually be performed. Point-based splitting involves dividing the graph along edges, resulting in multiple copies of vertices. The advantage is that this does not incur the overhead of cutting edges; however, the disadvantage is that multiple copies of vertices are generated, which may incur synchronization communication overhead.

[0080] Furthermore, during the graph data partitioning process, a graph data compression mechanism is run between and within partitions. For example, in a complete graph, the vertex ID is large and needs to be stored using the long data type, with each vertex occupying 8 bytes. After partitioning, the vertex can be<first_partition_id,second_partition_id,offset> Triple storage, data type is byte, each vertex occupies 3 bytes.

[0081] Based on the different characteristics of the data relied upon during training, the present invention adopts a cutting algorithm. The GPU is responsible for positive sampling (random walk), positive sample training, and negative sample training, while the CPU is responsible for negative sampling and cutting edge training. This balances the overhead of CPU computing, GPU computing, and CPU-GPU communication, thereby constructing a high-performance end-to-end training complete execution flow. Specifically, the CPU performs negative sampling and cutting edge training operations, the GPU performs positive sampling, positive sample training, and negative sample training operations, and the CPU-GPU communication is performed using PCI load.

[0082] The task allocation is shown in the following table:

[0083]

[0084]

[0085] In more detail, the CPU tasks specifically include:

[0086] Data loading: The first step in the training workflow is data loading. The data in this system consists of two types. The first type is graph-structured data (nodes and edges), which stores the structural relationships of the graph. This data is large in size and is read-only during training. The second type is graph-embedded data, which maintains an embedding vector for each graph vertex. This represents the training target and result, is frequently read and written, and has a smaller data size.

[0087] After being segmented, the graph structure data is stored on disk. During the loading process, the graph structure data is loaded from the disk into the Optane DIMM, and a metadata structure called Graph is abstracted to point to the specific data in the Optane DIMM. The metadata is stored in DRAM.

[0088] Graph embedding data is allocated memory and initialized in DRAM, and a metadata structure called Embedding is abstracted to point to specific data and stored in DRAM.

[0089] Negative sampling: The system starts FIRST_PARTITION_NUM threads. Each thread maintains a NEG_SAMPLE_POOL_NUM block sampling pool with a sampling pool size of NEG_SAMPLE_POOL_SIZE. Different threads perform negative sampling in parallel.

[0090] For each small sampling pool, the sampling strategy includes: counting the degree sum of the whole graph points and the degree sum of the subgraph points, and determining the number of samples for each subgraph based on the ratio of the subgraph degree sum to the whole graph degree sum, and meeting the following conditions:

[0091] subgraph i- degree_num / graph_degree_num

[0092] =subgraph i- neg_sam_mum / NEG_SAMPLE_POOL_SIZE

[0093] The negative sampling module uses a producer-consumer model, with the CPU sampling as the producer and the GPU training as the consumer. When the sampling pool is full, the CPU sampling thread is blocked, allowing the GPU training thread to execute. When the sampling pool is empty, the CPU sampling thread can execute, but the GPU training thread is blocked. The value of NEG_SAMPLE_POOL_NUM determines the load balance between the two, while NEG_SAMPLE_POOL_SIZE affects training quality. Both parameters are user-controllable.

[0094] Edge cutting training: For each edge cut generated during edge segmentation, the CPU is used for training. The CPU starts CROSS_EDGE_TRAIN_THREAD threads, each of which is responsible for training part of the edge cut.

[0095] The specific strategy is as follows: the CPU starts CROSS_EDGE_TRAIN_THREAD threads, each thread is responsible for training part of the cut edge; for each cut edge, a random walk is performed on source_vertex to generate positive samples; for each positive sample, a vertex is randomly sampled globally to replace target_vertex to generate a negative sample; for each positive sample, num_negative negative samples are sampled; gradient descent is performed on positive and negative samples to achieve the training effect.

[0096] Task Scheduling: This invention uses a cutting algorithm based on the different characteristics of the data used during training. The GPU is responsible for positive sampling (random walks), positive sample training, and negative sample training, while the CPU is responsible for negative sampling and edge cutting training. Algorithm scheduling and data communication are centrally controlled by the CPU.

[0097] Specific strategy: The first-layer subgraphs are trained in parallel on different GPUs, with data isolation between different GPUs and no data communication overhead. The second-layer subgraphs are trained serially. The negative sampling module adopts a producer-consumer model, with the CPU sampling as the producer and the GPU training as the consumer. When the sampling pool is full, the CPU sampling thread is blocked, and the GPU training thread can execute. When the sampling pool is empty, the CPU sampling thread can execute, and the GPU training thread is blocked. Asynchronous transmission overlaps the GPU training overhead with the CPU-GPU data transmission overhead. Specifically, while subgraph P1 is being trained on the GPU, subgraph P2 is transferred from the host (Optane DIMM) to the GPU.

[0098] Graph Embedding Evaluation: Use the generated node embeddings to run different types of machine learning tasks, such as node classification. Evaluate micro-f1 or macro-f1 and compare with other solutions.

[0099] In more detail, the GPU tasks include:

[0100] Positive sampling: For each edge, positive sampling is performed by random walk. The sampling pseudo code is as follows:

[0101]

[0102]

[0103] The input is vertex vid, and the output is a list of positively sampled edges. Line 4 initializes walk_length to 0. Line 5 enters a loop, which terminates when counter i reaches random_walk_length times. In the loop, line 9 obtains the list of neighbor vertices of vertex vid; line 10 determines whether the neighbor list is empty. If so, the loop is exited; line 13 randomly samples neighbor vertices, with the probability of being sampled determined by the vertex weight, and the sampled vertex is assigned to uid; line 14 assigns vid to head[i]; line 15 assigns uid to tail[i]; line 16 assigns uid to vid; and lines 17 and 18 increment the variables i and walk_length by one, respectively.

[0104] Training: For each edge in the subgraph, use the GPU for training. The specific strategy is as follows: the GPU starts NUM_EDGES threads, each responsible for training one edge; for each cut edge, perform a random walk on source_vertex to generate positive examples; for each positive example, replace target_vertex with a vertex sampled from the negative sample pool to generate a negative example; for each positive example, sample num_negative negative examples; and perform gradient descent on the positive and negative examples to achieve the training effect.

[0105] In more detail, the PCI load tasks include:

[0106] Graph structure data transmission: host (Optane DIMM) to GPU;

[0107] The graph structure data is copied from the Optane DIMM to the GPU for calculation via the PCIe protocol;

[0108] Negative sample transfer: host (DRAM) to GPU;

[0109] The graph structure data is copied from the host DRAM to the GPU for calculation via the PCIe protocol;

[0110] Embedded transfer: host (DRAM) to GPU;

[0111] The graph structure data is copied from the host DRAM to the GPU for calculation via the PCIe protocol;

[0112] Embedded transfer: GPU to host (DRAM);

[0113] The embedded data is copied from the GPU back to the host DRAM via the PCIe protocol.

[0114] The present invention also introduces a large-scale graph embedding training system based on Optane DIMM, comprising:

[0115] Original image processing module: processes the original image and generates image data that can be loaded into DRAM;

[0116] Data preprocessing module: This module splits the graph data into two layers based on features, splits the complete graph into subgraphs, and stores them on disk so that the subgraphs can be loaded into the GPU for partitioned training.

[0117] Graph training module: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. The algorithm is divided according to the different characteristics of the data relied on during the training process. The CPU and GPU are used to divide the training work, balancing the overhead of CPU computing, GPU computing, and CPU-GPU communication.

[0118] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0119] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A large-scale graph embedding training method based on Optane DIMM, characterized by: include: Original image processing step: Process the original image to generate image data that can be loaded into DRAM; Data preprocessing step: The graph data is segmented into two layers according to the features, the complete graph is split into subgraphs, and the subgraphs are stored on disk so that they can be loaded into the GPU for partitioned training; Graph training steps: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. Based on the different characteristics of the data relied on during training, a segmentation algorithm is used, and the CPU and GPU are used to divide the training work, balancing the overhead of CPU computing, GPU computing, and CPU-GPU communication. The CPU performs negative sampling and edge cutting training operations, the GPU performs positive sampling, positive sample training and negative sample training operations, and the CPU-GPU communication is performed using PCI load.

2. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The original image processing step includes the following sub-steps: S1: Initialize a hash table of size hash_table_size to store the mapping from vertex names to vertex IDs; S2: Enter the loop, the number of loops is the size of the original graph list; S3: Each loop reads an edge from the original graph, consisting of vertex v_name and target vertex u_name; S4: Check if v_name has appeared in the hash table. If so, return the mapped v_id. If not, call the hash_table.AddVertex(name_v, count_num_vertices) method to insert a new vertex. S5: If hash_table.AddVertex(name_v, count_num_vertices) is called, a new vertex is first created and added to the vertex set vertices, and then the counter count_num_vertices is incremented by one; if count_num_vertices exceeds the maximum capacity of the existing vertices, vertices will be automatically expanded; S6: Loop until an empty space in the hash table is found, and insert the mapping relationship between vertex name and ID into the hash table; S7: process u_name according to the steps in S4-S6; S8: Write the mapped edges to the output file.

3. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The graph data is stored on the disk in the form of a file, and the data format is source_vertex_id, destination_vertex_id; for an undirected graph, source_vertex_id, destination_vertex_id; destination_vertex_id, source_vertex_id two edges are stored at the same time to represent it.

4. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The splitting of the graph data includes: - Use edge splitting strategy: Divide the graph data by points, separate the edges, and divide the subgraph data into the number of GPUs. For devices with a single GPU, no edge splitting is performed; - Use point splitting strategy: Divide the graph data by edges. The number of subgraphs is determined by the GPU memory size. When the subgraph can be fully loaded into the GPU, the point splitting strategy is not used.

5. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The CPU tasks specifically include: Data loading: Load the data required for training, including graph structure data and graph embedding data; The graph structure data is loaded from the disk into the Optane DIMM, and a metadata structure called Graph is abstracted to point to the specific data in the Optane DIMM. The metadata is stored in DRAM. The graph embedding data allocates memory and initializes it in DRAM, abstracts the metadata structure Embedding, points to the specific data, and stores it in DRAM; Negative sampling: The system starts FIRST_PARTITION_NUM threads, each thread maintains NEG_SAMPLE_POOL_NUM block sampling pools, the sampling pool size is NEG_SAMPLE_POOL_SIZE, and different threads perform negative sampling in parallel; Edge training: For each edge generated during edge segmentation, use the CPU for training; the CPU opens CROSS_EDGE_TRAIN_THREAD threads, each thread is responsible for training part of the edge Task scheduling: The first layer of subgraphs are trained in parallel by different GPUs, with data isolation between different GPUs and no data communication overhead; the second layer of subgraphs are trained serially; Graph Embedding Evaluation: We use the generated node embeddings to run different types of machine learning tasks, evaluate micro-f1 or macro-f1, and compare with other solutions.

6. The Optane DIMM-based large-scale graph embedding training method according to claim 5, characterized in that: During the negative sampling process, the sampling strategy for each sampling pool includes: counting the degree sum of the whole image points and the degree sum of the sub-image points, and determining the number of samples for each sub-image based on the ratio of the sub-image degree sum to the whole image degree sum, and meeting the following conditions: subgraph i _degree_num / graph_degree_num =subgraph i _neg_sam_num / NEG_SAMPLE_POOL_SIZE。 7. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The GPU tasks include: Positive sampling: For each edge, positive sampling is performed by random walk. The input is the vertex vid and the output is a list of positively sampled edges. Training: For each edge in the subgraph, use GPU for training.

8. The Optane DIMM-based large-scale graph embedding training method according to claim 1, characterized in that: The PCI load tasks include: Graph structure data transmission: Graph structure data is copied from Optane DIMM to GPU for calculation via PCIe protocol; Negative sample transfer: The graph structure data is copied from the host DRAM to the GPU for calculation via the PCIe protocol; Embedded transmission: The graph structure data is copied from the host DRAM to the GPU for calculation via the PCIe protocol; Embedded transfer: The embedded data is copied from the GPU back to the host DRAM via the PCIe protocol.

9. A large-scale graph embedding training system based on Optane DIMM, characterized by: include: Original image processing module: processes the original image and generates image data that can be loaded into DRAM; Data preprocessing module: This module splits the graph data into two layers based on features, splits the complete graph into subgraphs, and stores them on disk so that the subgraphs can be loaded into the GPU for partitioned training. Graph training module: Based on the memory access characteristics of different media, the graph data used for training is stored in different physical media. Based on the different characteristics of the data relied on during training, a segmentation algorithm is implemented, and the CPU and GPU are used to divide the training work, balancing the overhead of CPU computing, GPU computing, and CPU-GPU communication. The CPU performs negative sampling and edge cutting training operations, the GPU performs positive sampling, positive sample training and negative sample training operations, and the CPU-GPU communication is performed using PCI load.

Images