Target feature information extraction method and system based on graph representation learning

Abstract

The application provides a target feature information extraction method and system based on graph representation learning. The method is applied to a NUMA architecture and comprises the following steps: determining graph data containing target information; reorganizing the graph data in descending order of vertex degree of the graph data, and merging vertices with the same vertex degree to obtain corresponding vertex blocks, recording the starting offset of each degree corresponding vertex block and the first-order neighbor vertices of each vertex in the vertex block, and storing the graph data in the memory in the format of compressed sparse blocks; reading the sparse matrix stored in the local and / or remote memory of the current NUMA node, multiplying the read sparse matrix with the dense matrix stored in the local memory, writing the intermediate matrix obtained by multiplication into the local memory, and finally merging the intermediate matrices stored on each node to obtain the target feature information; and performing task allocation for the parallel calculation of the sparse matrix and the dense matrix multiplication through an entropy-aware thread allocation strategy. The application improves the graph representation learning target feature extraction efficiency.

Description

Technical Field

[0001] This invention belongs to the field of target feature extraction, and more specifically, relates to a method and system for extracting target feature information based on graph representation learning. Background Technology

[0002] With the increasing informatization of society, graphs, as an important data structure, have been widely used in many fields. For example, in social networks, vertices can represent users, and edges can represent the following or interaction between users; in chemical molecular networks, vertices represent atoms, and edges represent bonds; in e-commerce networks, vertices represent users, and edges represent users' browsing and rating of products, and so on. In recent years, graph data analysis tasks have become increasingly important, and graph representation learning, as an effective graph analysis technique, has also attracted much attention. Techniques such as Deepwalk, Node2vec, LINE, and ProNE aim to map large-scale and high-dimensional graph data into a low-dimensional dense vector space to extract corresponding target feature information while preserving the original graph's structural and attribute features. More importantly, it can be effectively applied to machine learning tasks, such as link prediction, node classification, recommender systems, and visualization.

[0003] Graph representation learning has seen extensive research and application over the past decade. With the continuous growth of graph data scale, higher demands are placed on the efficiency and scalability of graph representation learning methods on large-scale graphs. For example, for graph data with 100 million vertices and 500 million edges, using 20 threads on a mainstream server, LINE takes weeks, while Deepwalk and Node2vec take months to complete graph representation learning. Recently, a fast and scalable graph representation learning method, ProNE, was proposed. This method first generates initial embedding vector representations quickly through sparse matrix factorization. Then, addressing the limitation that sparse matrix factorization can only capture local structural information, ProNE further modulates the spectral space of the graph structure using higher-order Cheeger inequalities. This results in a graph structure that emphasizes both local smoothness and global clustering information. Finally, by propagating the initial vertex embedding vector representations across the adjusted graph structure, local smoothness and global clustering information are fused, leading to the learning of better graph embedding vector representations. Compared to Deepwalk, Node2vec, and LINE, ProNE can provide a performance speedup of 10x to 400x. Similarly, for graph data with 100 million vertices and 500 million edges, ProNE can complete training in just 29 hours using a single thread. However, because ProNE is a graph representation learning technique based on matrix factorization, its training process requires more than 500GB of memory, far exceeding the capacity of ordinary commercial servers. Even with distributed memory support, DRAM faces serious scalability issues in terms of cost.

[0004] Persistent memory (PM), also known as non-volatile memory (NVM), is a new type of storage device that combines the byte-addressability and low latency of DRAM with the persistence and large capacity similar to SSDs. However, it offers even lower latency and higher read / write bandwidth compared to existing storage devices connected to external interfaces such as PCIe (including Optane SSDs). Currently, persistent memory devices (Intel Optane DCPersistent Memory Modules, Optane PM) are commercially available and supported by Intel Cascade Lake processors, appearing on the CPU memory bus as standard addressable DDR4 DIMMs. Like DRAM DIMMs, PMs communicate with the memory controller via a custom protocol compatible with DDR4. However, PMs offer significantly higher storage capacities than DRAM, providing 128GB, 256GB, and 512GB options, and their price per unit capacity is 2.1 times lower than DRAM. This presents a new opportunity for memory-dependent graph representation learning technologies. However, compared to DRAM, PM exhibits a significant performance gap, with write bandwidth and read bandwidth only 1 / 6 and 1 / 3 of DRAM, respectively. This severely impacts the efficiency of graph representation learning on PM. For example, running ProNE on the Twitter graph (11.316 million vertices and 127 million edges), while using PM can save more than 2 times the memory cost (DRAM is approximately $785, PM is approximately $312), its execution time is 4.96 times slower than in a DRAM environment. Therefore, considering and optimizing the performance gap between DRAM and PM is crucial for efficient and scalable graph representation learning on persistent memory.

[0005] Currently, most multi-core computers widely use Non-Uniform Memory Access (NUMA) architecture. In Optane PM, the processor and DRAM / PM are also grouped into nodes, and these nodes are interconnected via Intel Ultra Path Interconnect (UPI). For traditional DRAM-based NUMA systems, the memory access latency of remote NUMA nodes is typically higher than that of local NUMA nodes, and PM further exacerbates this asymmetry in memory access latency under NUMA architecture. For example, compared to local writes (on 8 threads), the peak bandwidth of remote write operations is reduced by 69.2%. More seriously, the remote and local latency of PM-based systems is 3.3 times and 4.2 times higher than that of DRAM-based systems, respectively, indicating that the performance gap between DRAM and PM is further amplified under NUMA architecture. Overall, the impact of NUMA architecture cannot be ignored in achieving efficient and scalable graph representation learning for persistent memory, especially considering the strong computational resource dependence of numerous parallel operations in graph representation learning. For example, in ProNE, parallel sparse and dense matrix multiplication operations account for more than 60% of its total overhead. Furthermore, since most graph data in the real world follows a power-law distribution, when performing matrix computations on such graph data, the inherent sparsity of graph data significantly increases the risk of workload imbalance and tail latency during parallel computation, further hindering the efficiency of graph representation learning and affecting the efficiency of target feature extraction.

[0006] Therefore, how to design an efficient graph representation learning method and system for persistent memory to meet the representation learning needs on large-scale graphs and achieve efficient extraction of target feature information is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for extracting target feature information based on graph representation learning, thereby solving the problem of low efficiency in existing target feature information extraction methods.

[0008] To achieve the above objectives, in a first aspect, the present invention provides a method for extracting target feature information based on graph representation learning, the method being applied to a non-uniform memory access NUMA architecture, the method comprising the following steps:

[0009] Determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is a relationship between the nodes. Neighboring nodes are nodes that have a direct or indirect relationship with the target node.

[0010] The graph data is reorganized in descending order of vertex degree. Vertices with the same degree are then merged to obtain corresponding vertex blocks. The starting offset of each degree-corresponding vertex block and the first-order neighbor vertices of each vertex within the vertex block are recorded. The sparse matrix corresponding to the graph data is stored in the corresponding memory in the format of compressed sparse blocks. The memory is either the local memory on the current NUMA node or the remote memory across NUMA nodes.

[0011] The sparse matrix stored in local memory and / or remote memory is read from the current NUMA node. The read sparse matrix is ​​multiplied by the dense matrix stored in local memory, and the intermediate matrix obtained by multiplication is written to local memory. Then, the intermediate matrices stored in the local memory of the current NUMA node are merged to obtain the feature information of the target. The dense matrix is ​​used to perform feature representation on the sparse matrix corresponding to the graph data. The sparse matrix is ​​stored in memory in the format of compressed sparse blocks so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of the NUMA node in the matrix operation process and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

[0012] Optionally, the multiplication operations of sparse and dense matrices on the NUMA node are performed in parallel. The thread tasks of the parallel operation are allocated in an entropy-aware manner to achieve workload balancing among different threads and reduce tail latency in the parallel computing process.

[0013] Optionally, the threads for parallel computation are allocated in an entropy-aware manner, specifically as follows:

[0014] The initial workload and workload entropy of unassigned threads are determined based on the dynamic load balancing scheme.

[0015] The execution efficiency of unallocated threads is determined based on the initial workload of unallocated threads, the dispersion of the sparse matrix, and the bandwidth of sequential access.

[0016] Based on the linear relationship between thread runtime and workload entropy, the execution efficiency of unallocated threads and workload entropy are combined to determine the correspondence between the workload entropy of unallocated threads and the discreteness of sparse matrices.

[0017] The dynamic average workload entropy value is determined based on the allocated workload entropy value and the number of allocated threads. The optimal workload for unallocated threads is determined based on the workload entropy value of unallocated threads and the initial workload of unallocated threads, combined with the relationship between the workload entropy value and the dispersion of the sparse matrix.

[0018] Optionally, the memory partitioning method for storing the sparse and dense matrices is determined based on the number of NUMA nodes.

[0019] Optionally, the data in the compressed sparse block format includes: Deg_ind, Deg_list, nnz_list, and col_list; wherein, Deg_list is a list of vertex degrees, Deg_ind is the row index of the sparse matrix composed of graph data, col_list is the column index of the sparse matrix composed of graph data, and nnz_list is the element value of the sparse matrix composed of graph data.

[0020] Optionally, the NUMA architecture includes: persistent memory (PM) and dynamic random access memory (DRAM); wherein, read-only metadata is preferentially stored in the PM; the read-only metadata includes: data in compressed sparse block format corresponding to the graph data; write-only metadata is preferentially stored in the DRAM; the write-only metadata includes: intermediate result matrices of the matrix calculation process and associated indexes.

[0021] Secondly, the present invention provides a target feature information extraction system based on graph representation learning, comprising:

[0022] The graph data determination unit is used to determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is a relationship between the nodes. Neighboring nodes are nodes that have a direct or indirect relationship with the target node.

[0023] The graph data storage unit is used to reorganize the graph data according to the descending order of the degree of each vertex in the graph data, and then merge vertices with the same degree to obtain the corresponding vertex blocks. It records the starting offset of the vertex block corresponding to each degree and the first-order neighbor vertices of each vertex in the vertex block, so as to store the sparse matrix corresponding to the graph data in the corresponding memory in the format of compressed sparse blocks; the memory is the local memory on the current NUMA node or the remote memory across NUMA nodes.

[0024] The target feature extraction unit is used to read sparse matrices stored in local memory and / or remote memory of the current NUMA node, multiply the read sparse matrices by dense matrices stored in local memory, and write the intermediate matrices obtained by multiplication into local memory. Then, the intermediate matrices stored in the local memory of the current NUMA node are merged to obtain the feature information of the target. The dense matrix is ​​used to perform feature representation on the sparse matrix corresponding to the graph data. The sparse matrix is ​​stored in memory in the format of compressed sparse blocks so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of the NUMA node in the matrix operation process and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

[0025] Optionally, the target feature extraction unit performs the multiplication operation of sparse and dense matrices in a parallel computing manner. The thread tasks of the parallel computing are allocated in an entropy-aware manner to achieve workload balancing among different threads and reduce tail latency in the parallel computing process.

[0026] Optionally, the target feature extraction unit determines the initial workload and workload entropy of the unallocated threads according to the dynamic load balancing scheme; determines the execution efficiency of the unallocated threads based on the initial workload of the unallocated threads, the dispersion of the sparse matrix, and the bandwidth of sequential access; combines the execution efficiency of the unallocated threads and the workload entropy based on the linear relationship between thread runtime and workload entropy to determine the correspondence between the workload entropy of the unallocated threads and the dispersion of the sparse matrix; and determines the dynamic average workload entropy based on the allocated workload entropy and the number of allocated threads, and determines the optimal load of the unallocated threads based on the workload entropy of the unallocated threads and the initial workload of the unallocated threads, combined with the relationship between the load entropy and the dispersion of the sparse matrix.

[0027] Thirdly, the present invention provides an electronic device comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any alternative method described in the first aspect.

[0028] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed on a processor, causes the processor to perform the method described in the first aspect or any alternative method of the first aspect.

[0029] Fifthly, the present invention provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any alternative method of the first aspect.

[0030] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0031] This invention provides a method and system for extracting target feature information based on graph representation learning. The proposed compressed sparse block storage format based on vertex degree takes into account the characteristic that graph data in the real world generally follows a power-law distribution. It constructs a data storage index with a vertex degree that is much smaller than the number of vertices, which can effectively reduce the memory occupation of data storage representation. It can also realize sequential reading operations on the sparse matrix of graph data in the subsequent calculation stage, and provides a series of operators that meet the relevant operation requirements of matrix calculation.

[0032] This invention provides a method and system for extracting target feature information based on graph representation learning. The proposed entropy-aware thread task allocation strategy achieves a good balance between workload balancing and reducing tail latency, thereby improving the efficiency of parallel computing in graph representation learning.

[0033] This invention provides a method and system for extracting target feature information based on graph representation learning. It proposes a NUMA-aware data placement strategy, placing read-only metadata in the persistent memory (PM) and write-only metadata in the DRAM. This strategy combines the NUMA access characteristics of persistent memory devices with the runtime characteristics of operators in graph representation learning, enabling global sequential reads and local writes. This reduces the adverse effects of the NUMA architecture on the PM and fully leverages the advantages of DRAM and PM in heterogeneous storage, providing system-level support for fast and large-capacity storage. Furthermore, it improves the efficiency of target feature information extraction by increasing the graph representation learning rate. Attached Figure Description

[0034] Figure 1 This is a flowchart of a target feature information extraction method based on graph representation learning provided in an embodiment of the present invention;

[0035] Figure 2 A flowchart of graph representation learning provided for embodiments of the present invention;

[0036] Figure 3 A schematic diagram of a vertex-degree-based compressed sparse block storage format provided as an example of the present invention;

[0037] Figure 4 This is a schematic diagram comparing persistent memory read / write bandwidth provided in an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of a NUMA-aware data placement strategy provided in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram illustrating the implementation of graph representation learning for persistent memory on a heterogeneous memory architecture of Intel Optane DC Persistent Memory Modules, as provided in an embodiment of the present invention.

[0040] Figure 7 This is a system architecture diagram for extracting target feature information based on graph representation learning, provided in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0043] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0044] Next, the technical solutions provided in the embodiments of the present invention will be introduced.

[0045] This invention discloses a method and system for extracting target feature information based on graph representation learning, belonging to the fields of graph representation learning and target feature extraction. It includes: addressing the significant performance gap between persistent memory (PM) and DRAM, which is further amplified under NUMA architecture, a NUMA-aware data placement strategy is proposed, combining the NUMA access characteristics of persistent memory devices and the runtime characteristics of operators in graph representation learning. This strategy aims to reduce the adverse effects of NUMA architecture on PM and fully leverage the advantages of DRAM and PM under heterogeneous storage, providing system-level support for fast and large-capacity storage in graph representation learning. To address the difficulties in ensuring load balancing and tail latency in parallel computation thread task allocation during graph representation learning, an entropy-aware thread task allocation strategy is proposed to further improve parallel computation efficiency. Furthermore, this invention designs a vertex-degree-based compressed sparse block storage format to reduce the size of data storage indexes and provides operators that satisfy matrix computation-related operations. This invention can efficiently and scalably perform representation learning on large-scale graph data, achieving high-efficiency target feature extraction.

[0046] It is understood that the target can be a user in a social network, an atom in a chemical molecular network, or a user in an e-commerce network, etc. This invention is used to extract feature information of a target based on the association between the target and its neighbors, and is not limited to any specific target. Those skilled in the art can choose according to actual needs.

[0047] This invention provides a method and system for extracting target feature information based on graph representation learning. The purpose of this method and system is to solve the performance gap between PM and DRAM in graph representation learning, and the problem that the performance gap is further amplified under NUMA architecture.

[0048] Figure 1 This is a flowchart of a target feature information extraction method based on graph representation learning provided in an embodiment of the present invention; as shown below. Figure 1 As shown, it includes the following steps:

[0049] S101, determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is a relationship between the nodes, and the neighboring nodes are nodes that have a direct or indirect relationship with the target node;

[0050] S102, reorganize the graph data according to the descending order of the vertex degrees in the graph data, then merge vertices with the same degree to obtain the corresponding vertex blocks, record the starting offset of the vertex block corresponding to each degree and the first-order neighbor vertices of each vertex in the vertex block, so as to store the sparse matrix corresponding to the graph data in the corresponding memory in the format of compressed sparse blocks; the memory is the local memory on the current NUMA node or the remote memory across NUMA nodes;

[0051] S103: Read the sparse matrix stored in local memory and / or remote memory on the current NUMA node, multiply the read sparse matrix by the dense matrix stored in local memory, and write the intermediate matrix obtained by multiplication into local memory. Then, merge the intermediate matrices stored in the local memory of the current NUMA node to obtain the feature information of the target. The dense matrix is ​​used to perform feature representation on the sparse matrix corresponding to the graph data. The sparse matrix is ​​stored in memory in the format of compressed sparse blocks so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of the NUMA node in the matrix operation process and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

[0052] like Figure 2 As shown, this invention discloses a graph representation learning method for persistent memory, used to extract feature information of a target. The execution of this method and system includes the following steps:

[0053] (1) By storing the graph data in compressed storage, the original graph data is stored in memory in a format of compressed sparse blocks based on vertex degree;

[0054] (2) Based on ProNE, representation learning is performed. When parallel computing is executed, the load tasks are allocated to each thread through an entropy-aware thread allocation strategy.

[0055] (3) The sparse matrix composed of graph data and the intermediate data of matrix calculation are placed locally and remotely through the NUMA-aware data placement strategy, and the characteristics of each storage device in the heterogeneous memory structure of Intel Optane DC Persistent Memory Modules are fully utilized; after ProNE is executed, a vector representation of each vertex is generated, thereby realizing the embedding of high-dimensional graph data into a low-dimensional vector space.

[0056] Specifically, step (1) includes:

[0057] (1.1) Considering that real-world graphs typically follow a power-law distribution, and that the degree of a vertex is much smaller than the number of vertices in real-world graph data, a Compressed Sparse Degree-Block (CSDB) storage format based on vertex degree is proposed. This format uses two index arrays, Deg_list and Deg_ind, to represent the graph structure in a more lightweight way. Specifically, Deg_list represents a list of vertex degrees in the graph data, while Deg_ind represents the starting offset of a vertex block with the same degree.

[0058] Understandably, the graph data is first reorganized in descending order of vertex degree, and then two index arrays are generated: Deg_list and Deg_ind. Based on the vertex's number (i.e., the row index of the sparse matrix composed of the graph data) and its position in the vertex degree block index Deg_ind, the degree (Degree(v)) of the vertex is further searched in the vertex degree list Deg_list. i );according to Determine the offset of a vertex in either the edge list (i.e., the column index of the sparse matrix composed of graph data, col_list) or the edge weight list (i.e., the element value of the sparse matrix composed of graph data, nnz_list), and use this data storage method to acquire and update the corresponding graph data during the graph representation learning process.

[0059] Furthermore, such as Figure 3 As shown, step (1.1) specifically includes:

[0060] (1.1.1) First, reorganize the graph data in descending order of vertex degree, then generate two index arrays: Deg_list and Deg_ind; for Figure 3 In example graph G in (a), the vertex list is [v0, v1, v2, v3, v4, v5, v6], and the corresponding col_list (i.e., the edge list of G) is [1, 2, 4, 6, 0, 3, 4, 6, 0, 3, 4, 5, 1, 2, 5, 0, 1, 2, 2, 3, 0, 1]. The elements in nnz_list are the weights of each edge, which are set to 1 by default. Figure 3 As shown in (b), Deg_list and Deg_ind in CSDB are [4, 3, 2, 0] and [0, 3, 5, 7], respectively;

[0061] It should be noted that Deg_list and Deg_ind are [4, 3, 2, 0] and [0, 3, 5, 7] respectively, indicating that: the starting vertex index offset of a vertex with a block degree of 4 is 0, which means that the first vertex with a block degree of 4 is the vertex with index 0 (i.e., the first vertex); the starting vertex index offset of a vertex with a block degree of 3 is 3, which means that the first vertex with a block degree of 3 is the vertex with index 3 (i.e., the 4th vertex), and so on.

[0062] (1.1.2) Based on the vertex's index (i.e., the row index of the sparse matrix composed of graph data) and its position in the vertex degree block index Deg_ind, further search for the vertex's degree (Degree(v)) in the vertex degree list Deg_list. i Suppose we need to access the neighbors of vertex v1, that is, obtain the non-zero elements with row 1 in the adjacency matrix composed of G. Note that the index of v1 is 1, then the vertex degree block index is within the range of [Deg-ind(0), Deg_ind(1)). Therefore, according to Deg_list(0), we can obtain that the vertex degree of v1 is 4 (i.e., Degree(v1)).

[0063] (1.1.3) According to Determine the offset of a vertex in either the edge list (i.e., the column indices of the sparse matrix composed of graph data, col_list) or the edge weight list (i.e., the element values ​​of the sparse matrix composed of graph data, nnz_list). The initial offset of v1 is 4. Therefore, by using Deg_ptr(v1) and Degree(v1), the neighbors of v1 are found in col_list as [v0, v3, v4, v6]. Similarly, the edge weights (i.e., the non-zero element values ​​of the matrix) can be obtained from nnz_list.

[0064] Specifically, step (2) includes:

[0065] (2.1) The sparse and dense matrix multiplication operations in ProNE adopt parallel computing methods and account for more than 60% of the total overhead. Therefore, parallel computing is performed through an entropy-aware thread allocation strategy (EaTA) to achieve workload balancing of the thread allocation scheme and reduce tail latency in the parallel computing process.

[0066] Furthermore, step (2.1) specifically includes:

[0067] (2.1.1) Obtain the initial workload Wi of thread pi according to the dynamic load balancing scheme (unallocated workload and unallocated thread count), and according to... Derive Hi (the entropy of the workload), where n and m represent the starting and ending row indices assigned to Wi in the sparse matrix A, respectively. |Row j | indicates the number of non-zero elements in the j-th row (i.e., the number of edges at vertex vj);

[0068] (2.1.2) Based on empirical analysis, there is a good linear relationship between the runtime (T(pi)) of each thread and the entropy value (Hi) of the workload, expressed as: T(pi) i )=K·H i ;

[0069] (2.1.3) Due to the inherent dispersion of the sparse matrix composed of graph data when performing multiplication calculations with dense matrices, (Determined by the dispersion of column indices in the sparse matrix) has a direct impact on access bandwidth, so thread p i The execution efficiency can be expressed as: Among them BW r_seq Indicates sequential access bandwidth;

[0070] (2.1.4) Based on (2.1.2), T(pi) is further quantized as follows: According to W i The impact of inherent dispersion on access bandwidth, H i and The relationship between them is: Where H i pass Standardize using (0≤Hi≤log|V|), where V is the number of vertices in the graph data or the number of rows in the sparse matrix formed by the graph data, and β is: (That is, the ratio of sequential access bandwidth to random access bandwidth of a storage device). Intuitively, if H i A value close to 1 indicates a large amount of random access, and the bandwidth of this thread is BWr_rand. Conversely, when Hi is 0, it indicates absolutely sequential access, and the bandwidth of this thread is BWr_seq.

[0071] (2.1.5) To mitigate tail latency caused by the inherent dispersion of workloads, the inherent dispersion of each workload should be relatively similar. Therefore, the optimal load for each thread is determined based on... To generate, where The dynamically averaged workload entropy value (i.e. ).

[0072] Specifically, step (3) includes:

[0073] (3.1) When performing parallel computing, the NUMA-aware Data Placement (NaDP) strategy is used to implement the access principle of global sequential read and local write, so as to reduce the adverse effects of NUMA architecture on PM.

[0074] Figure 4 This is a schematic diagram comparing persistent memory read / write bandwidth provided in an embodiment of the present invention; as shown below. Figure 4 As shown, the sequential (SEQ) / random (RAND) read / write bandwidth of PMs at different thread counts was tested on a system with 6 × 256GB PMs (3 PM DIMMs and 18 CPU cores per NUMA node) under different thread counts. It can be observed that for read operations, the peak bandwidth of remote sequential access is comparable to that of local sequential access, and significantly higher than the peak bandwidth of local and remote random access (2.41 times and 2.45 times, respectively). For write operations, local write operations consistently outperform remote write operations, regardless of whether they are sequential or random. Compared to local write operations, the peak bandwidth of remote PM writes is reduced by 69.2%. Figure 4 It is evident that high-performance PM systems should prioritize global sequential reads and local writes during parallel computing to minimize the adverse effects of NUMA on the PM. Indices in the sparse matrix represented by CSDB, such as Deg_list, Deg_ind, col_list, and nnz_list, can guarantee sequential reading of non-zero elements during the multiplication of the coefficient matrix and the dense matrix. For writing intermediate data, i.e., writing temporary values ​​to the result matrix, since the result matrix is ​​a dense matrix stored in column-major order, the result writing process is a sequential write operation. Therefore, this invention stores the sparse matrix corresponding to the graph data, etc., on the PM in local memory and / or remote memory, while storing the dense matrix in local memory.

[0075] Furthermore, the sparse and dense matrices are partitioned according to the number of processor nodes (i.e., NUMA nodes); the sub-sparse matrices stored on the local processor or across processor nodes are obtained by the allocated threads and multiplied with the locally stored sub-dense matrices; the intermediate results are written to two-dimensional arrays constructed locally for each sub-matrix multiplication; and the intermediate result matrices stored on each processor node are merged to obtain the final result matrix.

[0076] Furthermore, such as Figure 5 As shown, step (3.1) specifically includes:

[0077] (3.1.1) Given a sparse matrix M(7,7) and a dense matrix L(7,6), NaDP first divides M and L into M1(3,7), M2(4,7), L1(7,3), and L2(7,3), where M1 and L1 are placed on N0, and M2 and L2 are placed on N1. To store intermediate result data, NaDP also constructs R on N0. 11 (3,3) and R 21 (4,3), R is constructed on N1 22 (4,3) and R 12 (3, 3);

[0078] (3.1.2) Based on the entropy-aware thread task allocation strategy (EaTA), assuming the number of threads is 8 (P01, P02, P03, and P04 on N0, and P11, P12, P13, and P14 on N1), for the M1×L1 operation, P01 and P02 sequentially retrieve non-zero elements (nnz) from M1 locally, and then perform the multiplication operation with L1. Simultaneously, intermediate result data is written to R sequentially. 11 In the M2×L1 operation, P03 and P04 sequentially obtain nnz of M2 from another processor node N1, then perform a multiplication operation with L1 on N0, and the intermediate results are also written to R sequentially. 21 middle;

[0079] (3.1.3) Similarly, for the M2×L2 and M1×L2 operations, P11, P12, P13 and P14 use the same execution mechanism to complete the multiplication operation;

[0080] (3.1.4) NaDP simply requires merging the submatrices (R) on N1 and N0. 11 R 21 R 22 R 12 This will generate the final result matrix R(7,6).

[0081] Specifically, step (3.2) includes:

[0082] (3.2) Due to the performance difference between PM and DRAM, the characteristics of each storage device in the PM-DRAM heterogeneous memory structure of Intel Optane DC Persistent Memory Modules (Optane PM) should be fully utilized;

[0083] More specifically, such as Figure 6 As shown, step (3.2) specifically includes:

[0084] (3.2.1) For read-only metadata (e.g., Deg_ind, Deg_list, nnz_list, and col_list in CSDB), place this data in PM;

[0085] (3.2.2) Only write metadata is placed in DRAM to achieve better performance. The metadata includes intermediate result matrices of the computation process and associated indexes, such as graph reading and computational operations such as sparse matrix multiplication and dense matrix multiplication.

[0086] (3.2.3) Communication between DRAM and PM is carried out through memcpy in C++ and memcpy_persist in PMDK, without the need to create separate cross-device copies for data transfer.

[0087] (3.3) After the execution process of ProNE is completed, it generates the vector representation of the vertices, thus embedding the graph data into a low-dimensional vector space.

[0088] This invention employs the aforementioned graph representation learning method for persistent memory. Addressing the issue that matrix factorization-based graph representation learning methods are heavily reliant on memory during operation, thus affecting their scalability, this invention proposes an efficient and scalable graph representation learning method and system based on persistent memory (PM), a novel persistent memory device with higher storage density and lower cost than DRAM. This aims to alleviate the performance gap between PM and DRAM, and further address the problem that the performance gap is amplified under NUMA (Non-Uniform Memory Access) architectures.

[0089] In one specific embodiment, compared with DRAM only (ideal benchmark system), PM only (worst benchmark system), ProNE-DRAM (a representative matrix factorization-based graph representation learning system), and PM-based ProNE (ProNE-PM), the graph representation learning system of the present invention delivers an average performance speedup of 43.42 times on real graph datasets of different sizes, and significantly reduces the average performance gap between PM and DRAM from orders of magnitude to 57.6%, successfully bridging the performance gap between the two. Compared to the mainstream storage format CSR, the vertex-degree-based compressed sparse block storage format adopted in this invention reduces the indexing overhead of graph data organization and representation by nearly three orders of magnitude and provides an average processing efficiency 1.35 times faster. The entropy-aware thread allocation strategy proposed in this invention, compared to round-robin and load-balancing allocation schemes, not only provides a performance acceleration of 1.07 to 7.51 times but also effectively reduces thread tail latency during parallel computing. The graph representation learning system in this invention, equipped with a NUMA-aware Data Placement (NaDP) strategy, delivers a performance acceleration of 2.35 to 3.58 times compared to schemes without NaDP, reducing the adverse effects of NUMA architecture on PM. Furthermore, the graph representation learning system of this invention exhibits good scalability under different thread configurations and on synthetic graph data of different sizes. Therefore, the persistent memory-oriented graph representation learning method provided by this invention can efficiently and scalably represent graph data and provides system-level support for fast and large-capacity storage, greatly improving the efficiency of target feature information extraction.

[0090] Figure 7 This is a system architecture diagram for target feature information extraction based on graph representation learning provided in an embodiment of the present invention; as shown below. Figure 7 As shown, it includes:

[0091] The graph data determination unit 710 is used to determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is an association between the nodes, and the neighboring nodes are nodes that have a direct or indirect association with the target node;

[0092] Graph data storage unit 720 is used to reorganize the graph data according to the descending order of the degree of each vertex in the graph data, then merge vertices with the same degree to obtain corresponding vertex blocks, and record the starting offset of the vertex block corresponding to each degree and the first-order neighbor vertices of each vertex in the vertex block, so as to store the sparse matrix corresponding to the graph data in the corresponding memory in the format of compressed sparse blocks; the memory is the local memory on the current NUMA node or the remote memory across NUMA nodes;

[0093] The target feature extraction unit 730 is used to read sparse matrices stored in local memory and / or remote memory of the current NUMA node, multiply the read sparse matrices by dense matrices stored in local memory, write the intermediate matrices obtained by multiplication into local memory, and then merge the intermediate matrices stored in the local memory of the current NUMA node to obtain the feature information of the target. The dense matrix is ​​used for graph representation learning of graph data, and the sparse matrix is ​​stored in memory in the format of compressed sparse blocks so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of the NUMA node in the matrix operation process and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

[0094] It should be understood that the above system is used to execute the methods in the above embodiments. The corresponding program units in the system are similar in implementation principle and technical effect to those described in the above methods. The working process of the system can be referred to the corresponding process in the above methods, and will not be repeated here.

[0095] Based on the methods described in the above embodiments, this invention provides an electronic device. The device may include at least one memory for storing a program and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor performs the methods described in the above embodiments.

[0096] Based on the methods in the above embodiments, this embodiment of the invention provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0097] Based on the methods in the above embodiments, this embodiment of the invention provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0098] It is understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0099] The method steps in these embodiments of the invention can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), persistent memory (PM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC.

[0100] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0101] It is understood that the various numerical designations used in the embodiments of the present invention are merely for descriptive convenience and are not intended to limit the scope of the embodiments of the present invention.

[0102] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for extracting target feature information based on graph representation learning, wherein the method is applied to a non-uniform memory access NUMA architecture, characterized in that, Includes the following steps: Determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is a relationship between the nodes. Neighboring nodes are nodes that have a direct or indirect relationship with the target node. The graph data is reorganized in descending order of vertex degree. Vertices with the same degree are then merged to obtain corresponding vertex blocks. The starting offset of each degree-corresponding vertex block and the first-order neighbor vertices of each vertex within the vertex block are recorded to store the sparse matrix corresponding to the graph data in the corresponding memory in the format of compressed sparse blocks. The memory can be local memory on the current NUMA node or remote memory across NUMA nodes; Read the sparse matrix stored in local memory and / or remote memory on the current NUMA node, multiply the read sparse matrix by the dense matrix stored in local memory, write the intermediate matrix obtained by multiplication into local memory, and then merge the intermediate matrices stored in local memory on the current NUMA node to obtain the feature information of the target. The dense matrix is ​​used to represent the features of the sparse matrix corresponding to the graph data. The sparse matrix is ​​stored in memory in a compressed sparse block format so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of NUMA nodes during matrix operations and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

2. The method according to claim 1, characterized in that, The multiplication operations of sparse and dense matrices on the NUMA node are performed in parallel. The parallel computing thread tasks are allocated in an entropy-aware manner to achieve workload balancing among different threads and reduce tail latency in the parallel computing process.

3. The method according to claim 2, characterized in that, The threads for parallel computation are allocated in an entropy-aware manner, specifically as follows: The initial workload and workload entropy of unassigned threads are determined based on the dynamic load balancing scheme. The execution efficiency of unallocated threads is determined based on the initial workload of unallocated threads, the dispersion of the sparse matrix, and the bandwidth of sequential access. Based on the linear relationship between thread runtime and workload entropy, the execution efficiency of unallocated threads and workload entropy are combined to determine the correspondence between the workload entropy of unallocated threads and the discreteness of sparse matrices. The dynamic average workload entropy value is determined based on the allocated workload entropy value and the number of allocated threads. The optimal workload for unallocated threads is determined based on the workload entropy value of unallocated threads and the initial workload of unallocated threads, combined with the relationship between the workload entropy value and the dispersion of the sparse matrix.

4. The method according to claim 1, characterized in that, The memory partitioning method for storing the sparse and dense matrices is determined based on the number of NUMA nodes.

5. The method according to claim 1, characterized in that, The data in the compressed sparse block format includes: Deg_ind, Deg_list, nnz_list, and col_list; where Deg_list is a list of vertex degrees, Deg_ind is the row index of the sparse matrix composed of graph data, col_list is the column index of the sparse matrix composed of graph data, and nnz_list is the element value of the sparse matrix composed of graph data.

6. The method according to any one of claims 1-5, characterized in that, The NUMA architecture includes: persistent memory (PM) and dynamic random access memory (DRAM); wherein, read-only metadata is preferentially stored in PM; the read-only metadata includes: data in compressed sparse block format corresponding to graph data; write-only metadata is preferentially stored in DRAM; the write-only metadata includes: intermediate result matrices of matrix calculation process and associated indexes.

7. A target feature information extraction system based on graph representation learning, characterized in that, include: The graph data determination unit is used to determine graph data containing target information; the vertices of the graph data include target nodes and neighboring nodes, and the existence of edges between nodes indicates that there is a relationship between the nodes. Neighboring nodes are nodes that have a direct or indirect relationship with the target node. The graph data storage unit is used to reorganize the graph data according to the descending order of the degree of each vertex in the graph data. Then, vertices with the same degree are merged to obtain the corresponding vertex blocks. The starting offset of the vertex block corresponding to each degree and the first-order neighbor vertices of each vertex in the vertex block are recorded so as to store the sparse matrix corresponding to the graph data in the corresponding memory in the format of compressed sparse blocks. The memory can be local memory on the current NUMA node or remote memory across NUMA nodes; The target feature extraction unit is used to read the sparse matrix stored in the local memory and / or remote memory of the current NUMA node, multiply the read sparse matrix by the dense matrix stored in the local memory, write the intermediate matrix obtained by multiplication into the local memory, and then merge the intermediate matrices stored in the local memory of the current NUMA node to obtain the feature information of the target. The dense matrix is ​​used to represent the features of the sparse matrix corresponding to the graph data. The sparse matrix is ​​stored in memory in a compressed sparse block format so that the sparse matrix is ​​read sequentially, thereby improving the access bandwidth of NUMA nodes during matrix operations and reducing the impact of the performance gap between local and remote memory in the NUMA architecture on the target feature information extraction process.

8. The system according to claim 7, characterized in that, The target feature extraction unit performs sparse and dense matrix multiplication operations in parallel. The parallel operation thread tasks are allocated in an entropy-aware manner to achieve workload balancing among different threads and reduce tail latency in the parallel computing process.

9. The system according to claim 8, characterized in that, The target feature extraction unit determines the initial workload and workload entropy of the unallocated thread according to the dynamic load balancing scheme; and determines the execution efficiency of the unallocated thread based on the initial workload of the unallocated thread, the dispersion of the sparse matrix, and the bandwidth of sequential access. Based on the linear relationship between thread runtime and workload entropy, the execution efficiency of unassigned threads and workload entropy are combined to determine the correspondence between the workload entropy of unassigned threads and the dispersion of the sparse matrix. Furthermore, the dynamic average workload entropy is determined based on the workload entropy of assigned threads and the number of assigned threads. Finally, the optimal workload of unassigned threads is determined based on the workload entropy of unassigned threads and the initial workload of unassigned threads, combined with the relationship between workload entropy and the dispersion of the sparse matrix.

10. An electronic device, characterized in that, include: At least one memory for storing programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-6.

Images