Centrality-Guided Random Walks for Balanced Node Representation in Graph Embeddings

Abstract

A centrality-based random walk graph embedding technique for balanced node representation can overcome the limitations of handling long-tail distributions in random walk-based methods through a novel approach that incorporates node-specific structural importance into the random walk process.

Description

Technology Field

[0001] This document relates to a centrality-based random walk graph embedding technique for balanced node representation. Background Technology

[0002] In the real world, entities often form complex relationships, which naturally give rise to graph structures. Examples of these structures are abundant, ranging from social networks connecting individuals to biological networks mapping complex molecular interactions. These graphs serve as powerful tools for modeling diverse and dynamic relationships, enabling researchers to solve important challenges. Among these, node classification, which assigns labels to entities, and link prediction, which identifies hidden or potential connections, have received significant attention due to their wide range of applications in fields such as recommendation systems and knowledge discovery. The problem to be solved

[0003] According to one embodiment of the present document, a centrality-based random walk graph embedding technique for balanced node representation may be provided, which proposes a new approach to integrating structural importance per node into a random walk process.

[0004] The problem to be solved in one embodiment of this document is not limited to the problem mentioned above, and can be extended in various ways without departing from the spirit and scope of one embodiment of this document. means of solving the problem

[0005] A computing device according to one embodiment of the present document includes: a memory for storing one or more programs; and one or more processors that perform operations according to the one or more programs stored in the memory; wherein the processors calculate a structural importance score for each node in a graph and dynamically adjust parameters such as the number of walks and a search style based on the structural importance score.

[0006] The above structural importance score can be obtained based on three graph indicators consisting of Degree, PageRank, and Betweenness Centrality.

[0007] The above order is an indicator representing the number of neighbors a node has, the above PageRank is an indicator representing the global importance of a node based on its location within the network, and the above betweenness centrality may be an indicator representing how often a node acts as a bridge between other nodes.

[0008] The processor can calculate the structural importance score for a node by normalizing each of the order, page rank, and betweenness centrality scores to a range between 0 and 1, and averaging the normalized order score, the normalized page rank score, and the normalized betweenness centrality score.

[0009] The processor can classify each node in the graph into one of a high group, a medium group, and a low group using the structural importance score and a preset quantile, and can dynamically adjust parameters such as the number of works and search style for each of the high group, the medium group, and the low group.

[0010] The processor can adjust parameters so that the number of works increases in the order of the high group, the middle group, and the low group.

[0011] The processor can adjust search style parameters such that Breadth-First Search (BFS) is applied to the high group, Depth-First Search (DFS) is applied to the low group, and both Breadth-First Search (BFS) and Depth-First Search (DFS) are applied to the middle group. Effects of the invention

[0012] According to one embodiment of the present document, the limitations of long-tail distribution processing in random walk-based methods can be overcome.

[0013] The effects according to the various embodiments of this document are not limited to those described above, and it is obvious to a person skilled in the art that various effects are inherent in one embodiment of this document. Brief explanation of the drawing

[0014] FIG. 1 is a block diagram for explaining a computing device according to one embodiment of the present document. FIG. 2 is a diagram illustrating a node degree distribution (a) and classification performance (b) based on the node degree according to an embodiment of the present document. FIG. 3 is a diagram illustrating long-tail node visits in various datasets according to one embodiment of the present document. FIG. 4 is a diagram showing algorithm 1 for generating a random walk according to one embodiment of the present document. FIG. 5 is a diagram illustrating a node characteristic calculation (a) and a DFS / BFS style search strategy (b) according to one embodiment of the present document. Specific details for implementing the invention

[0015] Hereinafter, embodiments of this document will be described in detail with reference to the attached drawings. The advantages and features of the embodiments of this document, and the methods for achieving them, will become clear by referring to the details described below in conjunction with the attached drawings. However, the embodiments of this document are not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments of this document are defined only by the scope of the claims.

[0017] A centrality-based random walk graph embedding technique for balanced node representation according to one embodiment of the present document will be described in detail below with reference to the attached drawings.

[0019] First, a computing device according to one embodiment of the present document will be described with reference to FIG. 1.

[0020] FIG. 1 is a block diagram for explaining a computing device according to one embodiment of the present document.

[0021] Referring to FIG. 1, a computing device (100) according to one embodiment of the present document can perform operations using a centrality-based random walk graph embedding technique for balanced node representation.

[0022] To this end, the computing device (100) may include one or more processors (110), a computer-readable storage medium (130), and a communication bus (150).

[0023] The processor (110) can control the computing device (100) to operate. For example, the processor (110) can execute one or more programs (131) stored in a computer-readable storage medium (130). One or more programs (131) may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the computing device (100) to perform an operation using a centrality-based random walk graph embedding technique for balanced node representation when executed by the processor (110).

[0024] A computer-readable storage medium (130) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information for performing operations using a centrality-based random walk graph embedding technique for balanced node representation. A program (131) stored in the computer-readable storage medium (130) includes a set of instructions executable by a processor (110). In one embodiment, the computer-readable storage medium (130) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that are accessed by a computing device (100) and capable of storing desired information, or a suitable combination thereof.

[0025] The communication bus (150) interconnects various other components of the computing device (100), including the processor (110) and the computer-readable storage medium (130).

[0026] The computing device (100) may also include one or more input / output interfaces (170) and one or more communication interfaces (190) that provide interfaces for one or more input / output devices. The input / output interfaces (170) and the communication interfaces (190) are connected to a communication bus (150). An input / output device (not shown) may be connected to other components of the computing device (100) through the input / output interface (170).

[0028] Then, with reference to FIGS. 2 to 5, a centrality-based random walk graph embedding technique for balanced node representation according to one embodiment of the present document will be described.

[0029] FIG. 2 is a diagram illustrating a node degree distribution (a) and classification performance based on node degree (b) according to an embodiment of the present document, FIG. 3 is a diagram illustrating long-tail node visitation in various datasets according to an embodiment of the present document, FIG. 4 is a diagram showing an algorithm 1 for generating a random walk according to an embodiment of the present document, and FIG. 5 is a diagram illustrating node characteristic calculation (a) and a DFS / BFS style search strategy (b) according to an embodiment of the present document.

[0030] Introduction

[0031] In the real world, entities often form complex relationships, which naturally give rise to graph structures. Examples of these structures are abundant, ranging from social networks connecting individuals to biological networks mapping complex molecular interactions. These graphs serve as powerful tools for modeling diverse and dynamic relationships, enabling researchers to solve important challenges. Among these, node classification, which assigns labels to entities, and link prediction, which identifies hidden or potential connections, have received significant attention due to their wide range of applications in fields such as recommendation systems and knowledge discovery.

[0032] Early approaches to these challenges generally relied on manual feature engineering and domain-specific insights. For example, techniques such as matrix decomposition are frequently used for link prediction, and manual heuristics are employed for node classification. However, these methods face significant limitations. Matrix decomposition is difficult to scale to large-scale graphs due to computational complexity, and manual heuristics lack adaptability to heterogeneous structures or evolving dynamic graph structures.

[0033] To address these shortcomings, Graph Representation Learning has emerged as a promising solution. This technology embeds graph structures into low-dimensional vector spaces while preserving essential structural and semantic features as well as the unique properties of individual nodes. This approach overcomes the inefficiencies of existing methods and provides enhanced scalability, automation, and adaptability for various graph-based tasks.

[0034] Random walk-based methods and Graph Neural Networks (GNNs) offer complementary strengths in learning graph representations. Random walk-based approaches, such as DeepWalk and Node2Vec, excel at capturing the local structure of a graph by exploring neighbors and encoding proximity-based relationships. By focusing on local exploration, these methods effectively generalize node embeddings across the entire graph without relying on individual node features, making them particularly suitable for tasks such as node classification and clustering. In contrast, GNNs like GCNs utilize neighborhood aggregation to integrate both node features and structural information. This capability enables GNNs to model complex dependencies within the graph, making them well-suited for scenarios requiring a deeper understanding of graph properties.

[0035] Despite their excellent performance, both GNNs and random walk-based methods face significant limitations in long-tail scenarios where the majority of nodes have only a few neighbors. This limitation is further exacerbated by the power-law order distribution commonly observed in real-world networks, where a small number of nodes possess high connectivity while the majority remain sparsely connected. For example, as shown in Figure 2(a), the Cora citation dataset exhibits a typical long-tail order distribution, with a significant portion of nodes having very low orders. Nodes at the 'tail' of this distribution often lack sufficient structural connectivity.

[0036] These constraints have practical consequences for downstream tasks such as node classification and link prediction. As shown in Figure 2(b), prediction accuracy decreases sharply for lower-order nodes, highlighting that existing methods fail to robustly handle long-tail distributions. In particular, GNNs rely heavily on neighbor aggregation, which often fails to generate robust representations for nodes with limited connectivity. Specifically, low-order nodes suffer significant information loss during the aggregation process. Their sparse connections constrain the extraction of structural and semantic information. Furthermore, these nodes are unlikely to have 'labeled neighbors' that are critical for effective loss calculation. Consequently, their embeddings fail to capture meaningful information and remain unoptimized.

[0037] In contrast, random walk-based methods tend to disproportionately favor higher-order nodes during the random walk, as exemplified in Fig. 3. This figure represents the frequency of vertex visits during the random walk and demonstrates the clear dominance of higher-order nodes, consistent with the power-law distribution commonly observed in networks. This bias stems from the 'Friendship Paradox,' which states that a node's neighbors are, on average, more connected than the node itself. This results in higher-order nodes being visited much more frequently, even though lower-order nodes can serve as important bridges in the network. Consequently, random walk-based embeddings often fail to effectively capture the structural and semantic properties of lower-order nodes.

[0038] To address these challenges, a method is needed that not only improves representation learning for low-order nodes but also ensures robust performance for high-order nodes. Overemphasizing low-order nodes can lead to overfitting, which can reduce the model's ability to effectively generalize to the entire network. Therefore, a balanced approach is essential to uniformly improve graph representation across all node orders.

[0039] To overcome the limitations of traditional random walk-based methods in handling long-tail distributions, we propose a novel approach that incorporates node-specific structural importance into the random walk process. Unlike standard skip-gram-based random walks that operate independently of node characteristics, our method guides random walkers by utilizing key structural indicators. We use metrics such as Degree, PageRank, and Betweenness Centrality to coordinate neighbor influences and capture the unique structural role of each node.

[0040] Based on these importance scores, the random walk process dynamically adjusts parameters such as the search style (Depth-First Search (DFS) and Breadth-First Search (BFS)) and the number of walks assigned to each node. Low-order nodes with poor local connectivity benefit from DFS search, as it allows random workers to move deeper into the network to capture non-local structural information. Conversely, high-order nodes already possess rich local neighbor information, making BFS search more effective for fully utilizing the densely connected structure. For intermediate nodes, the operation of DFS and BFS is balanced.

[0041] We assign additional random walks (e.g., 5 additional walks per node) to all nodes in the lower-order group regardless of their individual importance scores. This uniform coordination helps offset the inherent imbalance of random walks favoring higher-order nodes due to dense local connectivity.

[0042] In summary, the key contributions of this document are as follows: (1) We propose a novel random walk-based approach that incorporates node-specific structural importance into the embedding process. (2) We designed a comprehensive evaluation framework for analyzing performance across head and tail nodes. (3) We demonstrated improvements in tail node and overall performance through extensive experiments on five benchmark datasets.

[0044] Preliminaries

[0045] We consider an undirected and unweighted graph G=(V, E) composed of a set of nodes V and a set of edges E. Each node v∈V is associated with a feature vector xv∈Rd in a d-dimensional feature space. While these features represent important characteristics of the nodes, the Random Walk-based embedding method described in this document relies entirely on the structural characteristics of the graph rather than node attributes. Specifically, the Random Walk process utilizes the graph's topology to learn embeddings based on connectivity between nodes and their structural roles within the network. For each node v∈V, Nv is defined as the set of neighbors of node v, and its degree |Nv| represents the number of neighbors, which is the size of the neighbor set. Node degree is an important structural characteristic of each node. Nodes are classified based on this degree to guide the Random Walk process. The Random Walk is controlled by two key parameters introduced from the node2vec algorithm: the return parameter p and the input-output parameter q. These parameters determine the search style of the Random Walk. The return parameter p influences the probability of returning to the previous node, making the search more like Depth-First Search (DFS), and the input / output parameter q influences the width of the search, making the Breadth-First Search (BFS) style more preferred when the value is high, and the DFS style more preferred when the value is low.

[0046] To account for different importance levels for each node, a structural importance score for each node is calculated based on various graph metrics that capture diverse aspects of the node's structural role within the network. These metrics include the following: PageRank PR(v): Measures a node's global importance by considering the link structure coming from other nodes within the graph. Betweenness Centrality BC(v): Quantifies the extent to which a node acts as a bridge between other nodes; defined by the number of shortest paths passing through the node, it indicates its role in facilitating communication across the network. Degree Deg(v): Represents the number of direct neighbors of a node and is used as a measure of local connectivity within the network. Each of these metrics is normalized to the range [0, 1] using the following Equation 1:

[0047] [Mathematical Formula 1]

[0048]

[0049] Here, x represents the set of values ​​for a specific metric across all nodes. This normalization process ensures that each metric contributes equally to the final importance score. Subsequently, the Pareto Principle is used to classify nodes into three categories based on their importance scores. Nodes are ranked according to their importance scores and assigned to one of the following groups based on these rankings: the top 20% of high-importance nodes (Vhigh), the bottom 20% of low-importance nodes (Vlow), and the remaining 60% of intermediate-importance nodes (Vintermediate). Formally, this is defined as shown in Equation 2 below:

[0050] [Mathematical Formula 2]

[0051]

[0052] K is a threshold value corresponding to the quantiles that divide the nodes into low, medium, and high importance groups.

[0054] Methodology

[0055] This section presents a methodology used to mitigate the tail node sparsity problem that occurs in graph representation learning. Our approach is based on a random walk method, but introduces a new mechanism that incorporates node-specific structural importance during the random walk process. By ensuring balanced representations for both high-order and low-order nodes, this method improves the overall effect of embeddings, particularly for tail nodes.

[0056] The core idea of ​​our approach is to modify the traditional random walk process to reflect the structural importance of each node. Standard random walk-based methods, such as DeepWalk and node2vec, typically traverse graphs based on a node's local neighbors but do not explicitly consider the varying importance of each node. However, in real-world networks, nodes differ significantly in terms of their structural roles. For example, high-order nodes have well-connected neighbors, whereas low-order nodes (tail nodes) can perform important structural roles despite having limited connectivity.

[0057] To address this, we propose a method to tune the random walk process based on three key graph metrics: Degree, PageRank, and Betweenness Centrality. These metrics guide the random walk by emphasizing the importance of each node within the graph. Degree reflects the number of neighbors a node has, PageRank captures the node's global importance based on its position within the network, and Betweenness Centrality quantifies how often a node acts as a bridge between other nodes. By utilizing these metrics, our method customizes the random walk process to match the unique structural characteristics of each node and improves the overall quality of node embeddings.

[0058] The first step of our approach is to calculate a structural importance score for each node within the graph. This score is derived from three key graph metrics: PageRank, betweenness centrality, and degree. PageRank reflects a node's global importance by evaluating the structure of incoming links, while betweenness centrality indicates its role as a connector by measuring how often a node lies on the shortest path between other nodes. Degree, on the other hand, represents a node's local connectivity, providing insight into its immediate network neighbors. To calculate the structural importance score for each node, these three metrics are first normalized to a range between 0 and 1 to ensure comparability. The normalized PageRank, betweenness centrality, and degree scores are averaged to obtain the joint importance score I(v) for each node. This score reflects both the local and global structural roles of the node, ensuring that nodes with significant influence within the graph are appropriately weighted. To classify nodes based on importance, the quantiles of the joint score are used to divide the nodes into three groups: High, Intermediate, and Low. Nodes with importance scores in the top 20% are classified as high-importance nodes, those in the middle 60% as middle nodes, and those in the bottom 20% as tail nodes. This classification is very important for coordinating subsequent random walk processes because nodes in different groups require different search strategies.

[0059] Our random walk approach is dynamically adjusted based on the structural importance of each node. Unlike traditional methods that apply the same search parameters to all nodes, we customize parameters such as the number of walks and search style according to the role of each node within the network. This enables more effective graph traversal, particularly for underrepresented or sparsely connected nodes, such as tail nodes. For each node, the number of random walks is first determined based on its structural importance score. Nodes with high importance scores are assigned fewer walks because they are already sufficiently well represented within the network's local structure. Conversely, nodes with low importance scores are assigned more walks to adequately capture their less represented structural roles. Additionally, nodes classified as tail nodes are assigned additional random walks to ensure that even sparsely connected nodes receive sufficient exploration for embedding improvement. The search style is also determined by the node's importance score. For central nodes, Breadth-First Search (BFS) is applied to focus on exploring wider neighbors to capture the dense local structure of these nodes. For tail nodes, Depth-First Search (DFS) is applied to compensate for the lack of immediate neighbors by allowing the random walk to explore deeper parts of the network. Intermediate nodes located in between utilize a balanced approach that adjusts BFS and DFS parameters based on their intermediate importance scores. This dynamic adjustment of search styles and walk counts ensures that the structural significance of all nodes—from highly connected central nodes to sparsely connected tail nodes—is effectively captured. This granular approach mitigates the inherent bias of traditional random walk methods (a phenomenon caused by the paradox of friendship) that tends to favor higher-order nodes.By ensuring that low-order nodes also receive sufficient attention during random walks, more robust node embeddings are achieved, which leads to improved performance in downstream tasks such as node classification and link prediction.

[0060] Once a random walk is generated, node embeddings are learned by applying a Skip-gram model, similar to the approach used in DeepWalk and node2vec. The Skip-gram model learns a vector representation for each node by maximizing the probability of predicting neighboring nodes during the random walk. This process enables the embeddings to capture both local and global structural information about the graph. Importantly, by tailoring the random walk process based on node importance, the embeddings for lower-order nodes become more robust and informative. The learned embeddings are subsequently used for downstream tasks such as node classification and link prediction. These tasks benefit from the improved embeddings, particularly from tail nodes that were often underrepresented in traditional embedding methods.

[0061] To evaluate the effectiveness of the proposed method, we designed a comprehensive framework that focuses on evaluating performance in various types of nodes, particularly tail nodes. Since tail nodes are often underrepresented in graph-based tasks, we placed special emphasis on their prediction accuracy.

[0063] The embodiments of this document are intended to illustrate technical concepts, and the scope of the technical concepts of the embodiments of this document is not limited by these embodiments. The scope of protection of the embodiments of this document shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the embodiments of this document. Explanation of the symbols

[0064] 100 : computing device, 110 : Processor, 130 : Computer-readable storage media, 131 : Program, 150 : Communication bus, 170 : Input / Output Interface, 190: Communication interface

Claims

Claim 1 A computing device comprising: a memory for storing one or more programs; and one or more processors that perform operations according to the one or more programs stored in the memory; wherein the processors calculate a structural importance score for each node in a graph and dynamically adjust parameters such as the number of walks and a search style based on the structural importance score. Claim 2 A computing device, wherein the structural importance score in claim 1 is obtained based on three graph indicators consisting of Degree, PageRank, and Betweenness Centrality. Claim 3 A computing device in paragraph 2, wherein the degree is an indicator representing the number of neighbors a node has, the PageRank is an indicator representing the global importance of a node based on its location within the network, and the betweenness centrality is an indicator representing how often a node acts as a bridge between other nodes. Claim 4 In paragraph 3, the processor calculates the structural importance score for a node by normalizing each score of the order, the page rank, and the betweenness centrality to a range between 0 and 1, and averaging the normalized score of the order, the normalized score of the page rank, and the normalized score of the betweenness centrality. Claim 5 In paragraph 4, the processor classifies each node in the graph into one of a high group, a medium group, and a low group using the structural importance score and a preset quantile, and dynamically adjusts parameters such as the number of works and search style for each of the high group, the medium group, and the low group, a computing device. Claim 6 In paragraph 5, the processor is a computing device that adjusts parameters such that the number of works increases in the order of the high group, the middle group, and the low group. Claim 7 In paragraph 5, the processor is a computing device that adjusts search style parameters such that Breadth-First Search (BFS) is applied to the high group, Depth-First Search (DFS) is applied to the low group, and Breadth-First Search (BFS) and Depth-First Search (DFS) are applied to the middle group.

Images