An Information-Driven Distributed Graph Representation Learning Method and System

Abstract

This invention discloses an information-driven distributed graph representation learning method and system, comprising: executing a multi-order proximity-aware streaming graph partitioning strategy from graph data, using dynamic load constraints to ensure load balancing among computing nodes; further, using an incremental information-centric computation mechanism to ensure dynamic measurement of the information effectiveness of random walk-generated paths and constant-level message communication among computing nodes; generating vertex embedding vectors through a distributed Skip-Gram learning model, specifically utilizing a global word vector matrix and local caches to implement vertex vector access and update operations, thereby improving access locality and reducing cache jitter during training and updates; then, fully utilizing CPU hardware resources through a multi-window shared negative sample computation mechanism; and employing a hotspot-based word block synchronization mechanism to ensure efficient parameter synchronization in a distributed environment. This invention efficiently and effectively represents graph data information.

Description

Technical Field

[0001] This invention belongs to the field of graph representation learning, and more specifically, relates to an information-driven distributed graph representation learning method and system. Background Technology

[0002] With the increasing informatization of society, graphs, as an important data structure, have been widely used in many fields. For example, in social networks, vertices can represent users, and edges can represent the following or interaction between users; in chemical molecular networks, vertices represent atoms, and edges represent bonds; in e-commerce networks, vertices represent users, and edges represent users' browsing and rating of products, and so on. In recent years, graph data analysis tasks have become increasingly important, and graph embedding, as an effective graph analysis technique, has also attracted much attention. Techniques such as Deepwalk, Node2vec, and VERSE aim to map large-scale and high-dimensional graph data into a low-dimensional dense vector space while preserving the original graph's structure and attribute features. More importantly, it can be effectively applied to machine learning tasks, such as link prediction, node classification, recommender systems, and visualization.

[0003] Random walks have been widely used in graph representation learning techniques due to their high flexibility, ease of parallelization, and good scalability. Specifically, a random walk is performed on the input graph data to collect vertex sequences. Starting from a vertex, a neighboring vertex is randomly selected and added to the walk path. This process is repeated to generate contextual samples. Finally, the samples are processed using a word embedding model (Skip-Gram) from natural language processing techniques to generate a vector representation for each vertex.

[0004] However, graphs often reach the scale of millions of vertices and billions of edges, undoubtedly increasing the challenge of representation learning for large-scale graphs. This leads to the common problems of high computational overhead and difficulty in ensuring scalability in graph embedding techniques. For example, Deepwalk was the first graph embedding technique proposed to learn the representation of vertices in a graph based on random walks. It obtains the node sequence by truncating the random walk and then uses the Skip-Gram model from natural language processing to learn the embedding vector representation of the graph vertices. Node2vec is an extension of Deepwalk, which introduces a biased random walk to capture the features of the vertices in the graph and then generates low-dimensional embedding vectors. However, graph embedding techniques based on random walks usually rely on the walk to collect a large number of vertex pairs to generate context samples to ensure the quality of the embedding. For example, it would take several months to complete the representation learning of a graph with 100 million vertices and 500 million edges on a typical commercial machine with 20 threads, which undoubtedly poses a great challenge to computing resources. The latest work, HuGE, aims to address the issues of effectiveness and efficiency in graph representation learning through an entropy-driven approach. However, it still requires more than a week to complete the representation learning of a Twitter graph with billions of edges on a state-of-the-art server, hindering its usability in real-world scenarios.

[0005] In addition, some works have attempted to accelerate representation learning using GPUs, performing random walks on CPUs and completing the embedding vector representation of graph vertices on GPUs. However, due to the performance gap between CPUs and GPUs and the limited memory capacity of GPUs, the performance of such systems is unsatisfactory. Recently, to address the scalability issue of graph representation learning, large-scale graph representation learning in a distributed environment has received considerable attention. For example, KnighKing focuses on node2vec, a representative graph embedding method, and implements a distributed random walk engine by optimizing the sampling mechanism of vertex transition probabilities. However, it neglects the efficiency of information transfer between computational nodes during the random walk process and the effectiveness of walk path acquisition. In addition, there are some distributed graph representation learning systems based on other technical approaches. PyTorch-BigGraph uses graph partitioning techniques and parameter servers to implement a distributed graph representation learning system based on a relational model. However, the parameter server used in this system needs to synchronize parameters with the client. For training large-scale graph data, network communication becomes its performance bottleneck, thus limiting its scalability. DistDGL is a recently proposed distributed graph representation system based on graph neural networks. However, its inefficient sampling efficiency (such as the sampling overhead accounting for more than 80% in GraphSAGE) and the latency caused by its mini-batch sampling in gradient updates lead to inefficient parameter synchronization.

[0006] Considering that distributed systems are an effective means of solving large-scale graph processing, the information-centered graph representation learning method HuGE was deployed to the distributed system KnightKing. It was found that because HuGE needs to measure the effectiveness of the generated path in real time during the walk, it inevitably introduces higher computation and communication costs in the distributed environment. The performance improvement brought by the method itself is severely limited by this overhead, making it difficult to give full play to the advantages of information-centered graph representation. In addition, the Skip-Gram model also faces problems such as a large number of random accesses, low computational throughput and low parameter synchronization efficiency in the distributed environment.

[0007] Therefore, how to design an efficient distributed graph representation learning method and system to meet the requirements of random walks on large-scale graphs, while ensuring the effectiveness and scalability of graph representation learning, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the present invention aims to provide an information-driven distributed graph representation learning method and system, which addresses the issues of load balancing of graph partitioning strategies and utilization of local node partition subsets in distributed environments, redundant computation and high communication costs in information-centric random walks, and random access, low computational throughput and low parameter synchronization efficiency during feature training.

[0009] To achieve the above objectives, in a first aspect, the present invention provides an information-driven distributed graph representation learning method, comprising the following steps:

[0010] Obtain social network graph data; in the social network graph data, vertices represent users, and edges represent social relationships between users.

[0011] The social network graph data is partitioned using a multi-order proximity-aware streaming method, dividing it into multiple computing nodes. The multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of each vertex with each computing node, so as to group vertices with similar characteristics into the same computing node, and to apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes.

[0012] Each computing node randomly selects a vertex from its assigned graph data as the starting point according to a mixed attribute-aware random walk strategy, and performs a random walk from the starting point using an incremental information-centered computing mechanism to generate path information for each vertex. The incremental information-centered computing mechanism refers to iteratively calculating the determination coefficient of the information entropy and path length of the new path obtained after adding a new vertex based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added to the walk path, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length, to determine whether the random walk starting from a certain vertex ends.

[0013] The walk paths generated by each computing node are input into the distributed Skip-Gram learning model to learn the vector representation of each user. During training, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text has been learned, it is updated in the global word vector matrix. This improves the locality of data access and avoids cache jitter caused by multi-threaded learning. In addition, multiple texts are trained simultaneously on one thread to make full use of CPU hardware resources. Furthermore, a synchronization mechanism based on hot word blocks is adopted to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

[0014] As an example, the social network graph data is subjected to multi-order proximity-aware streaming partitioning, dividing it into multiple computing nodes, specifically as follows:

[0015] The vertex flow of the social network graph data is traversed using a depth-first search method, and the vertex with the highest degree is selected as the candidate vertex in turn.

[0016] A multi-proximity-aware streaming graph partitioning strategy is used to calculate the score of each candidate vertex relative to each candidate computing node. The candidate computing node with the highest score is selected as the computing node to place the vertex. Specifically, by considering first-order and second-order attribute features, the characteristics of the random walk task are satisfied as much as possible, thereby improving the utilization rate of local computing nodes in graph representation learning in a distributed environment. At the same time, load balancing among computing nodes is considered. Dynamic load constraints are used in the graph partitioning process to ensure load balancing among computing nodes during the random walk. The first-order attribute features refer to the intersection of the candidate vertex's neighbors and the vertex set of the candidate computing node, and the second-order attribute features refer to the number of common neighbors of the candidate vertex and the vertices in the candidate computing node.

[0017] As an example, the generation of path information for each vertex specifically involves:

[0018] Each computing node performs a random walk, using a hybrid attribute of vertex common neighbors and degree as the measure of vertex proximity.

[0019] During the random walk, the information of the random walk path is evaluated based on incremental information related to information entropy in order to adaptively determine the length of the walk path. The information entropy is used to measure how much information is contained in the random process. When cross-vertex calculation occurs, the information entropy and path length of the path before crossing the node are referenced. The number of times the new vertex appears in the new path after the new vertex is added to the walk path and the length of the new path are used as incremental information to calculate the information entropy of the new path after crossing the vertex, so as to reduce redundant calculation and communication consumption.

[0020] After each computing node completes a sampling of all vertices in its local area, it measures the generated context samples based on relative entropy to determine whether the random walk sampling has ended.

[0021] As an example, the evaluation of the information of the random walk path based on information entropy to adaptively determine the length of the walk path specifically involves:

[0022] The random walk path starting from vertex u is Vertices are selected to join the network based on the vertex proximity metric. middle;

[0023] Based on the current path length L and the current information entropy and vertex v i The number of times n appears in a path of length L+1 after adding the walking path. L+1 (v i Iteratively calculate the latest entropy value

[0024] in, T is an intermediate parameter, if v i If it already exists in the current path of length L, then otherwise n L (v i ) is the vertex v i The number of times it appears in a path of length L;

[0025] Based on the average of the current path information entropy The mean of the current path length E(L), and the mean of the product of the current path information entropy and the path length. Mean of the square of the current path information entropy and the mean of the square of the current path length E(L) 2 The vertex v is obtained through iteration. i The coefficient of determination between the updated information entropy after adding the walking path and the path length L+1 To determine whether a random walk starting at vertex u terminates;

[0026] in, Find the coefficient of determination At that time, respectively Substitute L+1 into the above equation;

[0027]

[0028]

[0029] Where p represents the path length, E p (X) represents the mean of X when the path length is p. P E represents the value of X when the path length is p. P (Y) represents the mean value of Y when the path length is p. P This represents the value of Y when the path length is p, where X or Y refers to the information entropy or path length; E() is E p () is an abbreviation for ().

[0030] As an example, the score of the candidate vertex relative to the candidate compute node is calculated, with the score PS = (PS1 + PS2) × τ(P). i );

[0031] Where PS1 represents the first-order attribute feature score, PS2 represents the second-order attribute feature score, and dynamic load constraint... |P i | For the current computing node P i The number of vertices included, γ is a constraint coefficient, and n represents the total number of candidate computation nodes.

[0032] As an example, the distributed Skip-Gram learning model learns a vector representation for each user, specifically:

[0033] Before each computing node starts training, a global word vector matrix is ​​created to store each vertex and its initial vector, arranged in descending order of frequency based on the frequency of vertex occurrences in the text library.

[0034] A local cache is created for the currently trained text on each thread to improve the locality of context word data access and avoid cache thrashing caused by randomly generated negative sample words under multi-threaded execution;

[0035] For each text assigned on a thread, a multi-window shared negative sample computation mechanism is used to update the sample vector. Multiple texts are trained simultaneously on one thread to fully utilize the CPU's hardware resources. The center words of different windows are simultaneously used as negative samples of another window during the training process. Then, the two matrices are multiplied. The two matrices refer to the matrix composed of context words within the sliding window and the matrix composed of the center words and negative samples.

[0036] Before the current text training is completed, the center word vector is updated directly in the global word vector matrix, while the vectors of the context words and negative sample words are accessed and updated in the local cache. After the current text is learned, the vectors are updated in the global word vector matrix.

[0037] During text training, a synchronization mechanism based on hot word blocks is used. Each time, a word with the same frequency is randomly selected from the words to enter the synchronization word vector message block and then sent as a message to each computing node for parameter synchronization. This achieves the purpose of updating hot words, ensuring the timeliness of parameter updates, alleviating the pressure on communication bandwidth, and improving the efficiency of distributed training.

[0038] Secondly, the present invention provides an information-driven distributed graph representation learning system, comprising:

[0039] The graph data acquisition unit is used to acquire social network graph data; in the social network graph data, vertices represent users and edges represent social relationships between users.

[0040] The graph data partitioning unit is used to perform multi-order proximity-aware streaming partitioning on the social network graph data, dividing it into multiple computing nodes. The multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of each vertex with each computing node, so as to partition vertices with similar characteristics into the same computing node, and to apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes.

[0041] The path random walk unit is used by each computing node to randomly select a vertex as the starting point from the graph data it is assigned, based on a mixed attribute-aware random walk strategy. A computation mechanism centered on incremental information is then used to perform a random walk from the starting point to generate path information for each vertex. This incremental information-centered computation mechanism iteratively calculates the determination coefficient between the information entropy and path length of the new path obtained after adding a new vertex, based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length. This coefficient determines whether the random walk starting from a particular vertex has ended.

[0042] The walk path learning unit is used to input the walk paths generated by each computing node into the distributed Skip-Gram learning model to learn the vector representation of each user. During the training process, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text has been learned, it is updated in the global word vector matrix to improve the locality of data access and avoid cache jitter caused by multi-threaded learning. It also trains multiple texts on a single thread to make full use of CPU hardware resources. Furthermore, it adopts a synchronization mechanism based on hot word blocks to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

[0043] As an example, the graph data partitioning unit traverses the vertex flow of the social network graph data in a depth-first search manner and sequentially selects the vertex with the highest degree as a candidate vertex. A multi-order proximity-aware streaming graph partitioning strategy is used to calculate the score of each candidate vertex relative to each candidate computing node, and the candidate computing node with the highest score is selected as the computing node to which the vertex is placed. Specifically, by considering first-order and second-order attribute features, the characteristics of the random walk task are satisfied as much as possible, improving the utilization rate of local computing nodes for random walks in a distributed environment for graph representation learning. At the same time, load balancing among computing nodes is considered, and dynamic load constraints are used during the graph partitioning process to ensure load balancing among computing nodes during random walks. The first-order attribute feature refers to the intersection of the candidate vertex's neighbors and the vertex set of the candidate computing node, and the second-order attribute feature refers to the number of common neighbors between the candidate vertex and the vertices in the candidate computing node.

[0044] As an example, the path random walk unit controls each computing node to perform a random walk using a hybrid attribute of vertex common neighbors and degree as a measure of vertex proximity. During the random walk, the information of the random walk path is evaluated based on incremental information related to information entropy to adaptively determine the length of the walk path. The information entropy is used to measure the amount of information contained in the random process. When cross-vertex computation occurs, the information entropy and path length of the path before the cross-vertex occur are referenced, and the number of times the new vertex appears in the new path after the new vertex is added to the walk path and the length of the new path are used as incremental information to calculate the information entropy of the new path after the cross-vertex, so as to reduce redundant computation and communication consumption. Furthermore, after each computing node completes one sampling of all vertices in its local area, it measures the generated context samples based on relative entropy to determine whether the random walk sampling has ended.

[0045] As an example, the path random walk unit: the random walk path starting from vertex u is... Vertices are selected to join the network based on the vertex proximity metric. In the middle; through the current path length L, the current information entropy and vertex vi The number of times n appears in a path of length L+1 after adding the walking path. L+1 (v i Iteratively calculate the latest entropy value in, T is an intermediate parameter, if v i If it already exists in the current path of length L, then otherwise n L (v i ) is the vertex v i The number of times it appears in a path of length L; the mean of the information entropy of the current path. The mean of the current path length E(L), and the mean of the product of the current path information entropy and the path length. Mean of the square of the current path information entropy and the mean of the square of the current path length E(L) 2 The vertex v is obtained through iteration. i The coefficient of determination between the updated information entropy after adding the walking path and the path length L+1 To determine whether a random walk starting at vertex u terminates; where, Find the coefficient of determination At that time, respectively Substitute L+1 into the above equation; Where p represents the path length, E p (X) represents the mean of X when the path length is p. P E represents the value of X when the path length is p. p (Y) represents the mean value of Y when the path length is p. P This represents the value of Y when the path length is p, where X or Y refers to the information entropy or path length; E() is E p () is an abbreviation for ().

[0046] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method described in the first aspect or any possible example of the first aspect.

[0047] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the methods described in the first aspect or any possible example of the first aspect.

[0048] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible example of the first aspect.

[0049] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0050] This invention provides an information-driven distributed graph representation learning method and system. The proposed multi-order proximity-aware streaming graph partitioning strategy is more in line with the characteristics of random walk in a distributed environment, thereby ensuring the utilization rate of local partitions on each computing node and reducing cross-node computation. At the same time, the dynamic load constraint in this strategy can ensure the load balance of each computing node during the distributed random walk task.

[0051] This invention provides an information-driven distributed graph representation learning method and system. The proposed information-centered computation mechanism addresses the issue that existing full-path information computation mechanisms in a distributed environment generate significant redundant computations and high communication costs because the HuGE graph representation learning method, based on information-centered random walks, requires real-time evaluation of the information validity of generated paths and adaptively determines the random walk length. By proposing an incremental information-centered computation mechanism, the effectiveness evaluation of path information can be completed incrementally, thus avoiding redundant computations. Furthermore, communication between computing nodes only requires constant-level messages, thereby reducing communication message costs.

[0052] This invention provides an information-driven distributed graph representation learning method and system. Compared with existing work, the proposed distributed Skip-Gram learning model creates a global word vector matrix and a local cache to access and update word vectors, thereby improving access locality and reducing cache jitter during training and updates. Then, a multi-window shared negative sample computing mechanism is proposed to make full use of CPU hardware resources. In addition, a synchronization mechanism based on hot word blocks is proposed to ensure efficient parameter synchronization in a distributed environment. Attached Figure Description

[0053] Figure 1 A flowchart of an information-driven graph representation learning method provided in an embodiment of the present invention;

[0054] Figure 2 A flowchart illustrating another information-driven graph representation learning method provided in an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of an information-centered random walk provided as an example of the present invention;

[0056] Figure 4 A schematic diagram of the distributed Skip-Gram learning model provided in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the data flow of an information-driven graph representation learning system provided in an embodiment of the present invention;

[0058] Figure 6 This is a flowchart of an information-driven graph representation learning method provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] Next, the technical solutions provided in the embodiments of this application will be described.

[0061] This invention provides an information-driven graph representation learning method and system, comprising the following steps:

[0062] (1) By using a graph partitioning strategy, graph data is distributed to each computing node in a distributed environment;

[0063] (2) According to the random walk strategy, a vertex is randomly selected from the graph data allocated to each computing node as the starting point, and a random walk is performed under the computing mechanism centered on incremental information to generate the path information of each vertex.

[0064] (3) The generated vertex path information is sequentially input into the distributed word embedding model on each computing node to generate a vector representation of each vertex, thereby embedding high-dimensional graph data into a low-dimensional vector space.

[0065] Specifically, step (1) includes:

[0066] (1.1) Based on the number of computing nodes in the distributed environment, place each vertex and its edges in the optimal partition in a vertex flow manner;

[0067] Furthermore, step (1.1) specifically includes:

[0068] (1.1.1) The graph data is traversed in the order of the vertex flow by depth-first search, and the vertex with the highest degree is selected as the candidate vertex in turn;

[0069] (1.1.2) The scores of candidate vertices and each candidate partition are calculated by using a multi-order proximity-aware streaming graph partitioning strategy suitable for the characteristics of random walk tasks. The candidate partition with the highest score is selected as the partition where the vertex is placed. Specifically, by considering first-order attribute features (intersection of the neighbors of the candidate vertex and the vertex set of the candidate partition) and second-order attribute features (number of common neighbors of the candidate vertex and the vertex of the candidate partition), the characteristics of random walk tasks are satisfied as much as possible. This aims to improve the utilization rate of local partitions in random walks in a distributed environment. At the same time, the load balancing between partitions is considered. Dynamic load constraints are used in the graph partitioning process to ensure the load balancing of each computing node during random walks.

[0070] Specifically, step (2) includes:

[0071] (2.1) Implement HuGE-based hybrid attribute-aware random walks for graph data on each computing node;

[0072] Furthermore, step (2.1) specifically includes:

[0073] (2.1.1) Randomly select an unvisited vertex from the graph data as the starting vertex;

[0074] (2.1.2) If the current traversing vertex is u, then randomly select a vertex v from all of u's neighboring vertices;

[0075] (2.1.3) Under HuGE’s hybrid attribute-aware random walk strategy, the two attributes of common neighbors and vertex degree are used as the vertex proximity measure. That is, the probability p of vertex u moving to vertex v is determined by the number of common neighbors Cm(u,v) between vertex u and vertex v, as well as the degree of vertex u deg(u) and the degree of vertex v deg(v). If the vertex u accepts the move, vertex v is added to the walk path. If the vertex v rejects the move, the path moves to vertex v and then backtracks to vertex u.

[0076] (2.2) In the information-centric random walk process, it is also necessary to evaluate the information of the random walk path based on information entropy in order to adaptively determine the length of the walk path. In a distributed environment, to avoid redundant computation and high communication overhead in this process, an incremental information-centric computation mechanism is adopted to evaluate the effectiveness of the generated path information;

[0077] Specifically, step (2.2) includes:

[0078] (2.2.1) The random walk path starting from vertex u is: Select vertices to join via step (2.1) middle;

[0079] (2.2.2) As vertex v in the walking path i With the addition of v, calculate i In Frequency of occurrence Since information entropy can measure how much information a stochastic process contains, it can be based on... Determine the information entropy of the walk path as the walk length L changes. In HuGE's solution, each vertex added to the traversal path requires recalculating the value of each vertex v. i Frequency of occurrence This leads to the latest entropy value of path length L+1. In a distributed environment, once cross-node computation occurs, the path's information entropy needs to be recalculated, and the generated path needs to be sent as a message—a mechanism involving the calculation of the entire path information. As the path length increases, the communication overhead also increases. To avoid redundant computation and high communication costs in this process, the current path length L and the current entropy value can be used as a reference. and vertex v i The number of times it appears in a path of length L+1 after adding the walking path. The latest entropy value is calculated iteratively using incremental information.

[0080] (2.2.3) Information entropy through the walking path The coefficient of determination between the travel length L and the travel length L To determine whether a random walk starting at vertex u terminates, if... If 0.99 ≤ μ < 1, proceed to step (2.1.2); otherwise, record the vertex sequence of the traversal path and proceed to step (2.1.1). This process, like case (2.2.2), also produces redundant computation (requiring updates to the entropy value that increases with the path length) and high communication costs (cross-node computation requires sending the entire path as a message). Under a computation mechanism centered on incremental information, only the mean of the entropy needs to be used. The mean of path length E(L), and the mean of the product of entropy and path length. Mean of the square of the entropy and the mean of the squared path lengths E(L) 2 The iteration yields a path of length L+1.

[0081] (2.3) After all vertices in the graph have completed one sampling, the generated context samples are measured based on the relative entropy in HuGE to determine whether the random walk sampling has ended.

[0082] Furthermore, step (2.3) specifically includes:

[0083] (2.3.1) After all vertices in the graph data have completed one round of random walk sampling, the vertex v can be determined. i The frequency of occurrence q(v) in the collected context samples i Based on relative entropy, q(v) is characterized. i ) and the degree distribution p(v) of the vertices in the graph i The difference between q and q is D(q||q);

[0084] (2.3.2) As the iteration number r increases, observe the difference ΔD between two iterations of D(p||q). r (p||q), if ΔD r If (p||q)≥δ, where 0<δ≤0.01, then proceed to step (2.1); otherwise, the sampling ends.

[0085] Specifically, step (3) includes:

[0086] (3.1) The traversal path information generated on each computing node is sequentially input into the distributed Skip-Gram learning model (referred to as DSGL) on each computing node.

[0087] Furthermore, step (3.1) specifically includes:

[0088] (3.1.1) For each computing node, a walk path (hereinafter referred to as the text library) is generated. Before Skip-Gram training begins, DSGL creates a global word vector matrix that stores each vertex and its initial vector according to the descending order of the frequency of the vertices in the text library.

[0089] (3.1.2) Research found that during the learning process of a path (i.e. a text) by the Skip-Gram model, the context words are reused, and the randomly generated negative sample words will cause cache jitter under multi-threaded execution. To solve this problem, DSGL generates a local cache for the currently trained text during the training process. Before the training of the current text ends, the vectors of the context words and negative sample words are accessed and updated in the local cache. After the current text is learned, the global word vector matrix in (3.1.1) is updated.

[0090] (3.1.3) Further research revealed that when Skip-Gram trains a text at a time, the vector matrix composed of context words in the sliding window and the vector matrix composed of center words and negative sample words are too small to fully utilize the CPU's computing throughput during multiplication. To address this, DSGL uses a multi-window shared negative sample calculation mechanism, which trains multiple texts simultaneously on a single thread to fully utilize the CPU's hardware resources. For example, if the number of texts trained simultaneously is 2, the context words in the sliding windows of the two texts form a context word vector matrix, and a shared negative sample set and two center words form another vector matrix. It should be noted that the center words of different windows are simultaneously used as negative samples of another window during training. Then, the two matrices are multiplied, and the result is used to update the latest vector in (3.1.2).

[0091] (3.1.4) In a distributed environment, the word vectors in the global word vector matrix in (3.1.1) need to be synchronized on each computing node. Traditional global synchronization methods, due to the large number of intermediate parameters, put significant pressure on communication bandwidth, causing delays and affecting training efficiency. Therefore, DSGL employs a synchronization mechanism based on hot word blocks to ensure efficient parameter synchronization in a distributed environment. Specifically, research has found that the frequency of word occurrences in the generated text library follows a power-law distribution, meaning that high-frequency words are more likely to be updated than low-frequency words. Therefore, by creating a synchronization word vector message block, within a certain synchronization time period, a word with the same frequency is randomly selected from the words in the synchronization word vector block and sent as a message to each computing node for parameter synchronization. It should be noted that the number of high-frequency words in the text library is far less than that of low-frequency words, so their probability of being selected into the synchronization word vector message block is also much higher than that of low-frequency words, achieving the goal of updating hot words. This ensures the timeliness of parameter updates while alleviating the pressure on communication bandwidth, thereby improving the efficiency of distributed training.

[0092] (3.2) Generate a vector representation of each vertex through a word embedding model, embedding the graph data into a low-dimensional vector space.

[0093] Figure 1 A flowchart of an information-driven graph representation learning method provided in an embodiment of the present invention; as shown. Figure 1 As shown, it includes:

[0094] S101, Obtain social network graph data; in the social network graph data, vertices represent users, and edges represent social relationships between users.

[0095] S102, Perform multi-order proximity-aware streaming partitioning on the social network graph data and divide it into multiple computing nodes; the multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of the vertex with each computing node, so as to divide vertices with similar characteristics into the same computing node, and apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes;

[0096] S103, each computing node randomly selects a vertex as the starting point from the graph data it is assigned according to the mixed attribute-aware random walk strategy, and performs a random walk from the starting point using an incremental information-centered computing mechanism to generate path information for each vertex; the incremental information-centered computing mechanism refers to iteratively calculating the determination coefficient of the information entropy and path length of the new path obtained after adding a new vertex based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added to the walk path, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length, to determine whether the random walk starting from a certain vertex ends;

[0097] S104 inputs the walk path generated by each computing node into the distributed Skip-Gram learning model to learn the vector representation of each user. During training, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text is learned, it updates the global word vector matrix, which improves the locality of data access and avoids cache jitter caused by multi-threaded learning. It also trains multiple texts on one thread at the same time to make full use of CPU hardware resources. In addition, it adopts a synchronization mechanism based on hot word blocks to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

[0098] like Figure 2 As shown, this invention discloses another information-driven graph representation learning method, the execution of which includes the following steps:

[0099] (1) Perform multi-order proximity-aware streaming partitioning on the graph data. Following the vertex flow order of depth-first search traversal, select the vertex with the highest degree as the candidate vertex in turn. Assuming the number of candidate partitions is n, calculate the scores of the candidate vertices and candidate partitions in turn: PS = (PS1 + PS2) × τ(P i PS1 represents the first-order attribute score (the intersection of the candidate vertex's neighbors and the candidate partition vertex set), PS2 represents the second-order attribute score (the number of common neighbors of the candidate vertex and the candidate partition vertex set), and a dynamic load constraint. |P i| For the current partition P i The number of vertices included, where γ is a constraint coefficient; a larger γ will relax τ(P). i The load constraint is limited. The candidate partition with the maximum score PS with the candidate vertex is selected as the partition where the vertex is placed. This partitioning strategy, which takes into account the random walk characteristics, can ensure both the utilization rate of each partitioned subgraph when performing random walks and the load balance among the computing nodes.

[0100] (2) Randomly select an unvisited vertex u from the subgraph data divided into a node as the starting vertex for random walk, and set the evaluation threshold μ for the walk path length L.

[0101] (3) Start a random walk from the current vertex u. Randomly select a vertex v from the neighboring vertices of vertex u as the candidate vertex for the next hop. If vertex v is not in the current computed node, such as Figure 3 The diagram illustrates how cross-node computation can be performed in a distributed environment with two computing nodes using an incremental information-centric computation mechanism. Specifically, it involves sending incremental information evaluating the effectiveness of the traversal path information to the target vertex, namely: the current path length L and the current entropy value. Vertex v i The number of times it appears in a path of length L+1 after adding the walking path. mean of entropy The mean of path length E(L), and the mean of the product of entropy and path length. Mean of the square of the entropy and the mean of the squared path lengths E(L) 2 Then proceed to step (4) to perform a random walk;

[0102] (4) When selecting the next hop vertex, HuGE's hybrid attribute-aware random walk strategy is adopted, with vertex v selected with probability. The acceptance probability undergoes a walk-through path transition from u to v; otherwise, it follows a 1-P path. (u,v) Backtracking to vertex u; as vertices are added to the traversal path, such as... Figure 3 As shown, a calculation mechanism centered on incremental information is used, utilizing incremental information such as the current path length L and the current entropy value. and vertex v i The number of times it appears in a path of length L+1 after adding the walking path. Iteratively calculate the latest entropy value Right now Where v i If there is an already generated walk path, then otherwise The number of times a vertex appears in the generated path The occurrence count of all vertices assigned to this node will be maintained locally, so the vertex count only needs to be updated locally;

[0103] (5) As the walking path increases, the information entropy of the walking path and the determination coefficient of the path length L can be determined. like Figure 3 As shown, under a computational mechanism centered on incremental information, only the mean of entropy needs to be used. The mean of path length E(L), and the mean of the product of entropy and path length. Mean of the square of the entropy and the mean of the squared path lengths E(L) 2 The iteration yields a path of length L+1. in

[0104] like For example, if μ = 0.99, then record the vertex sequence of the wandering path into the current sample above and below, and proceed to step (3); otherwise, proceed to step (6); therefore, the determination coefficient between information entropy H and wandering length L is used. To determine whether a random walk starting at vertex u terminates;

[0105] (6) If there are still unvisited vertices, proceed to step (2); otherwise, the random walk of the current round ends and proceed to step (7).

[0106] (7) After all vertices have completed one round of sampling, the generated context samples are evaluated based on relative entropy to determine whether the random walk sampling has ended. The relative entropy D is the distribution of vertex occurrences in the context samples collected in the current round r compared to the distribution of vertex degrees in the original graph data. r Compared to the previous round D r-1 The change ΔD r If (p||q)≥δ, for example, if δ=0.01, then proceed to step (1) to start a new round of random walks; otherwise, the sampling ends.

[0107] (8) The collected upper and lower samples (hereinafter referred to as the text library) are input into the distributed word embedding model Skip-Gram (DSGL). The word embedding model generates a vector representation of each vertex, and the graph data is transformed into a low-dimensional vector space representation learning. The working diagram is shown below. Figure 4 As shown;

[0108] (9) Distribute the text to be trained to each computing node according to the number of computing nodes in the distributed environment;

[0109] (10) Before each computing node starts training, considering that the frequency of vertices in the generated text library follows a power law distribution and high-frequency vertices occupy most of the samples in the text library, DSGL creates a global word vector matrix to store each vertex and its initial vector according to the descending order of the frequency of vertices in the text library. This ensures that high-frequency word vertices are arranged more closely together in the cache, which is beneficial to ensuring the locality of data access.

[0110] (11) Create a local cache for the currently trained text on each thread. Research found that during the learning process of Skip-Gram model for a path (i.e. a text), context words are reused, and randomly generated negative sample words will cause cache jitter under multi-threaded execution. To solve this problem, DSGL generates a local cache for the currently trained text during training.

[0111] (12) For the text allocated on each thread, a multi-window shared negative sample calculation mechanism is used to update the vector of the sample. That is, multiple texts are trained on one thread at the same time to make full use of the CPU hardware resources. For example, if the number of texts trained at the same time is 2, then the context words in the sliding window of the two texts form a context word vector matrix, and a shared negative sample set and two center words form another vector matrix. Note that the center words of different windows are used as negative samples of another window at the same time during the training process, and then the two matrices are multiplied.

[0112] (13) Before the training of the current text ends, the center word vector is directly updated in the global word vector matrix in step (10), while the vectors of the context words and negative sample words are accessed and updated in the local cache created in step (11). After the current text is learned, the vectors are updated in the global word vector matrix in step (10).

[0113] (14) During the above steps (11) to (13), each computing node needs to use a synchronization mechanism based on hot word blocks to ensure efficient synchronization of parameters in the global word vector matrix in a distributed environment. Specifically, the study found that the number of words appearing in the generated text library follows a power law distribution, that is, the probability of high-frequency words being updated is greater than that of low-frequency words. Therefore, by creating a synchronization word vector message block, within a certain synchronization time period, such as t = 0.1s, a word is randomly selected from words with the same frequency each time to enter the synchronization word vector block and then sent as a message to each computing node for parameter synchronization. It should be noted that the number of high-frequency words in the text library is much less than that of low-frequency words, so the probability of them being selected into the synchronization word vector message block is also much higher than that of low-frequency words, thus achieving the purpose of updating hot words. This can ensure the timeliness of parameter updates, alleviate the pressure of communication bandwidth, and improve the efficiency of distributed training.

[0114] (15) After all the texts in the text library have been trained, a vector representation of each vertex in a low-dimensional space is generated.

[0115] like Figure 5 The diagram shown illustrates the information-driven distributed graph representation learning of the present invention, which can be specifically divided into: graph data input, multi-order proximity-aware streaming graph data partitioning, random walk sampling centered on incremental information computation, and a distributed Skip-Gram learning model.

[0116] The aforementioned information-driven distributed graph representation learning method can not only efficiently represent graph data information, but also exhibit good scalability and ensure the effectiveness of representation learning.

[0117] In a distributed environment with 8 computing nodes, this paper compares the performance of existing popular distributed graph representation learning systems: KnightKing released by Tsinghua University (K. Yang et al., 2019 paper KnightKing: A Fast DistributedGraph Random Walk Engine), DistDGL released by Amazon (D. Zheng et al., 2020 paper DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs), and PyTorch-BigGraph released by Facebook (A. Lerer et al., 2019 paper PyTorch-BigGraph: A Large Scale Graph Embedding). Compared to the previous three systems, the graph representation learning method proposed in this invention can achieve performance speedups of 2.33x to 129x on various graph datasets of different sizes (up to the Twitter dataset with 1.4 billion edges and 40 million vertices). Compared to load-balancing-based graph partitioning strategies, the multi-order proximity-aware streaming graph partitioning strategy proposed in this invention can reduce cross-node computation by an average of 45%, while also achieving an average performance improvement of 38.9% for random walk tasks. Compared to the graph partitioning time of the three systems mentioned above, the graph partitioning time of this invention can achieve a performance speedup of 25.1x. Furthermore, the distributed Skip-Gram learning model proposed in this invention is comparable to the advanced distributed learning model Pword2vec released by Intel (published in the 2019 paper Parallelizing Word2vec in Shared and Distributed...). Compared to other methods, this invention offers a performance acceleration of 17.37 to 27.95 times, with an average increase in computational throughput of 3.1 times. In downstream tasks, this invention improves prediction accuracy (AUC score) by an average of 11.7%, while for multi-label classification tasks, it improves classification accuracy (Macro-F1 and Micro-F1 scores) by an average of 9.2% and 3.3%, respectively. Furthermore, as the number of computing nodes increases, this invention exhibits a better linear relationship in execution time compared to the other three schemes. Simultaneously, this invention provides a general interface that supports the rapid deployment of traditional random walk-based graph representation learning methods, such as node2vec and Deepwalk, into the distributed system of this invention. Through efficient distributed implementation and a random walk effectiveness evaluation method centered on incremental information, traditional random walk methods can achieve performance and effectiveness improvements.Therefore, the information-driven distributed graph representation learning method and system provided by this invention can efficiently and effectively represent graph data information, and also provides good scalability and versatility.

[0118] Figure 6 The following is an architecture diagram of an information-driven distributed graph representation learning system provided in an embodiment of the present invention, such as... Figure 6 As shown, it includes:

[0119] The graph data acquisition unit 610 is used to acquire social network graph data; the vertices in the social network graph data represent users, and the edges represent social relationships between users.

[0120] The graph data partitioning unit 620 is used to perform multi-order proximity-aware streaming partitioning on the social network graph data, dividing it into multiple computing nodes. The multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of each vertex with each computing node, so as to partition vertices with similar characteristics into the same computing node, and to apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes.

[0121] The path random walk unit 630 is used by each computing node to randomly select a vertex as the starting point from the graph data it is assigned, based on a mixed attribute-aware random walk strategy. A computation mechanism centered on incremental information is used to perform a random walk from the starting point to generate path information for each vertex. The computation mechanism centered on incremental information refers to iteratively calculating the determination coefficient of the information entropy and path length of the new path obtained after adding a new vertex, based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length. This determines whether the random walk starting from a certain vertex has ended.

[0122] The walking path learning unit 640 is used to input the walking paths generated by each computing node into the distributed Skip-Gram learning model to learn the vector representation of each user. During the training process, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text is learned, it is updated in the global word vector matrix to improve the locality of data access and avoid cache jitter caused by multi-threaded learning. It also trains multiple texts on one thread at the same time to make full use of CPU hardware resources. Furthermore, it adopts a synchronization mechanism based on hot word blocks to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

[0123] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0124] Based on the methods described in the above embodiments, this application provides an electronic device. The device may include at least one memory for storing a program and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor performs the methods described in the above embodiments.

[0125] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0126] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0127] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0128] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0129] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0130] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0131] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An information-driven distributed graph representation learning method, characterized in that, Includes the following steps: Obtain social network graph data; in the social network graph data, vertices represent users, and edges represent social relationships between users. The social network graph data is partitioned using a multi-order proximity-aware streaming method, dividing it into multiple computing nodes. The multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of each vertex with each computing node, so as to group vertices with similar characteristics into the same computing node, and to apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes. Each computing node randomly selects a vertex from its assigned graph data as the starting point according to a mixed attribute-aware random walk strategy, and performs a random walk from the starting point using an incremental information-centered computing mechanism to generate path information for each vertex. The incremental information-centered computing mechanism refers to iteratively calculating the determination coefficient of the information entropy and path length of the new path obtained after adding a new vertex based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added to the walk path, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length, to determine whether the random walk starting from a certain vertex ends. The walk paths generated by each computing node are input into the distributed Skip-Gram learning model to learn the vector representation of each user. During training, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text has been learned, it is updated in the global word vector matrix. This improves the locality of data access and avoids cache jitter caused by multi-threaded learning. In addition, multiple texts are trained simultaneously on one thread to make full use of CPU hardware resources. Furthermore, a synchronization mechanism based on hot word blocks is adopted to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

2. The method according to claim 1, characterized in that, The social network graph data is partitioned using a multi-order proximity-aware streaming method, dividing it across multiple computing nodes, specifically as follows: The vertex flow of the social network graph data is traversed using a depth-first search method, and the vertex with the highest degree is selected as the candidate vertex in turn. A multi-proximity-aware streaming graph partitioning strategy is used to calculate the score of each candidate vertex relative to each candidate computing node. The candidate computing node with the highest score is selected as the computing node to place the vertex. Specifically, by considering first-order and second-order attribute features, the characteristics of the random walk task are satisfied as much as possible, thereby improving the utilization rate of local computing nodes in graph representation learning in a distributed environment. At the same time, load balancing among computing nodes is considered. Dynamic load constraints are used in the graph partitioning process to ensure load balancing among computing nodes during the random walk. The first-order attribute features refer to the intersection of the candidate vertex's neighbors and the vertex set of the candidate computing node, and the second-order attribute features refer to the number of common neighbors of the candidate vertex and the vertices in the candidate computing node.

3. The method according to claim 1, characterized in that, The generation of path information for each vertex specifically involves: Each computing node performs a random walk, using a hybrid attribute of vertex common neighbors and degree as the measure of vertex proximity. During the random walk, the information of the random walk path is evaluated based on the incremental information related to information entropy, so as to adaptively determine the length of the walk path; The information entropy is used to measure how much information a random process contains. When cross-vertex computation occurs, the information entropy and path length of the path before crossing the node are referenced. The number of times the new vertex appears in the new path after the new vertex is added to the walking path and the new path length are used as incremental information to calculate the information entropy of the new path after crossing the vertex, so as to reduce redundant computation and communication consumption. After each computing node completes a sampling of all vertices in its local area, it measures the generated context samples based on relative entropy to determine whether the random walk sampling has ended.

4. The method according to claim 3, characterized in that, The evaluation of the information of the random walk path based on information entropy to adaptively determine the length of the walk path is specifically as follows: The random walk path starting from vertex u is Vertices are selected to join the network based on the vertex proximity metric. middle; Based on the current path length L and the current information entropy and vertex v i The number of times n appears in a path of length L+1 after adding the walking path. L+1 (v i Iteratively calculate the latest entropy value in, T is an intermediate parameter, if v i If it already exists in the current path of length L, then otherwise n L (v i ) is the vertex v i The number of times it appears in a path of length L; Based on the average of the current path information entropy The mean of the current path length E(L), and the mean of the product of the current path information entropy and the path length. Mean of the square of the current path information entropy and the mean of the square of the current path length E(L) 2 The vertex v is obtained through iteration. i The coefficient of determination between the updated information entropy after adding the walking path and the path length L+1 To determine whether a random walk starting at vertex u terminates; in, Find the coefficient of determination At that time, respectively Substitute L+1 into the above equation; Where p represents the path length, E p (X) represents the mean of X when the path length is p. P E represents the value of X when the path length is p. p (Y) represents the mean value of Y when the path length is p. P This represents the value of Y when the path length is p, where X or Y refers to the information entropy or path length; E() is E p () is an abbreviation for ().

5. The method according to claim 2, characterized in that, Calculate the score of the candidate vertex relative to the candidate computation node, the score PS = (PS1 + PS2) × τ(P i ); Where PS1 represents the first-order attribute feature score, PS2 represents the second-order attribute feature score, and dynamic load constraint... |P i | For the current computing node P i The number of vertices included, γ is a constraint coefficient, and n represents the total number of candidate computation nodes.

6. The method according to any one of claims 1 to 5, characterized in that, The distributed Skip-Gram learning model learns the vector representation for each user, specifically as follows: Before each computing node starts training, a global word vector matrix is ​​created to store each vertex and its initial vector, arranged in descending order of frequency based on the frequency of vertex occurrences in the text library. A local cache is created for the currently trained text on each thread to improve the locality of context word data access and avoid cache thrashing caused by randomly generated negative sample words under multi-threaded execution; For each text assigned on a thread, a multi-window shared negative sample computation mechanism is used to update the sample vector. Multiple texts are trained simultaneously on one thread to fully utilize the CPU's hardware resources. The center words of different windows are simultaneously used as negative samples of another window during the training process. Then, the two matrices are multiplied. The two matrices refer to the matrix composed of context words within the sliding window and the matrix composed of the center words and negative samples. Before the current text training is completed, the center word vector is updated directly in the global word vector matrix, while the vectors of the context words and negative sample words are accessed and updated in the local cache. After the current text is learned, the vectors are updated in the global word vector matrix. During text training, a synchronization mechanism based on hot word blocks is used. Each time, a word with the same frequency is randomly selected from the words to enter the synchronization word vector message block and then sent as a message to each computing node for parameter synchronization. This achieves the purpose of updating hot words, ensuring the timeliness of parameter updates, alleviating the pressure on communication bandwidth, and improving the efficiency of distributed training.

7. An information-driven distributed graph representation learning system, characterized in that, include: The graph data acquisition unit is used to acquire social network graph data. In the social network graph data, vertices represent users, and edges represent social relationships between users. The graph data partitioning unit is used to perform multi-order proximity-aware streaming partitioning on the social network graph data, dividing it into multiple computing nodes. The multi-order proximity-aware streaming partitioning refers to traversing the vertices in the graph data according to the maximum depth-first search traversal method, considering the number of neighbors of each vertex in each computing node and the number of common neighbors of each vertex with each computing node, so as to partition vertices with similar characteristics into the same computing node, and to apply dynamic load constraints to each computing node to ensure load balance among multiple computing nodes. The path random walk unit is used by each computing node to randomly select a vertex as the starting point from the graph data it is assigned, based on a mixed attribute-aware random walk strategy. A computation mechanism centered on incremental information is then used to perform a random walk from the starting point to generate path information for each vertex. This incremental information-centered computation mechanism iteratively calculates the determination coefficient between the information entropy and path length of the new path obtained after adding a new vertex, based on the current path length, the current path information entropy value, the number of times the vertex appears in the path after being added, the mean information entropy of the current path, the mean length, the mean of the product of information entropy and path length, the mean squared information entropy, and the mean squared path length. This coefficient determines whether the random walk starting from a particular vertex has ended. The walk path learning unit is used to input the walk paths generated by each computing node into the distributed Skip-Gram learning model to learn the vector representation of each user. During the training process, the Skip-Gram learning model generates a local cache for the currently trained text. After the currently trained text has been learned, it is updated in the global word vector matrix, which improves the locality of data access and avoids cache jitter caused by multi-threaded learning. It also trains multiple texts on a single thread to make full use of CPU hardware resources, and adopts a synchronization mechanism based on hot word blocks to ensure that hot words are updated, alleviate communication bandwidth pressure, and improve training efficiency.

8. The system according to claim 7, characterized in that, The graph data partitioning unit traverses the vertex flow of the social network graph data using a depth-first search approach, and sequentially selects the vertex with the highest degree as a candidate vertex. A multi-order proximity-aware streaming graph partitioning strategy is used to calculate the score of each candidate vertex relative to each candidate computing node, and the candidate computing node with the highest score is selected as the computing node to place that vertex. Specifically, by considering first-order and second-order attribute features, the system aims to satisfy the characteristics of the random walk task as much as possible, improving the utilization rate of local computing nodes for random walks in a distributed environment. Simultaneously, load balancing among computing nodes is considered, and dynamic load constraints are used during the graph partitioning process to ensure load balancing among computing nodes during random walks. The first-order attribute features refer to the intersection of the candidate vertex's neighbors and the vertex set of the candidate computing node, while the second-order attribute features refer to the number of common neighbors between the candidate vertex and the vertices in the candidate computing node.

9. The system according to claim 7, characterized in that, The path random walk unit controls each computing node to perform a random walk using a hybrid attribute of vertex common neighbors and degree as the vertex proximity metric. During the random walk, the information of the random walk path is evaluated based on incremental information related to information entropy in order to adaptively determine the length of the walk path. The information entropy is used to measure how much information a random process contains. When cross-vertex computation occurs, the information entropy and path length of the path before crossing the node are referenced. The number of times the new vertex appears in the new path after the new vertex is added to the walking path and the new path length are used as incremental information to calculate the information entropy of the new path after crossing the vertex, so as to reduce redundant computation and communication consumption. And after each computing node completes a sampling of all vertices in its local area, it measures the generated context samples based on relative entropy to determine whether the random walk sampling has ended.

10. The system according to claim 9, characterized in that, The path random walk unit: a random walk path starting from vertex u. Vertices are selected to join the network based on the vertex proximity metric. In the middle; through the current path length L, the current information entropy and vertex v i The number of times n appears in a path of length L+1 after adding the walking path. L+1 (v i Iteratively calculate the latest entropy value in, T is an intermediate parameter, if v i If it already exists in the current path of length L, then otherwise n L (v i ) is the vertex v i The number of times it appears in a path of length L; the mean of the information entropy of the current path. The mean of the current path length E(L), and the mean of the product of the current path information entropy and the path length. Mean of the square of the current path information entropy and the mean of the square of the current path length E(L) 2 The vertex v is obtained through iteration. i The coefficient of determination between the updated information entropy after adding the walking path and the path length L+1 To determine whether a random walk starting at vertex u terminates; where, Find the coefficient of determination At that time, respectively Substitute L+1 into the above equation; Where p represents the path length, E p (X) represents the mean of X when the path length is p. P E represents the value of X when the path length is p. p (Y) represents the mean value of Y when the path length is p. P This represents the value of Y when the path length is p, where X or Y refers to the information entropy or path length; E() is E p () is an abbreviation for ().

Images