Self-supervised optimization unary path heterogeneous graph embedding method based on structural similarity

Abstract

The invention discloses a self-supervised optimization unary path heterogeneous graph embedding method based on structural similarity, and belongs to the technical field of computers. The method comprises the following steps of: balancing node distribution by using a node type selection strategy through a random walk sequence so as to screen out nodes of a target type; based on the target type, the structural similarity between the nodes is calculated by adopting the Euclidean distance and the Adamic-Adaar index, and a target node is screened out through an improved NSWOA multi-target optimization algorithm; generating a random walk node sequence according to the first two steps; and inputting the generated random walk node sequence into a Skip-gram model for training to obtain a final node embedding representation. According to the method, a traditional method depending on a meta-path is abandoned, a self-supervised learning mechanism is adopted to guide the walking process of the nodes, a node type selection strategy is designed, and equalization of node distribution is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a method for self-supervised optimization of heterogeneous graph embedding without meta-paths based on structural similarity. Background Art

[0002] Graph embedding is the most fundamental research process in graph learning. Graph embedding aims to learn an effective low-dimensional node representation for the nodes in a graph. An efficient node representation is low-dimensional and has rich expressive power, that is, it can save more effective information (such as structural information, attribute information, etc.) using low-dimensional data. Graph embedding is widely used in tasks such as node classification, clustering, and link prediction. The result of graph embedding has a very large impact on the accuracy of downstream tasks. How to obtain an effective low-dimensional embedding representation of nodes has always been a problem that researchers have been constantly exploring.

[0003] Networks in the real world are often diverse and complex, and most of them are heterogeneous graphs containing different types of nodes and edges. Heterogeneous graph embedding is to embed the rich and complex structural and semantic information in a heterogeneous graph into an effective low-dimensional node representation. In recent years, researchers have proposed some methods to solve the node embedding representation of heterogeneous graphs, aiming to capture more rich and complex semantic and structural information in heterogeneous graphs. However, most of these methods rely on meta-paths, and the way of randomly walking nodes is guided by pre-defining meta-paths to obtain node representations, and the performance of the method is related to the selection of meta-paths. These meta-paths are pre-defined manually and require prior knowledge of domain experts. There will be semantic biases between different meta-paths, and the selection of meta-paths is also a problem that needs to be considered. Improper selection of meta-paths may cause the problem of node semantic loss, resulting in the inability of meta-path-based methods to capture more complex semantic relationships. Summary of the Invention

[0004] In order to solve the above technical problems existing in the prior art, the present invention provides a method for self-supervised optimization of heterogeneous graph embedding without meta-paths based on structural similarity. The present invention abandons the traditional method that relies on meta-paths, adopts a self-supervised learning mechanism to guide the node walking process, realizes the equalization of node distribution by designing a node type selection strategy to screen out the target node type, and at the same time combines the evaluation of node structural similarity, uses the Euclidean distance and the Adamic-Adar index as similarity metrics, and integrates the NSWOA multi-objective optimization algorithm to accurately identify and screen out the most representative target nodes.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A method for self-supervised optimization of heterogeneous graph embedding without meta-paths based on structural similarity, comprising the following steps:

[0007] Step S1: Through a random walk sequence, the node type selection strategy is used to balance the node distribution and select the target type; specifically:

[0008] S 11 , the queue Q records the m nodes most recently visited in the walk sequence;

[0009] S 12 , introduce a smooth function g(n) = α n , where n is the number of nodes of type q in queue Q, and α is a random number between [0,1], which is used to control the speed of the curve movement;

[0010] S 13 , calculate the transition probability of all types, the calculation formula is as follows:

[0011] (1);

[0012] Among them, v i is the current node, node v i The neighborhood set is N(v i ), the neighborhood type set of node vi is ;

[0013] S 14 , if node v i The neighborhood type set R(v i ) has only type q, then the probability that the next hop node is type q is 1;

[0014] S 15 , if node v i The neighborhood type set R(v i ) If there is no type q, the probability that the next hop node type is type q is 0; otherwise, a smoothing function is used to calculate the probability of type q as the next hop type, and the type of the next hop node is selected according to the type probability;

[0015] S 16 , select the type with the largest transfer probability as the type of the next hop node;

[0016] Step S2: Filter out target nodes by calculating the structural similarity between nodes according to the target type;

[0017] Step S3, according to step S 1、 S2 generates a random walk node sequence;

[0018] In step S4, the generated random walk node sequence is input into the Skip-gram model for training to obtain the final node embedding representation.

[0019] Further, the step S2 filters the target nodes according to the target type, specifically as follows:

[0020] S 21 , If the type of the current node v i is inconsistent with the target type, then filter out the neighborhood nodes of the target type in the neighborhood of the current node v i and select nodes according to the degree values of the neighborhood nodes;

[0021] S 22 , if the type of the current node v i is consistent with the target type, then filter out the neighborhood nodes of the target type in the neighborhood of the current node v i and select nodes according to the structural similarity between the current node v i and the neighborhood nodes.

[0022] Further, the specific steps for calculating the structural similarity of the nodes in the step S2 are as follows:

[0023] Count the number of edges of each type of the node, calculate the Euclidean distance between node pairs, and the calculation formula is as follows:

[0024] (2);

[0025] where x β and y β respectively represent the number of edges of the β-th type containing node v i and node v j , and δ is the number of different types of edges in the network;

[0026] The calculated structural distance is normalized using the Softmax function, and the calculation formula is as follows:

[0027] (3);

[0028] Calculate the Adamic-Adar index between nodes, and the calculation formula is as follows:

[0029] (4);

[0030] where N(v i ) and N(v j ) respectively represent the neighbor node sets of node v i and node v j , N(v i ) ∩ N(v j ) represents their shared neighbor node set, and d(v u ) represents the degree of the shared node v u ;

[0031] According to the above formula, two optimization functions are constructed as follows:

[0032] (5);

[0033] According to the above model formula, the problem of selecting target nodes is transformed into an optimization problem, and the specific steps are as follows:

[0034] D1 Initialize the population; randomly generate an initial whale population, where each whale represents a potential solution, and initialize the algorithm parameters including population size, maximum number of iterations, and inertia weight parameter;

[0035] D2 Calculate fitness; for each whale, evaluate its fitness index under multiple objective functions, and these function values together form an objective vector, which is further integrated into a multi-objective fitness matrix;

[0036] D3 Non-dominated sorting; based on the multi-objective fitness matrix, perform non-dominated level division to stratify the whale population into different Pareto levels; within the same level, perform secondary sorting according to the degree of crowding;

[0037] D4 Calculate crowding degree; for each Pareto level, calculate the crowding degree between whales;

[0038] D5 Selection operation; comprehensively considering the results of non-dominated sorting and crowding degree evaluation, perform a screening operation to construct a new whale population;

[0039] D6 Update position; according to the selected whales, update the position in the following way to generate new candidate solutions;

[0040] Introduce adaptive weight update as follows:

[0041] (6);

[0042] where w(t) is the weight at the current iteration t, w min is the minimum weight value, taking the value of 0.1, w max is the maximum weight value, taking the value of 0.9, and t max is the maximum number of iterations;

[0043] When the whale surrounds and preys on the prey, its position update formula is:

[0044] (7);

[0045] where b is the logarithmic spiral shape constant, X * (t) is the position vector of the current optimal solution, X(t) is the position vector of the current whale, and X(t + 1) is the position vector of the whale at the next moment, D* is the distance between the current whale and the prey, which is the optimal solution so far. l is a random number between [0, 1], and A and C are correlation coefficients, calculated according to the following formula:

[0046] (8);

[0047] where r is a random number between [0, 1], and θ is the convergence factor;

[0048] When the whale approaches the prey in a spiral, the search process is as follows:

[0049] (9);

[0050] where b0 is the initial parameter value, and k is the rate of adjusting the parameter;

[0051] Introduce the optimal neighborhood perturbation strategy, and the formula is as follows:

[0052] (10);

[0053] where is the generated new position;

[0054] When the whale approaches the prey in a spiral, the search process is as follows:

[0055] D7 Repeat the iteration; repeat steps D2 to D6 until the maximum number of iterations is reached;

[0056] D8 Output the final non-dominated solution set, that is, the best trade-off between the distance between nodes and the Adamic-Adar index, and select the specific target nodes.

[0057] Furthermore, the specific steps for the step S3 to generate the node sequence are as follows:

[0058] S 31 , randomly select an initial node, filter out the target type according to the node type selection strategy, and filter out the target nodes according to the target type, and add the target nodes to the node sequence;

[0059] S 32 , sequentially add the filtered nodes to generate a node sequence with a fixed length.

[0060] Furthermore, the specific steps of the step S4 include:

[0061] Execute the Skip-gram model, calculate the node embedding by maximizing the co-occurrence probability of context node pairs, and the formula is as follows:

[0062] (11);

[0063] (12);

[0064] Among them, σ(x) is the sigmoid function, and are the node embeddings of node v i and node v j respectively;

[0065] The Skip-gram model is maximized through the log-likelihood function, and the calculation formula is as follows:

[0066] (13);

[0067] Among them, G = {v j |v j ∈W seq , |i - j| ≤ w size , i ≠ j} is the context node representation of node v i , W seq = (v1,..., v z ) is the walk sequence, with a length of z and a window size of w size ;

[0068] The Skip-gram model introduces negative sampling to maximize the probability that node v i and the randomly sampled negative node v N do not appear in the random walk sequence W seq , and approximates the objective function as the following formula:

[0069] (14);

[0070] Among them, S ∈ Z + represents the number of negative samples, and E represents the expectation;

[0071] The Skip-gram model uses asynchronous stochastic gradient descent to optimize the node embeddings in parallel.

[0072] The beneficial effects of the present invention are as follows: By designing a node type selection strategy, the present invention filters out the type of the next-hop node; and based on the filtered node type, the next-hop node is filtered out through the structural similarity between nodes. This self-supervised random walk method can represent heterogeneous graphs more flexibly and comprehensively, and is more suitable for complex heterogeneous network structures. The present invention does not require prior definition of meta-paths, avoiding the subjectivity and uncertainty of manually specifying meta-paths. The present invention has high flexibility and generalization ability, can adapt to different types of heterogeneous networks, and provides a more automated and universal solution for heterogeneous network embedding. The present invention uses a self-supervised method for random walk of nodes, making up for the limitation of the meta-path-based method that only learns the semantic information of specific paths.

[0073] The present invention is directed to a heterogeneous graph containing different types of nodes and edges, without pre-defining meta-paths, and uses a self-supervised method for random walk of nodes. During the node selection process, if the type of the current node is the same as the target type, the structural similarity of the nodes is used as a metric for selecting the next-hop node; otherwise, the degree value of the node is used as a metric for selecting the next-hop node. The present invention measures the similarity between nodes from the perspective of structural similarity, which can reflect the potential correlation relationship between nodes. Structural similarity has high applicability and robustness in dealing with complex network structures, and has good interpretability, which is helpful for further data analysis and mining tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 Flowchart of the present invention;

[0075] Figure 2 Schematic diagram for implementing the node type selection strategy in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0076] The following further describes the present invention in detail with reference to the drawings and specific embodiments.

[0077] As Figure 1 shown, Figure 1 is the flowchart of the present invention. A self-supervised optimization method for heterogeneous graph embedding based on structural similarity in the present invention specifically includes the following steps:

[0078] Step S1, through a random walk sequence, using the node type selection strategy to balance the node distribution and filter out the target type. Specifically as follows:

[0079] S 11 , the queue Q records the m nodes most recently visited in the walk sequence.

[0080] S 12 , introduce a smoothing function g(n) = α n, where n is the number of nodes of type q in queue Q, and α is a random number between [0, 1], which is used to control the curve movement speed; the smoothing function is used to balance the number of nodes of each node type in the walking sequence to prevent the local optimum of a certain node type.

[0081] S 13 , calculate the transition probabilities of all types, and the calculation formula is as follows:

[0082] (1);

[0083] Among them, v i is the current node, and the neighborhood set of node v i is N(v i ), and the neighborhood type set of node vi is .

[0084] S 14 , if the neighborhood type set R(v i ) of node v i only has type q, then the probability that the next-hop node type is type q is 1;

[0085] S 15 , if the neighborhood type set R(v i ) of node v i does not have type q, then the probability that the next-hop node type is type q is 0; otherwise, use the smoothing function to calculate the probability that type q is the next-hop type, and select the type of the next-hop node according to the type probability.

[0086] S 16 , select the type with the largest transition probability as the type of the next-hop node, which is the target type. Figure 2 shows a schematic diagram of implementing the node type selection strategy of the present invention.

[0087] Step S2, according to the target type, filter out the target nodes by calculating the structural similarity between nodes:

[0088] Since various types of nodes in the heterogeneous network are interconnected to form different types of edges, first count the number of each type of edge of the nodes.

[0089] Use the Euclidean distance to calculate the structural distance between each node and its neighborhood nodes, and the calculation method is as follows:

[0090] (2);

[0091] Among them, x β and y β respectively represent containing node v i and node v jThe number of edges of the β-th type, where δ is the number of different types of edges in the network.

[0092] The obtained structural distance is normalized using the Softmax function, and the calculation formula is as follows:

[0093] (3);

[0094] The greater the distance between nodes, the greater the difference in their structures; the smaller the distance, the more similar the structures between nodes.

[0095] Calculate the Adamic-Adar index between two nodes, and the calculation formula is as follows:

[0096] (4);

[0097] Where, N(v i ) and N(v j ) respectively represent the neighbor node sets of node v i and node v j , N(v i ) ∩ N(v j ) represents their shared neighbor node set, and d(v u ) represents the degree of the shared node v u ;

[0098] The greater the Adamic-Adar index between nodes, the closer the relationship between nodes and the more similar the structures; the smaller the Adamic-Adar index, the more distant the relationship between nodes and the less similar the structures.

[0099] According to the above formula, construct two optimization functions

[0100] (5);

[0101] According to the above model, the problem of selecting target nodes is transformed into an optimization problem of a function. The specific steps are as follows:

[0102] D1 Initialize the population. Randomly generate an initial whale population, and each whale represents a potential solution. Initialize the algorithm parameters including population size, maximum number of iterations, inertia weight and other parameters.

[0103] D2 Calculate the fitness. For each whale, evaluate its fitness indicators under multiple objective functions. These function values together form the objective vector, which is further integrated into the multi-objective fitness matrix.

[0104] D3 Non-dominated sorting. Based on the multi-objective fitness matrix, non-dominated levels are divided to stratify the whale population into different Pareto ranks. Within the same level, a secondary sorting is performed according to the degree of crowding.

[0105] D4 Calculate the crowding degree. For each Pareto level, calculate the crowding degree among whales. This index reflects the distribution compactness of whales in the objective space.

[0106] D5 Selection operation. Combining the results of non-dominated sorting and crowding degree evaluation, a screening operation is performed to construct a new whale population.

[0107] D6 Update the position. According to the selected whales, update the position in the following way to generate new candidate solutions.

[0108] To improve the global search ability in the early stage of algorithm optimization and avoid falling into local optimal situations, an adaptive weight update is introduced:

[0109] (6);

[0110] In the formula, w(t) is the weight at the current iteration number t, w min is the minimum weight, with a value of 0.1, w max is the maximum weight, with a value of 0.9, t max is the maximum number of iterations.

[0111] When a whale encircles and hunts prey, its position update formula is:

[0112] (7);

[0113] In the formula, b is the logarithmic spiral shape constant, X * (t) is the position vector of the current optimal solution; X(t) is the position vector of the current whale, X(t + 1) is the position vector of the whale at the next moment, D * is the distance between the current whale and the prey, which is the optimal solution so far; l is a random number between [0, 1], and A and C are correlation coefficients, calculated as follows:

[0114] (8);

[0115] In the formula, r is a random number between [0, 1], and θ is the convergence factor.

[0116] When a whale approaches the prey in a spiral, the search process is:

[0117] (9);

[0118] Among them, b0 is the initial parameter value, and k is the rate of adjusting the parameter;

[0119] Introduce the optimal neighborhood perturbation strategy, and the formula is as follows:

[0120] (10);

[0121] Wherein, is the newly generated position;

[0122] D7 Repeat the iteration. Repeat steps D2 to D6 until the maximum number of iterations is reached.

[0123] D8 Output the final non-dominated solution set, that is, the best trade-off between the distance between nodes and the Adamic-Adar index, and select specific target nodes.

[0124] Step S3, generate a random walk node sequence according to steps S1 and S2:

[0125] Randomly select an initial node, and filter out the target type according to the node type selection strategy. And filter out the target nodes according to the target type, and add the target nodes to the node sequence. Add the filtered nodes in turn to generate a node sequence with a fixed length.

[0126] Step S4, input the generated random walk node sequence into the Skip-gram model for training to obtain the final node embedding representation:

[0127] First, encode the nodes in the sequence using one-hot encoding according to the walked node sequence.

[0128] Execute the Skip-gram model, and learn the node embedding by maximizing the co-occurrence probability of the context node pairs. The calculation method is as follows:

[0129] (11);

[0130] (12);

[0131] Wherein, σ(x) is the sigmoid function, and are the node embeddings of node v i and node v j respectively.

[0132] The Skip-gram model is maximized through the log-likelihood function, that is, the objective function, and the calculation method is as follows:

[0133] (13);

[0134] Wherein, G = {v j |v j ∈Wseq , |i - j| ≤ w size , i ≠ j} is the context node representation of node v i W seq = (v1, …, v z ) is a random walk sequence with length z and window size w size ;

[0135] The Skip - gram model introduces negative sampling to maximize the probability that node v i and the negative nodes v sampled randomly N do not appear in the random walk sequence W seq , and approximates the objective function as the following formula:

[0136] (14);

[0137] where S ∈ Z + represents the number of negative samples, and E represents the expectation.

[0138] Finally, the Skip - gram model uses asynchronous stochastic gradient descent to optimize node embeddings in parallel.

Claims

1. A self-supervised optimization method for meta-path-free heterogeneous graph embedding based on structural similarity, characterized in that It includes the following steps: Step S1, through a random walk sequence, using a node type selection strategy to balance the node distribution and screen out the target type; specifically: S 11 , the queue Q records the m nodes that have been recently visited in the traversal sequence; S 12 , introduce a smoothing function g(n) = α n , where n is the number of nodes of type q in the queue Q, and α is a random number between [0, 1] used to control the curve movement speed; S 13 , calculate the transition probabilities of all types, and the calculation formula is as follows: (1)； Among them, v i is the current node, and the node v i has a neighborhood set of N(v i ), and the neighborhood type set of the node vi is ; S 14 , if node v i 's neighborhood type set R(v i ) has only type q, then the probability that the next-hop node type is type q is 1; S 15 If the neighborhood type set R(v i ) of node v i does not have type q, the probability that the next-hop node type is type q is 0; Otherwise, use a smoothing function to calculate the probability of type q as the probability of the next-hop type, and select the type of the next-hop node according to the type probability; S 16 , select the type with the highest transition probability as the type of the next-hop node; Step S2, according to the target type, screen out the target nodes by calculating the structural similarity between nodes; Step S3, according to step S 1、 S2, generate a random walk node sequence; Step S4, input the generated random walk node sequence into the Skip-gram model for training to obtain the final node embedding representation.

2. The method for self-supervised optimization of meta-pathless heterogeneous graph embedding based on structural similarity according to claim 1, wherein: The specific method for screening target nodes in step S2 according to the target type is: S 21 If the type of the current node v i is inconsistent with the target type, then filter out the neighboring nodes of the target type in the neighborhood of the current node v i and select a node according to the degree value of the neighboring nodes; S 22 If the type of the current node v i is the same as the target type, then filter out the neighboring nodes of the target type in the neighborhood of the current node v i and select nodes according to the structural similarity between the current node v i and the neighboring nodes.

3. The method for self-supervised optimization of heterogeneous graph embedding without meta-paths based on structural similarity according to claim 1, characterized in that: The specific steps for calculating the structural similarity of nodes in step S2 are as follows: Count the number of edges of each type of node, calculate the Euclidean distance between node pairs, and the calculation formula is as follows: (2)； where x β and y β represent the number of edges of the β-th type containing node v i and node v j respectively, and δ is the number of different types of edges in the network; The calculated structural distance is normalized using the Softmax function, and the calculation formula is as follows: (3)； Calculate the Adamic-Adar index between nodes, and the calculation formula is as follows: (4)； where N(v i ) and N(v j ) respectively represent the sets of neighbor nodes of node v i and node v j , N(v i ) ∩ N(v j ) represents their set of shared neighbor nodes, and d(v u ) represents the degree of the shared node v u ; According to the above formula, construct the following two optimization functions: (5)； According to the above model formula, transform the problem of selecting target nodes into an optimization problem, and the specific steps are as follows: D1 Initialize the population; randomly generate an initial whale population, each whale represents a potential solution, and initialize the algorithm parameters including the population size, the maximum number of iterations, and the inertia weight parameter; D2 Calculate the fitness; for each whale, evaluate its fitness index under multiple objective functions, and these function values together form an objective vector, which is further integrated into a multi-objective fitness matrix; D3 Non-dominated sorting; based on the multi-objective fitness matrix, perform non-dominated level division, and divide the whale population into different Pareto levels; within the same level, perform secondary sorting according to the degree of crowding; D4 Calculate the crowding degree; for each Pareto level, calculate the crowding degree between whales; D5 Selection operation; comprehensively consider the results of non-dominated sorting and crowding degree evaluation, and perform a screening operation to construct a new whale population; D6 Update the position; according to the selected whales, update the position in the following way to generate new candidate solutions; Introduce an adaptive weight update as follows: （6）； Among them, w(t) is the weight at the current iteration number t, and w min is the minimum weight value, with a value of 0.1, and w max is the maximum weight value, with a value of 0.9, and t max is the maximum number of iterations; When a whale surrounds and preys on prey, its position update formula is: （7）； where b is the logarithmic spiral shape constant, is the position vector of the current optimal solution, X(t) is the position vector of the current whale, and X(t + 1) is the position vector of the whale at the next moment. is the distance between the current whale and the prey, which is the optimal solution so far. l is a random number between [0, 1], and A and C are correlation coefficients, calculated as follows: （8）； where r is a random number between [0,1], and θ is the convergence factor; When a whale approaches the prey in a spiral manner, the search process is: （9）； where b0 is the initial parameter value, and k is the rate of adjusting the parameter; Introduce an optimal neighborhood perturbation strategy, and the formula is as follows: （10）； Among them, is the newly generated position; D7 Repeat iteration; repeat steps D2 to D6 until the maximum number of iterations is reached; D8 Output the final non-dominated solution set, that is, the best trade-off between the distance between nodes and the Adamic-Adar index, and select the specific target nodes.

4. The method for self-supervised optimization of meta-pathless heterogeneous graph embedding based on structural similarity according to claim 1, characterized in that: The specific steps for generating the node sequence in step S3 are: S 31 , randomly select an initial node, filter out the target type according to the node type selection strategy, and filter out the target nodes according to the target type, and add the target nodes to the node sequence; S 32 , sequentially add the selected nodes to generate a node sequence of fixed length.

5. The method for self-supervised optimization of meta-pathless heterogeneous graph embedding based on structural similarity according to claim 1, characterized in that: The specific steps of step S4 include: Execute the Skip-gram model, calculate the node embedding by maximizing the co-occurrence probability of context node pairs, and the formula is as follows: （11）； (12)； where σ(x) is the sigmoid function, and are the node embeddings of node v i and node v j respectively; The Skip-gram model realizes maximization through the log-likelihood function, and the calculation formula is as follows: (13)； Among them, G = {v j |v j ∈W seq , |i - j| ≤ w size , i ≠ j} is the context node representation of node v i , W seq = (v1, …, v z ) is a random walk sequence with length z and window size w size ; The Skip-gram model introduces negative sampling to maximize the probability that node v i and the randomly sampled negative node v N do not appear in the random walk sequence W seq and approximates the objective function as follows: (14)； where S ∈ Z + represents the number of negative samples, and E represents the expectation; The Skip-gram model uses asynchronous stochastic gradient descent to parallelly optimize the node embedding.

Images