Lightweight graph comparative learning method inspired by manifold learning

Abstract

The invention discloses a manifold learning inspired lightweight graph contrast learning method, which comprises the following steps: S1, in a neighbor sampling module, obtaining a positive and negative sample pair set of each node by using a graph topological structure, and calculating to obtain the weight of the positive and negative sample pairs by referring to a manifold learning idea, and S2, in a feature coding module, adopting a GCN as an encoder, and carrying out feature coding on the positive and negative sample pairs. S3, the weights of the positive and negative sample pairs are transmitted to a graph comparison module, comparison loss training is carried out, and encoder parameters are updated by minimizing a loss function; according to the method, through graph topology-based k-hop neighbor positive sample definition and manifold learning guided local linear reconstruction weight distribution, structural damage is avoided, key nodes are accurately distinguished, calculation complexity is remarkably reduced, unsupervised end-to-end training is achieved through weighted comparison loss, and while the graph semantic integrity is kept, the method has the advantages of being simple in structure and convenient to operate. The method improves the discrimination of node embedding and the expandability of the model, and is suitable for a large-scale scene.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and graph neural network, and in particular to a lightweight graph contrastive learning method inspired by manifold learning. Background Art

[0002] Graph representation learning aims to learn appropriate low-dimensional representations for graph-structured data to serve a variety of downstream tasks. It has shown strong potential and widespread application in areas such as social network analysis, molecular property prediction, and recommender systems. Compared to traditional Euclidean data, the unstructured nature and complex relationships of graphs pose unique challenges for representation learning. Despite extensive research, its reliance on task-specific labels limits its applicability in real-world scenarios. Inspired by the success of contrastive learning in the field of vision, graph contrastive learning has gradually emerged and achieved excellent performance.

[0003] Graph contrastive learning uses an unsupervised approach to shorten the distance between pairs of positive samples and extend the distance between pairs of negative samples, thereby learning node embedding representations suitable for downstream tasks. Although its performance is comparable to that of supervised learning methods, key challenges still exist in data augmentation strategies and node sampling methods. Commonly used random augmentation strategies may destroy the structural integrity and semantic consistency of the graph. Existing node sampling methods mainly rely on the similarity of structural relationships and node embeddings or probabilistic models to estimate the probability of positive and negative sample pairs. This simple and direct strategy may result in high computational costs and is difficult to apply to large-scale scenarios. In addition, existing methods ignore the differences in the importance of different nodes, a feature that is particularly significant in social networks. Therefore, the present invention proposes a lightweight graph contrastive learning method inspired by manifold learning to solve the problems existing in the prior art. Summary of the Invention

[0004] In response to the above problems, the purpose of the present invention is to propose a lightweight graph contrastive learning method inspired by manifold learning. This manifold learning-inspired lightweight graph contrastive learning method avoids structural destruction and accurately distinguishes key nodes through the k-hop neighbor positive sample definition based on graph topology and manifold learning-guided local linear reconstruction weight distribution. Combined with uniform negative sample weights and GCN lightweight encoding, it significantly reduces the computational complexity and finally achieves unsupervised end-to-end training through weighted contrast loss. While retaining the semantic integrity of the graph, it improves the discriminability of node embedding and the scalability of the model. It is suitable for large-scale scenarios such as social networks and molecular analysis, and effectively solves the problems of semantic perturbation caused by random data enhancement, high computational cost caused by complex negative sampling, and ignoring node importance differences in traditional graph contrastive learning.

[0005] To achieve the purpose of the present invention, the present invention is implemented by the following technical solution: a lightweight graph contrast learning method inspired by manifold learning, comprising the following steps:

[0006] S1. In the neighbor sampling module, the graph topology is used to obtain the set of positive and negative sample pairs for each node. Drawing on the manifold learning idea, the weights of the positive and negative sample pairs are calculated.

[0007] S2. In the feature encoding module, GCN is used as the encoder to obtain the node features of the encoded graph;

[0008] S3. Pass the weights of the positive and negative sample pairs into the graph comparison module, perform contrastive loss training, and update the encoder parameters by minimizing the loss function.

[0009] A further improvement is that: in S1, node v i The k-hop nearest neighbor nodes are its positive pair set Π i , the remaining nodes are its negative pair set N i .

[0010] A further improvement is that: in S1, it is assumed that there are enough nodes, each node and its neighboring nodes are located in a local linear region of the manifold or close to this region, and the local geometric structure of these nodes is characterized by a set of linear coefficients. These coefficients can be used to reconstruct them using the positive sample pairs of each anchor point. The optimization objective can be expressed as follows:

[0011]

[0012] Here, λ is a positive hyperparameter used to balance the reconstruction error and the regularization term.

[0013] A further improvement is that: for a given anchor point v in S1 i , its positive weight is calculated by the following formula:

[0014]

[0015] in, v i The reconstruction weight vector of v i The corresponding embedding matrix of v i The embedding vector of .

[0016] A further improvement is that: for a given anchor point v in S1 i , and set the weights of all its negative pairs to:

[0017]

[0018] A further improvement is that in S2, the graph adjacency matrix A and node features X are input into the GCN encoder to obtain the feature matrix Z, which is formulated as follows:

[0019]

[0020] where Z (l) is the node feature matrix of the lth layer, Z (0) =X, Add self-loops to the normalized adjacency matrix A, W (l) is the learnable weight matrix of layer l, and σ is the activation function.

[0021] A further improvement is that: in the S3, the node v i As an anchor point, its feature in the feature matrix Z is represented as z i , the contrast loss function is:

[0022]

[0023] Among them, τ∈[0,1] is the adjustable temperature coefficient;

[0024] The total loss function can be expressed as:

[0025]

[0026] Further improvements are: it includes a nearest neighbor sampling module, a feature encoding module and a graph comparison module. The nearest neighbor sampling module is used to generate positive and negative sample pairs for each node and calculate weights based on the graph topology structure and manifold learning ideas. The feature encoding module is used to aggregate multi-hop neighbor information through a graph convolutional network to generate low-dimensional node embeddings. The graph comparison module is used to bring positive samples closer and push negative samples further away through weighted contrast loss to achieve unsupervised representation learning.

[0027] The beneficial effects of the present invention are as follows: the present invention avoids structural destruction and accurately distinguishes key nodes through the k-hop neighbor positive sample definition based on graph topology and the local linear reconstruction weight distribution guided by manifold learning. It combines uniform negative sample weights and GCN lightweight encoding to significantly reduce computational complexity, and finally realizes unsupervised end-to-end training through weighted contrast loss. While retaining the semantic integrity of the graph, it improves the discriminability of node embedding and the scalability of the model. It is suitable for large-scale scenarios such as social networks and molecular analysis, and effectively solves the problems of semantic perturbation caused by random data enhancement, high computational cost caused by complex negative sampling, and ignoring node importance differences in traditional graph contrast learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of the steps of the present invention;

[0029] Figure 2 This is a model framework diagram of the present invention. DETAILED DESCRIPTION

[0030] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the examples. The examples are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0031] Example 1

[0032] according to Figure 1 、 Figure 2 As shown, this embodiment provides a lightweight graph contrast learning method inspired by manifold learning, including the following steps:

[0033] S1. In the neighbor sampling module, the graph topology is used to obtain the set of positive and negative sample pairs for each node. Drawing on the manifold learning idea, the weights of the positive and negative sample pairs are calculated.

[0034] The node v i The k-hop nearest neighbor nodes are its positive pair set Π i , the remaining nodes are its negative pair set N i , preserves the semantics of the graph structure and avoids the semantic destruction that may be caused by random enhancement. By defining the k-hop neighbors of a node as positive samples (rather than generating enhanced views), the topology and semantics of the original graph are preserved, solving the problem that existing methods rely on random data enhancement, which may destroy the structure or semantic information of the graph;

[0035] Assuming that there are enough nodes, each node and its neighboring nodes are located in or close to a local linear region of the manifold. The local geometric structure of these nodes is characterized by a set of linear coefficients. These coefficients can be used to reconstruct them using the positive sample pairs of each anchor point. The weights of positive samples are calculated based on local linear reconstruction to avoid the noise introduced by random enhancement and ensure the semantic contribution of important neighbors. Local linear reconstruction for manifold learning is introduced to linearly combine the features of neighboring nodes to approximate the features of the central node. By optimizing the reconstruction error and regularization term, sparse and robust weights are obtained. The optimization objective can be expressed as follows:

[0036]

[0037] Here, λ is a positive hyperparameter used to balance the reconstruction error and the regularization term.

[0038] For a given anchor point v i , its positive weight is calculated by the following formula:

[0039]

[0040] in, v i The reconstruction weight vector of v i The corresponding embedding matrix of v iThe embedded vector of is obtained by quickly calculating the weight through closed-form solution in the formula derivation, avoiding iterative optimization and improving efficiency.

[0041] For a given anchor point v i , and set the weights of all its negative pairs to:

[0042]

[0043] S2. In the feature encoding module, GCN is used as the encoder to obtain the node features of the encoded graph;

[0044] Input the graph adjacency matrix A and node features X into the GCN encoder to obtain the feature matrix Z, as follows:

[0045]

[0046] where Z (l) is the node feature matrix of the lth layer, Z (0) =X, Add self-loops to the normalized adjacency matrix A, W (l) is the learnable weight matrix of the lth layer, σ is the activation function, and the graph convolutional network (GCN) is used to encode the original node features and adjacency matrix. Through multiple layers of nonlinear transformations, high-order structural information and feature information are extracted to obtain a low-dimensional and dense node feature matrix Z. This step converts the graph data into an embedding space suitable for contrastive learning.

[0047] S3 passes the weights of the positive and negative pairs into the graph comparison module for contrastive loss training. By minimizing the loss function, the encoder parameters are updated. Using the weights from S1, a weighted contrastive loss function is designed to maximize the similarity of positive pairs and suppress negative pairs. By minimizing the total loss, the GCN parameters are updated to align positive samples and separate negative samples in the encoded embedding space, thereby improving the discriminative power of node representations.

[0048] Take node v i As an anchor point, its feature in the feature matrix Z is represented as z i , the contrast loss function is:

[0049]

[0050] Among them, τ∈[0,1] is the adjustable temperature coefficient;

[0051] The total loss function can be expressed as:

[0052]

[0053] Example 2

[0054] according to Figure 1 、 Figure 2 As shown, this embodiment provides a lightweight graph contrast learning method inspired by manifold learning, including a neighbor sampling module, a feature encoding module and a graph contrast module. The neighbor sampling module is used to generate positive and negative sample pairs for each node and calculate weights based on the graph topology and manifold learning ideas. Traditional methods rely on random enhancement, which may destroy the graph structure or semantic information. Positive samples are directly defined based on the graph topology, retaining the original structure and avoiding semantic perturbations. Through the local linear assumption of manifold learning, the weights of different neighbors in the positive sample pairs are calculated to distinguish important nodes from non-important nodes. Negative samples use uniform weights to avoid complex negative sampling strategies and reduce computational overhead. The feature encoding module is used to aggregate multi-hop neighbor information through a graph convolutional network to generate low-dimensional node embeddings. GCN uses a message passing mechanism to fuse the node's own features with neighbor features layer by layer to capture the local and global structures of the graph. The original graph structure is directly used to avoid additional view encoding calculations. The graph contrast module is used to bring positive samples closer and push negative samples farther away through weighted contrast loss to achieve unsupervised representation learning. Positive sample weights and uniform negative sample weights are introduced to strengthen the influence of important neighbors and avoid simple average comparison.

[0055] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight graph contrastive learning method inspired by manifold learning, including the following steps: S1. In the neighbor sampling module, the graph topology is used to obtain the set of positive and negative sample pairs for each node. Drawing on the manifold learning idea, the weights of the positive and negative sample pairs are calculated. S2. In the feature encoding module, GCN is used as the encoder to obtain the node features of the encoded graph; S3. Pass the weights of the positive and negative sample pairs into the graph comparison module, perform contrastive loss training, and update the encoder parameters by minimizing the loss function.

2. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: The node v in S1 i The k-hop nearest neighbor nodes are its positive pair set Π i , the remaining nodes are its negative pair set N i .

3. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: In S1, it is assumed that there are enough nodes, and each node and its neighboring nodes are located in a local linear region of the manifold or close to this region. The local geometric structure of these nodes is characterized by a set of linear coefficients. These coefficients can be reconstructed using the positive sample pairs of each anchor point. The optimization objective can be expressed as follows: Here, λ is a positive hyperparameter used to balance the reconstruction error and the regularization term.

4. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: For a given anchor point v in S1 i , its positive weight is calculated by the following formula: in, v i The reconstruction weight vector of v i The corresponding embedding matrix of v i The embedding vector of .

5. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: For a given anchor point v in S1 i , and set the weights of all its negative pairs to:

6. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: In S2, the graph adjacency matrix A and node features X are input into the GCN encoder to obtain the feature matrix Z, which is formulated as follows: where Z (l) is the node feature matrix of the lth layer, Z (0) =X, Add self-loops to the normalized adjacency matrix A, W (l) is the learnable weight matrix of layer l, and σ is the activation function.

7. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: In S3, node v i As an anchor point, its feature in the feature matrix Z is represented as z i , the contrast loss function is: Among them, τ∈[0,1] is the adjustable temperature coefficient; The total loss function can be expressed as:

8. The lightweight graph contrastive learning method inspired by manifold learning according to claim 1, characterized in that: It includes a nearest neighbor sampling module, a feature encoding module and a graph comparison module. The nearest neighbor sampling module is used to generate positive and negative sample pairs for each node and calculate the weights based on the graph topology structure and manifold learning ideas. The feature encoding module is used to aggregate multi-hop neighbor information through a graph convolutional network to generate low-dimensional node embeddings. The graph comparison module is used to bring positive samples closer and push negative samples further away through weighted contrast loss to achieve unsupervised representation learning.

Images