A graph contrastive learning method and system based on redundancy reduction and cluster allocation
By generating cross-view similarity matrices and node clustering, the problem of selecting positive and negative samples in graph contrastive learning is solved, achieving efficient redundancy reduction and cluster allocation, and improving the accuracy of graph embedding representation and global information preservation.
Patent Information
- Application Number
- CN202211728660.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-30
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2042-12-30
AI Technical Summary
Existing graph contrastive learning methods require additional pre-tasks when selecting positive and negative samples, resulting in low efficiency and large space consumption. They also lack guidance from global information and perform poorly, especially on complex high-dimensional datasets.
By generating a cross-view similarity matrix and performing node clustering, and reassigning the centroid matrix, redundant information is reduced and consistency of identical nodes is enforced, avoiding the step of defining positive and negative samples. Data augmentation and cluster allocation techniques are employed.
It effectively reduces the consumption of time and space resources, improves the accuracy of node classification and clustering, preserves global information, and enhances the quality of graph embedding representation.
Smart Images
Figure CN115965800B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of deep learning and graph contrastive learning, and in particular to a graph contrastive learning method and system based on redundancy reduction and cluster allocation. Background Technology
[0002] After its success in computer vision, contrastive learning has been gradually introduced into graph representation learning. However, current graph contrastive learning is mainly based on the idea of MOCO, which generates two views through data augmentation, with identical nodes as positive samples and different nodes as negative samples, such as GRACE and GCA. BGRL completely discards negative samples and only selects positive samples for contrastive learning. Barlow Twins removes the concept of positive and negative samples by using the correlation matrix obtained from the two views.
[0003] The method of comparative learning by defining positive and negative samples requires an additional pre-task in selecting these samples, and is susceptible to false positives and negatives influencing the judgment. This coarse method of selecting positive and negative samples is inefficient and consumes a lot of space. The method of reducing redundancy through correlation matrices eliminates the step of defining positive and negative samples, but the fundamental idea remains unchanged. Lacking guidance from global information, it performs poorly on complex and high-dimensional datasets.
[0004] Therefore, a graph contrastive learning method and system based on redundancy reduction and cluster allocation is proposed to solve the problem that selecting positive and negative samples requires additional pre-tasks, saving a lot of time and space resources. This is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a graph contrastive learning method and system based on redundancy reduction and cluster allocation, which solves the problem that selecting positive and negative samples requires additional pre-tasks and saves a lot of time and space resources.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] A graph contrastive learning method based on redundancy reduction and cluster allocation includes the following steps:
[0008] S101. Obtain the graph dataset;
[0009] S201. A graph data is augmented to generate two distinct views, and an encoder is used to generate embedded representations of the two views.
[0010] S301. Generate a similarity matrix of cross views using the embedding representation of two views, where each value of the similarity matrix is the similarity of nodes in the two views. By having the diagonal elements in the similarity matrix tend to 1 and the off-diagonal elements tend to 0, the redundancy between unrelated nodes is reduced, and the same nodes are forced to tend to be consistent.
[0011] S401. By clustering nodes, find the centroids of each cluster, reassign values to each node based on the centroid matrix, and then force the new values of the two views to be the same.
[0012] The above method, optionally, includes the following specific content in S201:
[0013] Data augmentation consists of two parts: feature masking of the feature matrix and edge masking of the adjacency matrix; firstly, for random vectors... Sampling is performed, where the values of each dimension are... They all independently start from probability 1-p x Extracted from the Bernoulli distribution, thus generating the feature masking matrix:
[0014]
[0015] Randomly delete a portion of edges from the original graph, and first sample a masking matrix M∈{0,1}. N×N If A ij If M = 1, then the value of the edge is M. ij It comes from a probability of 1-p e The Bernoulli distribution is followed if it is true, otherwise it is 0; generate the edge masking matrix:
[0016]
[0017] The encoder generates the embedding Z-axis of two views. 1 Z 2 ;
[0018] Where, x i Embed the i-th row of the representation matrix X for the original view. For the generated feature masking matrix, Z represents the generated edge masking matrix, N represents the number of data samples, D represents the feature dimension of the samples, and Z represents the edge masking matrix. 1 Z 2 This indicates the embedding of the first and second enhanced views.
[0019] Optionally, in S301, the formula for the similarity matrix of the cross views in the above method is as follows:
[0020]
[0021] in, and Let (·) represent the i-th node of the first view and the j-th node of the second view. T This indicates the matrix transpose.
[0022] Optionally, in S301, the node similarity calculation formula is as follows:
[0023]
[0024] Optionally, in S401, the centroid of each cluster is found through node clustering, as described above.
[0025] C = Kmeans(Z)
[0026] Where C is the cluster centroid matrix of the view, obtained by the K-means clustering method, and Z represents the embedding representation matrix of the view;
[0027] Reassign values to each node based on the centroid matrix:
[0028] Q = Z·C T
[0029] Where Q represents the node assignment for the cluster centroid reassignment, (·) T Indicates matrix transpose;
[0030]
[0031] Where P is the soft distribution of node assignments for the reassignment of the cluster centroid, Softmax(·) is the soft assignment function, and τ is the temperature coefficient;
[0032] The difference in the probability distribution of reassigned nodes in the two views is represented by the following formula:
[0033]
[0034] A graph contrastive learning system based on redundancy reduction and cluster allocation, used to implement any of the above-mentioned graph contrastive learning methods based on redundancy reduction and cluster allocation, includes a data acquisition module, a data augmentation module, a redundancy reduction module, and a cluster exchange assignment module connected in sequence.
[0035] The data acquisition module is used to acquire graph datasets;
[0036] The data augmentation module is used to generate two different views from a graph by data augmentation, and the encoder generates the embedding representations of the two views.
[0037] The redundancy reduction module is used to generate a similarity matrix of cross views using the embedding representations of two views. Each value of the similarity matrix is the similarity of nodes in the two views. By having the diagonal elements in the similarity matrix tend to 1 and the off-diagonal elements tend to 0, the redundancy between unrelated nodes is reduced, and the same nodes are forced to tend to be consistent.
[0038] The cluster exchange assignment module is used to find the centroid of each cluster by clustering nodes, reassign values to each node according to the centroid matrix, and then force the new assignments of the two views to be the same.
[0039] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device containing the storage medium executes the aforementioned graph contrastive learning method based on redundancy reduction and cluster allocation.
[0040] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the graph contrast learning method based on redundancy reduction and cluster allocation described above.
[0041] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a graph contrastive learning method and system based on redundancy reduction and cluster allocation: by jointly learning the cross-view similarity matrix and cluster allocation, it not only reduces the redundant information of the data, but also makes the embedded representation have global similarity node consistency; there is no need to define the contrastive learning algorithm of positive and negative samples, which solves the problem of needing additional pre-tasks to select positive and negative samples, saving a lot of time and space resources. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0043] Figure 1 A flowchart of a graph contrastive learning method based on redundancy reduction and cluster allocation provided by the present invention;
[0044] Figure 2 This is a framework diagram of the graph contrastive learning algorithm (CGRLC) based on redundancy reduction and cluster allocation provided by the present invention;
[0045] Figure 3 Comparison results of CGRLC with algorithms such as GRACE, GCA and CCA-SSG in node classification provided by this invention;
[0046] Figure 4 The comparison results of CGRLC with algorithms such as GRACE, GCA and CCA-SSG provided by this invention in node clustering;
[0047] Figure 5 The comparison results of CGRLC provided by this invention with algorithms such as GRACE, GCA and CCA-SSG in similarity search;
[0048] Figure 6 The present invention provides a structural block diagram of a graph contrastive learning system based on redundancy reduction and cluster allocation. Detailed Implementation
[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0050] This invention can be used in a wide variety of general-purpose or special-purpose computing environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.
[0051] Reference Figure 1 As shown, this invention discloses a graph contrastive learning method based on redundancy reduction and cluster allocation, comprising the following steps:
[0052] S101. Obtain the graph dataset;
[0053] S201. A graph data is augmented to generate two distinct views, and an encoder is used to generate embedded representations of the two views.
[0054] S301. Generate a similarity matrix of cross views using the embedding representation of two views, where each value of the similarity matrix is the similarity of nodes in the two views. By having the diagonal elements in the similarity matrix tend to 1 and the off-diagonal elements tend to 0, the redundancy between unrelated nodes is reduced, and the same nodes are forced to tend to be consistent.
[0055] S401. By clustering nodes, find the centroids of each cluster, reassign values to each node based on the centroid matrix, and then force the new values of the two views to be the same.
[0056] Furthermore, the specific content of S201 is as follows:
[0057] Data augmentation consists of two parts: feature masking of the feature matrix and edge masking of the adjacency matrix; firstly, for random vectors... Sampling is performed, where the values of each dimension are... They all independently start from probability 1-p x Extracted from the Bernoulli distribution, thus generating the feature masking matrix:
[0058]
[0059] Randomly delete a portion of edges from the original graph, and first sample a masking matrix M∈{0,1}. N×N If A ij If M = 1, then the value of the edge is M. ij It comes from a probability of 1-p e The Bernoulli distribution is followed if it is true, otherwise it is 0; generate the edge masking matrix:
[0060]
[0061] The encoder generates the embedding Z-axis of two views. 1 Z 2 ;
[0062] Where, x i Embed the i-th row of the representation matrix X for the original view. For the generated feature masking matrix, Z represents the generated edge masking matrix, N represents the number of data samples, D represents the feature dimension of the samples, and Z represents the edge masking matrix. 1 Z 2 This indicates the embedding of the first and second enhanced views.
[0063] Furthermore, in S301, the formula for the similarity matrix of the cross views is as follows:
[0064]
[0065] in, and Let (·) represent the i-th node of the first view and the j-th node of the second view. T This indicates the matrix transpose.
[0066] Furthermore, in S301, the formula for calculating node similarity is as follows:
[0067]
[0068] Furthermore, in S401, the centroid of each cluster is found through node clustering:
[0069] C = Kmeans(Z)
[0070] Where C is the cluster centroid matrix of the view, obtained by the K-means clustering method, and Z represents the embedding representation matrix of the view;
[0071] Reassign values to each node based on the centroid matrix:
[0072] Q = Z·C T
[0073] Where Q represents the node assignment for the cluster centroid reassignment, (·) T Indicates matrix transpose;
[0074]
[0075] Where P is the soft distribution of node assignments for the reassignment of the cluster centroid, Softmax(·) is the soft assignment function, and τ is the temperature coefficient;
[0076] The difference in the probability distribution of reassigned nodes in the two views is represented by the following formula:
[0077]
[0078] This application addresses the issue from the perspective of embedding relevance, reducing the similarity of irrelevant nodes and increasing the consistency of similar nodes. Specifically, firstly, data augmentation is performed on the original view to obtain two contrasting views. An encoder is then used to obtain the embedding representations of the two views, which are multiplied and normalized into a cosine similarity matrix. Then, redundancy is reduced by ensuring that the similarity of identical nodes in the two views approaches 1 and the correlation between different nodes approaches 0. Secondly, the graph embedding representations are clustered to obtain the centroid matrix for each cluster. The centroid matrix is then multiplied by the embedding representation matrix to obtain a new assignment. Finally, soft assignments are used to calculate cross-entropy between the assignment matrix of one view and the assignment matrix of the other contrasting view, forcing nodes distributed in the same cluster to have the same representation. Considering the ambiguity of positive and negative comparison samples during graph-to-graph comparison, the definition of positive and negative samples is effectively solved by constraining the global consistency of the cross-views and reducing noise and redundant information. Therefore, the learned latent representations are more conducive to constructing the intrinsic similarity of data for node classification tasks. Based on redundancy reduction and cluster assignment, a more accurate embedding representation can be constructed for classification. Figure 2 The diagram shows the algorithm block diagram of the CGRLC method.
[0079] In one specific embodiment, for real-world graph data, noise is inevitably mixed into the original features. Therefore, the graph embedding representation learned from the original features is inaccurate. To address this issue, noise in the original data is eliminated through node feature masking and edge removal:
[0080]
[0081] For random vectors Sampling is performed, where the values of each dimension are... They all independently start from probability 1-p x Extracted from the Bernoulli distribution;
[0082] Randomly delete a portion of edges from the original graph, and first sample a masking matrix M∈{0,1}. N×N If A ij =1, then the value of the edge is M. ij It comes from a probability of 1-p e The Bernoulli distribution is followed if it is true, otherwise it is 0; generate the edge masking matrix:
[0083]
[0084] The encoder generates the embedding Z-axis of two views. 1 Z 2 ;
[0085] Where, x i Embed the i-th row of the representation matrix X into the original view; For the generated feature masking matrix, Z represents the generated edge masking matrix, N represents the number of data samples, D represents the feature dimension of the samples, and Z represents the edge masking matrix. 1 Z 2 This indicates the embedding of the first and second enhanced views.
[0086] Redundancy reduction phase:
[0087] Directly classifying nodes using the generated embedding representations yields too low an accuracy, while traditional contrastive learning has a rather coarse definition of positive samples. This method uses the similarity between two nodes as the basis for graph comparison. First, a similarity matrix of the cross-view is generated using the embeddings of the two views, as shown in the following formula:
[0088]
[0089] In this matrix, each value represents the similarity between nodes in two views. and Let (·) represent the i-th node of the first view and the j-th node of the second view. T This represents the matrix transpose; redundancy reduction requires bringing identical nodes in two views closer together and distancing dissimilar nodes further apart; the node similarity calculation formula is as follows:
[0090]
[0091] By making the diagonal elements in the cross-view similarity matrix tend to 1 and the off-diagonal elements tend to 0, redundancy between unrelated nodes is reduced, and identical nodes are forced to tend to be consistent.
[0092] Cluster exchange assignment
[0093] Redundancy reduction ignores potentially similar nodes. By clustering nodes, the centroids of each cluster are found, and each node is reassigned based on the centroid matrix. Then, the reassigned values are forced to be the same in both views.
[0094] C = Kmeans(Z)
[0095] Where C is the cluster centroid matrix of the view, obtained by the K-means clustering method, and Z represents the embedding representation matrix of the view.
[0096] Q = Z·C T
[0097] Where Q is the value assigned to the node whose centroid has been reassigned in the cluster.
[0098]
[0099] Where P is the soft distribution of node assignments for the reassignment of the cluster centroid, Softmax(·) is the soft assignment function, and τ is the temperature coefficient;
[0100] The difference in the probability distribution of reassigned nodes in the two views is represented by the following formula:
[0101]
[0102] The overall loss function is:
[0103] L = L F +λL s
[0104] Where λ is the balance coefficient.
[0105] Furthermore, this application optimizes the system equipment and process flow, thereby addressing the shortcomings of the existing system and achieving process optimization, with the following results:
[0106] To fully verify the effectiveness of the CGRLC method of this invention, the performance of the CGRLC method was first tested on seven commonly used basic datasets (cora, citeseer, pubmed, Amazon.computers, Amazon.photo, Coauthor.CS, Coauthor.Physics), and compared with the following four supervised graph representation learning algorithms and ten currently popular graph representation learning algorithms.
[0107] Results analysis:
[0108] (1) See Figure 3 (Figures 3.1 and 3.2 show the results of node classification, used to measure the accuracy of graph embedding representation.) As shown, the CGRLC algorithm significantly outperforms other comparative algorithms. Taking the computer dataset as an example, the CGRLC algorithm outperforms the second-best CCA-SSG algorithm by 0.33 percentage points in the ACC metric. This verifies the advantages and effectiveness of the method proposed in this embodiment. The CGRLC algorithm achieves better clustering results mainly for the following two reasons: First, the CGRLC algorithm preserves the global features of nodes through clustering and increases the accuracy of comparison through cluster allocation; second, simultaneously performing cluster allocation and redundancy reduction learning can obtain excellent graph embedding representations of node-level and semantic information.
[0109] (2) See Figure 4 As shown, the CGRLC algorithm outperforms the three contrastive graph representation learning algorithms: GRACE, GCA, and CCA-SSG. GRACE and GCA algorithms learn graph representations based on traditional contrastive learning. However, redundant information in the embedded representations makes the learned embeddings insufficient to reveal the inherent structure of the data, thus limiting their classification performance. The CCA-SSG algorithm mines in-view and out-of-view correlations but does not consider global information in the graph data. Therefore, its clustering performance is also limited.
[0110] (3) See Figure 5 As shown, the CGRLC algorithm outperforms the three contrastive graph representation learning algorithms GRACE, GCA, and CCA-SSG on the similarity search task. This demonstrates that, compared to traditional contrastive learning, it can better reduce redundancy in the cross-view similarity matrix and better preserve global information through cluster allocation.
[0111] and Figure 1 Correspondingly, this invention also provides a graph contrastive learning system based on redundancy reduction and cluster allocation, used to implement a graph contrastive learning method based on redundancy reduction and cluster allocation, the specific structural block diagram of which is as follows: Figure 6 As shown, it includes a data acquisition module, a data enhancement module, a redundancy reduction module, and a cluster exchange assignment module connected in sequence.
[0112] The data acquisition module is used to acquire graph datasets;
[0113] The data augmentation module is used to generate two different views from a graph by data augmentation, and the encoder generates the embedding representations of the two views.
[0114] The redundancy reduction module is used to generate a similarity matrix of cross views using the embedding representations of two views. Each value of the similarity matrix is the similarity of nodes in the two views. By having the diagonal elements in the similarity matrix tend to 1 and the off-diagonal elements tend to 0, the redundancy between unrelated nodes is reduced, and the same nodes are forced to tend to be consistent.
[0115] The cluster exchange assignment module is used to find the centroid of each cluster by clustering nodes, reassign values to each node according to the centroid matrix, and then force the new assignments of the two views to be the same.
[0116] This invention also provides a storage medium, which includes stored instructions, wherein, when the instructions are executed, the device where the storage medium is located executes the above-described graph comparison learning method based on redundancy reduction and cluster allocation.
[0117] This invention also provides an electronic device, specifically including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors to perform a graph contrast learning method based on redundancy reduction and cluster allocation.
[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0119] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both.
[0120] To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality above. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0121] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A graph contrastive learning method based on redundancy reduction and cluster assignment, characterized in that, The method comprises the following steps: S101. Obtain a graph data set; S201. One graph data generates two different views through data augmentation, and embedding representations of the two views are generated through an encoder; S301. A similarity matrix of cross views is generated using the embedding representations of the two views, wherein each value of the similarity matrix is the similarity of nodes in the two views, diagonal elements in the similarity matrix tend to 1, and non-diagonal elements tend to 0, thereby reducing the redundancy between unrelated nodes and forcing the same nodes to be consistent; S401. The centroids of each cluster are found through node clustering, each node is re-assigned according to the centroid matrix, and the new assignments of the two views are forced to be the same; In S301, the formula of the similarity matrix of the cross views is as follows: wherein and denotes the i-th node of the first view and the j-th node of the second view, denotes matrix transposition; In S301, the formula for calculating the similarity of nodes is as follows: ； In S401, the centroids of each cluster are found through node clustering: wherein, is the matrix of cluster centroids of views, where is obtained by the clustering method, is the matrix of embedding representations of views; Each node is re-assigned according to the centroid matrix: wherein reassigning values to the nodes for the cluster centroids, denotes matrix transposition; where P is the soft distribution of the node assignment of the cluster centroid reassignment, is a soft assignment function, is a temperature coefficient; The difference between the probability distributions of the re-assigned nodes in the two views is represented by the following formula: 。 2. A graph contrastive learning system based on redundancy reduction and cluster assignment, characterized in that, The method for realizing the graph contrast learning method based on redundancy reduction and cluster allocation of claim 1 comprises a data acquisition module, a data augmentation module, a redundancy reduction module, and a cluster exchange assignment module connected in sequence; The data acquisition module is used to obtain a graph data set; The data augmentation module is used to generate two different views through data augmentation for one graph data, and embedding representations of the two views are generated through an encoder; The redundancy reduction module is used to generate a similarity matrix of cross views using the embedding representations of the two views, wherein each value of the similarity matrix is the similarity of nodes in the two views, diagonal elements in the similarity matrix tend to 1, and non-diagonal elements tend to 0, thereby reducing the redundancy between unrelated nodes and forcing the same nodes to be consistent; The cluster exchange assignment module is used to find the centroids of each cluster through node clustering, re-assign each node according to the centroid matrix, and force the new assignments of the two views to be the same.