Siamese Network Contrastive Representation Learning Method and Device Optimized Based on Graph Clustering

Abstract

The present invention provides a Siamese network contrastive representation learning method and device based on graph clustering optimization. The method includes: obtaining an original graph, and respectively inputting the original graph into a trained online network model and a target network model to obtain a first embedding representation and a second embedding representation; wherein, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph; the parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism; obtaining a finally learned node embedding representation based on the first embedding representation and the second embedding representation, and performing downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation. The present invention achieves a good representation learning effect without label information and a large number of negative samples.

Description

Technical Field

[0001] The present invention relates to the technical field of graph learning, and in particular to a twin network contrast representation learning method and device based on graph clustering optimization. Background Art

[0002] The graph representation learning method is a method that encodes graph nodes into a low-dimensional representation space, which can effectively improve the graph information representation ability while reducing the information dimension. This method has attracted extensive research and exploration, and has been widely applied and developed in fields such as literature indexing networks, bioinformatics networks, and social media networks. Common graph representation learning methods can be classified into traditional methods based on matrix factorization, deep learning methods based on graph neural networks, especially graph convolutional networks (GCNs), and deep learning methods based on graph contrast learning according to their development time and technical characteristics. The representation method based on graph contrast learning introduces the idea of contrast learning on the basis of the original deep learning method. By comparing the similarity of graph node representations, it effectively reduces the dependence on graph label information and is an important self-supervised learning method, achieving good research results in computer vision and text processing.

[0003] Although the existing graph contrast learning methods have achieved certain research results, there are still some defects: for example, the existing contrast learning methods based on mutual information (MI) need to maintain an additional MI estimator to evaluate positive and negative sample pairs, which brings a large amount of additional computational overhead. At the same time, the existing graph contrast learning methods lack design for the selection of negative samples, resulting in the model introducing a considerable number of pseudo-negative samples, which affects the contrast learning ability of the existing methods. In addition, the existing graph contrast learning methods rely on a large number of negative sample pairs to avoid model collapse during the training process, which also limits the generalization ability of the model. Summary of the Invention

[0004] The present invention provides a twin network contrast representation learning method and device based on graph clustering optimization, which are used to solve the defect in the prior art that relying on a large number of negative sample pairs limits the generalization ability of the model, and achieve good representation learning effects without label information and a large number of negative samples.

[0005] The present invention provides a twin network contrast representation learning method based on graph clustering optimization, including:

[0006] Obtain an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0007] Wherein, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained without supervision based on the first enhanced graph and the second enhanced graph;

[0008] The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism;

[0009] Based on the first embedding representation and the second embedding representation, the finally learned node embedding representation is obtained, and downstream tasks such as node classification, node clustering, and visualization are performed based on the node embedding representation.

[0010] According to a method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention, the training process of the online network model includes:

[0011] Obtain an original graph sample, and obtain a first enhanced graph and a second enhanced graph based on the original graph sample;

[0012] Input the first enhanced graph and the second enhanced graph into the online network model to obtain a first online representation and a second online representation, and input the first enhanced graph and the second enhanced graph into the target network model to obtain a first target representation and a second target representation;

[0013] Based on the first online representation, the second online representation, the first target representation, and the second target representation, obtain the comprehensive loss of the online network model and the target network model;

[0014] Based on the comprehensive loss, perform backpropagation gradient optimization to update the parameter gradient of the online network model until the training is completed.

[0015] According to a method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention, the training process of the target network model includes:

[0016] Obtain a fixed ratio of momentum update and the training process parameters of the online network model;

[0017] Based on the fixed ratio and the training process parameters of the online network model, perform iterative update on the target network model until the training is completed.

[0018] According to a method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention, obtaining the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation includes:

[0019] Perform clustering on the first online representation to obtain a clustering result, and obtain a clustering loss;

[0020] Based on the clustering result, the first online representation, and the second online representation, obtain a cross-graph contrast loss;

[0021] Obtain a cross-network contrast loss based on the clustering result, the first online representation, and the first target representation;

[0022] Obtain a comprehensive loss of the online network model and the target network model based on the clustering loss, the cross-graph contrast loss, and the cross-network contrast loss.

[0023] According to a method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention, the online network model includes a first graph neural network, a first multi-layer perceptron, and a clustering layer;

[0024] Cluster the first online representation to obtain a clustering result and obtain a clustering loss, including:

[0025] Input the first enhanced graph and the second enhanced graph into the first graph neural network, and output a first online representation through the first multi-layer perceptron;

[0026] Input the first online representation into the clustering layer to output a clustering result of positive samples, and obtain a clustering loss according to the clustering result.

[0027] According to a method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention, after clustering the first online representation to obtain a clustering result and obtaining a clustering loss, it further includes:

[0028] Obtain a first similarity of the first online representation at the target clustering point and a second similarity of the first target representation at the target clustering point;

[0029] Obtain an average similarity based on the first similarity and the second similarity, obtain an auxiliary target distribution, calculate the KL divergence based on the average similarity and the auxiliary target distribution, and obtain the target point clustering to which it belongs;

[0030] Obtain the similarity of the first enhanced graph through contrastive learning by the online network model and the target network model based on the target point clustering to which it belongs;

[0031] Obtain the similarity of the first enhanced graph and the second enhanced graph through contrastive learning by the online network model based on the target point clustering to which it belongs.

[0032] The present invention also provides a device for contrastive representation learning of a siamese network optimized based on graph clustering, including:

[0033] An acquisition module, configured to acquire an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0034] Among them, both the online network model and the target network model obtain the first enhanced graph and the second enhanced graph from the original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph;

[0035] The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism;

[0036] An application module, configured to obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method for contrastive representation learning of a twin network optimized based on graph clustering as described in any one of the above.

[0038] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for contrastive representation learning of a twin network optimized based on graph clustering as described in any one of the above.

[0039] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for contrastive representation learning of a twin network optimized based on graph clustering as described in any one of the above.

[0040] The method and device for contrastive representation learning of a twin network optimized based on graph clustering provided by the present invention obtain an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation; among them, both the online network model and the target network model obtain the first enhanced graph and the second enhanced graph from the original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph; the parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism; obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation. The present invention achieves a good representation learning effect without label information and a large number of negative samples. Description of the Drawings

[0041] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0042] Figure 1 It is one of the schematic flowcharts of the method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention;

[0043] Figure 2 It is the framework diagram of the method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention;

[0044] Figure 3 It is the schematic structural diagram of the device for the method for contrastive representation learning of a siamese network optimized based on graph clustering provided by the present invention;

[0045] Figure 4 It is the schematic structural diagram of the electronic device provided by the present invention. Specific Embodiments

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0047] The following will describe Figures 1 - 4 the method for contrastive representation learning of a siamese network optimized based on graph clustering of the present invention.

[0048] Referring to Figure 1 , the present invention provides a method for contrastive representation learning of a siamese network optimized based on graph clustering, including:

[0049] Step 110, obtain an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0050] Among them, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained without supervision based on the first enhanced graph and the second enhanced graph;

[0051] The parameters of the target network model are updated by the momentum update mechanism to update the parameters of the online network model;

[0052] Specifically, this embodiment is about the application process of the online network model and the target network model. The original graph is input into the trained online network model and the target network model respectively to obtain the first embedding representation and the second embedding representation.

[0053] It should be noted that the online network model in this embodiment is obtained by enhancing the original graph samples to obtain enhanced graphs and performing unsupervised training on the enhanced graphs. Specifically, it is embodied as follows: two enhanced graphs are obtained by combining four basic graph enhancement strategies, that is, the first enhanced graph is obtained by the combination of sub-sampling + graph diffusion + node feature masking, and the second enhanced graph is obtained by the combination of sub-sampling + edge replacement + node feature masking. The four basic enhancement strategies are as follows:

[0054] 1. Graph Diffusion (GD) is an enhancement strategy that introduces the global information of the graph into the original graph. By performing a series of operations on the adjacency matrix and degree matrix of the original graph, the obtained graph contains graph topology information at various levels. Specifically, the graph diffusion strategy adopted in this scheme can be described as:

[0055] S = α(I - (1 - α)D -1 / 2 AD -1 / 2 ) -1 (1)

[0056] Among them, S represents the adjacency matrix of the graph obtained by graph diffusion, α is a predefined hyperparameter used to balance the proportion of local information and global information, I represents the identity matrix, A represents the adjacency matrix of the original graph, and D represents the degree matrix of the original graph.

[0057] 2. Sub Sampling (SS) is an enhancement strategy that obtains a smaller subgraph by cropping the original graph. A certain number of nodes are randomly selected from the original graph, and these nodes and their topological information and feature information are retained, thereby obtaining a subgraph of a fixed size. The sub-sampling enhancement strategy can not only effectively reduce the subsequent calculation cost of the model, but also, similar to the pooling layer in computer vision, sub-sampling will also improve the representation learning ability of the model to a certain extent.

[0058] 3. Edge Modification (EM) is an enhancement strategy that randomly removes a certain number of edges from the original graph and adds an equal number of edges. Through this strategy, while retaining the attribute (feature) information of the original graph, the topological structure of the graph is made more complex. Specifically, for the adjacency matrix A of the original graph, according to the given edge replacement ratio P (for example, 0.2), P / 2 edges are deleted from the original graph, and then P / 2 new edges are added.

[0059] 4. Node Feature Masking (NFM) is an augmentation strategy that randomly selects some nodes in the original graph and sets all their feature information to 0. Specifically, nodes are sampled from the attribute matrix X of the original graph at a fixed ratio P (e.g., 0.2), and all features (attributes) of the sampled nodes are set to 0.

[0060] Step 120: Obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation.

[0061] Specifically, in this embodiment, the output results of the online network model and the target network model are processed, and the sum of the two obtained embedding representations is used as the finally learned representation and applied to downstream tasks such as node classification and node clustering, that is:

[0062] g = g θ + g ξ (2)

[0063] where g represents the finally learned node embedding representation of this solution, and g θ and g ξ represent the embedding representations obtained by the original graph passing through the trained online network and the target network respectively.

[0064] The twin network contrastive representation learning method based on graph clustering optimization provided by the present invention obtains the original graph and inputs the original graph into the trained online network model and the target network model respectively to obtain the first embedding representation and the second embedding representation; wherein, both the online network model and the target network model obtain the first enhanced graph and the second enhanced graph based on the original graph sample and perform unsupervised training based on the first enhanced graph and the second enhanced graph; the parameters of the target network model are updated by the momentum update mechanism to update the parameters of the online network model; the finally learned node embedding representation is obtained based on the first embedding representation and the second embedding representation, and node classification and node clustering are performed based on the node embedding representation. The present invention achieves a good representation learning effect without label information and a large number of negative samples.

[0065] Based on the above embodiments, the training process of the online network model includes:

[0066] Obtain the original graph sample, and obtain the first enhanced graph and the second enhanced graph based on the original graph sample;

[0067] Input the first enhanced graph and the second enhanced graph into the online network model to obtain the first online representation and the second online representation, and input the first enhanced graph and the second enhanced graph into the target network model to obtain the first target representation and the second target representation;

[0068] Obtain the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation;

[0069] Based on the comprehensive loss, perform backpropagation gradient optimization to update the parameter gradients of the online network model until the training is completed.

[0070] Specifically, this embodiment provides a training method for the online network model, that is, the obtaining process of the online network model.

[0071] First, use the original graph sample as a positive sample and perform graph augmentation to obtain a first augmented graph and a second augmented graph.

[0072] Then, input the two augmented graphs (view 1, view 2) into the online network model and the target network model respectively to obtain four graph node representations, namely the first online representation, the second online representation, the first target representation, and the second target representation (online h1, online h2, target h1, target h2).

[0073] Furthermore, based on the four graph node representations, obtain the comprehensive loss of the two network models, including the clustering loss, the cross-graph loss, and the cross-network loss.

[0074] Finally, based on the comprehensive loss, perform backpropagation gradient optimization to adjust the parameters of the online network until the online network model converges, indicating that the training is completed.

[0075] Based on the above embodiments, the training process of the target network model includes:

[0076] Obtain the fixed ratio of momentum update and the training process parameters of the online network model;

[0077] Based on the fixed ratio and the training process parameters of the online network model, perform iterative updates on the target network model until the training is completed.

[0078] Specifically, this embodiment provides a training method for the target network model, that is, the obtaining process of the target network model.

[0079] The target network model in this embodiment is updated based on the parameter update of the online network model, and iterative updates need to be performed according to the parameters of the online network model and the fixed ratio of momentum update.

[0080] The momentum update mechanism is a mechanism that updates based on the historical parameters of the model at a fixed ratio (momentum m). By adopting this update mechanism, the target network model can be kept relatively stable, thereby ensuring that the online network model iterates along a relatively stable direction. Specifically, the momentum update mechanism adopted in this solution can be described by the following formula:

[0081] ξ t = m·ξ t-1 +(1 - m)·θ t (3)

[0082] Where ξ t-1 represents the parameters of the target network after the (t - 1)th update, θ t represents the parameters of the online network after t times of gradient updates, and ξ t represents the new parameters of the target network. The momentum m is a predefined hyperparameter used to adjust the momentum update ratio. When m is set to 0, it is equivalent to the target network directly copying the parameters of the online network in each iteration; when m is set to 1, it is equivalent to the target network keeping the initial random parameters and stopping the update. In this embodiment, m is set to 0.5.

[0083] It should be noted that both the online network model and the target network model in this embodiment are composed of a graph convolutional network and a 512 - layer multi - layer perceptron. The parameters of the two network models are initialized to the same random values. Among them, the parameters of the online network model are updated with gradient descent during model training, while the parameters of the target network model are not updated with gradients, but are gradually updated using the momentum update mechanism.

[0084] Based on the above embodiments, obtaining the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation includes:

[0085] Performing clustering on the first online representation to obtain a clustering result and obtaining a clustering loss;

[0086] Obtaining a cross - graph contrast loss based on the clustering result, the first online representation, and the second online representation;

[0087] Obtaining a cross - network contrast loss based on the clustering result, the first online representation, and the first target representation;

[0088] Obtaining the comprehensive loss of the online network model and the target network model based on the clustering loss, the cross - graph contrast loss, and the cross - network contrast loss.

[0089] Specifically, this embodiment provides a cross - graph contrast loss and a cross - network contrast loss, and the process of obtaining the comprehensive loss according to the cross - graph contrast loss and the cross - network contrast loss.

[0090] Using the clustering results obtained in the above embodiments, a debiasing mechanism is used to perform contrastive learning on the low-dimensional representations of nodes, which is divided into two parts: cross-augmented graph contrast loss and cross-network contrast loss.

[0091] 1. Cross-graph contrast loss. According to the obtained clustering results, positive and negative samples of the representations online h1 and online h2 obtained by the online network for two debiased selected augmented graphs are calculated for their contrast loss, denoted as L clv (x i ).

[0092] 2. Cross-network contrast loss. According to the obtained clustering results, positive and negative samples of the representations online h1 and target h1 obtained by the online network and the target network for the debiased selected augmented Figure 1 (view 1) are calculated for their contrast loss, denoted as L cln (x i ).

[0093] The calculation formulas for the contrast loss in cross-network contrast and cross-graph contrast are as follows:

[0094]

[0095]

[0096] Among them, g θ (x i ) represents the representation of node x Figure 1 in the augmented i (view 1) learned by the online network, represents the representation of node Figure 1 in the augmented learned by the target network, represents the representation of node Figure 2 in the augmented learned by the online network in

[0097] L cln (x i ) and L clv (x i ) are calculated in the same way, the difference being the input network node representations. Taking cross-network contrast as an example, where g θ (xk) represents the embedding representation of node x i in online h1, represents the embedding representation of the positive sample with the same clustering pseudo-label as x i selected from targeth1, represents the one selected from target h1 that is the same as xi The embedding representations of negative samples with different clustering pseudo - labels, where M is the predefined number of negative samples.

[0098] 3. Comprehensive loss. By combining the cross - graph contrast loss and the cross - network contrast loss obtained, as well as the clustering loss, the overall loss function of the model is obtained. The overall objective function L of the model is defined as follows:

[0099] L = β * L kl + L cln + L clv (6)

[0100] Among them, L kl 、L cln and L clv represent the clustering loss, the cross - network contrast loss, and the cross - graph contrast loss obtained in the third step respectively, and β represents the hyperparameter for adjusting the clustering loss and the de - biased contrast loss.

[0101] Referring to Figure 2 , based on the above embodiments, the online network model includes a first graph neural network 210, a first multi - layer perceptron 220, and a clustering layer 230; the target network model includes a second graph neural network 240 and a second multi - layer perceptron 250;

[0102] Clustering the first online representation to obtain a clustering result and obtaining a clustering loss, including:

[0103] Inputting the first enhanced graph and the second enhanced graph into the first graph neural network, and outputting the first online representation through the first multi - layer perceptron;

[0104] Inputting the first online representation into the clustering layer to output the clustering result of positive samples, and obtaining the clustering loss according to the clustering result.

[0105] Specifically, in this embodiment, two enhanced graphs are input into the first graph neural network, and the first multi - layer perceptron outputs the first online representation (online h1), and the first online representation is input into the clustering layer to cluster the first online representation (online h1) obtained by the online network for the enhanced Figure 1 (view1), and the corresponding clustering loss L kl .

[0106] Based on the above embodiments, after clustering the first online representation to obtain a clustering result and obtaining a clustering loss, it further includes:

[0107] Obtaining the first similarity of the first online representation at the target clustering point, and the second similarity of the first target representation at the target clustering point;

[0108] Obtain the average similarity based on the first similarity and the second similarity, and obtain the auxiliary target distribution. Calculate the KL divergence based on the average similarity and the auxiliary target distribution, and obtain the target point clustering to which it belongs;

[0109] Based on the target point clustering to which it belongs, obtain the similarity of the first enhanced graph through contrastive learning by the online network model and the target network model;

[0110] Based on the target point clustering to which it belongs, obtain the similarity of the first enhanced graph and the second enhanced graph through contrastive learning by the online network model.

[0111] Specifically, this embodiment describes the similarity of the representations learned by the enhanced Figure 1 (view 1) through the online network and the target network respectively.

[0112] First, calculate the soft label distributions Q1 and Q2 of the representations (online h1 and target h1) learned by the enhanced Figure 1 (view1) through the online network model and the target network learning model respectively. Taking online h1 as an example, calculate its soft label distribution Q1 measured by the student-t distribution, indicating the similarity (the first similarity) that the node x k in online h1 belongs to the j-th clustering (target clustering point), and its definition is as follows:

[0113]

[0114] where g θ (x k ) represents the embedding representation of the node x k in online h1, and μ j represents the j-th clustering center (target clustering point).

[0115] At the same time, calculate the soft label distribution Q2 of target h1 measured by the student-t distribution, indicating the similarity (the second similarity) that the node x k in target h1 belongs to the j-th clustering (target clustering point), and its definition is as follows:

[0116]

[0117] Define Introduce the auxiliary target distribution P, which is derived from Q1:

[0118]

[0119] Calculate the average distribution Q of Q1 and Q2 m and the KL divergence of the target P distribution for enhancement Figure 1 (view 1) Measure the similarity of the representations (online h1, target h1) obtained by the online network model and the target network model. The objective function is defined as:

[0120]

[0121] Q m =(Q1 + Q2) / 2 (10)

[0122] where Qm is the mean of Q1 and Q2, represents the node x k belonging to the average similarity of the j-th cluster.

[0123] Next, the twin network contrastive representation learning device based on graph clustering optimization provided by the present invention will be described. The twin network contrastive representation learning device based on graph clustering optimization described below can be correspondingly referred to the twin network contrastive representation learning method based on graph clustering optimization described above.

[0124] Refer to Figure 3 , the present invention also provides a twin network contrastive representation learning device based on graph clustering optimization, including:

[0125] An acquisition module 310, configured to acquire an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0126] wherein, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through the original graph sample, and are trained without supervision based on the first enhanced graph and the second enhanced graph;

[0127] The parameters of the target network model are updated by the momentum update mechanism to update the parameters of the online network model;

[0128] An application module 320, configured to obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation.

[0129] Based on the above embodiments, the training process of the online network model includes:

[0130] An enhanced graph acquisition module, configured to acquire an original graph sample, and obtain a first enhanced graph and a second enhanced graph based on the original graph sample;

[0131] A representation acquisition module, configured to input the first enhanced graph and the second enhanced graph into the online network model to obtain a first online representation and a second online representation, and input the first enhanced graph and the second enhanced graph into the target network model to obtain a first target representation and a second target representation;

[0132] A comprehensive loss acquisition module, configured to obtain the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation;

[0133] An update module, configured to perform parameter gradient update on the online network model by performing backpropagation gradient optimization based on the comprehensive loss until the training is completed.

[0134] Based on the above embodiments, the training process of the target network model includes:

[0135] A parameter acquisition module, configured to obtain a fixed ratio for momentum update and the training process parameters of the online network model;

[0136] An iteration module, configured to perform iterative update on the target network model based on the fixed ratio and the training process parameters of the online network model until the training is completed.

[0137] Based on the above embodiments, the comprehensive loss acquisition module is specifically configured to:

[0138] Perform clustering on the first online representation to obtain a clustering result, and obtain a clustering loss;

[0139] Obtain a cross-graph contrast loss based on the clustering result, the first online representation, and the second online representation;

[0140] Obtain a cross-network contrast loss based on the clustering result, the first online representation, and the first target representation;

[0141] Obtain the comprehensive loss of the online network model and the target network model based on the clustering loss, the cross-graph contrast loss, and the cross-network contrast loss.

[0142] Based on the above embodiments, the comprehensive loss acquisition module is specifically configured to:

[0143] Input the first enhanced graph and the second enhanced graph into a first graph neural network, and output a first online representation through a first multi-layer perceptron;

[0144] Input the first online representation into the clustering layer to output the clustering result of positive samples, and obtain a clustering loss according to the clustering result.

[0145] Based on the above embodiments, it further includes:

[0146] A similarity acquisition module for obtaining a first similarity of the first online representation at the target clustering point and a second similarity of the first target representation at the target clustering point;

[0147] An average similarity acquisition module for obtaining an average similarity based on the first similarity and the second similarity, obtaining an auxiliary target distribution, calculating the KL divergence based on the average similarity and the auxiliary target distribution, and obtaining the target point cluster to which it belongs;

[0148] A similarity acquisition module for obtaining the similarity of the first enhanced graph obtained by contrastive learning of the online network model and the target network model based on the target point cluster to which it belongs;

[0149] Obtaining the similarity of the first enhanced graph and the second enhanced graph obtained by contrastive learning of the online network model based on the target point cluster to which it belongs.

[0150] Figure 4 An example of the physical structure diagram of an electronic device is shown as Figure 4 shown. The electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 complete mutual communication through the communication bus 440. The processor 410 may call the logical instructions in the memory 430 to execute the twin network contrastive representation learning method based on graph clustering optimization, and this method includes:

[0151] Obtaining an original graph and inputting the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0152] Among them, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples and are trained unsupervised based on the first enhanced graph and the second enhanced graph;

[0153] The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism;

[0154] Obtaining the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and performing downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation.

[0155] In addition, when the logical instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0156] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for contrastive representation learning of a Siamese network optimized based on graph clustering provided by the above-mentioned various methods. The method includes:

[0157] Obtain an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0158] Among them, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph;

[0159] The parameters of the target network model are updated by updating the parameters of the online network model through a momentum update mechanism;

[0160] Obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation.

[0161] On yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for contrastive representation learning of a Siamese network optimized based on graph clustering provided by the above-mentioned various methods. The method includes:

[0162] Obtain an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation;

[0163] Among them, both the online network model and the target network model are obtained by acquiring a first enhanced graph and a second enhanced graph from the original graph samples, and are trained without supervision based on the first enhanced graph and the second enhanced graph;

[0164] The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism;

[0165] Based on the first embedding representation and the second embedding representation, the finally learned node embedding representation is obtained, and downstream tasks such as node classification, node clustering, and visualization are performed based on the node embedding representation.

[0166] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0167] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A twin network contrastive representation learning method based on graph clustering optimization, characterized in that Including: Obtain the original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation; Among them, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph; The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism; Based on the first embedding representation and the second embedding representation, obtain the finally learned node embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation; The training process of the online network model includes: Obtain the original graph sample, and obtain a first enhanced graph and a second enhanced graph based on the original graph sample; Input the first enhanced graph and the second enhanced graph into the online network model to obtain a first online representation and a second online representation, and input the first enhanced graph and the second enhanced graph into the target network model to obtain a first target representation and a second target representation; Obtain the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation; Perform backpropagation gradient optimization based on the comprehensive loss to update the parameter gradients of the online network model until the training is completed; The online network model includes a first graph neural network, a first multi-layer perceptron, and a clustering layer; Clustering the first online representation to obtain a clustering result, and obtaining a clustering loss, including: Input the first enhanced graph and the second enhanced graph into the first graph neural network, and output the first online representation after passing through the first multi-layer perceptron; Input the first online representation into the clustering layer to output the clustering result of positive samples, and obtain the clustering loss according to the clustering result.

2. The method for contrastive representation learning of Siamese network optimized based on graph clustering according to claim 1, wherein The training process of the target network model includes: Obtain the fixed ratio of momentum update and the training process parameters of the online network model; Perform iterative update on the target network model based on the fixed ratio and the training process parameters of the online network model until the training is completed.

3. The method for contrastive representation learning of a Siamese network optimized based on graph clustering according to claim 1, characterized in that, The obtaining of the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation includes: Clustering the first online representation to obtain a clustering result, and obtaining a clustering loss; Obtain a cross-graph contrast loss based on the clustering result, the first online representation, and the second online representation; Obtain a cross-network contrast loss based on the clustering result, the first online representation, and the first target representation; Obtain the comprehensive loss of the online network model and the target network model based on the clustering loss, the cross-graph contrast loss, and the cross-network contrast loss.

4. The method for contrastive representation learning of a Siamese network optimized based on graph clustering according to claim 1, wherein After clustering the first online representation to obtain a clustering result and obtaining a clustering loss, it further includes: Obtain the first similarity of the first online representation at the target clustering point, and the second similarity of the first target representation at the target clustering point; Obtain the average similarity based on the first similarity and the second similarity, and obtain the auxiliary target distribution. Calculate the KL divergence based on the average similarity and the auxiliary target distribution, and obtain the target point clustering to which it belongs; Based on the target point clustering to which it belongs, obtain the similarity of the first enhanced graph through contrastive learning by the online network model and the target network model; Based on the target point clustering to which it belongs, obtain the similarity of the first enhanced graph and the second enhanced graph through contrastive learning by the online network model.

5. A twin network contrastive representation learning device based on graph clustering optimization, characterized in that, Includes: An acquisition module, configured to acquire an original graph, and input the original graph into a trained online network model and a target network model respectively to obtain a first embedding representation and a second embedding representation; Among them, both the online network model and the target network model obtain a first enhanced graph and a second enhanced graph through original graph samples, and are trained unsupervised based on the first enhanced graph and the second enhanced graph; The parameters of the target network model are obtained by updating the parameters of the online network model through a momentum update mechanism; An application module, configured to obtain the finally learned node embedding representation based on the first embedding representation and the second embedding representation, and perform downstream tasks such as node classification, node clustering, and visualization based on the node embedding representation; The training process of the online network model includes: Obtain an original graph sample, and obtain a first enhanced graph and a second enhanced graph based on the original graph sample; Input the first enhanced graph and the second enhanced graph into the online network model to obtain a first online representation and a second online representation, and input the first enhanced graph and the second enhanced graph into the target network model to obtain a first target representation and a second target representation; Obtain the comprehensive loss of the online network model and the target network model based on the first online representation, the second online representation, the first target representation, and the second target representation; Perform backpropagation gradient optimization based on the comprehensive loss to update the parameter gradient of the online network model until the training is completed; The online network model includes a first graph neural network, a first multi-layer perceptron, and a clustering layer; Cluster the first online representation to obtain a clustering result, and obtain a clustering loss, including: Input the first enhanced graph and the second enhanced graph into the first graph neural network, and output the first online representation through the first multi-layer perceptron; Input the first online representation into the clustering layer to output the clustering result of positive samples, and obtain the clustering loss according to the clustering result.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the graph clustering optimization-based contrastive representation learning method for siamese networks according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the graph clustering optimization-based contrastive representation learning method for siamese networks according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the graph clustering optimization-based contrastive representation learning method for siamese networks according to any one of claims 1 to 4.

Images