Automatic construction method of self-supervised graph representation learning framework based on evolution

Abstract

The invention discloses a self-supervised graph representation learning framework automatic construction method based on evolution, and mainly solves the problems that manual construction of a self-supervised graph representation learning framework wastes time and labor and manual optimization of model hyper-parameters is difficult to achieve in the prior art. The method comprises the following implementation steps: 1) acquiring a reference data set of self-supervised node classification; 2) defining a framework search space and constructing a plurality of self-supervised graph representation learning frameworks; 3) calculating node classification precision of the frameworks; 4) obtaining a plurality of sub-generation frameworks through crossover and variation, and calculating respective node classification precision; 5) selecting a plurality of high-precision frames from a plurality of original frames and a plurality of sub-generation frames as a new-generation frame; and repeating the step 4) and the step 5), and selecting the frame with the highest node classification precision from the last generation of frames as the optimal frame. According to the method, the self-supervised graph representation learning framework is optimized, so that the time and energy spent by professionals in constructing, adjusting and optimizing the framework are reduced, the use threshold is reduced, and the method can be used for graph machine learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and further designs a method for automatically constructing an evolution-based self-supervised graph representation learning framework, which can be used for graph machine learning. Background Art

[0002] Graph data can vividly represent the relationship between entities. It has been widely used in computer vision, natural language processing, interdisciplinary science and other fields. Graph neural networks have demonstrated excellent graph data analysis and mining capabilities in modeling and representing the association information between graph nodes. Although classic graph neural network models have achieved good performance in graph downstream tasks, in order to achieve more ideal results, it is necessary to design the architecture of graph neural networks based on the specific characteristics of graph datasets. At present, the design of automatic construction of graph neural networks mainly focuses on the search direction of graph neural architecture in supervised or semi-supervised scenarios. There is insufficient research on the automatic construction of graph representation learning models in self-supervised scenarios. The design of self-supervised graph representation learning models is manually designed by researchers with rich knowledge of graph neural networks. In practice, most developers do not have such knowledge, so the automatic design of self-supervised graph representation learning models is increasingly valued in the field of machine learning.

[0003] Liu Y et al. published a graph neural architecture search method in their paper "Automatic search of architecture and hyperparameters of graph convolutional networks for node classification. 2023, 53(9): 11104-11119." The implementation scheme of this method is: select ResGCN, JKNET-M, JKNET-C, IncepGCN as the network structure search space, hidden dim, learning rate, weight decay, dropout as the hyperparameter search space, and then use genetic algorithm to search for graph neural network models suitable for different data sets. Although this method has achieved good results for node classification tasks in both supervised and semi-supervised environments, it requires a large amount of labeled data to train the model and requires manual labeling of the data, which is not only time-consuming and labor-intensive, but also cannot guarantee the accuracy.

[0004] Guangzhou University disclosed a graph contrast learning classification method and system based on asymmetric weighting of twin networks in the patent document with application number 202311215750.6. Its implementation scheme is: for the obtained benchmark data set of self-supervised node classification, two views are generated by designing two data enhancement methods of topological augmentation and feature augmentation, and then two different representations are obtained using a twin encoder with shared weights, and a multi-layer perceptron is used as a predictor to obtain the final node representation, and finally a symmetrical cross-matching is performed between the node representations. This method can complete the node classification task well, but because the design of data enhancement methods and graph contrast learning classification methods requires experienced researchers to design, the hyperparameter adjustment for different data sets also requires the participation of professionals, and it cannot guarantee that the graph contrast learning classification performance is optimal. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a method for automatically constructing an evolutionary self-supervised graph representation learning framework to reduce the heavy reliance on professional and technical personnel, improve the accuracy of the optimal model for a specific graph data set, and improve the performance of graph comparative learning methods in node classification tasks.

[0006] To achieve the above object, the technical solution of the present invention includes:

[0007] (1) Obtain a benchmark dataset for self-supervised node classification from a public website, which includes a training set and a test set;

[0008] (2) Define the component search space of the self-supervised graph contrastive learning framework, which includes: multiple data augmentation methods, multiple proxy tasks, and multiple hyperparameters;

[0009] (3) Build a component search space module for data augmentation methods, proxy tasks, and hyperparameter selection, and connect it with the existing graph convolutional neural network encoder and logistic regression classifier in sequence to form a single self-supervised graph representation learning framework;

[0010] (4) Repeat step (3) N times to form a self-supervised graph representation learning framework set, N ≥ 30;

[0011] (5) Input the training set into a single self-supervised graph representation learning framework, train the graph convolutional neural network using the gradient descent method, and input the test set into the trained graph convolutional neural network to obtain a node representation, and the node representation is obtained by the node classification accuracy obtained by the trained logistic regression classifier;

[0012] (6) Repeat step (5) N times to obtain the node classification accuracy of each framework in the self-supervised graph representation learning framework set;

[0013] (7) Select two frames with high node classification accuracy from the current self-supervised graph representation learning frame set as the first parent frame A and the second parent frame B, and perform crossover and mutation operations on the two parent frames in turn to obtain the first child frame C and the second child frame D;

[0014] (8) Repeat step (7) N times to obtain the learning framework set of offspring self-supervised graph representations;

[0015] (9) Inputting the training set into each child self-supervised graph representation learning framework, training the graph convolutional neural network using the gradient descent method, and inputting the test set into the trained graph convolutional neural network to obtain a node representation, and the node representation is obtained by the node classification accuracy obtained by the logistic regression classifier;

[0016] (10) Repeat step (9) N times to obtain the node classification accuracy of each frame in the offspring self-supervised graph representation learning frame set;

[0017] (11) selecting a framework with high node classification accuracy from the self-supervised graph representation learning framework set before the crossover operation and the offspring self-supervised graph representation learning framework set as the new generation self-supervised graph representation learning framework E;

[0018] (12) Repeat step (11) N times to obtain a new generation of self-supervised graph representation learning framework set;

[0019] (13) Repeat steps (7)-(12) M times, M ≥ 20, and select the framework with the highest node classification accuracy as the best framework in the last generation of self-supervised graph representation learning frameworks.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] First, the present invention can quickly construct a self-supervised graph representation learning framework by defining a component search space of a self-supervised graph comparison learning framework, thereby reducing the time for professionals to build the framework;

[0022] Secondly, the present invention automatically constructs a self-supervised graph representation learning framework, which not only reduces the dependence on professionals, but also allows amateurs to easily use the self-supervised graph representation learning framework, and reduces the amount of labeled data used by the graph representation learning framework;

[0023] Thirdly, the present invention obtains a new framework through multiple crossover and mutation operations and calculates the node classification accuracy of the self-supervised graph representation learning framework multiple times, which can automatically find the optimal framework, reduce the time for professionals to adjust the framework and framework hyperparameters, and improve the accuracy of node classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1It is a flow chart of the implementation of the present invention;

[0025] Figure 2 It is the self-supervised graph representation learning framework diagram in the present invention. DETAILED DESCRIPTION

[0026] The technical scheme and effects of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments.

[0027] Reference Figure 1 , the implementation steps of this example are as follows:

[0028] Step 1: Obtain a benchmark dataset for self-supervised node classification from a public website, which includes a training set and a test set.

[0029] The benchmark dataset includes various citation network datasets. Each node in each citation network dataset represents a paper, and each edge represents the citation relationship between papers. Each paper cites or is used by at least one other paper. The attribute of each node is a description of the content of the paper. Nodes can be divided into different categories according to the content of the paper. This dataset adopts public division in a semi-supervised environment and has a divided training set and test set.

[0030] Step 2: Define the component search space of the self-supervised graph representation learning framework.

[0031] The self-supervised graph representation learning framework is a framework that maximizes the similarity between positive samples and minimizes the similarity between negative samples. It does not rely on external labels, but constructs proxy tasks from the data itself to learn useful feature representations. This feature representation is used for downstream tasks such as node classification. The construction of the self-supervised graph representation learning framework requires selecting components from its component search space.

[0032] This step defines the component search space of the self-supervised graph representation learning framework, including: multiple data augmentation methods, multiple proxy tasks, and multiple hyperparameters, where:

[0033] The multiple data enhancement methods include random mask, edge perturbation, subgraph, node deletion, empty, personalized PageRank and hot core. Each enhancement method generates an enhanced view through the input graph data. The enhanced view includes a feature matrix and an adjacency matrix. Each enhancement method has different operations.

[0034] The various proxy tasks include node-node comparison, node-subgraph comparison, and node-graph comparison. Each proxy task obtains the graph embedding representation of each view by using graph neural networks on different views. In this framework, the original view (A, X) and the negative sample view and Enhanced View (A m ,X m) are used on three views, and each proxy task has a different operation method.

[0035] The various hyperparameters include learning rate, weight decay rate and hidden layer dimension. Each hyperparameter is defined as follows:

[0036] Learning rate, including parameters such as 0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.007, 0.008, 0.009, and 0.1;

[0037] Weight decay rate, including parameters such as 1e-5, 3e-5, 5e-5, 7e-5, 9e-5, and 1e-4;

[0038] Hidden layer dimensions include parameters such as 128, 256, 512, and 1024.

[0039] Each hyperparameter is randomly selected and put into the self-supervised graph representation learning framework.

[0040] Step 3: Implementation of multiple data enhancement methods.

[0041] Random masking is to generate the feature matrix of the enhanced view by randomly masking a part of the feature matrix of the input graph data, and the adjacency matrix of the enhanced view is the same as that of the input data;

[0042] Edge perturbation is to generate the adjacency matrix of the enhanced view by randomly blocking a part of the adjacency matrix of the input graph data. The feature matrix of the enhanced view is the same as that of the input data.

[0043] A subgraph is constructed from the input graph data by random walk. The subgraph is the generated enhanced view. The adjacency matrix of the enhanced view is the topology of the subgraph. The feature matrix of the enhanced view is the node features of all nodes in the subgraph.

[0044] Node deletion is to delete some nodes from the adjacency matrix and feature matrix of the input graph data to generate the adjacency matrix and feature matrix of the enhanced view;

[0045] Empty means that the input graph data is not changed, and the adjacency matrix and feature matrix of the generated enhanced view are the same as those of the input data;

[0046] Personalized PageRank is to change the adjacency matrix of the input graph data, and obtain the adjacency matrix and feature matrix τ of the enhanced view P (A,X):

[0047] τ P (A,X)=(α(I-(1-α)D -1 / 2 AD -1 / 2 ) -1,X)

[0048] Where A is the adjacency matrix of the input data, X is the feature matrix of the input data, I is an identity matrix, D is the degree matrix of the input data, and α is the probability of random walk;

[0049] The heat kernel changes the adjacency matrix of the input graph data to obtain the adjacency matrix and feature matrix τ of the enhanced view 热 (A,X):

[0050] τ 热 (A,X)=(exp(tAD -1 -t),X)

[0051] Where t is the diffusion time and exp is an exponential function with the natural constant e as the base.

[0052] Step 4: Implementation of various proxy tasks.

[0053] The proxy task in this example is to find the original view (A, X) and the negative sample view and Enhanced View (A m ,X m ) are used on three views. These three views are obtained by taking the input image data as the original view (A, X), and performing feature rearrangement on the original view (A, X) to obtain the negative sample view: The original view (A, X) is enhanced to obtain the enhanced view: (A m ,X m )=τ(A,X), where idx is a list of randomly arranged numbers from 1 to N, N is the number of rows of the feature matrix X, and τ indicates the use of data augmentation.

[0054] The three types of proxy tasks are implemented as follows:

[0055] Node-node comparison, the first node in the original view (A, X) and the negative view The node representation is obtained by graph convolutional neural network, and its second node is in the enhanced view (A m ,X m ) to obtain node representation through graph convolutional neural network;

[0056] Node-graph comparison, where nodes are in the original view (A, X) and the negative view The node representation is obtained by graph convolutional neural network, and its graph is enhanced in view (A m ,X m ) First, we use the graph convolutional neural network to obtain the node representation H G , and then use the readout function to get the graph representation G(H):

[0057]

[0058] in is the node representation of node i;

[0059] The node-subgraph comparison shows the nodes in the original view (A, X) and the negative sample view. The node representation is obtained by graph convolutional neural network, and its subgraph is in enhanced view (A m ,X m ) first obtain multiple clusters through K-means clustering, and then obtain the node representation H of each cluster through graph convolutional neural network. S Finally, the subgraph representation S(H) is obtained through the readout function:

[0060]

[0061] in is the node representation of node i;

[0062] The K-means clustering is implemented as follows:

[0063] 4.1) Randomly select K nodes as the initial cluster centroids;

[0064] 4.2) Assign each node to the nearest cluster based on the Euclidean distance between it and the centroid of each cluster;

[0065] 4.3) Recalculate the centroid of each cluster, that is, take the average of the Euclidean distances from all data points in the cluster to the original cluster center as the new cluster center;

[0066] Repeat the above steps 4.2) and 4.3) until the set number of iterations is met, and K clusters are obtained. The cluster contains multiple nodes, and the nodes in each cluster are closest to the centroid of the current cluster.

[0067] Step 5: Construct a single self-supervised graph representation learning framework based on the component search space.

[0068] 5.1) Establish the component search space module of the data enhancement submodule, proxy task submodule and hyperparameter selection submodule:

[0069] The data enhancement submodule is used to generate an enhanced view (A) by randomly selecting one of the data enhancement methods. m ,X m ), and generate negative sample views and the original view (A,X);

[0070] The proxy task submodule is used to obtain the node representation h of the original view. i , node representation of negative sample view and the graph embedding representation of the enhanced view π k , where k is the selected proxy task;

[0071] The hyperparameter selection submodule is used to randomly select a hyperparameter from the learning rate, weight decay rate, and hidden layer dimension space;

[0072] 5.2) Use the existing graph convolutional neural network encoder and logistic regression classifier:

[0073] The graph convolutional neural network encoder consists of a linear layer and a PReLU activation function to convert the view into a node representation;

[0074] The logistic regression classifier consists of a linear layer that converts the input node representation into a predicted category;

[0075] 5.3) Connect the data enhancement submodule, proxy task submodule, hyperparameter selection submodule, graph convolutional neural network and logistic regression classifier in sequence to construct a single self-supervised graph representation learning framework, such as Figure 2 shown.

[0076] Step 6: Repeat step 5 N times to form a self-supervised graph representation learning framework set, where N = 30.

[0077] Step 7: Train a single self-supervised graph representation learning framework to obtain the node classification accuracy of the framework.

[0078] 7.1) Input the training set data into a single self-supervised graph representation learning framework. The training set data is used as the original view (A, X). The negative sample view is first obtained through the enhancement module. and Enhanced View (A m ,X m ), and then obtain the node representation h of the original view through the proxy task submodule i , node representation of negative sample view and the graph embedding representation π of the enhanced view k , and then obtain the hyperparameters for training the graph convolutional neural network through the hyperparameter selection submodule;

[0079] 7.2) The standard binary cross entropy loss function is used as the loss function of the graph convolutional neural network, and its loss function value is calculated through the graph embedding representation of the three views:

[0080]

[0081] Where N is the number of positive samples, M is the number of negative samples, and E (A,X) [log f(h i ,π k)] is the expected value of the average loss calculated on multiple samples when the adjacency matrix A and the feature matrix X are fixed, and f is a bilinear layer network;

[0082] 7.3) Initialize the number of iterations t, and set the maximum number of iterations T = 500;

[0083] 7.4) The original image (A, X) and the negative sample image and Enhanced View (A m ,X m ) is input into the graph convolutional neural network and the gradient descent method is used to minimize its loss function value;

[0084] 7.5) Repeat step 7.4) until the number of iterations t reaches the set maximum number of iterations T, completing the training of the graph convolutional neural network;

[0085] 7.6) Input the training set data into the trained graph convolutional neural network to obtain the node representation of the training set, and use the cross entropy loss function as the loss function of the logistic regression classifier;

[0086] 7.7) Input the node representation of the training set and the labels of the training set data into the logistic regression classifier, and use the gradient descent method to minimize its loss function value;

[0087] 7.8) Repeat step 7.7) 50 times to complete the training of the logistic regression classifier;

[0088] 7.9) Input the test set into the trained graph convolutional neural network to obtain the node representation of the training set;

[0089] 7.10) Input the node representation of the test set into the trained logistic regression classifier to obtain the predicted classification type of each node. Compare the predicted category with the category marked by the test set data and calculate the node classification accuracy Acc:

[0090]

[0091] Among them, TP is the number of positive classes predicted as positive classes, FN is the number of positive classes predicted as negative classes, FP is the number of negative classes predicted as positive classes, and TN is the number of negative classes predicted as negative classes.

[0092] Step 8: Repeat step 7 30 times to obtain the node classification accuracy of each framework in the self-supervised graph representation learning framework set.

[0093] Step 9, select two frameworks with high node classification accuracy from the current self-supervised graph representation learning framework set, and obtain the first child framework C and the second child framework D through crossover and mutation operations.

[0094] 9.1) First, randomly select two frameworks from the current self-supervised graph representation learning framework set and compare the node classification accuracy of the two frameworks:

[0095] If the node classification accuracy of the two frames is different, the frame with higher node classification accuracy is selected as the first parent frame A;

[0096] If the node classification accuracy of the two frames is the same, a frame is randomly selected as the first parent frame A;

[0097] 9.2) Second, randomly select two frameworks from the current self-supervised graph representation learning framework set and compare the node classification accuracy of the two frameworks:

[0098] If the node classification accuracy of the two frames is different, the frame with higher node classification accuracy is selected as the second parent frame B;

[0099] If the node classification accuracy of the two frames is the same, a frame is randomly selected as the second parent frame B;

[0100] 9.3) Set the crossover probability P c =0.9, mutation probability P m =0.2;

[0101] 9.4) Use the random function to generate a random number r c , where r c ∈[0,1], and compare r c With the crossover probability P c :

[0102] If r c <P c , then select two different points from the selected first parent frame A, and exchange the components between these points with the components at the same positions on the second parent frame B, to obtain the first parent frame A′ and the second parent frame B′;

[0103] If r c ≥P c , then the selected first parent frame A is used as the first parent frame A′, and the second parent frame B is used as the second parent frame B′;

[0104] 9.5) Use the random function to generate a random number r m , where r m ∈[0,1], and compare r m With mutation probability P m :

[0105] If r m <P m, then randomly select a point on the first parent frame A′ and the second parent frame B′ obtained by the cross, and replace the component at the point with another component of the same type to obtain the first child frame C and the second child frame D;

[0106] If r m ≥P m , then the first parent frame A′ and the second parent frame B′ are used as the first child frame C and the second child frame D.

[0107] Step 10: Repeat step 9 for 15 times to obtain the offspring self-supervised graph representation learning framework set.

[0108] Step 11, train each child self-supervised graph representation learning framework according to the method in step 7 to obtain the node classification accuracy of the framework.

[0109] Step 12, repeat step 11 30 times in total, and obtain the node classification accuracy of each framework in the offspring self-supervised graph representation learning framework set.

[0110] Step 13, selecting a framework with high node classification accuracy from the self-supervised graph representation learning framework set in step 8 and the descendant self-supervised graph representation learning framework set as a new generation of self-supervised graph representation learning framework E.

[0111] 13.1) The self-supervised graph representation learning framework set and the descendant self-supervised graph representation learning framework set in step 8 are merged into one framework set, and two frameworks are randomly selected from the framework set. After this use, these two frameworks are removed from the dataset;

[0112] 13.2) Compare the node classification accuracy of the two frameworks:

[0113] If the node classification accuracy of the two frameworks is different, the framework with higher node classification accuracy is selected as the new generation of self-supervised graph representation learning framework E.

[0114] If the node classification accuracy of the two frameworks is the same, a framework is randomly selected as the new generation of self-supervised graph representation learning framework E.

[0115] Step 14: Repeat step 13 30 times to obtain a new generation of self-supervised graph representation learning framework set.

[0116] Step 15: Repeat steps 9 to 14 20 times in total, and select the framework with the highest node classification accuracy as the best framework in the last generation of self-supervised graph representation learning framework set.

[0117] The effect of the present invention can be further illustrated by the following simulation experiment results.

[0118] 1. Simulation conditions:

[0119] Simulation experiment hardware platform: Processor is AMD EPYC 7Y43 48-Core Processor

[0120] The main frequency is 3.6GHz, and the memory is 251.5GB;

[0121] Simulation experiment software platform: Ubuntu 22.04.3 and python 3.8;

[0122] The simulation experiment data sets are three citation network data sets: Cora, Citeseer and Pubmed. Both Cora and Citeseer are data sets about mutual citations between scientific publications. The scientific publications in Cora are divided into seven categories: cases, genetic algorithms, neural networks, probabilistic methods, reinforcement learning, rule learning and theory. The scientific publications in Citeseer are divided into six categories: agents, artificial intelligence, databases, human-computer interaction, information retrieval and machine learning. Pubmed is a data set about mutual citations between scientific publications on diabetes, which is divided into three categories: experimental diabetes, type 1 diabetes and type 2 diabetes.

[0123] 2. Simulation content and results:

[0124] The present invention and nine prior arts are used to classify nodes in three citation network datasets, namely, Cora, Citeseer and Pubmed, respectively, and the node classification accuracy of each dataset is shown in Table 1:

[0125] Table 1: Node classification accuracy Acc (unit: percentage)

[0126] method Cora Citeseer Pubmed DGI 82.3 71.8 76.8 MVGRL 83.4 73.2 80.1 GRACE 81.8 71.2 80.6 GDJ 83.9 73.0 81.3 SUBG-CON 83.5 73.2 81.0 Graph-NAS 83.7 73.5 80.5 PaSca 84.6 73.4 80.8 Genetic-GNN 83.8 73.5 79.2 GCN-GA 83.1 70.7 79.3 The present invention 83.1 74.0 80.9

[0127] The nine existing technologies include: self-supervised graph representation learning methods DGI, MVGRL, GRACE, GGD, SUBG-CON, semi-supervised neural architecture search methods Graph-NAS, PaSca, Genetic-GNN, GCN-GA, the specific sources are as follows:

[0128] DGI refers to the node classification method proposed in "Petar Velickovic, William Fedus, William L. Hamilton, Pietro Lió, Yoshua Bengio, R. Devon Hjelm. Deep Graph Infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019", referred to as DGI;

[0129] MVGRL refers to the node classification method proposed in "Hassani K, Khasahmadi AH. Contrastive multi-view representation learning on graphs[C] / / International conference on machine learning. PMLR, 2020:4116-4126.", abbreviated as MVGRL;

[0130] GRACE refers to the node classification method proposed in "You Y, Chen T, Sui Y, et al. Graph contrastive learning with augmentations[J]. Advances in neural information processing systems, 2020, 33: 5812-5823.", referred to as GRACE;

[0131] GGD refers to the node classification method proposed in "Zheng Y, Pan S, Lee V, et al. Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination [J]. Advances in Neural Information Processing Systems, 2022, 35: 10809-10820.", referred to as GGD;

[0132] SUBG-CON refers to the node classification method proposed in "Peng Z, Dong Y, Luo M, et al. Self-supervised graph representation learning via global context prediction.arXiv preprint arXiv:2003.01604,2020", referred to as SUBG-CON;

[0133] Graph-NAS refers to the node classification method proposed in "Gao Y, Yang H, Zhang P, et al. Graph neural architecture search [C] / / International joint conference on artificial intelligence. International Joint Conference on Artificial Intelligence, 2021.", referred to as Graph-NAS;

[0134] PaSca refers to the node classification method proposed in "Zhang W, Shen Y, Lin Z, et al. Pasca: A graph neural architecture search system under the scalable paradigm [C] / / Proceedings of the ACM WebConference 2022.2022: 1817-1828.", referred to as PaSca;

[0135] Genetic-GNN refers to the node classification method proposed in "Shi M, Tang Y, Zhu X, et al. Genetic-GNN: Evolutionary architecture search for graph neural networks[J]. Knowledge-Based Systems, 2022, 247: 108752.", referred to as Genetic-GNN;

[0136] GCN-GA refers to the node classification method proposed in "Liu Y, Liu J, Li Y. Automatic search of architecture and hyperparameters of graph convolutional networks for node classification[J]. Applied Intelligence, 2023, 53(9): 11104-11119.", referred to as GCN-GA;

[0137] As can be seen from Table 1, the present invention outperforms 2, 9 and 7 existing technologies on the Cora, Citeseer and Pubmed datasets respectively. Compared with the self-supervised graph representation learning method and the semi-supervised graph neural architecture search method, the present invention achieves similar or even higher performance, which verifies the effectiveness of the present invention.

[0138] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

[0139] It should be noted that the step numbers in the specification and claims of the present invention are only for a clear description of the implementation scheme of the present invention to facilitate understanding, and the order of the step numbers is not limited.

Claims

1. A method for automatically constructing an evolution-based self-supervised graph representation learning framework, characterized in that: The following steps are involved: (1) Obtain a benchmark dataset for self-supervised node classification from a public website, which includes a training set and a test set; (2) Define the component search space of the self-supervised graph representation learning framework, which includes: multiple data augmentation methods, multiple proxy tasks, and multiple hyperparameters; (3) Build a component search space module to select data augmentation methods, proxy tasks, and hyperparameters, and connect it with the existing graph convolutional neural network and logistic regression classifier in sequence to form a single self-supervised graph representation learning framework; (4) Repeat step (3) N times to form a self-supervised graph representation learning framework set, N ≥ 30; (5) Input the training set into a single self-supervised graph representation learning framework, train the graph convolutional neural network using the gradient descent method, and input the test set into the trained graph convolutional neural network to obtain a node representation, and the node representation is obtained by the node classification accuracy obtained by the trained logistic regression classifier; (6) Repeat step (5) N times to obtain the node classification accuracy of each framework in the self-supervised graph representation learning framework set; (7) Select two frames with high node classification accuracy from the current self-supervised graph representation learning frame set as the first parent frame A and the second parent frame B, and perform crossover and mutation operations on the two parent frames in turn to obtain the first child frame C and the second child frame D; (8) Repeat step (7) Second, we obtain a learning framework set of offspring self-supervised graph representations; (9) Inputting the training set into each child self-supervised graph representation learning framework, training the graph convolutional neural network using the gradient descent method, and inputting the test set into the trained graph convolutional neural network to obtain a node representation, and the node representation is obtained by the node classification accuracy obtained by the logistic regression classifier; (10) Repeat step (9) N times to obtain the node classification accuracy of each frame in the offspring self-supervised graph representation learning frame set; (11) selecting a framework with high node classification accuracy from the self-supervised graph representation learning framework set before the crossover operation and the offspring self-supervised graph representation learning framework set as the new generation self-supervised graph representation learning framework E; (12) Repeat step (11) N times to obtain a new generation of self-supervised graph representation learning framework set; (13) Repeat steps (7)-(12) M times, M ≥ 20, and select the framework with the highest node classification accuracy as the best framework in the last generation of self-supervised graph representation learning frameworks.

2. The method according to claim 1, characterized in that: The multiple data enhancement methods defined in step (2) include random mask, edge perturbation, subgraph, node deletion, empty, personalized PageRank and heat kernel. Each enhancement method generates an enhanced view through the input graph data. The enhanced view includes a feature matrix and an adjacency matrix, where: Random masking is to generate the feature matrix of the enhanced view by randomly masking a part of the feature matrix of the input graph data, and the adjacency matrix of the enhanced view is the same as that of the input data. Edge perturbation is to generate the adjacency matrix of the enhanced view by randomly blocking a part of the adjacency matrix of the input graph data. The feature matrix of the enhanced view is the same as that of the input data. A subgraph is constructed from the input graph data by random walk. The subgraph is the generated enhanced view. The adjacency matrix of the enhanced view is the topology of the subgraph. The feature matrix of the enhanced view is the node features of all nodes in the subgraph. Node deletion is to delete some nodes from the adjacency matrix and feature matrix of the input graph data to generate the adjacency matrix and feature matrix of the enhanced view. Empty means that the input graph data is not changed, and the adjacency matrix and feature matrix of the generated enhanced view are the same as those of the input data. Personalized PageRank is to change the adjacency matrix of the input graph data, and obtain the adjacency matrix and feature matrix τ of the enhanced view P (A,X): t P (A,X)=(α(I-(1-α)D -1 / 2 AD -1 / 2 ) -1 ,X) Where A is the adjacency matrix of the input data, X is the feature matrix of the input data, I is an identity matrix, D is the degree matrix of the input data, and α is the probability of random walk; The heat kernel changes the adjacency matrix of the input graph data to obtain the adjacency matrix and feature matrix τ of the enhanced view 热 (A,X): τ 热 (A,X)=(exp(tAD -1 -t),X) Where t is the diffusion time and exp is an exponential function with the natural constant e as the base.

3. The method according to claim 1, characterized in that The multiple proxy tasks defined in step (2) include node-node comparison, node-subgraph comparison, and node-graph comparison. Each proxy task obtains a graph embedding representation of each view by using a graph neural network on different views, and its implementation is as follows: Perform feature rearrangement and data enhancement on the original view (A, X) to obtain the negative sample view and Enhanced View (A m ,X m ); Node-node comparison, the first node in the original view (A, X) and the negative view The node representation is obtained by graph convolutional neural network, and its second node is in the enhanced view (A m ,X m ) to obtain node representation through graph convolutional neural network; Node-subgraph comparison, where the nodes are in the original view (A, X) and the negative view The node representation is obtained by graph convolutional neural network, and its subgraph is in enhanced view (A m ,X m ) is first clustered by K-means, and then the cluster nodes are represented by subgraphs through graph convolutional neural networks and readout functions; Node-graph comparison, where nodes are in the original view (A, X) and the negative view The node representation is obtained by graph convolutional neural network, and its graph is enhanced in view (A m ,X m ) first obtains the node representation through the graph convolutional neural network, and then obtains the graph representation through the readout function.

4. The method according to claim 1, characterized in that: The multiple hyperparameters defined in step (2) include learning rate, weight decay rate and hidden layer dimension, each of which is randomly selected and put into the self-supervised graph representation learning framework, where: Learning rate, including parameters such as 0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.007, 0.008, 0.009, and 0.1; Weight decay rate, including parameters such as 1e-5, 3e-5, 5e-5, 7e-5, 9e-5, and 1e-4; Hidden layer dimensions include parameters such as 128, 256, 512, and 1024.

5. The method according to claim 1, characterized in that: The component search space module established in step (3) includes: The established data enhancement submodule is used to generate an enhanced view (A m ,X m ), and generate negative sample views and the original view (A,X); The proxy task submodule is used to obtain the node representation h of the original view. i , node representation of negative sample view and the graph embedding representation of the enhanced view π k , where k is the selected proxy task; The hyperparameter selection submodule is used to select hyperparameters from the learning rate, weight decay rate, and hidden layer dimension space; The three submodules are cascaded in sequence.

6. The method according to claim 1, characterized in that In step (5), the training set is input into a single self-supervised graph representation learning framework, and the graph convolutional neural network is trained using the gradient descent method, which is implemented as follows: (5a) Take the training set data as the original view (A, X) and obtain the negative sample view through the feature rearrangement method The enhanced view (A) is obtained by the enhancement method selected in the framework. m ,X m ), the node representation h of the original view is obtained through the proxy task selected in the framework i , node representation of negative sample view and the graph embedding representation π of the enhanced view k , the standard binary cross entropy loss function is used as the loss function of the graph convolutional neural network; (5b) The loss function value is calculated by the graph embedding representation of the three views: Where N is the number of positive samples, M is the number of negative samples, and E (A,X) [logf(h i ,π k )] is the expected value of the average loss calculated on multiple samples when the adjacency matrix A and the feature matrix X are fixed, f is a bilinear layer network, and log is the logarithmic function; (5c) Initialize the number of iterations t, the maximum number of iterations is T and is at least 500; (5d) The original image (A, X) and the negative sample image and Enhanced View (A m ,X m ) is input into a graph convolutional neural network consisting of a linear layer and a PReLU activation function, and the gradient descent method is used to minimize its loss function value; (5e) Repeat step (5d) until the number of iterations t reaches the set maximum number of iterations T, completing the training of the graph convolutional neural network.

7. The method according to claim 1, characterized in that In step (5), the node representation of the test set is passed through the trained logistic regression classifier to obtain the node classification accuracy, which is achieved as follows: (5f) Input the training set data into the trained graph convolutional neural network to obtain the node representation of the training set, and use the cross entropy loss function as the loss function of the logistic regression classifier; (5g) Input the node representation of the training set and the label of the training set data into a logistic regression classifier composed of a linear layer, and use the gradient descent method to minimize its loss function value; (5h) Repeat step (5g) 50 times to complete the training of the logistic regression classifier; (5i) Input the node representation of the test set into the trained logistic regression classifier to obtain the predicted classification type of each node. Compare the predicted category with the category marked by the test set data and calculate the node classification accuracy Acc: Among them, TP is the number of positive classes predicted as positive classes, FN is the number of positive classes predicted as negative classes, FP is the number of negative classes predicted as positive classes, and TN is the number of negative classes predicted as negative classes.

8. The method according to claim 1, characterized in that In step (7), two frameworks with high node classification accuracy are selected from the current self-supervised graph representation learning framework set as the first parent framework A and the second parent framework B, respectively, and the implementation is as follows: (7a) First, randomly select two frameworks from the current set of self-supervised graph representation learning frameworks and compare the node classification accuracy of the two frameworks: If the node classification accuracy of the two frames is different, the frame with higher node classification accuracy is selected as the first parent frame A; If the node classification accuracy of the two frames is the same, a frame is randomly selected as the first parent frame A; (7b) The second time, two frameworks are randomly selected from the current self-supervised graph representation learning framework set and the node classification accuracy of the two frameworks is compared: If the node classification accuracy of the two frames is different, the frame with higher node classification accuracy is selected as the second parent frame B; If the node classification accuracy of the two frames is the same, a frame is randomly selected as the second parent frame B.

9. The method according to claim 1, characterized in that: In step (7), the two parent frames are subjected to crossover and mutation operations in sequence to obtain a first child frame C and a second child frame D, which is implemented as follows: (7c) Select two different points on the selected first parent frame A, and exchange the components between these points with the components at the same positions on the second parent frame B to obtain the first parent frame A′ and the second parent frame B′, thus completing the crossover operation; (7d) Then randomly select a point on a single frame and replace the component at that point with another component of the same type to complete the mutation operation. (7e) Perform mutation operations on the first parent frame A′ and the second parent frame B′ obtained by the crossover, respectively, to obtain the first child frame C and the second child frame D.

10. The method according to claim 1, characterized in that In the step (11), a framework with high node classification accuracy is selected from the self-supervised graph representation learning framework set before the crossover operation and the offspring self-supervised graph representation learning framework set as a new generation of self-supervised graph representation learning framework E, which is implemented as follows: (11a) The self-supervised graph representation learning framework set before the crossover operation and the offspring self-supervised graph representation learning framework set are merged into a framework set, and two frameworks are randomly selected from the framework set. After this use, these two frameworks are removed from the dataset; (11b) Comparison of the node classification accuracy of the two frameworks: If the node classification accuracy of the two frameworks is different, the framework with higher node classification accuracy is selected as the new generation of self-supervised graph representation learning framework E. If the node classification accuracy of the two frameworks is the same, a framework is randomly selected as the new generation of self-supervised graph representation learning framework E.

Images