Algorithm model migration method from convolutional neural network to graph neural network

Abstract

The invention belongs to the technical field of artificial intelligence neural networks, and particularly relates to an algorithm model migration method from a convolutional neural network to a graph neural network. The invention provides an innovative concept of a convolutional bridge, which is an important mechanism for optimizing a cross-domain migration process from a convolutional neural network to a graph neural network. The bridge effectively aligns a data structure between the convolutional neural network and the graph neural network, and promotes seamless migration of a model based on the convolutional neural network to the graph field. Through the convolutional bridge, the Inception architecture and the U-Net architecture are successfully migrated from the convolutional neural network to the graph neural network, and a GraInc model and a GraU-Net model are formed. The GraInc model and the GraU-Net model show remarkable competitiveness, especially on a dense graph data set.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence neural networks, and in particular relates to a method for migrating an algorithm model from a convolutional neural network to a graph neural network. Background Art

[0002] Graphs, as a subset of non-Euclidean structures, are widely used in data representation in multiple fields, including social networks, neuroscience, physical systems, and knowledge graphs. Interestingly, both images and texts can be regarded as special cases of graph structures. Images can be regarded as regularized two-dimensional graph structures, while texts can be regarded as regularized one-dimensional graph structures. However, due to the limitations of application scenarios and technological development, the emergence of Graph Neural Networks (GNNs) is much later than the rapid rise of Convolutional Neural Networks (CNNs) in image processing and the rise of Recurrent Neural Networks (RNNs) in text processing. When the concept of GNNs was proposed, the Convolutional Neural Network model had already faced many challenges and achieved remarkable success.

[0003] In order to accelerate the development of GNNs, many excellent convolutional neural network models are considered to migrate to the field of graph neural networks. For example, Szegedy et al. introduced the Inception architecture to capture multi-scale features in images. On this basis, Rossi et al. developed the Scalable Inception Graph Network (SIGN) model, which showed strong competitiveness in feature extraction and achieved excellent results in node classification tasks. Similarly, the encoder-decoder architecture of U-Net has achieved success in various pixel-level prediction tasks in convolutional neural networks. Considering that similar ideas are still lacking in GNNs, Gao et al. designed Graph U-Net, which combines the architecture of U-Net architecture and the characteristics of graphs. Experimental results show that Graph U-Net performs well in graph-level and node-level classification tasks.

[0004] Cross-domain migration has effectively promoted the development of GNNs. However, all of the above models are migrations of ideas. Although many graph neural networks have been proposed for various scenarios, including spectral graph neural networks, heterogeneous graph neural networks, knowledge graph neural networks, and spatiotemporal graph neural networks, their information aggregation process does not have the advantages of convolution. Some of these works use normalized adjacency matrices to guide information aggregation, but their graph information aggregation process is not learnable. Other works calculate the information aggregation strength through the attention mechanism, which has high computational complexity and is more likely to cause overfitting problems. In the convolution operation of CNN, the weights of the convolution kernel are directly adjusted through the back-propagation algorithm, so that the model can adapt to different tasks and extract the required features. In addition, the convolution kernel shares parameters across the entire input image, which means that the same filter extracts specific features at different locations, thereby improving the generalization ability of CNN. The introduction of a learnable and parameter-sharing convolution operator can effectively improve the flexibility and generalization ability of the graph information aggregation process and promote the effect of model migration.

[0005] A potential solution for cross-domain model transfer is to align the data representations of CNNs and GNNs so that convolution operators can be used directly in graph-related tasks. This approach bypasses the challenges caused by the structural differences between data types and simplifies the transfer process. Gao et al. proposed a large-scale learnable graph convolutional network (LGCN) as a method to align graph and image data. LGCN treats nodes as pixels and selects adjacent nodes based on numerical order, effectively converting graph data into a two-dimensional grid-like structure, allowing ordered convolution operations to be performed on general graphs. Although this ensures ordered convolution, erasing nodes and ignoring edges will result in information loss and may not achieve the expected transfer effect.

[0006] It is important to note that image pixels are not equivalent to graph nodes. For images, the connections between pixels are limited to distance proximity and there is no additional information. The information required for the task is composed between adjacent pixels, which requires convolution to weight all pixels within the receptive field for filtering. In contrast, the edges in the graph have clear relationships, and indiscriminate aggregation of local information on known relationships using randomly initialized learnable weights may lead to overfitting to the training data and ignore the relationship and intensity properties of the graph edges. The key challenge in migrating convolutions from CNNs to GNNs is to find convolutional units suitable for graphs.

[0007] As a rising star in the field of machine learning, graph neural networks (GNNs) are widely used in relational data models and have achieved outstanding performance in graph tasks. Graph neural networks continue to receive mature models from other fields such as computer vision and natural language processing, which has promoted the development of graph algorithms. However, due to the different data structures in different fields, the cross-domain migration of models must go through a long process of disassembly and reconstruction, and may not achieve ideal results. Summary of the invention

[0008] The present invention proposes a method for migrating an algorithm model from a convolutional neural network to a graph neural network, aligning the grid data in traditional convolution with the irregular data in the graph, so that the convolution operator applied to the grid data can be directly applied to the graph, further promoting the migration application of convolution on the graph, and solving the above-mentioned problems.

[0009] The technical solution of the present invention is achieved in this way:

[0010] A method for migrating an algorithm model from a convolutional neural network to a graph neural network, including the following:

[0011] (1) Convolution on the graph: A k-order neighbor set is defined, where the k-order neighbor represents the set of nodes that can reach the target node through k hops. The pixel and the neighbor set are regarded as equivalent convolution units, realizing the migration application of convolutional neural networks to graphs.

[0012] (2) Perform neighbor-level convolution:

[0013] The process of L-order neighbor-level graph convolution is as follows:

[0014]

[0015] in, represents the k-order association score between node i and node j, θ k represents the learnable weight of the k-th neighbor, N k (i) represents the k-th neighbor set of node i, x j represents the node characteristics of node j, represents the features of node i after the L-order neighbor-level graph convolution update;

[0016] The L-order neighbor-level convolution is defined as follows:

[0017]

[0018] Among them, σ(·) is the activation function, W is the learnable weight matrix used to extract and compress node features, X represents the feature matrix of all nodes, represents the adjacency matrix of order k;

[0019] The process of the C-channel graph convolution operator is as follows:

[0020] ConvG(X,L,C)=Concatenation(ConvG 1 (X,L),ConvG 2 (X,L),…,ConvG C (X,L));

[0021] Among them, ConvG(X,L,C) is a basic neighbor-level graph convolution module that can be extended to various graph models.

[0022] Optionally, a graph Inception architecture is built through a convolutional bridge, called the GraInc model. The steps to migrate the Inception architecture from a convolutional neural network to a graph neural network are as follows:

[0023] S1. Replace the 1x1 convolution with ConvG (L=0) to extract and compress the intrinsic features of the node;

[0024] S2, replace the 3x3 convolution with the first-order neighbor-level convolution ConvG (L=1);

[0025] S3, replace the 5x5 convolution with the third-order neighbor-level convolution ConvG (L=3);

[0026] S4, remove the maximum pooling branch to ensure that the node neighborhood features are not directly covered by the salient features;

[0027] S5. Obtain the GraInc model.

[0028] 4. Optionally, a graph U-Net architecture is constructed through a convolutional bridge, called the GraU-Net model, to migrate the U-Net architecture from a convolutional neural network to a graph neural network. The steps of migration are as follows:

[0029] S6, retains the four-stage encoder structure, four-stage decoder structure and skip connections between each stage of U-Net;

[0030] S7. In the convolutional layer of each stage, the second-order neighbor-level convolution ConvG (L=2) is used instead of the standard 33x3 convolution to extract local information from the graph;

[0031] S8, in the pooling layer, the Top-K pooling method is used to replace the traditional maximum pooling technique used in the U-Net architecture;

[0032] S9, before each downsampling operation, record the graph structure so as to perform graph upsampling in the decoding stage;

[0033] S10. Get the GraU-Net model.

[0034] After adopting the above technical solution, the beneficial effects of the present invention are:

[0035] In order to assist the effective cross-domain migration of convolutional neural networks to graph neural networks, the present invention proposes a novel graph and image data alignment strategy, named convolutional bridge. Based on the convolutional bridge, neighbor-level graph convolution is introduced, named ConvG. ConvG is the first work to use neighbor sets as convolution units. It exhibits the significant characteristics of traditional convolution and meets the characteristics of graphs. Experiments show that ConvG is interpretable, scalable and effective in graph tasks. Based on ConvG, the present invention migrated the Inception architecture from CNN for node-level tasks, and migrated the U-Net architecture for graph-level tasks, named GraInc and GraU-Net respectively. Compared with previous migration models and current mainstream models, the method of the present invention has advanced performance, which strongly proves the effectiveness of the migration method of the present invention.

[0036] This paper proposes the innovative concept of "convolutional bridge", an important mechanism designed to optimize the cross-domain migration process from convolutional neural networks to graph neural networks. The bridge effectively aligns the data structures between convolutional neural networks and graph neural networks, facilitating the seamless migration of convolutional neural network-based models to the graph domain. Through this strategy, the paper successfully migrated the Inception architecture and U-Net architecture from convolutional neural networks to graph neural networks, creating the GraInc model and GraU-Net model. The GraInc model and GraU-Net model show significant competitiveness, especially on dense graph datasets. These models consistently outperform advanced opponents, providing strong evidence of the effectiveness of the convolutional bridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 It is a schematic diagram of the traditional convolution operator;

[0039] Figure 2 It is a schematic diagram of the neighborhood-level graph convolution operator;

[0040] Figure 3 This is a schematic diagram of the Inception architecture;

[0041] Figure 4 It is a schematic diagram of the GraInc model;

[0042] Figure 5 It is a schematic diagram of the U-Net architecture;

[0043] Figure 6 It is a schematic diagram of the GraU-Net model. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] An embodiment of the present application discloses a method for migrating an algorithm model from a convolutional neural network to a graph neural network.

[0046] Example

[0047] according to Figures 1 to 6 As shown in the figure, a method for migrating algorithm models from convolutional neural networks to graph neural networks.

[0048] 1. Related Work

[0049] In order to apply convolution operators on graphs, Bruna et al. combined graph signal processing technology to map the graph to the graph Fourier domain (also known as the graph spectrum domain). In the graph Fourier domain, the components of the graph signal are regularly distributed, so the one-dimensional convolution operation can be directly implemented. However, this convolution operation is non-locally connected, and the model cannot focus on the local information of the graph. In addition, in each iteration, the graph signal needs to be mapped to the frequency domain and mapped back to the original domain after convolution, which results in a large computational overhead. To solve these problems, Defferrard et al. proposed the ChebyNet model. They assumed that the parameters of the graph convolution kernel are related to the graph frequency, used the K-order frequency response function to fit the parameters of the graph convolution kernel, and proved through mathematical derivation that the ChebyNet model has K localization characteristics.

[0050] Based on the above research, Kipf et al. introduced the graph convolutional network (GCN), which achieved state-of-the-art performance in multiple node classification tasks. The graph convolutional network guides the information aggregation process through a diagonally normalized adjacency matrix, sets a convolution kernel with a fixed rule, and ensures that the way the target node aggregates information is controlled by its degree and the degree of its first-order neighbor nodes. These weights are not trainable. However, not all tasks are suitable for these aggregation rules, and the untrainable convolution kernel limits the model capabilities of the graph convolutional network. To solve this problem, Velivckovic et al. proposed the graph attention network (GAT). GAT adaptively adjusts the aggregation rules of local information by learning the attention weights between the target node and its neighbor nodes. Although GAT provides trainable weights for different neighbor nodes, its learning process is different from that of conventional convolutional neural networks (CNNs), where the weights in the local filters of convolutional neural networks are learned directly. In addition, the attention mechanism leads to practical problems of memory and computing resources due to the additional calculations between nodes in each iteration.

[0051] The graph convolutional network model is a message passing framework based on the graph structure, which mainly includes two modules: information aggregation layer and feature extraction layer. The information aggregation layer is responsible for fusing information from related nodes, while the feature extraction layer is responsible for extracting the features of nodes. It is worth mentioning that the feature extraction layer only extracts information for each node and cannot directly affect the aggregation process. The convolution of an image is the weighting of adjacent pixels, so we focus on the information aggregation process. The information aggregation layer of the graph convolutional network is a dynamic non-learnable convolution operation, and the weights in the convolution kernel are determined by the number of neighboring nodes. The information aggregation layer of GAT is a dynamic learnable convolution operation, and the weights in the convolution kernel are learned according to the features of neighboring nodes, which means that the weights vary from node to node. However, the conventional convolution operation is a static learnable convolution operation, and the weights in the convolution kernel are trainable and shared for all nodes. Static learnable convolution kernels are more conducive to learning the general distribution of data in the feature space, and are suitable for a variety of tasks compared to non-learnable convolution kernels, and save computing resources compared to dynamically learnable convolution kernels. Introducing static learnable convolution operations in convolutional neural networks can enhance the adaptability and generalization ability of the information aggregation process and is conducive to the development of graph convolutional networks.

[0052] After the introduction of GCN and GAT, graph neural network models have sprung up and shined in various relational tasks. In the graph space domain, models such as GraphSAGE, JK-NET, Graph U-Net, SIGN, Dir-GNN, and A-DGN[ were proposed; in the graph spectrum domain, models such as GraphHeat, FAGCN, GPR-GNN, BernNet, and AF-GCN were proposed. These models show strong advantages in relational data that are sensitive to topological information, far exceeding the effects of non-graph neural network models. At the same time, GNNs continue to draw on the structure of CNN to build graph models, such as Graph U-Net, SIGN, and JK-NET. However, the unification of CNN and GNN has not yet been established, so that these graph models do not have static learnable convolutional layers, so these graph models cannot enjoy the advantages of generalization and robustness due to the lack of shared learnable weights. Nevertheless, GNNs that draw on the CNN structure have achieved excellent results in graph tasks, which inspires us to apply convolution operations on graphs.

[0053] 2. Methods

[0054] This paper proposes a strategy to align the data of convolutional neural networks and graph neural networks, combining the characteristics of graphs, called convolutional bridge. By using the convolutional bridge, the migration from convolutional neural networks to graph neural networks is achieved.

[0055] 2.1 Convolutional Bridge

[0056] In previous studies of convolutional neural networks, pixels and nodes are generally considered equivalent. However, when the nodes in the graph are treated as pixels for convolution, a series of problems will arise due to the inherent differences in the data structure between graphs and images. First, the number of neighbors of each node in the graph is different, resulting in an uncertain number of parameters for the graph convolution kernel. Second, the neighbors of the node are unordered, and different arrangements of neighbors will result in different node features calculated by the same convolution operator, which in turn leads to uncertainty in node representation. Finally, the relationship between nodes in the graph is explicitly defined by edges, and using randomly initialized learnable weights to adjust node relationships can easily cause the model to overfit the training set and lose fairness guided by the graph topology. Therefore, convolution in nodes does not conform to the characteristics of the graph and is difficult to implement. Due to the mismatch between nodes and pixels, nodes are not an ideal choice as convolution units.

[0057] In order to solve the above problems and implement convolution on the graph, the present invention defines a k-order neighbor set, where the k-order neighbor represents a set of nodes that can reach the target node through k hops. At the same time, the neighbor set is used to represent the set of neighbors of each order. In order to implement convolution on the graph, the present invention introduces an innovative idea, that is, to regard the neighbor set as a convolution unit. After adding self-loops to the nodes of the graph, each target node has an infinite-order neighbor set, which means that they can share the same size of convolution kernel. In addition, these neighbor sets can be classified according to the order, namely first-order neighbors, second-order neighbors, and so on, until K-order neighbors. Most importantly, convolution based on neighbor sets can effectively mine neighborhood information. The target node and its neighbor sets of different orders together constitute the local structure of the graph, and different order neighbor sets contribute different information to the target node, and these contributions need to be determined by learning. In addition, the interaction between neighbor sets of different orders also needs to be considered. Unlike node-based convolution, convolution based on neighbor sets can effectively capture the implicit relationship between neighbor sets of different orders.

[0058] In order to bridge the gap between convolutional neural networks and graph neural networks and enable the convolutional layer to smoothly migrate to the graph structure without being affected by the difference in data structure, this paper proposes to treat pixels and neighbor sets as equivalent convolutions. In this way, the convolutional layer can be seamlessly migrated to the graph domain.

[0059] 2.2 Neighbor Level Convolution

[0060] A graph is defined as G = (V, E), where V is the set of nodes, |V| = N, and E is the set of edges. Let v i ∈V represents the i node in the graph. If there is an edge between nodes i and j, then (v i ,v j )∈E. Let A∈R N×d Represents the adjacency matrix of graph G, where a ij >0 indicates the correlation score of the edge between nodes i and j, and 0 if there is no correlation. In addition, assume that each node has d-dimensional features, and represent the node feature matrix as X∈R N×d .

[0061] Convolution is a mathematical operation that aims to obtain a new output signal by systematically responding to an input signal. The traditional convolution process can be expressed by the following formula (1):

[0062]

[0063] Where p(·) is the input signal, h(·) represents the impulse response function, also known as the convolution kernel, and y(n) is the output signal. Similar to CNN, the learnable parameters are used as the convolution kernel. Let h(·) = {θ0, θ1, …, θL} represents the impulse response function of different order neighbor information, where θ i ∈h(·) is a learnable parameter. Through the gradient descent algorithm, h(·) can learn the combination pattern of different order neighbor information according to the task. In order to preserve the association score between nodes when performing neighbor-level convolution, a diagonally normalized adjacency matrix is ​​used To guide the node-level aggregation, and use the aggregation information of different orders as the neighbor-level convolution unit p(·). The process of L-order graph convolution is as follows:

[0064]

[0065] in, represents the k-order association score between node i and node j, θ k represents the learnable weight of the k-th neighbor, N k (i) represents the k-th neighbor set of node i, x j represents the node characteristics of node j, It represents the features of node i after the L-order neighbor-level graph convolution update.

[0066] In order to explain the traditional convolution process and the neighbor-level graph convolution process, the present invention visualizes the 3×3 traditional convolution operator and the 1×3 neighbor-level graph convolution operator, such as Figure 1 and Figure 2 As shown. Figure 2 In the above paper, θ0, θ1 and θ2 are learnable parameters that adaptively weight the target node, first-order neighbor nodes and second-order neighbor nodes respectively. The neighbor-level graph convolution operator can work effectively in local relationship exploration while following the basic characteristics of traditional convolution, such as local connection, weight sharing and hierarchical expression.

[0067] In more detail, the basic unit of traditional convolution is pixel, which can describe the color of a certain point in the image. For neighbor-level convolution, the basic unit is a neighbor set of different orders. Each neighbor set contains many nodes with node features. Compared with pixels, neighbor sets contain more complex information. Therefore, compared with traditional convolution, neighbor-level convolution requires additional feature extraction and dimensionality reduction processing. To this end, the present invention introduces a fully connected layer in the convolution layer. The L-order neighbor-level convolution is defined as follows:

[0068]

[0069] Among them, σ(·) is the activation function, W is the learnable weight matrix used to extract and compress node features, X represents the feature matrix of all nodes, Represents the adjacency matrix of order k.

[0070] Neighbor-level convolution can also be viewed as a method of integrating high-order neighbor information. It learns how to combine neighbor information through shared learnable weights. Compared with the method of utilizing high-order neighbor information by applying different learnable weight matrices for different order neighbor sets in traditional graph neural networks, ConvG learns more general weight parameters with lower computational complexity.

[0071] It is worth mentioning that through the optimization of the gradient descent algorithm, different random initializations of θ will produce different convolution kernels, which are task-oriented and useful for model prediction. In order to make full use of this valuable information, the present invention adopts a multi-channel strategy from CNN, in which each convolution operation is repeated multiple times and the results are then spliced ​​together. This multi-channel strategy provides the model with a broader feature perspective, which is conducive to the analysis and solution of graph tasks. The process of the C-channel graph convolution operator is as follows:

[0072] ConvG(X,L,C)=COncatenation(ConvG 1 (X,L),ConvG 2 (X,L),…,ConvG C (X,L)); (4)

[0073] Among them, ConvG(X,L,C) is a basic neighbor-level graph convolution module that can be extended to various graph models.

[0074] For a given graph task, L and C are selected according to the task conditions and goals. For the hyperparameter L, when it is necessary to capture smaller local features, it is a better choice to choose L=1. When the task needs to increase the receptive field and capture a wider range of graph features, you can consider using L=2 or a larger value. For the hyperparameter C, if the task is easier to learn, just choose C=1. When the task is more complex and more parameters are needed to learn to adapt to the task, the value of C should be increased. If there is no prior knowledge of the target graph task, the hyperparameters L and C should be selected through grid search. The running time for searching for the optimal hyperparameters in ConvG is linear in the search range of L. However, since C determines the number of repeated runs of ConvG, the search time is quadratic in the search range of C.

[0075] 3. GraInc Model

[0076] Deeper and wider neural networks usually mean greater capacity and expressiveness, which helps to better capture data patterns and features in complex tasks. However, improving model performance by increasing depth and width also brings two disadvantages: ① As the depth and width increase, the number of parameters to be learned also increases, and the huge number of parameters easily leads to overfitting problems. ② The increase in network size will also bring about the problem of computing resource consumption. The fundamental way to solve these two problems is to eventually shift from convolutional and fully connected architectures to sparsely connected architectures. However, existing computing frameworks are inefficient when processing non-uniform sparse data. A large number of literatures on sparse matrix computing show that aggregating sparse matrices into relatively dense sub-matrices can often achieve state-of-the-art practical performance in sparse matrix multiplication. In the field of computer vision, Szegedy et al. proposed the Inception architecture. The core idea of ​​the Inception architecture is to use dense components to approximate or replace the best local sparse features, for example, by splicing and compressing features from different receptive fields to replace features from a single receptive field.

[0077] In order to introduce the advantages of the Inception architecture into graph neural networks and evaluate the feasibility and effectiveness of the migration strategy, the present invention migrated the Inception architecture from convolutional neural networks to graph neural networks and conducted experiments in node classification tasks.

[0078] This paper builds a graph Inception architecture through a convolutional bridge, called the GraInc model, which draws on the Inception architecture in the field of computer vision to improve the performance of graph tasks. Figure 3 As shown. The migration process includes the following steps: Replace the 1x1 convolution with ConvG (L=0) to extract and compress the inherent features of the node. Considering that in the node classification task, if a high-order neighbor set is used in the information aggregation process, the node features tend to be biased towards local features, resulting in excessive smoothing of the neighbor node features. Therefore, the present invention uses low-order neighbor-level convolutions during the migration process. The present invention replaces the 3x3 convolution with the first-order neighbor-level convolution ConvG (L=1), and replaces the 5x5 convolution with the third-order neighbor-level convolution ConvG (L=3). Finally, the GraInc model is obtained, as shown in Figure 4 shown.

[0079] It is worth noting that Szegedy et al. believed that max pooling has the ability to extract features, so they used the max pooling branch in the Inception architecture. However, using max pooling in node-level tasks causes the salient features of neighboring nodes to replace the features of the target node. However, in heterogeneous graph tasks, neighboring nodes often have different labels, and their node representations are significantly different. The propagation of salient features will cause node features to be incorrectly mixed, which is detrimental to node classification tasks. Therefore, GraInc omits the max pooling branch.

[0080] 4. GraU-Net model

[0081] In the field of computer vision, the U-Net architecture based on the encoder-decoder architecture has made significant progress in image semantic segmentation. The U-Net architecture extracts and enhances image features through a two-stage approach. In the encoder part, the U-Net architecture extracts features from the input image by stacking multiple convolutional layers and max pooling layers, which can capture different levels of features, from low-level details (such as edges and textures) to high-level semantic features. In the decoder part, the U-Net architecture uses deconvolution layers to upsample the features extracted by the encoder and restore them to the original image size. In addition, the decoder also combines feature cascading operations, that is, splicing the output of each deconvolution layer with the corresponding encoder feature layer. This method can retain more contextual information during the upsampling process. The unique U-shaped structure of the U-Net architecture facilitates intuitive and reasonable splicing between features of different granularities, combining contextual information and fine details to achieve more accurate global image feature extraction.

[0082] In order to introduce the advantages of the U-Net architecture into the graph neural network (GNN) and verify the feasibility and effectiveness of the migration strategy, the present invention migrated the U-Net architecture from the convolutional neural network (CNN) to the graph neural network and conducted experiments in the graph classification task.

[0083] The present invention constructs a graph U-Net architecture, called GraU-Net, through a convolutional bridge, which draws on the U-Net architecture in the field of computer vision. Specifically, the present invention treats neighbor sets of different orders as pixels, so that learnable convolution operations can be applied directly on the graph. By replacing the convolution applied to the image in the U-Net architecture with neighbor-level convolution, the GraU-Net model can obtain neighbor combination features of different granularities to assist in graph-level tasks. The U-Net architecture is as follows Figure 5As shown. The migration process includes the following steps: In terms of model architecture, the present invention retains the four-stage encoder structure, four-stage decoder structure and jump connections between each stage of the U-Net architecture. In the convolution layer of each stage, the present invention uses the second-order neighbor-level convolution ConvG (L=2) to replace the standard 33x3 convolution in order to extract local information from the graph. In the pooling layer, the present invention uses the Top-K pooling method to replace the traditional maximum pooling technology used in the U-Net architecture. This improvement can effectively downsample the graph while retaining key graph information. In addition, before each downsampling operation, the graph structure is recorded so that the graph can be upsampled in the decoding stage. The complete GraU-Net model is shown in the figure. Figure 6 shown.

[0084] 5. Experiment

[0085] The present invention evaluates the performance of the convolutional bridge proposed in the present invention by comparing the performance of the neighbor-level convolutional model ConvG, the cross-domain model GraInc and the GraU-Net model with the previous state-of-the-art models in node classification and graph classification tasks. First, the real datasets and experimental settings used are introduced, and then the experimental results of these models on multiple real datasets are presented and analyzed. All experiments are performed on a machine equipped with NVIDIA GeForce RTX 4090 (24G memory), 13th generation Intel Core i9-13900K CPU (3.00GHz) and 32G RAM.

[0086] 5.1 Dataset

[0087] In order to comprehensively evaluate the performance of the model, the present invention uses real datasets covering multiple fields. In the node classification task, the present invention uses five homogeneous datasets from the Pytorch Geometric library. The connected nodes in these datasets usually have the same label. Including citation graphs (such as Cora, CiteSeer, PubMed), and co-purchase graphs from Amazon (such as Computers and Photo) datasets. In addition, the present invention also introduces three heterogeneous benchmark datasets, in which the connected nodes in these datasets often have different labels. These datasets include Actor co-occurrence graphs and web page graphs (Texas and Cornell) from WebKB. In order to fully verify the generalization of the model, the present invention also introduces a large-scale dataset OGBN-Arxiv. Table 1 summarizes the statistical information of the dataset, where H(G) is an indicator proposed by Pei et al. to measure the homogeneity of the dataset. The higher the H(G) value, the stronger the homogeneity. The graph density is calculated by the ratio of edges to nodes.

[0088] Table 1: Datasets used in node classification experiments

[0089]

[0090] In the graph classification task, the present invention uses two datasets from the field of bioinformatics: PROTEINS and D&D. The former is used to predict whether a protein functions as an enzyme. The present invention also uses two datasets from the field of small molecules: NCI1, which is from the field of compounds and is used to detect whether the screening compound has anticancer activity against non-small cell lung cancer cells; MUTAG is used to determine whether the molecule is a steroid compound. In addition, the present invention also uses four datasets from the field of social networks: Reddit-BINARY, which is used to classify posts based on user interactions; COLLAB, which classifies research fields by analyzing the egocentric network graphs of researchers; IMDB-BINARY and IMDB-MULTI, which classify movie types by mining the actor egocentric network in the Internet Movie Database (IMDB). Table 2 summarizes the statistics of the datasets in detail.

[0091] Table 2: Datasets used in graph classification experiments

[0092]

[0093] 5.2 Comparison Methods and Experimental Settings

[0094] In the performance comparison of node classification and graph classification tasks, the present invention uses the cutting-edge models in recent years as the benchmark baseline. In the node classification task, the present invention conducts extensive comparisons involving eleven classic models. These models are divided into two groups: nine models based on the graph space domain, including MLP, GraphSAGE, GAT, LGCN, SIGN, Dir-GNN, A-DGN, Ordered GNN and M2M-GNN; and four models based on the spectrum domain, including GCN, FAGCN, GPR-GNN and BernNet. It is worth noting that the LGCN model is a node-level graph convolution model migrated from traditional convolution. Compared with the ConvG model, the present invention can clearly identify the advantages of neighbor-level feature aggregation. In addition, the SIGN model introduces an Inception architecture similar to GraInc, but its convolution process is completed by GCN, and due to the untrained convolution kernel, it faces the problem of limited fitting ability. By comparing with the GraInc model, the significant competitiveness of the neighbor-level graph convolution kernel in cross-domain model migration can be more clearly observed.

[0095] The present invention conducts a fully supervised node classification experiment for each model. The node set is randomly divided into a training set, a validation set, and a test set with a ratio of 60%, 20%, and 20%. In order to ensure the fairness of the experiment, ten different seeds are randomly generated for data set partitioning. All models are evaluated under the same data set partitioning and repeated ten times under different random seeds. The final results are obtained through these partitionings. All compared models are designed based on a unified Pytorch framework and use Adam as a gradient descent optimizer. The hyperparameters are set to: 32 hidden units, 2 layers, 500 training rounds, and early stopping is set to 200. Grid search is used to select the learning rate {0.001, 0.002, 0.003, 0.01, 0.02, 0.03, 0.1} and dropout {0.0, 0.5, 0.6, 0.7, 0.8, 0.9}.

[0096] In the graph classification task, the present invention compares ten classic models, including GCN, GraphSAGE, DiffPool, EdgePool, GlobalAttention, Graph U-Net, SAGPool, ASAP, Cuco

[55] , AF-GCN, ENGAGE and Tree-G. Graph U-Net migrates the U-Net architecture from the CV field and uses GCN as the graph convolution kernel. By comparing with GraphU-Net, we can observe the effectiveness of learnable neighbor-level convolution in graph-level tasks. The present invention uses ten-fold cross validation to measure the performance of the model. All models are designed based on the Pytorch framework, the training rounds are set to 200, and the early stopping is set to 100. The hyperparameters are set as: learning rate 0.0005, weight decay 0.0005, dropout is set to 0.5, and batch size is 60. GCN and GraU-Net both use Top-K pooling, GraphSAGE uses global average pooling, and the remaining models use the pooling strategy proposed in the original setting.

[0097] The evaluation indicators in the node classification and graph classification experiments are calculated using the same process. After calculating the accuracy of ten divisions, the bootstrap algorithm is used to randomly sample 1000 times to construct a normal distribution of the model classification accuracy. The average accuracy (%) ± 95% confidence interval of the normal distribution is used as the evaluation indicator to evaluate the accuracy and error range of the model.

[0098] 5.3 Ablation Experiment

[0099] In order to comprehensively evaluate the impact of neighborhood-level convolution and cross-domain transfer, ablation experiments are conducted on node classification and graph classification tasks.

[0100] (1) Ablation study of node classification task: In the node classification experiment, eight control groups were set up, including the traditional graph neural network GCN, the SIGN model based on the cross-domain migration Inception architecture of GCN, the single-channel neighborhood-level convolution ConvG (C = 1), the three-channel neighborhood-level convolution ConvG (C = 3), and the GraInc model based on the cross-domain migration Inception architecture of ConvG (C = 3). In order to show the influence of different convolution scales, different scales are set for the three convolution branches of GraInc, including GraIn (L = [0, 1, 2]), GraInc (L = [0, 1, 3]) and GraInc (L = [0, 2, 4]). In order to further verify the influence of the maximum pooling branch, GraInc (L = [0, 1, 3]) containing the maximum pooling branch is set, which is called GraInc (Max pooling). The experimental results are shown in Table 3. Comparing GCN and ConvG, we can clearly see the improvement in accuracy, highlighting the effectiveness of neighborhood-level graph convolution in node classification tasks. In addition, the migration from GCN to SIGN and from ConvG to GraInc significantly improved the accuracy of the model. Both GCN and ConvG introduced the Inception architecture to perceive multi-scale features and showed better results. However, SIGN uses different learnable weight matrices for different neighborhood sets, which has higher computational complexity and parameter count, while ConvG is more competitive, which further emphasizes the advantage of neighborhood-level convolution in cross-domain migration. Comparing GraInc (L = [0, 1, 3]) with and without max pooling, we can intuitively see that GraInc performs poorly on heterogeneous datasets after adding the max pooling branch. This is because the use of max pooling will cause the salient features of the neighborhood nodes to replace their own nodes, and in heterogeneous datasets, neighborhood nodes often have different labels. The propagation of salient features will instead lead to the convergence of neighborhood nodes, affecting the accuracy of node classification. GraInc models with different perception scales show different results. GraInc(L=[0,1,2]) performs well overall but not outstandingly, while GraInc(L=[0,2,4]) performs best on the PubMed and Computers datasets and is slightly inferior to GraInc(L=[0,1,3]) on other datasets.

[0101] Table 3: Ablation study on node classification benchmark datasets

[0102]

[0103] (2) Ablation study of graph classification task: In the graph classification experiment, seven control groups were set up, including the traditional graph neural network GCN, the Graph U-Net architecture based on the cross-domain transfer U-Net architecture of GCN, the single-channel neighborhood-level convolution ConvG (C=1), the three-channel neighborhood-level convolution ConvG (C=3), and the GraU-Net model based on the cross-domain transfer U-Net architecture of ConvG (C=1). In order to further demonstrate the impact of convolution scale, different scales are set for the convolution layer of GraU-Net, including GraU-Net (L=1), GraU-Net (L=2) and GraU-Net (L=3). The experimental results are shown in Table 4. Compared with GCN, ConvG shows significant improvement in graph classification tasks, which can be attributed to the superiority of neighborhood-level graph convolution in processing local graph information compared to node-level graph convolution. Compared with the Graph U-Net architecture, GraU-Net shows more significant improvement on the basis of ConvG. This is due to the synergistic combination of learnable neighborhood-level convolution and U-Net architecture, which enables more comprehensive contextual information extraction and fusion in the graph. The above findings further demonstrate the applicability of neighborhood-level convolution in cross-domain migration. In the comparison of GraU-Net models with different convolution scales, GraU-Net$(L=2)$ performed better overall. Other models also showed excellent performance. For example, GraU-Net(L=1) outperformed the GraU-Net(L=2) model by 3.73% on the MUTAG dataset.

[0104] Table 4: Ablation study on graph classification benchmark datasets

[0105]

[0106] 5.4 Node Classification Task

[0107] According to the aforementioned node classification experimental settings, the present invention compares ConvG and GraInc with 11 other state-of-the-art node classification models. The relevant results are summarized in Table 5. Bold letters represent the best results within a given confidence interval.

[0108] (1) From the perspective of node classification accuracy. The results in Table 5 highlight the excellent performance of GraInc, which achieved leading results in six of the nine datasets. Even in the Cora dataset, GraInc is only 0.35% lower than the leading model. The GraInc model is based on the neighbor-level graph convolution operator, which significantly improves the prediction accuracy in these real-world datasets, by nearly 1.3% in the Actor dataset, 0.9% in the OGBN-Arxiv dataset, and 0.6% in the Pubmed dataset, all exceeding the second-best model. It is worth noting that GraInc's performance is particularly outstanding in dense datasets because neighbor-level convolutions rely more on rich neighbor information.

[0109] Table 5: Performance study of node classification benchmark datasets

[0110]

[0111] (2) From the perspective of migration effect. Although LGCN and ConvG are both migrated from CNN, the LGCN model that uses nodes as convolution units is far inferior to the ConvG model that uses neighbor sets of different orders as convolution units. This observation provides strong evidence that neighbor-level convolution is more in line with the intrinsic properties of the graph. In addition, although the SIGN model and GraInc are also based on the concept of the Inception architecture, SIGN's performance is still slightly inferior. This difference stems from the fact that SIGN uses GCN with fixed, non-trainable convolution kernels. Although using multiple receptive fields helps capture more graph information, it is still a challenge to obtain excellent results using fixed convolution kernels.

[0112] (3) From the perspective of applicability. Models such as GPR-GNN, FAGCN, and BernNet can dynamically adjust graph signals in the graph domain to adapt to task requirements. Regardless of whether the dataset is homogeneous or heterogeneous, these models can always achieve high accuracy in node classification prediction. The neighbor-level graph convolution can dynamically adjust the information aggregation of neighbors of different orders in the graph space domain, which also enables ConvG and GraInc to adapt to a variety of tasks. In contrast, models such as GraphSAGE, GCN, GAT, and LGCN are only applicable to homogeneous datasets.

[0113] (4) From the perspective of training time. In order to comprehensively evaluate the advantages and disadvantages of these models, the training time of each method is reported in Table 6. It is worth noting that in each convolution iteration, reconstructing the graph structure requires a lot of CPU operations, making LGCN the slowest of all models. In theory, the propagation step of GCN, GRP-GNN, Ordered GNN, M2M-GNN and ConvG has linear time complexity, while BernNet has quadratic time complexity. If models such as SIGN and GraInc, which are composed of multiple component models, are not considered, BernNet is the second slowest model. The time complexity of GraInc is consistent with ConvG, but due to the multiple stacking of ConvG, GraInc has a longer training time. Models such as GAT and FAGCN develop information aggregation strategies by calculating attention scores between nodes. Their time complexity is positively correlated with the network density and the dimension of node features, resulting in longer training time. Overall, models that implement learnable information aggregation strategies tend to achieve higher accuracy, but at the cost of increased time and space complexity. GraInc achieves impressive results on real-world datasets without significantly sacrificing training efficiency.

[0114] Table 6: Model training time on the node classification benchmark dataset

[0115]

[0116] 5.5 Image Classification Task

[0117] According to the above graph classification experimental settings, the present invention compares GraInc with ten other advanced graph classification models. The relevant results are summarized in Table 7. Bold words represent the best results within a given confidence interval.

[0118] (1) From the perspective of classification accuracy. It is obvious from Table 7 that GraU-Net performs best in four of the eight datasets. It is worth noting that in the graph classification task, GraU-Net shows a strong correlation between the density of the benchmark datasets. Compared with the second-place model, GraU-Net shows a significant improvement, with an improvement of nearly 10% in COLLAB and approximately 7% in DD, IMDB-MULTI, and IMDB-BINARY respectively. These advances significantly surpass the current baseline models. The success of GraU-Net can be attributed to two main factors. On the one hand, in the graph classification task, information from different neighborhoods in the graph plays a key role. Neighbor-level convolution can extract local information more efficiently. On the other hand, by combining the U-Net architecture from the field of computer vision, the model facilitates intuitive and reasonable splicing of features of different granularities. These enhancement factors jointly promote the excellent performance of GraU-Net in graph classification tasks.

[0119] Table 7: Performance study of graph classification benchmark datasets

[0120]

[0121] (2) From the perspective of migration effect. The comparative model Graph U-Net also borrows from the U-Net architecture and has achieved remarkable results in graph classification tasks. However, the convolution operation of Graph U-Net is completed through GCN, which still faces the problem of limited fitting ability due to the non-learnable convolution kernel. Therefore, its performance is not as good as the GraU-Net model.

[0122] (3) From the perspective of training time. In order to comprehensively evaluate the advantages and disadvantages of GraU-Net, the present invention reports the training time of each method in Table 8. Among these models, EdgePool, ASAP, Cuco and Tree-G are the most time-consuming. Tree-G splits graph data into decision trees. In order to achieve the best division of subgraphs, it must continuously perform grid search on the hyperparameters of the segmentation function, which is a time-consuming process. EdgePool needs to calculate the scores of all edges during the downsampling process to determine which nodes need to be shrunk. This complex process causes significant time overhead, making it slower than other comparison models. ASAP initially aggregates the network into clusters, and then uses an attention mechanism to fuse the features of all nodes in each cluster, which also consumes a lot of time. Cuco requires a lot of CPU operations when performing graph enhancement to obtain positive and negative samples. As the size of the graph increases, its training time increases significantly. As for Graph U-Net and GraU-Net, both use Top-K Pooling as GCN, which theoretically has the same time complexity. However, the Graph U-Net and GraU-Net models based on the U-Net framework require stacking multiple graph convolutional units, resulting in longer training time.

[0123] Table 8: Model training time on graph classification benchmark datasets

[0124]

[0125] The present invention solves the problem of migration of convolutional neural networks to graph neural networks, which is hindered by differences in data structures. The present invention constructs a data alignment bridge between convolutional neural networks and graph neural networks, named convolution bridge, by treating neighbor node sets of different orders as convolution units. In this way, the learnable and parameter-sharing convolution operator can be smoothly migrated to the graph while retaining excellent adaptability and generalization capabilities. On this basis, a neighbor-level graph convolution operation is designed, named ConG, and a graph Inception architecture is constructed for node-level tasks, and a graph U-Net architecture is constructed for graph-level tasks, named GraInc model and GraU-Net model respectively. In the node classification task, the GraInc model surpasses the current advanced models. Similarly, the GraU-Net model also shows excellent accuracy in the graph classification model, proving the effectiveness of the proposed neighbor-level convolution. Compared with SIGN and Graph U-Net, which are based on the idea of ​​migration from CNN, the designed GraInc model and GraU-Net model have higher performance, proving the feasibility of the data alignment migration strategy.

[0126] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for migrating an algorithm model from a convolutional neural network to a graph neural network, characterized in that: Includes the following: (1) Convolution on the graph: A k-order neighbor set is defined, where the k-order neighbor represents the set of nodes that can reach the target node through k hops. The pixel and the neighbor set are regarded as equivalent convolution units, realizing the migration application of convolutional neural networks to graphs. (2) Perform neighbor-level convolution: The process of L-order neighbor-level graph convolution is as follows: in, represents the k-order association score between node i and node j, θ k represents the learnable weight of the k-th neighbor, N k (i) represents the k-th neighbor set of node i, x j represents the node characteristics of node j, represents the features of node i after the L-order neighbor-level graph convolution update; The L-order neighbor-level convolution is defined as follows: Among them, σ(·) is the activation function, W is the learnable weight matrix used to extract and compress node features, X represents the feature matrix of all nodes, represents the adjacency matrix of order k; The process of the C-channel graph convolution operator is as follows: ConvG(X,L,C)=Concatenation(ConvG 1 (X,L),ConvG 2 (X,L),…,ConvG C (X,L)); Among them, ConvG(X,L,C) is a basic neighbor-level graph convolution module that can be extended to various graph models.

2. According to claim 1, a method for migrating an algorithm model from a convolutional neural network to a graph neural network is characterized in that: The graph Inception architecture, called the GraInc model, is built through the convolutional bridge. The steps to migrate the Inception architecture from the convolutional neural network to the graph neural network are as follows: S1. Replace the 1x1 convolution with ConvG (L=0) to extract and compress the intrinsic features of the node; S2, replace the 3x3 convolution with the first-order neighbor-level convolution ConvG (L=1); S3, replace the 5x5 convolution with the third-order neighbor-level convolution ConvG (L=3); S4, remove the maximum pooling branch to ensure that the node neighborhood features are not directly covered by the salient features; S5. Obtain the GraInc model.

3. The method for migrating an algorithm model from a convolutional neural network to a graph neural network according to claim 2, characterized in that: The graph U-Net architecture, called the GraU-Net model, is constructed through the convolutional bridge to migrate the U-Net architecture from the convolutional neural network to the graph neural network. The migration steps are as follows: S6, retains the four-stage encoder structure, four-stage decoder structure and skip connections between each stage of U-Net; S7. In the convolutional layer of each stage, the second-order neighbor-level convolution ConvG (L=2) is used instead of the standard 33x3 convolution to extract local information from the graph; S8, in the pooling layer, the Top-K pooling method is used to replace the traditional maximum pooling technique used in the U-Net architecture; S9, before each downsampling operation, record the graph structure so as to perform graph upsampling in the decoding stage; S10. Get the GraU-Net model.

Images