Method and device for searching graph neural architecture under distribution offset

Abstract

A method for designing a graph neural network (GNN) through a graph neural architecture search under a distribution offset between a training graph and a test graph is provided. The method includes obtaining, by a graph encoder module, a graph representation of an input graph in a decoupled implicit space; obtaining, by an architecture customization module, a searched GNN architecture for the input graph based on a probability of an operation selected from a set of candidate operations in a layer of the searched GNN architecture, wherein the probability of the operation is a function of a similarity between the obtained graph representation and a learnable prototype vector representation of the operation; and obtaining a weight of the searched GNN architecture through a super network module, wherein different operations in the candidate operation set are mixed into a continuous space.

Description

Technical Field

[0001] The present disclosure generally relates to the field of artificial intelligence technology, and more specifically, to graph neural architecture search technology. Background Art

[0002] In recent years, graph-structured data has attracted much attention due to its flexible representation capabilities in various fields. Graph neural network (GNN) models have been proposed and have achieved great success in many graph tasks. In order to save manpower in designing GNN architectures for different tasks and automatically design more powerful GNNs, graph neural architecture search (GNAS) has been utilized to search for the best GNN architecture. Under the independent and identical distribution (IID) assumption, i.e., the training and test graphs are independently sampled from the same distribution, these automatically designed architectures have achieved competitive or better performance than manually designed GNNs on datasets with the same distribution.

[0003] However, in real-world graph applications where there are a large number of unforeseen and uncontrollable hidden factors, distribution shift is ubiquitous and inevitable. Existing GNAS methods under the IID assumption only search for a single fixed GNN architecture based on the training set before directly applying the selected architecture on the test set, which cannot handle the distribution shift of changes in the out-of-distribution setting. Since the single GNN architecture discovered by the existing methods may overfit the distribution of the training graph data, it may not be able to accurately predict the test graph data with various distributions different from the training graph data.

[0004] Therefore, there is a need for an improved method and apparatus for performing graph neural architecture search under distribution shift between training graph data and test graph data. Summary of the invention

[0005] A simplified summary of one or more aspects of the present disclosure is presented below to provide a basic understanding of these aspects. This summary is not an extensive overview of all contemplated aspects, and is neither intended to identify key or important elements of all aspects, nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.

[0006] In one aspect of the present disclosure, a method for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between a training graph and a test graph is disclosed. The method includes obtaining a graph representation of an input graph in a decoupled latent space through a graph encoder module; obtaining a searched GNN architecture for the input graph based on the probability of an operation selected from a candidate operation set in a layer of the searched GNN architecture through an architecture customization module, wherein the probability of the operation is a function of the similarity between the obtained graph representation and a learnable prototype vector representation of the operation; and obtaining a weight of the searched GNN architecture through a hypernetwork module, wherein different operations in the candidate operation set are mixed into a continuous space.

[0007] In another aspect of the present disclosure, a device for searching a GNN designed by a graph neural architecture under a distribution shift between a training graph and a test graph is disclosed. The device includes: a graph encoder module for obtaining a graph representation of an input graph in a decoupled latent space; an architecture customization module for obtaining the searched GNN architecture for the input graph based on the probability of an operation selected from a candidate operation set in a layer of the searched GNN architecture, wherein the probability of the operation is a function of the similarity between the obtained graph representation and a learnable prototype vector representation of the operation; and a hypernetwork module for obtaining the weights of the searched GNN architecture, wherein different operations in the candidate operation set are mixed into a continuous space in the hypernetwork module.

[0008] In another aspect of the present disclosure, a device for designing a GNN by searching a graph neural architecture under a distribution shift between a training graph and a test graph is disclosed. The device may include a memory and at least one processor coupled to the memory. The at least one processor may be configured to: obtain a graph representation of an input graph in a decoupled latent space through a graph encoder module; obtain the searched GNN architecture for the input graph based on the probability of an operation selected from a candidate operation set in a layer of the searched GNN architecture through an architecture customization module, wherein the probability of the operation is a function of the similarity between the obtained graph representation and the learnable prototype vector representation of the operation; and obtain the weights of the searched GNN architecture through a hypernetwork module, wherein different operations in the candidate operation set are mixed into a continuous space.

[0009] In another aspect of the present disclosure, a computer-readable medium storing computer code for designing a GNN by searching a graph neural architecture under a distribution shift between a training graph and a test graph is disclosed. The computer code, when executed by a processor, can cause the processor to: obtain a graph representation of an input graph in a decoupled latent space through a graph encoder module; obtain the searched GNN architecture for the input graph through an architecture customization module based on the probability of an operation selected from a candidate operation set in a layer of the searched GNN architecture, wherein the probability of the operation is a function of the similarity between the obtained graph representation and the learnable prototype vector representation of the operation; and obtain the weights of the searched GNN architecture through a hypernetwork module, wherein different operations in the candidate operation set are mixed into a continuous space.

[0010] In another aspect of the present disclosure, a computer program product for designing a GNN by searching a graph neural architecture under a distribution shift between a training graph and a test graph is disclosed. The computer program product may include a processor executable computer code for: obtaining a graph representation of an input graph in a decoupled latent space through a graph encoder module; obtaining a searched GNN architecture for the input graph based on the probability of an operation selected from a candidate operation set in a layer of the searched GNN architecture through an architecture customization module, wherein the probability of the operation is a function of the similarity between the obtained graph representation and a learnable prototype vector representation of the operation; and obtaining the weights of the searched GNN architecture through a hypernetwork module, wherein different operations in the candidate operation set are mixed into a continuous space.

[0011] Other aspects or variations of the present disclosure will become apparent by consideration of the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The following drawings depict various embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the methods and structures disclosed herein can be implemented without departing from the spirit and principles of the present disclosure described herein.

[0013] Figure 1 A simplified example of a graph according to one aspect of the present disclosure is shown.

[0014] Figure 2 A schematic model of graph neural architecture search under distribution shift according to one aspect of the present disclosure is shown.

[0015] Figure 3 A flowchart of a method for designing a GNN through GNAS under distribution shift according to one aspect of the present disclosure is shown.

[0016] Figure 4A block diagram of an apparatus for designing a GNN through GNAS under distribution shift according to one aspect of the present disclosure is shown.

[0017] Figure 5 A block diagram of an apparatus for designing a GNN through GNAS under distribution shift according to one aspect of the present disclosure is shown. DETAILED DESCRIPTION

[0018] Before explaining any embodiment of the present disclosure in detail, it should be understood that the application of the present disclosure is not limited to the structural details and feature arrangements set forth in the following description. The present disclosure is capable of other embodiments and can be implemented or carried out in various ways.

[0019] Figure 1 A simplified example of a graph according to one aspect of the present disclosure is shown. A graph is a nonlinear data structure consisting of nodes and edges. A node may also be referred to as a vertex, and an edge is a line or arc connecting any two nodes in the graph. For example, Figure 1 As shown, a simple graph 100 includes nodes n1-n7 and edges e1-e6, edge e1 connects nodes n1 and n3, edge e2 connects nodes n2 and n4, and so on. Graph structured data can be used in various fields that cannot be structured in Euclidean space, including social networks, information networks, biological networks, infrastructure networks, etc. In one example, graph 100 can be an assembly line layout diagram, each node in the graph can represent a workshop, and each edge can represent the association between two workshops. In another example, graph 100 can be a circuit design layout diagram, each node in the graph can represent an electronic device of a circuit module, and each edge can represent the association between two electronic devices or circuit modules.

[0020] Graph neural networks (GNNs) can learn node representations through a recursive message passing scheme, where nodes iteratively aggregate information from their neighbors. Then, taking a graph classification task as an example, GNNs can use pooling methods to derive graph-level representations. Different GNN architectures differ mainly in their message passing mechanisms, i.e., how to exchange information, to adapt to the needs of different graph scenarios. Graph Neural Architecture Search (GNAS) can be used to automatically design GNN architectures for various graph tasks. However, when there is a distribution shift between training and test graphs, existing methods cannot handle the problem of adapting to unknown test graph structures because they only search for fixed architectures for all graphs. Taking drug discovery as an example, there is only a limited amount of training data for experiments, and due to the complex chemical properties of different molecules, their interaction mechanisms vary greatly. Therefore, GNN models designed for drug discovery must usually be tested on data with distribution shifts.

[0021] In the present disclosure, an improved graph neural architecture search method under distribution shift is provided. This graph neural architecture search method can capture key information about graphs with widely varying distributions in an out-of-distribution setting by customizing a unique GNN architecture for each graph instance.

[0022] Specifically, a self-supervised decoupled graph encoder is designed, which can project a graph into a decoupled latent space, where each decoupled factor is trained simultaneously by a supervised task and a corresponding self-supervised learning task. The design can capture the key information hidden in the graph in a more controllable way via a self-supervised decoupled graph representation, thereby improving the ability of the representation to generalize under distribution shift. Then, a prototype-based architecture customization is used to customize a specialized GNN architecture for the graph based on the similarity of its representation to the prototype vector in the latent space, where each prototype vector corresponds to a different operation. Next, a customized supernetwork with differentiable weights on a mixture of different operations is designed, which has great flexibility to integrate different combinations of operations and enables the disclosed GNAS model to be easily optimized in an end-to-end manner by a gradient-based method. The design of decoupled graph representation and learnable prototype operation mapping can enhance the generalization ability of the disclosed GNAS model under distribution shift. Extensive experiments on both synthetic and real-world graph datasets also show the superiority of the disclosed GNAS model over existing GNAS baselines.

[0023] For ease of description, the graph space is represented as The label space is represented as The training graph dataset is represented as The corresponding training label set is denoted as The test image dataset is represented as And the corresponding test label set is represented as The goal of GNAS under distribution shift is to use G tr and Y tr To design the model Its P(G tr ,Y tr )≠P(G te ,Y te ) works well on G te and Y te , where P(G tr ,Y tr ) represents the probability distribution of the training graph dataset, and P(G te ,Y te ) represents the probability distribution of the test image dataset, that is,

[0024]

[0025] in, is the loss function. In a common but challenging setting, during the training phase, Y te and unlabeled G te None of them are available. F can be a GNN for graph machine learning. A typical GNN consists of two parts: architecture and learnable weights in and Represent the architecture space and weight space respectively. Therefore, GNN can be expressed as the following mapping function

[0026] The present disclosure mainly focuses on different GNN layers, i.e., message passing functions, for searching GNN architectures. Therefore, a search space of layer-by-layer architectures without complex connections (such as residual or skip connections) is adopted, although the disclosed method can be easily generalized. In an embodiment, five widely used GNN layers can be used as the operation candidate set These include graph convolutional networks (GCN), graph attention networks (GAT), graph isomorphism networks (GIN), graph sampling and aggregation (SAGE), and GraphConv. In addition, multi-layer perceptrons (MLP) that do not consider graph structure can also be used. The pooling layer can also be fixed at the end of the GNN architecture as standard global mean pooling.

[0027] In the present disclosure, instead of using a fixed GNN architecture for all graphs as in the existing GNAS method, a GNN architecture can be customized for each graph. In this way, the disclosed GNAS method is more flexible and can better handle test graphs under distribution shift, because different GNN architectures are known to be suitable for different graphs. Therefore, it is necessary to learn an architecture mapping function and the weight mapping function These functions can automatically generate the best GNN for different graphs, including the architecture and its weights. Since the architecture depends only on the graph, the weight mapping function can be further simplified to Therefore, formula (1) can be transformed into the following objective function:

[0028]

[0029] in, is a regularization term, and γ is a hyperparameter representing the weight of the main loss function. Figure 2 Detailed description for proper design of Φ A , Φ w and The specific embodiment of the disclosed GNAS method can generalize under distribution shift.

[0030] Figure 2A schematic model of graph neural architecture search under distribution shift according to one aspect of the present disclosure is shown. Figure 2 As shown, the GNAS model 200 includes three cascaded modules, namely, a self-supervised decoupled graph encoder module 210, an architecture customization module 220 with a prototype strategy, and a customized super network module 230, to customize a unique GNN architecture for each graph instance, so that the model 200 can handle generalization under distribution shift with a non-IID setting. The graph encoder module 210 can capture diverse graph structures through self-supervision and supervised losses. Then, the architecture customization module 220 can customize the most suitable GNN architecture based on the learned graph representation. Finally, the customized super network module 230 can achieve efficient training through weight sharing. Each of these modules will be described in detail below.

[0031] like Figure 2 As shown, graphs g1, g2, and g3 with different structures are input into the graph encoder module 210. It can be understood that during the training phase, more input graphs can be input into the graph encoder module 210. These input graphs can have various graph structures from different distributions. In order to capture such various graph structures, the graph encoder module 210 can learn a low-dimensional representation of the graph. In an embodiment, K GNNs can be used to learn K-chunk graph representations:

[0032]

[0033] in, is the kth block of the node representation at the lth layer, A is the adjacency matrix of the graph, and ∥ represents concatenation. Different latent factors of the input graph can be captured by using these decoupled GNN layers. Then, a readout layer can be adopted to aggregate the node-level representation into a graph-level representation:

[0034] h=Readout(H (L) ). (4)

[0035] To learn the parameters of the self-supervised decoupled graph encoder module 210, both graph supervised learning tasks and self-supervised learning tasks may be used simultaneously.

[0036] The downstream target graph task naturally provides a supervisory signal for learning the graph encoder module 210. Therefore, a classification layer can be placed after the obtained graph representation to obtain predictions for the graph classification task. i The graph representation of can be expressed as h i The supervised learning loss is as follows:

[0037]

[0038] in, is the classification layer.

[0039] Graph self-supervised learning (SSL) aims to learn information graph representations through pretext tasks, which have shown several advantages, including reduced label dependency, enhanced robustness, and model generalization capabilities. Therefore, graph SSL can also be used to supplement supervised learning tasks. Specifically, SSL auxiliary tasks can be set by generating pseudo-labels from graph structures, and pseudo-labels can be used as additional supervisory signals. In addition, different pseudo-labels can be used for different chunks of the decoupled GNN, so that the decoupled graph encoder 210 can capture different factors of the graph structure. In an embodiment, the graph encoder 210 can focus on the degree distribution of the graph as a representation and interpretable structural feature, while generalization to other graph structures is simple and straightforward. Specifically, for the kth GNN block, a pseudo-label can be generated by calculating the ratio of nodes that have exactly degree k. Then, the SSL objective function can be formulated as:

[0040]

[0041] in, is a pseudo-label, and h can be represented by i A regression function (such as a linear layer followed by an activation function) is used on the k-th block of . In an embodiment, the last block can be kept without the SSL task to allow more flexible learning of disentangled graph representations.

[0042] like Figure 2 As shown, after obtaining the graph representations h1, h2, and h3, they can be input into the architecture customization module 220, which maps the representations into different customized GNN architectures with prototype strategies. Specifically, the probability of selecting operation o in the i-th layer of the searched architecture can be expressed as where i∈{1,2,…,N}, N is the number of layers, and The probability can be calculated as follows:

[0043]

[0044] in, is a learnable prototype vector representation of operation o. L2 normalization of q can be adopted to ensure numerical stability and fair competition between different operations. In the architecture customization module 220, a prototype vector can be learned for each candidate operation, and the operation can be selected based on the preference of the graph, that is, if the graph representation has a large projection on the prototype vector, its corresponding operation is more likely to be selected. In addition, by using an exponential function, the length of h can be determined. The shape of , that is, the larger the ∥h∥2, The more likely it is to be dominated by a minority of values, the more likely it is that the graph needs specific manipulation.

[0045] In one embodiment, to avoid the mode collapse problem, i.e., vectors of different operations are similar and thus become indistinguishable, a regularization term based on the cosine distance between vectors may be employed as follows to maintain the diversity of operations:

[0046]

[0047] The architecture customization with prototypes in module 220 can customize the most suitable GNN architecture for different input graphs based on the graph representation. In addition to the GNN architecture, the weights of the architecture also need to be learned.

[0048] like Figure 2 As shown, a hypernetwork module 230 may be used to obtain the weight of the architecture. Specifically, in the hypernetwork, all possible operations are jointly considered by mixing different operations into a continuous space as follows:

[0049]

[0050] where x is the input to the i-th layer, and f i (x) is the output. Then, the gradient descent method can be used to optimize all weights. In addition, since the weights of different architectures are shared, the training will be more efficient than training the weights for different architectures separately.

[0051] It should be noted that in most NAS models, by selecting for all layers the The architecture is discretized at the end of the search phase by using the operation of . Then, the weights of the selected architecture are retrained. However, retraining is not feasible in the disclosed novel GNAS model because the test graph can be customized with an architecture different from the one used for training the graph. Therefore, the weights of the hypernetwork can be directly used as the weights in the searched architecture. In addition, the continuous architecture is kept without the discretization step, thereby enhancing the flexibility of architecture customization and simplifying the optimization strategy. In addition, the customized hypernetwork can be used as a strong ensemble model, where are the integration weights, which can also benefit out-of-distribution generalization.

[0052] like Figure 2 As shown, the GNAS model 200 can be optimized by using a gradient descent method based on the following loss function:

[0053]

[0054] in, is the supervision loss of the custom architecture in formula (2), that is, the supervision loss of the final prediction given by the hypernetwork module 230, and are the supervision loss and self-supervision loss of the self-supervised decoupled graph encoder module 210, is the cosine distance loss of the architecture customization module 230, and β1 and β2 are hyperparameters. and are three additional loss functions introduced as regularization terms in Equation (2), and γ is a hyperparameter to control the contribution of the regularization term.

[0055] For the overall optimization, there are two sets of loss functions: classification loss and regularization term. In the early stages of the training process, the self-supervised decoupled graph encoder may not have been trained correctly, and the learned graph representation is also not informative, resulting in unstable architecture customization. Therefore, initially a larger weight can be set for the regularization term, i.e., a smaller initial γ in formula (2) to force the self-supervised decoupled graph encoder to learn through its supervised learning and SSL tasks. As the training process continues, by increasing γ as follows, we can gradually focus more on training the architecture customization module and the super network module:

[0056] γ t =γ0+tΔγ t , (11)

[0057] Among them, γ t is the hyperparameter value for the tth epoch, and Δγ is a small constant.

[0058] The overall training process is as follows. The most suitable GNN architecture and its parameters can be directly generated for the test graph without retraining.

[0059] Input: training dataset G tr and Y tr , hyperparameters γ0, Δγ t , β1, β2

[0060] Initialize all learnable parameters and set γ = γ0

[0061] while does not converge do

[0062] Using formula (3) and formula (4), we can calculate the graph representation h

[0063] Use formula (5) and formula (6) to calculate and

[0064] Use formula (7) to calculate the architecture probability

[0065] Use formula (8) to calculate

[0066] Based on formula (9), the parameters are obtained from the hypernetwork

[0067] Calculate the total loss in formula (2)

[0068] Use gradient descent to update the parameters

[0069] Update γ = γ - Δγ

[0070] end while

[0071] In an embodiment, different learning rates may be used for the three modules 210-230. For example, the learning rate of the self-supervised decoupled encoder module may be 1.5e -4 The learning rate of the architecture customization module can be 1e -4 The training process of these two modules is a sinusoidal annealing schedule. The learning rate of the customized hypernetwork module can be 2e -3 γ can be initialized to 0.07 and linearly increased to 0.5. In addition, β1 can be set to 0.05, and β2 can be set to 0.002. The number of layers can be set to 2 or 3. The disclosed GNAS method is not limited to such settings.

[0072] Figure 3 A flow chart of a method 300 for designing a GNN by GNAS under distribution shift according to one aspect of the present disclosure is shown. In an embodiment, the method 300 may be used to design a graph neural network for a graph classification task by graph neural architecture search under distribution shift between training and test graphs. The method 300 may also be used for other graph machine learning tasks. In an embodiment, the method 300 may be a computer-implemented method.

[0073] In box 310, method 300 can obtain a graph representation of the input graph in the decoupled latent space through a graph encoder module. The input graph can be an assembly line layout diagram. The input graph can also be a circuit design layout diagram, such as a printed circuit board design or a chip design. The GNN designed by method 300 can be used to classify the input assembly line layout diagram or the circuit design layout diagram under distribution shift. For example, the GNN can classify the input assembly line layout diagram or the circuit design layout diagram as whether the assembly line layout is reasonable or whether the circuit is valid, etc. The graph encoder module can be a self-supervised decoupled graph encoder, which can characterize the invariant factors hidden in different graph structures. The graph encoder module can use formulas (3) and (4) to calculate the graph representation of the input graph. The graph encoder module can be trained by supervised learning tasks and self-supervised learning tasks. The graph encoder module can use formulas (5) and (6) to calculate and

[0074] In box 320, method 300 can obtain a GNN architecture for searching the input graph based on the probability of the operation selected from the candidate operation set in the layer of the searched GNN architecture through the architecture customization module. The probability of the operation is a function of the similarity between the obtained graph representation and the learnable prototype vector representation of the operation. In an embodiment, for each layer of the searched GNN architecture, the probability of the operation can be calculated using formula (7). The candidate operation set can include at least one of a graph convolutional network (GCN), a graph attention network (GAT), a graph isomorphism network (GIN), and a graph sample and aggregation (SAGE). The candidate operation set can include other GNN layers, such as a graph convolution (GraphConv). Depending on different graph tasks, base graph shapes and / or data sets, an MLP that does not consider the graph structure can be used, the pooling layer can be fixed at the end of the searched GNN architecture as a global average pooling, or the GIN can be fixed at the first layer for the spurious-Motif data set.

[0075] In box 330, method 300 can obtain the weights of the searched GNN architecture through a hypernetwork module, where different operations in the candidate operation set are mixed into a continuous space, such as using formula (9). In an embodiment, the weights of the hypernetwork can be directly used as weights in the searched GNN architecture. The weights of the hypernetwork can be shared between different GNN architectures, and therefore training will be more efficient than training weights for different architectures separately. Then, in the testing phase, the trained weights can be directly obtained by the hypernetwork for the searched GNN architecture.

[0076] Despite Figure 3 Not shown, but during the training phase, method 300 may include optimizing the GNN based on a main loss function and a regularization term, which may include repeating blocks 310-330 until convergence. The main loss function may be a supervised loss of the searched GNN architecture such as in formula (2), and the regularization term may be based on a supervised learning loss function such as in formula (5) and a self-supervised learning objective function for the graph encoder such as in formula (6) and a cosine distance loss function between learnable prototype vector representations of different operations such as in formula (8). In addition, since the self-supervised decoupled graph encoder may not have been properly trained in the early stages of the training process and the learned graph representation is not informative, a small initial weight may be set for the main loss function, and the weight may be gradually increased during the training process.

[0077] Figure 4 4 is a block diagram of an apparatus 400 for designing a GNN using GNAS under distribution shift according to one aspect of the present disclosure. Figure 4As shown, the apparatus 400 may include a graph encoder module 410 , an architecture customization module 420 , and a hypernetwork module 430 .

[0078] The graph encoder module 410 can be used to obtain a graph representation of the input graph in the decoupled latent space. The architecture customization module 420 can be used to obtain the searched GNN architecture of the input graph based on the probability of the operation selected from the candidate operation set in the layer of the searched GNN architecture. The probability of the operation is a function of the similarity between the obtained graph representation and the learnable prototype vector representation of the operation, such as in formula (7). The super network module 430 can be used to obtain the weights of the searched GNN architecture. Different operations in the candidate operation set are mixed into a continuous space in the super network module 430. In the training phase, the operations performed by the graph encoder module 410, the architecture customization module 420 and the super network module 430 can be repeated to optimize the GNN based on the main loss function and the regularization term using formula (2).

[0079] The apparatus 400 can also be used for GNNs designed by graph neural architecture search under distribution shift between training graphs and test graphs. For example, in the test phase, the graph encoder module 410 can obtain a graph representation of the input graph using parameters learned for a specific graph task. The architecture customization module 420 can obtain different GNN architectures for different input graphs based on the probability or weight of the operations in each layer. The super network module 430 can obtain the weights learned in the training phase and share these weights for different GNN architectures.

[0080] Figure 5 1 shows a block diagram of an apparatus 500 for designing a GNN by GNAS under distribution shift according to one aspect of the present disclosure. The apparatus 500 may include a memory 510 and at least one processor 520. The processor 520 may be coupled to the memory 510 and configured to execute the above reference Figure 3 The method 300 described. The processor 520 can be a general-purpose processor, or can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The memory 510 can store input data, output data, data generated by the processor 520, and / or instructions executed by the processor 520. According to the present disclosure, the apparatus 500 can also be used to search for a GNN designed by a graph neural architecture under a distribution shift between a training graph and a test graph.

[0081] The various operations, modules, and networks described in conjunction with the present disclosure may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. According to an embodiment of the present disclosure, a computer program product for computer vision processing may include a computer program for executing the above reference. Figure 3According to another embodiment of the present disclosure, a computer readable medium may store computer code for computer vision processing, and when the computer code is executed by the processor, the processor may execute the method 300 described above. Figure 3 The method 300 described herein. Computer-readable media includes non-transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Any connection may be properly referred to as a computer-readable medium. Other embodiments and implementations are within the scope of the present disclosure.

[0082] The foregoing description of the disclosed embodiments is provided to enable those skilled in the art to make or use the various embodiments. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the various embodiments. Therefore, the claims are not intended to be limited to the embodiments shown herein, but should be given the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A method for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between a training graph and a test graph, comprising: Obtain the graph representation of the input graph in the decoupled latent space through the graph encoder module; Obtaining, by an architecture customization module, the searched GNN architecture for the input graph based on a probability of an operation selected from a set of candidate operations in a layer of the searched GNN architecture, wherein the probability of the operation is a function of a similarity between the obtained graph representation and a learnable prototype vector representation of the operation; as well as The weights of the searched GNN architecture are obtained through a hypernetwork module, where different operations in the candidate operation set are mixed into a continuous space.

2. The method according to claim 1, wherein: The input graph is an assembly line layout graph or a circuit design layout graph, and the designed GNN is used to classify the assembly line layout graph or the circuit design layout graph under distribution shift.

3. The method according to claim 1, wherein: The graph encoder module is trained through supervised learning tasks and self-supervised learning tasks.

4. The method according to claim 3, wherein: The GNN is optimized based on a main loss function and a regularization term, wherein the main loss function is the supervision loss of the searched GNN architecture, and the regularization term is based on the supervised learning loss function and the self-supervised learning objective function of the graph encoder and the cosine distance loss function between the learnable prototype vector representations of the different operations.

5. The method according to claim 4, wherein: The weight of the main loss function is gradually increased during the training process.

6. The method according to claim 1, wherein: The candidate operation set includes at least one of a graph convolutional network (GCN), a graph attention network (GAT), a graph isomorphism network (GIN), and a graph sample and aggregation (SAGE).

7. The method according to claim 1, wherein: The pooling layer is fixed at the end of the searched GNN architecture as global average pooling.

8. An apparatus for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between a training graph and a test graph, comprising: A graph encoder module to obtain a graph representation of the input graph in a disentangled latent space; an architecture customization module for obtaining the searched GNN architecture for the input graph based on a probability of an operation selected from a set of candidate operations in a layer of the searched GNN architecture, wherein the probability of the operation is a function of a similarity between the obtained graph representation and a learnable prototype vector representation of the operation; as well as A hypernetwork module is used to obtain the weights of the searched GNN architecture, wherein different operations in the candidate operation set are mixed into a continuous space in the hypernetwork module.

9. The device according to claim 8, wherein: The input graph is an assembly line layout graph or a circuit design layout graph, and the GNN is used to classify the assembly line layout graph or the circuit design layout graph under distribution shift.

10. The device according to claim 8, wherein: The graph encoder module is trained through supervised learning tasks and self-supervised learning tasks.

11. The device according to claim 10, wherein: The GNN is optimized based on a main loss function and a regularization term, wherein the main loss function is the supervision loss of the searched GNN architecture, and the regularization term is based on the supervised learning loss function and the self-supervised learning objective function of the graph encoder and the cosine distance loss function between the learnable prototype vector representations of the different operations.

12. The device according to claim 11, wherein The weight of the main loss function is gradually increased during the training process.

13. The device according to claim 8, wherein: The candidate operation set includes at least one of a graph convolutional network (GCN), a graph attention network (GAT), a graph isomorphism network (GIN), and a graph sample and aggregation (SAGE).

14. The device according to claim 8, wherein: The pooling layer is fixed at the end of the searched GNN architecture as global average pooling.

15. An apparatus for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between a training graph and a test graph, comprising: Memory; as well as At least one processor coupled to the memory and configured to perform the method of one of claims 1-7.

16. A computer-readable medium storing computer code for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between training and test graphs, the computer code, when executed by a processor, causes the processor to perform a method as described in one of claims 1-7.

17. A computer program product for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between training and test graphs, comprising: Processor executable computer code for performing the method of one of claims 1-7.

18. A computer-implemented method for designing a graph neural network (GNN) by graph neural architecture search under a distribution shift between training and test graphs, comprising the steps of the method described in one of claims 1-7.

Images