A wasserstein centroid matching layer method and product for message passing neural networks

Abstract

The application provides a message passing neural network-oriented Wasserstein centroid matching layer method and product, relates to the technical field of neural networks, and comprises the following steps: obtaining an original graph signal; inputting the original graph signal into the message passing neural backbone network to obtain a filtered graph signal; inputting the filtered graph signal into a Wasserstein centroid matching layer network to obtain the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtaining a graph representation vector, wherein each class centroid is used for representing the centroid of the vector representation of each graph category in the Wasserstein space; and inputting the graph representation vector into a classifier network to obtain the predicted category corresponding to the original graph signal. The application obtains the graph representation vector in the Wasserstein space by setting the Wasserstein centroid matching layer network, so that the message passing neural network has a controllable convergence rate independent of the dimension of the graph signal generation space, overcomes the curse of dimensionality of the graph signal generation space, and significantly improves the size generalization ability of the message passing neural network.

Description

[0001] Cross Reference to Related Applications

[0002] The present disclosure claims priority to the Chinese patent application No. 2023109191840, filed on July 25, 2023, entitled “A Wasserstein Centroid Matching Layer Method for Message Passing Neural Network and Product”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] Embodiments of the present application relate to the technical field of neural network, in particular, to a Wasserstein centroid matching layer method for message passing neural network and product. BACKGROUND

[0004] Message passing neural network is a representative method of graph classification. However, the existing message passing neural network has poor size generalization ability, that is, the graph classifier trained on the graph training set with a small average number of graph nodes performs poorly on the graph test set with a large average number of graph nodes. The main reason why the existing message passing neural network has poor size generalization ability is that it has a slow learning convergence rate related to the dimension of the generated graph signal space that cannot be controlled relative to the number of graph nodes.

[0005] In order to improve the size generalization ability of the message passing neural network, the existing scheme usually makes strong assumptions on the generation process of the graph, and focuses on various and diverse graph distribution shifts to improve the size generalization ability of the message passing neural network. However, the strong assumptions make the applicability of the graph in the real world decline, and the performance on the real world graph is often observed to be relatively consistent with the performance on the synthetic graph under the assumption. Therefore, how to effectively improve the size generalization ability of the message passing neural network has become a problem to be solved by the current technical personnel in the field. SUMMARY

[0006] Embodiments of the present application provide a Wasserstein centroid matching layer method for message passing neural network and product, aiming to solve the problem of how to effectively improve the size generalization ability of the message passing neural network.

[0007] The first aspect of the embodiments of the present application provides a Wasserstein centroid matching layer method for message passing neural network, applied to a message passing neural network, the message passing neural network comprising a message passing neural backbone network, a Wasserstein centroid matching layer network and a classifier network, and the method comprising:

[0008] obtaining an original graph signal;

[0009] Inputting the original graph signal into the message passing neural backbone network to obtain a filtered graph signal;

[0010] Inputting the filtered graph signal into the Wasserstein centroid matching layer network, obtaining the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtaining a graph representation vector, wherein each class centroid is used to represent the centroid of the embedding of each graph category in the Wasserstein space;

[0011] The graph representation vector is input into the classifier network to obtain the predicted category corresponding to the original graph signal.

[0012] In an optional embodiment, the multiple class centroids are obtained as follows:

[0013] Obtaining a training data set, wherein the training data set includes a plurality of training image subsets of different categories;

[0014] Based on each training graph subset, determining an embedding of the training graph data in each training graph subset in the Wasserstein space;

[0015] Based on the embedding corresponding to each training graph subset, a class centroid of the category corresponding to the training graph subset is determined, where the class centroid is the average value of the embedding in the Wasserstein space.

[0016] In an optional embodiment, the filtered graph signal is input into the Wasserstein centroid matching layer network, and the Wasserstein distance between the filtered graph signal and a plurality of class centroids is obtained to obtain a graph representation vector, including:

[0017] Inputting the filtered graph signal into the Wasserstein centroid matching layer network to obtain the embedding of the original graph signal in the Wasserstein space as the target embedding;

[0018] Calculate the Wasserstein distance between the target embedding and each class centroid in the Wasserstein space;

[0019] All Wasserstein distances in the Wasserstein space are combined into the graph representation vector.

[0020] In an optional embodiment, the training process of the message passing neural network is as follows:

[0021] Obtaining a training data set, wherein the training data set includes a plurality of training image subsets of different categories;

[0022] During each training, obtaining a class centroid prediction value based on each of the training graph subsets in the training data set;

[0023] Inputting the training graph data in the training data set into the initial message passing neural network to obtain a category prediction value;

[0024] Calculating the cross entropy loss between the category prediction value and the category true value of the training graph data;

[0025] Calculating the matching loss between the class centroid prediction value and the class centroid true value of the training image subset;

[0026] Optimizing learnable parameters of the initial message passing neural network based on the cross entropy loss and the matching loss;

[0027] The initial message passing neural network is trained based on the training data set to obtain the message passing neural network.

[0028] In an optional embodiment, the message passing neural network has a convergence error upper bound, and the inequality corresponding to the convergence error upper bound is as follows:

[0029]

[0030]

[0031]

[0032] in, Characterize the convergence error of the message passing neural network, For the message passing neural network, Message passing neural network for graph primitives; Characterize the upper bound of the convergence error of the message passing neural network, η is the confidence hyperparameter, η∈(0,1), N is the number of nodes in the original graph signal, C is the number of training graph subsets in the training dataset, n j Represents the number of training graph data in the j-th training graph subset in the training dataset, j = 1,…, C; C1, C2, C2 ′ ,K ′ , C3 is a constant related to the probability concentration inequality and the smoothness of the message passing neural network; F L is the vector dimension of the Wasserstein centroid matching layer network vectorization; O(·) represents the complexity of the calculation;

[0033] The probability that the inequality corresponding to the upper bound of the convergence error is true is:

[0034]

[0035] Wherein, P1 is the probability that the inequality corresponding to the upper bound of the convergence error holds true, and e is the natural base.

[0036] In an optional embodiment, the message passing neural network has a generalization error upper bound, and the inequality corresponding to the generalization error upper bound is as follows:

[0037]

[0038] in, Characterizing a generalization error of the message passing neural network; is the empirical risk, R exp is the generalization risk; l is the loss function of the message passing neural network; n represents the number of the training graph data in the training dataset;

[0039] The probability that the inequality corresponding to the generalization error upper bound is true is:

[0040]

[0041] Wherein, P2 is the probability that the inequality corresponding to the upper bound of the generalization error holds.

[0042] A second aspect of an embodiment of the present application provides a Wasserstein centroid matching layer device for a message passing neural network, which is applied to a message passing neural network. The message passing neural network includes a message passing neural backbone network, a Wasserstein centroid matching layer network, and a classifier network. The device includes:

[0043] An acquisition module, used to acquire the original image signal;

[0044] a preprocessing module, configured to input the original graph signal into the message passing neural backbone network to obtain a filtered graph signal;

[0045] a centroid matching module, configured to input the filtered graph signal into the Wasserstein centroid matching layer network, obtain the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtain a graph representation vector, wherein each class centroid is used to represent the centroid of the embedding of each graph category in the Wasserstein space;

[0046] A classification module is used to input the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal.

[0047] In an optional implementation, the centroid matching module includes:

[0048] The training set acquisition submodule is configured to acquire a training data set, and the training data set includes multiple training image subsets of different categories.

[0049] The embedding submodule is configured to determine, based on each training image subset, an embedding of training image data in the training image subset in the Wasserstein space.

[0050] The centroid module is configured to determine, based on the embedding corresponding to each training image subset, a centroid of a category corresponding to the training image subset, and the centroid is an average value of the embedding in the Wasserstein space.

[0051] In an optional implementation, the centroid matching module further includes:

[0052] The target embedding submodule is configured to input the filtered image signal into the Wasserstein centroid matching layer network to acquire an embedding of the original image signal in the Wasserstein space as a target embedding.

[0053] The Wasserstein distance submodule is configured to calculate a Wasserstein distance between the target embedding and each centroid in the Wasserstein space.

[0054] The graph representation vector submodule is configured to combine all Wasserstein distances in the Wasserstein space into the graph representation vector.

[0055] In an optional implementation, the apparatus further includes:

[0056] The training set acquisition module is configured to acquire a training data set, and the training data set includes multiple training image subsets of different categories.

[0057] The centroid prediction module is configured to acquire, based on each training image subset in the training data set, a centroid prediction value at each time of training.

[0058] The category prediction module is configured to input training image data in the training data set into an initial message passing neural network to acquire a category prediction value.

[0059] The first loss module is configured to calculate a cross-entropy loss between the category prediction value and a category true value of the training image data.

[0060] The second loss module is configured to calculate a matching loss between the centroid prediction value and a centroid true value of the training image subset.

[0061] an optimization module, configured to optimize the learnable parameters of the initial message passing neural network based on the cross entropy loss and the matching loss;

[0062] A training module is used to train the initial message passing neural network based on the training data set to obtain the message passing neural network.

[0063] In an optional embodiment, the device further includes:

[0064] The convergence error upper bound module is used to determine the convergence error upper bound of the message passing neural network. The inequality corresponding to the convergence error upper bound is as follows:

[0065]

[0066]

[0067]

[0068] in, Characterize the convergence error of the message passing neural network, For the message passing neural network, Message passing neural network for graph primitives; Characterize the upper bound of the convergence error of the message passing neural network, η is the confidence hyperparameter, η∈(0,1), N is the number of nodes in the original graph signal, C is the number of training graph subsets in the training dataset, n j Represents the number of training graph data in the j-th training graph subset in the training dataset, j = 1,…, C; C1, C2, C2 ′ ,K ′ , C3 is a constant related to the probability concentration inequality and the smoothness of the message passing neural network; F L is the vector dimension of the Wasserstein centroid matching layer network vectorization; O(·) represents the complexity of the calculation;

[0069] The probability that the inequality corresponding to the upper bound of the convergence error is true is:

[0070]

[0071] Wherein, P1 is the probability that the inequality corresponding to the upper bound of the convergence error holds true, and e is the natural base.

[0072] In an optional embodiment, the device further includes:

[0073] A generalization error upper bound module is used to determine the generalization error upper bound of the message passing neural network. The message passing neural network has a generalization error upper bound. The inequality corresponding to the generalization error upper bound is as follows:

[0074]

[0075] in, Characterizing a generalization error of the message passing neural network; is the empirical risk, R exp is the generalization risk; l is the loss function of the message passing neural network; n represents the number of the training graph data in the training dataset;

[0076] The probability that the inequality corresponding to the generalization error upper bound is true is:

[0077]

[0078] Wherein, P2 is the probability that the inequality corresponding to the upper bound of the generalization error holds.

[0079] A third aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps in the Wasserstein centroid matching layer method for message passing neural networks described in any one of the first aspects.

[0080] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the Wasserstein centroid matching layer method for a message passing neural network as described in any one of the first aspects.

[0081] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the Wasserstein centroid matching layer method for a message passing neural network described in any one of the first aspects.

[0082] Beneficial effects:

[0083] The present application provides a Wasserstein centroid matching layer method and product for a message passing neural network, which is applied to a message passing neural network, wherein the message passing neural network includes a message passing neural backbone network, a Wasserstein centroid matching layer network, and a classifier network. The method includes: obtaining an original graph signal; inputting the original graph signal into the message passing neural backbone network to obtain a filtered graph signal; inputting the filtered graph signal into the Wasserstein centroid matching layer network, obtaining the Wasserstein distance between the filtered graph signal and multiple class centroids, and obtaining a graph representation vector, wherein each class centroid is used to represent the embedded centroid of each graph category in the Wasserstein space; inputting the graph representation vector into the classifier network to obtain the predicted category corresponding to the original graph signal. The embodiment of the present application obtains a graph representation vector in the Wasserstein space by setting a Wasserstein centroid matching layer network, so that the message passing neural network has a controllable convergence rate that is independent of the dimension of the graph signal generation space, overcomes the dimensionality curse of the graph signal generation space, and thus significantly improves the size generalization ability of the message passing neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0085] Figure 1 This is a flow chart of a Wasserstein centroid matching layer method for a message passing neural network proposed in one embodiment of the present application;

[0086] Figure 2 This is a schematic diagram of the network architecture of a Wasserstein centroid matching layer for a message passing neural network proposed in one embodiment of the present application;

[0087] Figure 3 Schematic diagram of a Wasserstein centroid matching layer device for a message passing neural network proposed in one embodiment of the present application;

[0088] Figure 4 FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0089] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0090] Message passing neural networks are a representative method for graph classification. However, existing message passing neural networks suffer from poor scalability. This means that a graph classifier trained on a training set of graphs with a small average number of nodes performs poorly on a test set of graphs with a large average number of nodes. This poor scalability is primarily due to their slow and uncontrollable learning convergence rate, which is related to the spatial dimensionality of the graph signal generation space, relative to the number of graph nodes.

[0091] In related technologies, to improve the size generalization ability of message passing neural networks, one approach is to minimize the difference in local structure between small and large graphs. Their framework is based on the assumption that the test graph or domain label is accessible during training, which is often not true in practice. Another approach is to assume a complex causal model to describe the generation process of graphs of different sizes and design a size-invariant learning model based on it. However, the performance of size-invariant models degrades from synthetic graphs to real-world graphs. In addition, there is also the approach of simulating size changes through graph coarsening and penalizing changes in node embedding distributions. However, the underlying invariance assumptions and application scope of this approach are unclear. Therefore, existing solutions usually make strong assumptions about the generation process of graphs and focus on various diverse graph distribution shifts to improve the size generalization ability of message passing neural networks. However, these strong assumptions reduce the applicability to real-world graphs. At the same time, it is often observed that the performance on synthetic graphs, which is relatively consistent with the assumptions, degrades dramatically on real-world graphs.

[0092] In view of this, the embodiment of the present application proposes a Wasserstein centroid matching layer method for a message passing neural network, which is applied to a message passing neural network. The message passing neural network includes a Wasserstein centroid matching layer network and a classifier network. Figure 1 FIG. 1 shows a flow chart of a Wasserstein centroid matching layer method for a message passing neural network proposed in an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0093] S101: Obtain original image signal.

[0094] S102: Input the original graph signal into the message passing neural backbone network to obtain a filtered graph signal.

[0095] S103: Input the filtered graph signal into the Wasserstein centroid matching layer network, obtain the Wasserstein distance between the filtered graph signal and multiple class centroids, and obtain a graph representation vector.

[0096] The original graph signal is input into the Wasserstein centroid matching layer network, and the distance between the original graph signal and multiple class centroids is obtained to obtain a graph representation vector, wherein each class centroid is used to represent the centroid of the vector representation of each graph category in the Wasserstein space.

[0097] S104: Input the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal.

[0098] In order to enable those skilled in the art to better understand the Wasserstein centroid matching layer method for message passing neural networks provided in this application, Figure 2 A schematic diagram of the network architecture of a Wasserstein centroid matching layer for a message passing neural network proposed in one embodiment of the present application is shown, combined with Figure 2 , the following is a detailed introduction to the program:

[0099] When specifically implementing step S101, first obtain the original graph signal to be classified, the original graph signal is a graph signal matrix corresponding to the original graph data, wherein the graph signal matrix corresponding to the original graph signal includes the graph node information in the original graph signal and the edge information between the graph nodes, the graph node information is the attribute vector of the graph node in the original graph signal, and the edge information between the graph nodes is used to characterize the connection between different graph nodes. The original graph signal can be a large-sized original graph signal or a small-sized original graph signal. The large-sized original graph signal contains more graph node information and / or edge information than the small-sized original graph signal. Exemplarily, the original graph signal can be a consumer network graph composed of shopping data, a social network graph composed of social data, a biochemical molecular structure graph existing in nature, etc. It should be noted that the above examples are only some optional situations given to enable those skilled in the art to better understand the scheme of this application. The specific type of original graph signal can be determined according to actual conditions, and this application does not make specific restrictions here.

[0100] For large-size original graph signals, since they contain more graph node information and / or edge information, the cost of annotation and training is high. The training cost of directly optimizing the message passing network through large-size training graph data is high. Therefore, the message passing neural network used in this application is trained based on small-size training graph data with less average graph node information and / or edge information, while ensuring that the message passing network has good generalization ability on original graph signals with more average graph node information and / or edge information.

[0101] When implementing step S102, after obtaining the original image signal, it is necessary to preprocess the image signal matrix corresponding to the original image signal to remove redundant information in the original image signal that is irrelevant to image classification, thereby obtaining the preprocessed original image signal as the filtered image signal. Specifically, the original image signal is input into the message passing neural backbone network, and the image signal matrix corresponding to the original image signal is subjected to image filtering processing to obtain the image signal matrix corresponding to the filtered original image signal. By preprocessing the original image signal, the accuracy of image classification performed by the message passing network can be effectively improved.

[0102] Specifically, when step S103 is implemented, while obtaining the filtered graph signal, multiple class centroids are obtained based on the Wasserstein centroid matching layer network. In the embodiment of the present application, the Wasserstein centroid matching layer network is part of a preset message passing neural network, and the backbone network in the preset message passing neural network is connected to a network layer with learnable parameters to obtain the Wasserstein centroid matching layer network. The preset message passing neural network can be a common message passing neural network, such as a graph convolutional network (GCN), a principal neighborhood aggregation network (PNA), etc. After constructing the Wasserstein centroid matching layer network based on the backbone network in the preset message passing neural network, it is necessary to obtain multiple class centroids of different categories. The class centroids are the signal centroids of the embedded graph data of different categories under the measurement of the Wasserstein space. The centroids represent the signal centers of the graph data of different categories. The class centroids can be used as a consensus of graph granularity to control and accelerate the convergence speed of the graph classification network during the learning process, so that when the Wasserstein centroid matching layer for the message passing neural network is trained based on small-size training graph data, it overcomes the dimensionality curse of the graph signal generation space and removes the dimensionality limitation of the training process, thereby effectively improving the convergence rate of the message passing neural network, improving the performance of the trained message passing neural network on large-size graph data, and improving the size generalization ability of the message passing neural network.

[0103] In an optional embodiment, the multiple class centroids are obtained as follows: first, a training dataset is obtained, wherein the training dataset contains multiple training graph subsets of different categories, each training graph subset corresponding to a specific graph category, and each training graph subset contains multiple training graph data belonging to the graph category corresponding to the training graph subset; then, based on the multiple training graph data belonging to the same category in each training graph subset, a vector representation of the embedding points of the graph node information in each training graph data in the same training graph subset in Wasserstein space is obtained as the embedding corresponding to each graph node information; based on these embeddings, the average value of the embedding points (embeddings) of the graph node information (graph signal matrix) of the training graph data corresponding to the same category in the same training graph subset in Wasserstein space is calculated as the class centroid corresponding to the category, wherein the class centroid is the graph embedding in Wasserstein space that is closest to the embedding points corresponding to the training graph data of the same category under the Wasserstein metric. For a given centroid node number hyperparameter, the above process is repeated until the class centroids corresponding to all categories in the training dataset are obtained, and the centroid signal matrix is ​​constructed.

[0104] After determining the class centroids (centroid signal matrices) corresponding to multiple categories, the filtered graph signal is input into the Wasserstein centroid matching layer network. At this time, the centroid signal matrices of the class centroids corresponding to multiple categories have been constructed in the Wasserstein centroid matching layer network. The original graph signal is vectorized through the multiple class centroids in the Wasserstein centroid matching layer network to obtain a graph representation vector for the Wasserstein space. Specifically, the filtered graph signal is first input into the Wasserstein centroid matching layer network to obtain the embedding point of the original graph signal in the Wasserstein space as the target embedding, such as Figure 2 As shown, the signal matrix of all graph nodes in the original graph signal is mapped by the message passing neural backbone network and the Wasserstein centroid matching layer network to form an empirical distribution. The empirical distribution is the target embedding embedded in the Wasserstein space. The target embedding is expressed as follows:

[0105]

[0106] in, is the target embedding, Θ A (f) is the message passing neural network, and N is the number of nodes in the original graph signal; is the Dirac (unit impulse) function, and X is observed i1 if the function of time, otherwise 0, X i is the i-th graph node.

[0107] Subsequently, in the Wasserstein space, the Wasserstein distance between the target embedding and each class centroid is calculated, which is used to represent the values of different dimensions of the vectorized original graph signal; finally, all the Wasserstein distances in the Wasserstein space are combined into the graph representation vector wherein the Wasserstein distance is a 2-Wasserstein distance, which is used to represent the wasserstein distance with the second order Euclidean distance as the ground metric. As shown in the formula (2), the 2-Wasserstein distance between the target embedding and each class centroid is represented as follows: Figure 2

[0108]

[0109] wherein W2 is the 2-Wasserstein distance, is the class centroid corresponding to the class C, j = 1, …, C; is the target embedding.

[0110] In the implementation of step S104, the graph representation vector corresponding to the original graph signal is input into a classifier network, which is a neural network of fully connected layers. The classifier network predicts the classification of the image based on the graph representation vector and outputs the predicted class of the original graph signal.

[0111] In an optional embodiment, the message passing neural network is obtained by training and optimizing an initial message passing neural network. The initial message passing neural network includes an initial Wasserstein centroid matching layer network and an initial classifier network. The training process of the message passing neural network is as follows: first, a training data set is obtained, which includes multiple training graph subsets of different classes. Each training graph subset corresponds to a specific graph class, and each training graph subset includes multiple training graph data belonging to the graph class corresponding to the training graph subset. Each training graph subset is also configured with a corresponding class centroid true value, and each training graph data is configured with a corresponding class true value.

[0112] ​During each training, each of the training graph subsets in the training data set is input into the initial Wasserstein centroid matching layer network to obtain the class centroid prediction value corresponding to each training graph subset. The specific process of obtaining the class centroid prediction value can be found in the content of the above step S102, which will not be repeated here. After obtaining the class centroid prediction value, the matching loss is obtained based on the class centroid prediction value and the class centroid true value corresponding to the training graph subset. The matching loss is used to optimize the learnable parameters of the initial Wasserstein centroid matching layer network so that the initial Wasserstein centroid matching layer network can effectively learn the signal centroid of the graph signal matrix in the Wasserstein space of different categories of graph training subsets, so that the optimized Wasserstein centroid matching layer network can obtain multiple different categories of class centroids with higher accuracy, and use the class centroid as the consensus of the graph granularity to control and accelerate the convergence speed of graph classification learning.

[0113] At the same time, the training graph data in the training data set is input into the initial message passing neural network, first input into the initial message passing neural backbone network to obtain the filtered graph signal corresponding to the training graph data, and input the filtered graph signal into the initial Wasserstein centroid matching layer network to obtain the graph representation vector corresponding to the training graph data; then the graph representation vector is input into the initial classifier network to obtain the category prediction value corresponding to the training graph data; the cross entropy loss between the category prediction value and the category true value of the training graph data is calculated, and the cross entropy loss is used to optimize the learnable parameters in the initial message passing neural network except the initial Wasserstein centroid matching layer network (including the learnable parameters in the initial message passing backbone network and the initial classifier network), so that the initial message passing neural network can effectively filter the graph node granularity consensus of the classification gain, thereby improving the accuracy of the classification results of the message passing neural network.

[0114] Finally, the learnable parameters of the initial message passing neural network are optimized based on the cross entropy loss and the matching loss, and the initial message passing neural network is trained based on the training dataset to obtain the message passing neural network. Because the backbone network of the message passing neural network is a Wasserstein centroid matching layer network, the centroid is used as the consensus of the graph granularity during each training process, which enables the convergence rate to be controlled and accelerated during the training optimization process.

[0115] In one optional embodiment, the message passing neural network has an upper bound on the convergence error. For a given message passing neural network skeleton network Θ, with l as the loss function of the message passing neural network, for a set of original graph signals with N nodes, assuming that the graph signal matrix f corresponding to the original graph signals follows a sub-Gaussian distribution. Under the standard graph neural network regularity assumption, the inequality corresponding to the upper bound on the convergence error is as follows:

[0116]

[0117]

[0118]

[0119] in, Characterize the convergence error of the message passing neural network, For the message passing neural network, Message passing neural network for graph primitives; Characterize the upper bound of the convergence error of the message passing neural network, η is the confidence hyperparameter, η∈(0,1), N is the number of nodes in the original graph signal, C is the number of training graph subsets in the training dataset, n j represents the number of training graph data in the j-th training graph subset in the training dataset, j = 1, ..., C; C1, C2, C′2, K′, C3 are constants related to the probability concentration inequality and the smoothness of the message passing neural network; F L is the vector dimension of the Wasserstein centroid matching layer network vectorization; O(·) represents the complexity of the calculation;

[0120] The probability that the inequality corresponding to the upper bound of the convergence error is true is:

[0121]

[0122] Wherein, P1 is the probability that the inequality corresponding to the upper bound of the convergence error holds true, and e is the natural base.

[0123] In an optional embodiment, the message passing neural network has a generalization error upper bound, and the inequality corresponding to the generalization error upper bound is as follows:

[0124]

[0125] in, Characterizing a generalization error of the message passing neural network; is the empirical risk, R expwherein, l is a loss function of the message passing neural network; n represents a number of the training graph data in the training data set;

[0126] A probability that the inequality corresponding to the generalization error upper bound is established is:

[0127]

[0128] wherein, P2 is the probability that the inequality corresponding to the generalization error upper bound is established.

[0129] It can be known from the convergence error upper bound and the generalization error upper bound obtained through the above mathematical reasoning that the Wasserstein centroid matching layer method for the message passing neural network provided in the embodiments of the present application has stronger generalization ability on a single original graph signal and / or a sample set containing graph data of multiple different sizes, thereby greatly improving the graph size generalization ability of the current message passing neural network, so that the message passing neural network based on the embodiments of the present application can be labeled at a very low cost on a small graph to obtain a graph classifier that can be generalized to a large graph.

[0130] In addition, the scheme of the embodiments of the present application does not make strong assumptions on the graph generation process, only assumes a random graph model of graph elements, and the effectiveness of the method can be theoretically proved, which covers the Erdos-Renyi random graph, random block graph, random geometric graph and other common random graph types in practice.

[0131] The present application provides a Wasserstein centroid matching layer method and product for a message passing neural network, applied to a message passing neural network, the message passing neural network comprising a message passing neural backbone network, a Wasserstein centroid matching layer network and a classifier network, the method comprising: obtaining an original graph signal; inputting the original graph signal into the message passing neural backbone network to obtain a filtered graph signal; inputting the filtered graph signal into the Wasserstein centroid matching layer network to obtain a Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtaining a graph representation vector, wherein each class centroid is used to represent a centroid of embedding of each graph category in a Wasserstein space; inputting the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal. The embodiments of the present application obtain a graph representation vector in a Wasserstein space by setting a Wasserstein centroid matching layer network, so that the message passing neural network has a controllable convergence rate independent of the dimension of the graph signal generation space, overcomes the curse of dimensionality of the graph signal generation space, and thereby significantly improves the size generalization ability of the message passing neural network.

[0132] Based on the same inventive concept, the embodiment of the present application discloses a device of a Wasserstein centroid matching layer for a message passing neural network, applied to a message passing neural network, wherein the message passing neural network comprises a Wasserstein centroid matching layer network and a classifier network, Figure 3 An embodiment of the present application shows a device of a Wasserstein centroid matching layer for a message passing neural network, as shown in the figure, Figure 3 The device comprises:

[0133] An acquisition module is configured to acquire an original graph signal;

[0134] A preprocessing module is configured to input the original graph signal into the message passing neural backbone network to obtain a filtered graph signal;

[0135] A centroid matching module is configured to input the filtered graph signal into the Wasserstein centroid matching layer network, acquire a Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtain a graph representation vector, wherein each class centroid is used to represent a centroid of embedding of each graph category in a Wasserstein space;

[0136] A classification module is configured to input the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal.

[0137] In an optional implementation, the centroid matching module comprises:

[0138] A training set acquisition submodule is configured to acquire a training data set, wherein the training data set comprises a plurality of training graph subsets of different categories;

[0139] An embedding submodule is configured to determine, based on each training graph subset, an embedding of training graph data in the each training graph subset in the Wasserstein space;

[0140] A class centroid submodule is configured to determine, based on the embedding corresponding to the each training graph subset, a class centroid of a category corresponding to the training graph subset, wherein the class centroid is an average value of the embedding in the Wasserstein space.

[0141] In an optional implementation, the centroid matching module further comprises:

[0142] A target embedding submodule is configured to input the filtered graph signal into the Wasserstein centroid matching layer network to acquire an embedding of the original graph signal in the Wasserstein space as a target embedding;

[0143] Wasserstein distance submodule, used to calculate the Wasserstein distance between the target embedding and the centroid of each class in the Wasserstein space;

[0144] A graph representation vector submodule is configured to combine all Wasserstein distances in the Wasserstein space into the graph representation vector.

[0145] In an optional embodiment, the device further includes:

[0146] A training set acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of training graph subsets of different categories;

[0147] A class centroid prediction module, configured to obtain a class centroid prediction value based on each of the training graph subsets in the training data set during each training session;

[0148] A category prediction module, configured to input the training graph data in the training data set into an initial message passing neural network to obtain a category prediction value;

[0149] A first loss module, configured to calculate a cross entropy loss between the class prediction value and the class true value of the training graph data;

[0150] A second loss module is used to calculate the matching loss between the class centroid prediction value and the class centroid true value of the training image subset;

[0151] an optimization module, configured to optimize the learnable parameters of the initial message passing neural network based on the cross entropy loss and the matching loss;

[0152] A training module is used to train the initial message passing neural network based on the training data set to obtain the message passing neural network.

[0153] In an optional embodiment, the device further includes:

[0154] The convergence error upper bound module is used to determine the convergence error upper bound of the message passing neural network. The inequality corresponding to the convergence error upper bound is as follows:

[0155]

[0156]

[0157]

[0158] in, Characterize the convergence error of the message passing neural network, For the message passing neural network, Message passing neural network for graph primitives; Characterize the upper bound of the convergence error of the message passing neural network, η is the confidence hyperparameter, η∈(0,1), N is the number of nodes in the original graph signal, C is the number of training graph subsets in the training dataset, n j Represents the number of training graph data in the j-th training graph subset in the training dataset, j = 1,…, C; C1, C2, C2 ′ ,K ′ , C3 is a constant related to the probability concentration inequality and the smoothness of the message passing neural network; F L is the vector dimension of the Wasserstein centroid matching layer network vectorization; O(·) represents the complexity of the calculation;

[0159] The probability that the inequality corresponding to the upper bound of the convergence error is true is:

[0160]

[0161] Wherein, P1 is the probability that the inequality corresponding to the upper bound of the convergence error holds true, and e is the natural base.

[0162] In an optional embodiment, the device further includes:

[0163] A generalization error upper bound module is used to determine the generalization error upper bound of the message passing neural network. The message passing neural network has a generalization error upper bound. The inequality corresponding to the generalization error upper bound is as follows:

[0164]

[0165] in, Characterizing a generalization error of the message passing neural network; is the empirical risk, R exp is the generalization risk; l is the loss function of the message passing neural network; n represents the number of the training graph data in the training dataset;

[0166] The probability that the inequality corresponding to the generalization error upper bound is true is:

[0167]

[0168] Wherein, P2 is the probability that the inequality corresponding to the upper bound of the generalization error holds.

[0169] Based on the same inventive concept, the present application discloses an electronic device. Figure 4 A schematic diagram of an electronic device proposed in an embodiment of the present application is shown. Figure 4As shown, the electronic device 100 includes: a memory 110 and a processor 120, and the memory 110 and the processor 120 are connected to each other through a bus communication, and the memory 110 stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set can be loaded and run on the processor 120 to implement the steps in the Wasserstein centroid matching layer method for message passing neural networks disclosed in the embodiment of the present application.

[0170] Based on the same inventive concept, an embodiment of the present application discloses a computer-readable storage medium on which is stored at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the steps in the Wasserstein centroid matching layer method for message passing neural networks disclosed in an embodiment of the present application.

[0171] Based on the same inventive concept, an embodiment of the present application discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the Wasserstein centroid matching layer method for message passing neural networks disclosed in an embodiment of the present application.

[0172] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0173] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0176] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0177] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements that are inherent to such process, method, article, or terminal device. In the absence of further restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0178] The above is a detailed introduction to the Wasserstein centroid matching layer method and product for message passing neural networks provided by this application. Specific examples are used in this article to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core ideas. At the same time, for general technicians in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A Wasserstein centroid matching layer method for message passing neural networks, characterized in that: Applied to a message passing neural network, the message passing neural network includes a message passing neural backbone network, a Wasserstein centroid matching layer network, and a classifier network, the method includes: Get the original image signal; Inputting the original graph signal into the message passing neural backbone network to obtain a filtered graph signal; Inputting the filtered graph signal into the Wasserstein centroid matching layer network, obtaining the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtaining a graph representation vector, wherein each class centroid is used to represent the centroid of the embedding of each graph category in the Wasserstein space; Inputting the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal; Inputting the filtered graph signal into the Wasserstein centroid matching layer network, obtaining the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtaining a graph representation vector, including: Inputting the filtered graph signal into the Wasserstein centroid matching layer network to obtain the embedding of the original graph signal in the Wasserstein space as the target embedding; Calculate the Wasserstein distance between the target embedding and each class centroid in the Wasserstein space; Combining all Wasserstein distances in the Wasserstein space into the graph representation vector; The target embedding is expressed as follows: in, For the target embedding, For the message passing neural network, is the number of nodes in the original graph signal; is the Dirac function, and we observe The function takes the value 1 when , otherwise it takes the value 0. is the i-th graph node; The 2-Wasserstein distance between the target embedding and the centroid of each class is expressed as follows: in, is the 2-Wasserstein distance, is the centroid of the class corresponding to category C, ; Embed for the target.

2. The Wasserstein centroid matching layer method for message passing neural networks according to claim 1, characterized in that The multiple class centroids are obtained as follows: Obtaining a training data set, wherein the training data set includes a plurality of training image subsets of different categories; Based on each training graph subset, determining an embedding of the training graph data in each training graph subset in the Wasserstein space; Based on the embedding corresponding to each training graph subset, a class centroid of the category corresponding to the training graph subset is determined, where the class centroid is the average value of the embedding in the Wasserstein space.

3. The Wasserstein centroid matching layer method for message passing neural networks according to claim 1, characterized in that: The training process of the message passing neural network is as follows: Obtaining a training data set, wherein the training data set includes a plurality of training image subsets of different categories; During each training, obtaining a class centroid prediction value based on each of the training graph subsets in the training data set; Inputting the training graph data in the training data set into the initial message passing neural network to obtain a category prediction value; Calculating the cross entropy loss between the category prediction value and the category true value of the training graph data; Calculating the matching loss between the class centroid prediction value and the class centroid true value of the training image subset; Optimizing learnable parameters of the initial message passing neural network based on the cross entropy loss and the matching loss; The initial message passing neural network is trained based on the training data set to obtain the message passing neural network.

4. The Wasserstein centroid matching layer method for message passing neural networks according to claim 1, characterized in that The message passing neural network has a convergence error upper bound, and the inequality corresponding to the convergence error upper bound is as follows: ； , ； ； in, Characterize the convergence error of the message passing neural network, For the message passing neural network, Message passing neural network for graph primitives; Characterize the upper bound of the convergence error of the message passing neural network, is the confidence hyperparameter, , is the number of nodes in the original graph signal, is the number of training graph subsets in the training dataset, Indicates the training data set The number of training graph data in the class training graph subset, ; is a constant related to the probability concentration inequality and the smoothness of the message passing neural network; The vector dimension of the Wasserstein centroid matching layer network vectorization; Represents the complexity of calculation; The probability that the inequality corresponding to the upper bound of the convergence error is true is: ； in, is the probability that the inequality corresponding to the upper bound of the convergence error holds true, is the natural base.

5. The Wasserstein centroid matching layer method for message passing neural networks according to claim 4, characterized in that: The message passing neural network has a generalization error upper bound, and the inequality corresponding to the generalization error upper bound is as follows: in, Characterizing a generalization error of the message passing neural network; is the empirical risk, For generalization risk; is the loss function of the message passing neural network; Indicates the number of the training graph data in the training dataset; The probability that the inequality corresponding to the generalization error upper bound is true is: ； in, is the probability that the inequality corresponding to the upper bound of the generalization error holds.

6. A Wasserstein centroid matching layer device for message passing neural networks, characterized in that: Applied to a message passing neural network, the message passing neural network includes a message passing neural backbone network, a Wasserstein centroid matching layer network and a classifier network, and the device includes: An acquisition module, used to acquire the original image signal; a preprocessing module, configured to input the original graph signal into the message passing neural backbone network to obtain a filtered graph signal; a centroid matching module, configured to input the filtered graph signal into the Wasserstein centroid matching layer network, obtain the Wasserstein distance between the filtered graph signal and a plurality of class centroids, and obtain a graph representation vector, wherein each class centroid is used to represent the centroid of the embedding of each graph category in the Wasserstein space; a classification module, configured to input the graph representation vector into the classifier network to obtain a predicted category corresponding to the original graph signal; The centroid matching module is further used to input the filtered graph signal into the Wasserstein centroid matching layer network, obtain the Wasserstein distance between the filtered graph signal and multiple class centroids, and obtain a graph representation vector, including: inputting the filtered graph signal into the Wasserstein centroid matching layer network, obtaining the embedding of the original graph signal in the Wasserstein space as the target embedding; calculating the Wasserstein distance between the target embedding and each class centroid in the Wasserstein space; combining all Wasserstein distances in the Wasserstein space into the graph representation vector; wherein the target embedding is expressed according to the following formula: in, For the target embedding, For the message passing neural network, is the number of nodes in the original graph signal; is the Dirac function, and we observe The function takes the value 1 when , otherwise it takes the value 0. is the i-th graph node; The 2-Wasserstein distance between the target embedding and the centroid of each class is expressed as follows: in, is the 2-Wasserstein distance, is the centroid corresponding to category C, ; Embed for the target.

7. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the Wasserstein centroid matching layer method for message passing neural networks according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the Wasserstein centroid matching layer method for message passing neural networks according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the Wasserstein centroid matching layer method for message passing neural networks according to any one of claims 1 to 5 are implemented.

Images