node disambiguation
By assigning labels to the set of nodes in a graph neural network and processing their neighbor information, the problem of node ambiguity is solved, the recognition and computation capabilities of graph neural networks on non-isomorphic graphs are improved, and higher accuracy in machine learning tasks is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2019-09-25
- Publication Date
- 2026-05-01
AI Technical Summary
Existing graph neural networks struggle to effectively distinguish nodes with the same attributes, leading to performance loss in graph machine learning tasks, particularly in classification accuracy and regression loss.
By assigning different labels to a set of nodes with the same attributes and using a permutation-invariant function to process information about these nodes and their neighbors to form aggregate values, and then combining this with a gradient descent optimization algorithm to train the neural network weights, node disambiguation can be achieved.
Effectively separating nodes with the same attributes improves the recognition ability and target calculation accuracy of graph neural networks on non-isomorphic graphs, enabling them to better perform machine learning tasks.
Smart Images
Figure CN113692591B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to graph neural networks, and more particularly to eliminating ambiguity in such networks when nodes have the same properties. Background Technology
[0002] Many machine learning researchers believe that the ability to learn correct representations is the main reason behind the field's tremendous success in recent years. In fields such as image analysis, natural language processing, and reinforcement learning, groundbreaking results rely on efficient and flexible deep learning architectures that can transform complex inputs into simple vectors while retaining most of their valuable features.
[0003] The graph representation solves the problem of mapping high-dimensional objects to simple vectors through local aggregation steps in order to perform machine learning tasks such as regression or classification.
[0004] Some works studying the use of graph neural networks use recurrent neural networks to represent acyclic directed graphs, for example, as described in the following literature: Alessandro Sperduti and Antonina Starita, “Supervised neural networks for the classification of structures,” IEEE Transactions on Neural Networks, 8(3):714-735, 1997; and Paolo Frasconi, Marco Gori and Alessandro Sperduti, “A general framework for adaptive processing of data structures,” IEEE Transactions on Neural Networks, 9(5):768-786, 1998.
[0005] More general graph neural networks are described in the following literature: Marco Gori, Gabriele Monfardini, and Franco Scarselli, “A new model for learning in graph domains,” Proceedings of the IEEE International Joint Conference on Neural Networks, 2005, Vol. 2, pp. 729–734. IEEE, 2005 and Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, 20(1):61-80, 2009.
[0006] These general approaches can generally be divided into two categories. First, spectral methods, described in the following literature: Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun, “Spectral networks and locally connected networks on graphs,” ICLR, 2014; and Mikael Henaff, Joan Bruna, and Yann LeCun, “Deepconvolutional networks on graph-structured data,” arXiv preprint arXiv:1506.05163, 2015. These methods perform convolutions in the Fourier domain of the graph through the spectral decomposition of the graph Laplacian matrix. However, these methods lack spatial localization and have high computational complexity. The second category includes methods based on aggregation using neighbor information obtained through local iterative processes. For example, message-passing neural networks (MPNNs) are described in the following literature: Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl, “Neural message passing for quantum chemistry”, ICML, 2017; or neighbor aggregation schemes are described in the following literature: Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka, “How powerful are graph neural networks?”, ICLR, 2019.
[0007] This second category encompasses most existing graph representation methods, including DeepWalk (described in: Bryan Perozzi, Rami Al-Rfou, and Steven Skiena, “Deepwalk: Online learning of social representations,” Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 701-710, ACM, 2014), graph attention networks (GAT) (described in: Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio, “Graph attention networks,” ICLR, 2018); or graphSAGE (described in: Will Hamilton, Zhitao Ying, and Jure Leskovec, “Inductive representation learning on large graphs”). (graphs), Advances in Neural Information Processing Systems, pp. 1024-1034, 2017.
[0008] However, these processes may suffer performance losses (e.g., classification accuracy, regression loss, or more generally, any quality metric for the machine learning task) because the similarity of node attributes makes them difficult for neural networks to distinguish.
[0009] Therefore, despite their practical efficiency and strong correlation with the Weisfeiler-Lehman test in terms of graph isomorphism, techniques such as message-passing neural networks may not be able to distinguish simple graph structures and thus may not be adequately represented to provide good performance on any graph machine learning task.
[0010] The goal is to disambiguate nodes in graph neural networks so that they can be accurately applied to any machine learning task. Summary of the Invention
[0011] According to a first aspect, a data processing system for implementing a machine learning process based on a graph neural network is provided. The system is configured to receive multiple input graphs, each input graph having multiple nodes, at least some of which have attributes. The system is configured to: for at least one graph in the input graphs, determine one or more sets of nodes among the multiple nodes, each set containing nodes with the same attributes; for each set, assign a label to each node in the set such that each node in the set has a label different from the other nodes in the set; process the sets to form aggregate values and / or aggregate values for each set; and implement the machine learning process using the following as input: (i) input graphs other than the sets and (ii) the aggregate values and / or the aggregate values for each set.
[0012] The system provides a method for distinguishing objects with the same attributes within the context of structured data in a general graph representation. Labels are used to effectively separate nodes with the same attributes in a graph neural network. Disambiguation of nodes using this scheme allows for the separation of non-isomorphic graphs and enables the neural network to better identify each node and perform the target computation.
[0013] The system can be used to process each set to form an aggregated value by using a permutation-invariant function to process the neighboring nodes of each node in the set. This allows for the aggregation of information from the node itself and its neighbors.
[0014] The permutation-invariant function can be one of the sum, mean, or maximum. Other convenient functions can also be used.
[0015] The system can be used to process the set by assigning weights to the nodes, where the weights are parameters of a neural network. This allows the network to learn an optimal set of weights.
[0016] The system can also be used to iteratively update the weights. This can improve accuracy.
[0017] Each attribute and / or label can be a vector. Each label can be an additional attribute. Each label can be a color. A color can be represented as a one-hot encoded vector, or more generally as any finite set of k elements. Using color as a label can effectively separate nodes with the same attributes in a graph neural network.
[0018] The label can be randomly assigned to the determined node. This may be an efficient way to assign labels to nodes.
[0019] According to a second aspect, a method is provided for implementing a machine learning process based on a graph neural network in a data processing system, the system being used to receive a plurality of input graphs, each input graph having a plurality of nodes, at least some of the nodes having attributes, the method comprising: for at least one graph in the input graphs, determining one or more sets of nodes among the plurality of nodes, each set having nodes with the same attributes; for each set, assigning a label to each node in the set such that each node in the set has a label different from the other nodes in the set; processing the sets to form aggregate values and / or aggregate values for each set; and implementing the machine learning process using the following as input: (i) input graphs other than the sets and (ii) the aggregate values and / or the aggregate values for each set.
[0020] The method provides a way to distinguish objects with the same attributes within the context of structured data in a general graph representation. Labels are used to effectively separate nodes with the same attributes in a graph neural network. Disambiguation of nodes using the scheme allows for the separation of non-isomorphic graphs and allows the neural network to better identify each node and perform the target computation.
[0021] Each set can be processed to form an aggregated value by using a permutation-invariant function to process the neighboring nodes of each node in the set. This allows for the aggregation of information from the node itself and its neighbors.
[0022] The permutation-invariant function can be one of the sum, mean, or maximum. Other convenient functions can also be used.
[0023] The method may further include processing the set by assigning weights to the nodes, wherein the weights are parameters of the neural network. This allows the network to learn an optimal set of weights.
[0024] The method may also include iteratively updating the weights. This can improve accuracy.
[0025] Each label can be a color. A color can be represented as a one-hot encoded vector, or more generally as any finite set of k elements. Using colors as labels can effectively separate nodes with the same properties in a graph neural network.
[0026] The label can be randomly assigned to the determined node. This may be an efficient way to assign labels to nodes.
[0027] According to a third aspect, a computer program is provided that, when executed by a computer, causes the computer to perform the methods described above. The computer program may be provided on a non-transitory computer-readable storage medium. Attached Figure Description
[0028] The invention will now be described by way of example with reference to the accompanying drawings. In the drawings:
[0029] Figure 1 An overview diagram of a data processing system is shown, which assigns labels to the same node attributes to eliminate their ambiguity.
[0030] Figure 2 An example of an iteration method that uses color to distinguish the same nodes is shown.
[0031] Figure 3 This illustrates the cascading relationship between node attributes and assigned colors.
[0032] Figure 4 and 5 This demonstrates the application of this technology to malware classification tasks.
[0033] Figure 6 A method for implementing machine learning processes based on graph neural networks in a data processing system is shown.
[0034] Figure 7 An example of a data processing system is shown.
[0035] Figure 8 The results of using this method to distinguish structure graph characteristics on three synthetic datasets are shown.
[0036] Figure 9 The results are shown using this method on five real-world graph classification datasets extracted from standard social networks (IMDBb and IMDBm) and bioinformatics databases (MUTAG, PROTEINS, and PTC). Detailed Implementation
[0037] This invention proposes a technical solution to address node ambiguity in graph neural networks. The system described herein can learn representations of structured data to perform machine learning (ML) tasks using that data. The system computes a disambiguation scheme to effectively separate identical node attributes before applying any machine learning algorithm.
[0038] The definition of a graph with node attributes will now be described. Consider a dataset of n interacting objects (e.g., users of a social network), where each object... Has vector properties And it has an adjacency matrix The nodes in an undirected graph G.
[0039] The space of a graph of size n with m-dimensional node attributes is defined by the quotient space:
[0040]
[0041] Where A is the adjacency matrix of the graph, v contains the m-dimensional representation of each node in the graph, and the set of permutation matrices. The following equation applies to (v,A):
[0042]
[0043] The graph has the largest size n max In the case where n max It is a large integer, which allows for the consideration of functions on graphs of different sizes without requiring infinite-dimensional spaces and infinitely complex functions that would be impossible to learn from a finite number of samples. Therefore, Graph m The definition is as follows:
[0044]
[0045] The system described in this paper utilizes a general machine learning pipeline to process structured data in graphs, employs a labeling scheme to separate nodes with similar attributes, and combines the outputs from all labeled graphs to return a single output. This process is able to capture more complex structured graph features than traditional MPNNs.
[0046] like Figure 1 As shown in the overview, the system assigns labels to the same node attributes to eliminate their ambiguity. Attributes can be any quality, feature, or characteristic of a node. As shown in 101, the same node attributes are first identified. At 102, distinct labels (preferably represented as vectors) are appended to each node in the same set of nodes, such that all (attribute, label) pairs are distinct. Then, as shown in 103, an aggregation scheme is used to collect a graph of all labels and return a single output value that can be used for the ML task considered at 104.
[0047] In one embodiment, a process is employed that uses color as a label to distinguish nodes with the same attributes in order to differentiate non-isomorphic graphs. Figure 2 The steps of this preferred embodiment are shown in more detail below.
[0048] The iterative method includes the following steps. A graph with node attributes is provided at 201. In step 202, the system first clusters the nodes of the graph into sets of nodes with the same node attributes. Then, for each set, the system generates a fixed number of colors, each color being a random color assigned to each node in the set. A random number generator is shown at 203 for randomly assigning a color to each node. For each color, each node concatenates its attributes with the color it is assigned. Preferably, colors are randomly assigned to nodes. Figure 3The color concatenation is illustrated in Figure 204. The graph with node attribute 301 enhanced using color 302 is referred to herein as a shading graph. Each shading graph is then processed using the same neural network, as shown in Figure 204, including several iterative aggregations of neighboring shading attributes. At Figures 205 and 206, the final output is obtained by aggregating all outputs of the neural networks for all shading graphs using permutation-invariant functions (such as maximum or minimum). The model is trained using a gradient descent-based optimization algorithm, and backpropagation is performed on the output of this method to learn the neural network weights and optimal graph representation for the ML task under consideration, as shown in Figure 207.
[0049] More precisely, consider a set V of n nodes, and for each node, consider a graph G = (V, E) along with its eigenvectors. Let d > . This method calculates the graph. The projection onto the surface ensures that important relationships related to ML tasks are maintained.
[0050] A global workflow can be represented by the following:
[0051]
[0052] This method aims to learn neural network weights in order to compute vector representations of graphs. The labeling method does not rely on the weights or structure of the neural network, but rather eliminates ambiguity in node representations by concatenating labels with node features. Weights can be learned using any gradient descent-based optimization algorithm until a sufficiently accurate model is achieved for a given ML task.
[0053] The mathematical formulas for each step of the method will now be described for the case where the label is a color.
[0054] In the color generation / feature enhancement stage, for any Let C k It is a set of k colors. This set of k different colors is preferably selected randomly and uniformly. These colors can be represented as a one-hot encoded vector (Ci). k yes (the natural basis) or more generally as a finite set of any k elements.
[0055] Group nodes with the same attributes into partitions. In the middle. Then, for size |V k | set V k Each node in this set is assigned C. |Vk| The different colors in the graph. More precisely, the colors of graph G = (v, A). The set is defined as follows:
[0056]
[0057] Therefore, for each color c∈C k The node representation is initialized with its node attributes concatenated with the color:
[0058] In aggregation and combination schemes, each local aggregation step will combine a pair of... For input, where It is the representation of node i. It is the set of vector representations of the neighbors of node i.
[0059] The set of neighboring nodes for an m-dimensional node attribute is defined as follows:
[0060]
[0061] Where the permutation matrix The set is accessed through P·v = Pv To exert an effect.
[0062] The main difficulty in designing a general neighbor representation lies in the fact that, as defined in equation (6), the attributes of a node's neighbors are permutation-invariant relative to its adjacent nodes, thus requiring a permutation-invariant representation. The neural network described in this paper is a separable permutation-invariant network with a multilayer perceptron (MLP), which aggregates information from the node itself and its neighbors. The network is defined as follows:
[0063]
[0064] Wherein, ψ and It is an MLP with continuous non-polynomial activation functions.
[0065] In the color aggregation stage, for all colors generated in the previous step... Use neural networks to aggregate enhanced feature vectors (i.e., a concatenation of a node's attributes with its corresponding color):
[0066]
[0067] This function is a general neighbor representation.
[0068] During the color readout phase, from the aggregation, the enhancement vector of the transformation is selected by replacing the invariant function (e.g., the maximum value) with a coefficient-based approach. For example:
[0069]
[0070] Here, ψ is an MLP with a continuous non-polynomial activation function.
[0071] Therefore, this step performs a maximum value (or other function) over all possible colors to obtain a final color-independent graph representation. To maintain stability through cascading, a maximum value in coefficient form is used.
[0072] vector x G Then it is processed by any ML algorithm, and backpropagation is used to update the weights of the neural network.
[0073] When performing local iterations T times at each node, the complexity of the aggregation depends on the number of neighbors of the node being considered, and is proportional to the number of edges E in the graph and the number of steps T. Furthermore, this iterative aggregation is performed for each color, and the algorithm's complexity is also proportional to the number of colors k = |C| selected. k The time complexity is proportional to the time complexity. Therefore, the algorithm's time complexity is O(kET).
[0074] The above method can be performed by a data processing system such as a server or a combination of servers, or by a portable device such as a cellular communication device. The system can implement a machine learning process based on a graph neural network. The system can have inputs (e.g., internal or network inputs), thereby receiving multiple input graphs. Each graph can have multiple nodes, at least some of which can have attributes. After receiving a graph, the system can determine one or more sets of nodes for at least one input graph. The set can be determined such that some or all of the nodes in the set have the same attribute. Then, for each of these sets, the system can assign a label to each of their nodes. Nodes can be selected such that each node in the set has a different label than the other nodes in the set. The system can then process these sets to form aggregate values for all sets, or a series of aggregate values for each set. The system can then use the following as input to implement the machine learning process: (i) input graphs other than the sets and (ii) the aggregate values already formed therein or each of the aggregate values. This approach simplifies graph processing.
[0075] The systems and methods described in this paper can be applied to many technical fields that require data processing. For example, in the telecommunications field, many datasets to be processed are constructed as graphs. Some examples include process execution graphs for malware identification, handover graphs for wireless applications such as service forecasting on a single base station, or parameter tuning of a wireless base station. Other areas where graphs can be used include protein interactions, egocentric networks in social networks, and user-item pairs in recommender systems. Regression of graph features can be used, for example, to learn missing information in social or communication networks, or for regression of temporal data in a region, such as weather forecasting.
[0076] Figure 4 and 5 This demonstrates the application of this technique to a malware classification task. The sequence of events is generated by the execution of a software program (e.g., an application programming interface (API) call), and it is necessary to determine whether the software is malware.
[0077] This sequence of events can be formatted as an execution graph, where API (for this particular example) is a node attribute, such as... Figure 4 and 5 As shown. In these graphs, the execution trajectory is formatted as an execution graph. In this case, there are six nodes 401-406. The nodes are divided into four groups, V0, V1, V2, V3, respectively. Figure 5 Points 501, 502, 503, and 504 are shown in the diagram.
[0078] Because all groups except V3 have a cardinality of one, in this case, the aforementioned color-generating procedurally sampled colors are used only on nodes 404, 405, and 406 in group V3. This process allows all nodes in V3 to be distinguishable. The general mathematical approach described earlier is then followed. The model's input is a representation of the API, which can be one-hot encoded or derived from another algorithm (e.g., Word2vec representation). The method then outputs a vector used to learn a classifier to predict whether the software is malware.
[0079] Figure 6 A method for implementing a machine learning process based on a graph neural network in a data processing system is summarized. The system receives multiple input graphs, each having multiple nodes, at least some of which have attributes. For at least one input graph, the method includes: step 601, determining one or more sets of nodes from the multiple nodes, each set having nodes with the same attributes. Step 602, the method includes, for each set, assigning a label to each node in that set such that each node in the set has a label different from the other nodes in the set. Step 603, the method includes processing these sets to form aggregated values. Step 604, the method includes using the following as input to implement the machine learning process: (i) input graphs other than the sets and (ii) the aggregated values.
[0080] Figure 7A schematic diagram of a data processing system 700 for implementing the aforementioned network and its related components is shown. The system may include a processor 701 and non-volatile memory 702. The system may include more than one processor and more than one memory. The memory may store data executable by the processor. The processor may be used to run according to a computer program stored in a non-transitory form on a machine-readable storage medium. The computer program may store instructions for causing the processor to perform its methods in the manner described herein. The components may be implemented in physical hardware or may be deployed on various edge or cloud devices.
[0081] exist Figure 8 and 9 The results of two sets of experiments are shown, which were used to compare the method described herein with existing methods in supervised learning settings. Both sets of experiments followed the same protocol as described in the following literature: KeyuluXu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka, “How powerful are graph neural networks?”, ICLR, 2019 (10-fold cross validation with grid search hyper-parameter optimization).
[0082] Figure 8 The results are shown by using our method (CLIP) on three synthetic datasets to distinguish structural graph properties. Graph properties are a set of graphs that are closed under graph isomorphism. The performance of our method (e.g., Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka, “How powerful are graph neural networks?”, ICLR, 2019) for binary classification of three different structural properties is evaluated against Graph Isomorphism Networks (GINs).
[0083] Figure 8The table shows the classification accuracy of the synthetic dataset. For k-CLIP, colors k>0 are randomly selected for the CLIP model computation. These results demonstrate that the method described here can capture structural information regarding connectivity, bisection, and triangle independence in some implementations. One-hot encoding (equivalent to 1-CLIP) can improve accuracy. Furthermore, using more colors yields even better accuracy. In this implementation, high accuracy is achieved for as few as k=16 colors.
[0084] Figure 9 Results using this method on five real-world graph classification datasets extracted from standard social networks (IMDBb and IMDBm) and bioinformatics databases (MUTAG, PROTEINS, and PTC) are shown. Following standard practices for graph classification on these datasets, one-hot encoding of node degree as a node attribute was used for IMDBb and IMDBm, and single-label multi-class classification was performed on all datasets.
[0085] This method (CLIP) was compared with six existing baseline algorithms: Weisfeiler-Lehman (WL) subtree kernels (e.g., Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt, “Weisfeiler-lehman graph kernels”, Journal of Machine Learning Research, 2011); Anonymous Walk Embeddings (AWL) (e.g., Sergey Ivanov and Evgeny Burnaev, “Anonymous walk embeddings”, ICML, 2018); Diffusion-convolutional neural networks (DCNN) (e.g., James Atwood and Don Towsley, “DCNN” ... Networks), Conference on Advances in Neural Information Processing Systems, 2016; Patchy-SAN (PS) (e.g., Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov, “Learning convolutional neural networks for graphs”, International conference on machine learning, 2016); Deep Graph CNN (DGCNN) (e.g., Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen, “An end-to-end deep learning architecture for graph classification”, Proceedings of AAAI Conference on Artificial Intelligence, 2018); and GIN.WL and AWL are representative unsupervised methods coupled with SVM classifiers, while DCNN, PS, DGCNN, and GIN are four deep learning architectures.
[0086] Figure 9 A table showing the classification accuracy of the compared methods on the benchmark dataset is presented. The best performer in terms of mean is highlighted with an asterisk. An unpaired t-test, with asymptotic significance of 0.1 relative to the best performer, is highlighted in bold; the difference is not statistically significant.
[0087] In this implementation, our method (CLIP) achieves state-of-the-art performance on three out of five benchmark datasets and is comparable to competitors on the others. For the PTC dataset, our method significantly outperforms its competitors, which may indicate that this classification task requires more structural information about the graph. The high variability of most methods on MUTAG and PTC is likely due to the smaller number of graphs.
[0088] Therefore, this invention provides a method for distinguishing objects with the same attributes within the context of structured data in a general graph representation. Labels are used to effectively separate nodes with the same attributes in a graph neural network. In practice, this method involves concatenating different vectors to similar node attributes. Node disambiguation using this scheme allows for the separation of non-isomorphic graphs.
[0089] The method described here allows neural networks to better identify each node and perform target computations. As experimental results show, in some implementations, this method can achieve state-of-the-art results on classic datasets, separate any non-isomorphic graph pairs, extract any valuable patterns from structured data, and successfully learn any machine learning task given a sufficient amount of data. This method can compute complex structural features of graphs, such as the number of triangles or other small-scale patterns, which can be important for the machine learning task under consideration.
[0090] This method can be applied to data structures such as directed or weighted graphs with node attributes, graphs with node labels, graphs with edge attributes, or graphs with additional attributes at the graph level.
[0091] The applicant hereby discloses individually each individual feature described herein, as well as any combination of two or more such features. With ordinary knowledge of those skilled in the art, such features or combinations can be implemented as a whole based on this specification, regardless of whether such features or combinations of features solve any of the problems disclosed herein; and without limiting the scope of the claims. This application demonstrates that aspects of the invention can be constituted by any such individual features or combinations of features. Various modifications that can be made within the scope of the invention, given the foregoing description, will be apparent to those skilled in the art.
Claims
1. A data processing system for implementing machine learning processes based on graph neural networks, characterized in that, The system is used to receive multiple input graphs, each input graph having multiple nodes, at least some of which have attributes, and the system is used to: For at least one graph in the input graph: Determine one or more sets of nodes from the plurality of nodes, wherein the nodes in each set have the same attributes; For each set, assign a label to each node in the set such that each node in the set has a label different from the other nodes in the set; The set is processed by assigning weights to the nodes to form an aggregate value; wherein the weights are parameters of the neural network. Each set is processed to form an aggregate value by using a permutation-invariant function to process the neighboring nodes of each node in the set; as well as The machine learning process is implemented using the following as inputs: (i) an input graph other than the set and (ii) the aggregated values; The system is also used to iteratively update the weights.
2. The system as described in claim 1, characterized in that, The permutation-invariant function is one of the sum, mean, or maximum value.
3. The system as described in claim 1 or 2, characterized in that, Each attribute and / or tag is a vector.
4. The system as described in claim 1 or 2, characterized in that, Each label is a color.
5. The system as described in claim 1 or 2, characterized in that, The label is randomly assigned to the determined node.
6. A method for implementing a machine learning process based on a graph neural network in a data processing system, characterized in that, The system is used to receive multiple input graphs, each input graph having multiple nodes, at least some of the nodes having attributes, and the method includes: For at least one graph in the input graph: Determine one or more sets of nodes from the plurality of nodes, wherein the nodes in each set have the same attributes; For each set, assign a label to each node in the set such that each node in the set has a label different from the other nodes in the set; The set is processed by assigning weights to the nodes to form an aggregate value; wherein the weights are parameters of the neural network. Each set is processed to form an aggregate value by using a permutation-invariant function to process the neighboring nodes of each node in the set; as well as The machine learning process is implemented using the following as inputs: (i) an input graph other than the set and (ii) the aggregated values; The method further includes iteratively updating the weights.
7. The method as described in claim 6, characterized in that, The permutation-invariant function is one of the sum, mean, or maximum value.
8. The method as described in claim 6 or 7, characterized in that, Each label is a color.
9. The method as described in claim 6 or 7, characterized in that, The label is randomly assigned to the determined node.
10. A computer program product, characterized in that, The computer program product includes: a computer program or instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 6-9.