Method, system, device and medium for classifying target nodes in graph structure data

Abstract

The present invention discloses a method, system, device and medium for classifying target nodes in graph structured data. The classification task of target nodes in graph structured data is converted into the classification task of subgraphs corresponding to the target nodes. An embedded subgraph representation is generated for the target node by selecting fixed-order neighbor nodes of the target node, and then a reinforcement learning method is used to search for the optimal subgraph and the optimal graph neural network structure in a given search space to classify the subgraph. This not only makes full use of the feature information of the node and part of the structural information of the graph, thereby facilitating the improvement of the node classification accuracy, but also reduces the workload of designing the graph neural network model. The node classification accuracy can also be guaranteed by obtaining the optimal graph neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of target node classification in graph structured data, and in particular to a method, system, device and medium for classifying target nodes in graph structured data. Background Art

[0002] Graphs, as data structures that effectively model a set of objects (nodes) and the complex relationships (edges) between them, have gained widespread application in the real world due to their powerful expressive capabilities. Generally speaking, this can be categorized into two main aspects. First, many real-world systems can be represented as graphs or networks, such as power grids, communication networks, and the World Wide Web. Second, much real-world data can be organized and stored using graphs, including social network data like Facebook, Twitter, and Weibo, bioinformatics data like biomacromolecules and PPI, and e-commerce data like Taobao, JD.com, and Amazon.

[0003] Node classification in graph data is a crucial task. Generally speaking, given the labels of some nodes in a graph, the goal is to predict labels for the remaining unlabeled nodes in the graph. The node label category is often related to the specific application scenario. For example, when predicting malicious fraud users in financial networks, a node label can indicate whether a user is malicious or not.

[0004] Numerous node classification methods have been proposed from various perspectives, including: 1) Using traditional machine learning algorithms such as KNN (K-Nearest Neighbors), SVM (Support Vector Machine), and LR (Linear Regression), these methods utilize raw node feature information and predict labels for nodes through a discriminator, thereby distinguishing nodes. 2) Starting from random walks on graph-structured data, drawing on Word2Vec, a key WordEmbedding method in natural language processing, these methods capture the co-occurrence relationships of nodes in graph-structured data to generate embedding vectors for nodes. These embeddings are then used to predict labels for node classification. 3) Improving random walk strategies allows for flexible adjustment to prioritize capturing the homogeneity or structure of nodes in the graph, thereby generating embedding vectors for node label prediction and classification. 4) Defining first-order and second-order similarity on graph-structured data, these two similarities are maximized when generating embedding vectors for node classification, and the resulting node embeddings are used for classification. 5) A graph convolutional neural network is defined based on a message passing mechanism. It aggregates features of neighboring nodes and then classifies nodes based on these features. 6) An attention mechanism is introduced on top of the graph convolutional neural network. By considering the different influences of different neighboring nodes on the current node, the weighted summation of neighboring node features is performed to obtain an embedded vector representation for node classification.

[0005] The aforementioned methods also have some problems: 1) They fail to fully utilize the node feature information and graph structure information in the graph data, resulting in low node classification accuracy. 2) Designing the optimal network structure for a specific task often requires a large amount of prior knowledge, otherwise it is difficult to obtain the optimal network structure, which in turn affects node classification accuracy. Summary of the Invention

[0006] The purpose of the present invention is to provide a method, system, device and medium for classifying target nodes in graph structure data, which searches for the optimal subgraph construction method and the graph neural network architecture for subgraph classification through reinforcement learning methods, so that the optimal design details can be obtained without a large amount of prior knowledge, greatly reducing the time and effort spent on designing the network structure, and also improving the accuracy of node classification.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] A method for classifying target nodes in graph structured data, comprising:

[0009] Obtain graph structure data consisting of a node set and an edge set;

[0010] For the target node set to be classified, extracting neighbor nodes of corresponding fixed orders from the graph structure data with each target node as the center according to a set fixed order to construct a subgraph, thereby forming an initial subgraph set corresponding to the target node set to be classified;

[0011] Using reinforcement learning methods combined with an initial set of subgraphs, we search for the optimal strategy in the state space to obtain the optimal order and number of layers of the graph neural network for each target node. Each state in the state space is determined by the features of all nodes in a single subgraph.

[0012] For each target node, the corresponding optimal order of neighbor nodes is extracted from the graph structure data with the corresponding target node as the center to construct an optimal subgraph according to the corresponding optimal order, and the graph neural network with the corresponding optimal number of layers that has been trained is used to obtain a vector representation describing the characteristics of the optimal subgraph, and classification is performed to obtain the category of each target node.

[0013] A target node classification system in graph structured data, comprising:

[0014] A data acquisition unit, used to acquire graph structure data consisting of a node set and an edge set;

[0015] a subgraph initialization unit for extracting neighbor nodes of a corresponding fixed order from the graph structure data with each target node as the center according to a set fixed order for the target node set to be classified, thereby forming an initial subgraph set corresponding to the target node set to be classified;

[0016] An optimal strategy search unit, which uses reinforcement learning methods combined with an initial subgraph set to search for the optimal strategy in the state space, obtaining the optimal order and the optimal number of layers of the graph neural network corresponding to each target node. Each state in the state space is determined using the features of all nodes in a single subgraph.

[0017] The node classification unit is used to extract neighbor nodes of the corresponding optimal order from the graph structure data with the corresponding target node as the center according to the corresponding optimal order for each target node to construct an optimal subgraph, and use the graph neural network with the corresponding optimal number of layers that has been trained to obtain a vector representation describing the characteristics of the optimal subgraph, and perform classification to obtain the category of each target node.

[0018] A processing device comprising: one or more processors; a memory for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0020] A readable storage medium stores a computer program, which implements the aforementioned method when the computer program is executed by a processor.

[0021] It can be seen from the technical solution provided by the present invention that the classification task of the target node in the graph structure data is converted into the classification task of the subgraph corresponding to the target node. By selecting the fixed-order neighbor nodes of the target node, an embedded subgraph representation is generated for the target node, and then the reinforcement learning method is used to search for the optimal subgraph and the optimal graph neural network structure in the given search space to classify the subgraph. This not only makes full use of the feature information of the node and part of the structural information of the graph, which is conducive to improving the node classification accuracy, but also reduces the workload of designing the graph neural network model. The node classification accuracy can also be guaranteed by the obtained optimal graph neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 An overall flow chart of a method for classifying target nodes in graph structure data provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a model structure of a target node classification method in graph structure data provided by an embodiment of the present invention;

[0025] Figure 3 A basic flow chart of a method for classifying target nodes in graph structure data provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a target node classification system in graph structure data provided by an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of a processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] First, the following terms may be used in this article:

[0030] The term “and / or” means that either or both of them can be realized at the same time. For example, X and / or Y includes both “X” or “Y” and “X and Y”.

[0031] The terms "include," "comprises," "contains," "has," or other similar expressions should be interpreted as non-exclusive. For example, "including certain technical features (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products, or manufactured articles, etc.) should be interpreted as including not only the technical features explicitly listed, but also other technical features known in the art that are not explicitly listed.

[0032] The target node classification method, system, device and medium in a graph structure data provided by the present invention are described in detail below. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professionals in this field. If specific conditions are not specified in the embodiments of the present invention, they are carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer. If the manufacturer of the reagents or instruments used in the embodiments of the present invention is not specified, they are all conventional products that can be purchased commercially.

[0033] Example 1

[0034] The embodiment of the present invention provides a method for classifying target nodes in graph structure data, such as Figure 1 As shown, it mainly includes the following steps:

[0035] Step 1: Obtain graph structure data consisting of a node set and an edge set.

[0036] In the embodiment of the present invention, graph structure data is technical data with a specific meaning in this field, which can show the relationship between multiple nodes. Depending on the specific application scenario, the node can be specific information of a user, a device or certain objects.

[0037] In an embodiment of the present invention, each node feature in the graph structure data is a feature vector extracted based on the node information, and the node information includes but is not limited to: text information and / or image information of the node; for example, when the node is a user, relevant features can be extracted from the user information, and the user information can be relevant introduction text for the user (for example, age, hobbies, transaction records, etc.) and / or photos (image information), etc.

[0038] Taking the task of detecting malicious accounts on social networks as an example, a social network can be viewed as a graph consisting of a vast number of users and the complex relationships between them. The nodes in the graph can be users' social accounts, and the edges between nodes can represent the social relationships between accounts (for example, follower / follower relationships). Node information generally includes the following three types: 1) social account metadata, such as the number of followers, number of followers, and account creation time; 2) social account text information, such as the account profile and text content posted by the account; and 3) social account image, video, and audio information, such as the account's profile picture, posted videos, and songs. Node features can be derived by extracting some or all of these three types of information separately and then combining them.

[0039] Another example is fraud detection in financial transaction networks. The graph data construction process is as follows: Financial accounts can be considered nodes, and transaction transfers between different accounts constitute connections between accounts, or edges. Specifically, if there is a transfer between Account A and Account B, an edge can be used to connect the nodes corresponding to the two financial accounts. The initial features of the nodes can be extracted from relevant information about the corresponding accounts, such as the account opening date, the account holder's personal information, the account balance, and the account's credit rating.

[0040] After constructing the graph and extracting initial node features, we can use the designed graph neural network to detect and classify suspicious nodes (which can be different entities depending on the specific application scenario). In addition to the application scenarios mentioned above, potential applications of this invention also include spam email detection and medical insurance fraud detection.

[0041] In the embodiment of the present invention, graph structure data refers to graph structure data including target nodes to be classified.

[0042] Step 2: For the target node set to be classified, extract neighbor nodes of corresponding fixed order from the graph structure data with each target node as the center according to the set fixed order to construct a subgraph, forming an initial subgraph set corresponding to the target node set to be classified.

[0043] In the embodiment of the present invention, the target node set to be classified in the graph structure data is recorded as Each v i′ Represents a target node to be classified, the subscript is the node number, i∈{1′,2′,…,c′}, then there are c′ target nodes to be classified. According to the set fixed order, extract the neighbor nodes of each target node, construct a subgraph, and obtain the initial subgraph set Among them, each Represents a subgraph, the subscript is the subgraph number, and the subgraph number corresponds to the node number one by one. For example, is the target node v to be classified i′ The corresponding subgraph.

[0044] Step 3: Use reinforcement learning methods combined with the initial subgraph set to search for the optimal strategy in the state space to obtain the optimal order corresponding to each target node and the optimal number of layers of the graph neural network. Each state in the state space is determined using the features of all nodes in a single subgraph.

[0045] In an embodiment of the present invention, the initial representation vectors of all subgraphs in the initial subgraph set are used as input, and a reinforcement learning method is used to train the intelligent agent to obtain the optimal decision, that is, to determine the optimal subgraph order corresponding to the target node and the optimal number of layers of the graph neural network.

[0046] The training process is as follows: a subgraph is randomly selected at the initial time step, and its initial representation vector is used as the state at the initial time step. Afterwards, the transition probability is calculated based on the connection relationship between the subgraph corresponding to the previous time step and other subgraphs. The corresponding subgraph at the next time step is determined according to the transition probability, and the initial subgraph representation vector of the corresponding subgraph is used as the state of the next time step. Under the state of the current time step, different actions are selected from the action space, and corresponding reward feedback (environmental feedback) is obtained after executing different actions and the corresponding state-action value is calculated. The final action is selected in combination with the strategy function, and the state of the current time step, the final action, the reward value and the state of the next time step are combined to form the record data of the current time step; the recorded data of multiple time steps are used to train the agent until convergence, and the training of the agent is completed; wherein, each action includes a sub-action for selecting the order value and a sub-action for selecting the layer value of the graph neural network; at the same time, in each time step, the corresponding order and number of layers are determined based on the final action, and a new subgraph of the corresponding order is extracted from the graph structure data with the target node corresponding to the corresponding time step as the center, and then the new subgraph and the corresponding number of layers are stored in the cache, and the new subgraph is used to train the graph neural network of the corresponding number of layers, and applied to the subsequent reward feedback calculation process. Specifically:

[0047] The state of the next time step is determined as follows: for the current time step t, the number of connections (i.e., the number of edges) between each other subgraph and the subgraph corresponding to the current time step t accounts for the number of connections between the subgraph corresponding to the current time step t and the external nodes as the transition probability P i , and then according to the transition probability P i Select a subgraph from all other subgraphs and use the initial subgraph representation vector of the selected subgraph as the state s in the next time step t+1 t+1 .

[0048] The final action selection method is: each action a in the current time step t t By binary action Composition, by executing the first sub-action Get the value of the order by executing the second sub-action Get the value of the number of layers of the graph neural network and select different sub-actions from the given action space With sub-actions After executing them separately in the state of the current time step, the reward feedback of each action is determined according to the classification accuracy of the subgraph corresponding to the current time step t, and the state-action value obtained after executing each action at the current time step t is determined in combination with the cumulative state-action value from the given next time step t+1 to the maximum time step T. The strategy function is used to judge whether the current time step t meets the set conditions. If so, an action is randomly selected as the final action of the current time step t. If not, the action corresponding to the maximum state-action value is selected as the final action of the current time step t.

[0049] Step 4: For each target node, extract neighbor nodes of the corresponding optimal order from the graph structure data with the corresponding target node as the center according to the corresponding optimal order to construct an optimal subgraph, and use the graph neural network with the corresponding optimal number of layers that has been trained to obtain a vector representation describing the characteristics of the optimal subgraph, and perform classification to obtain the category of each target node.

[0050] In the embodiment of the present invention, a vector representation describing subgraph features can be directly obtained through graph pooling. Specifically: for each optimal subgraph, all node features contained therein are input into the graph neural network of the optimal number of layers, and all node features are aggregated into a feature vector as the vector representation (embedded vector representation) of the optimal subgraph feature, where the vector representation of the optimal subgraph feature corresponding to the i-th target node is described as:

[0051]

[0052] Where L is the last layer of the graph neural network, that is, the optimal number of layers corresponding to the i-th target node, z i Represents the vector representation of the optimal subgraph feature corresponding to the i-th target node, It represents the feature vector representation of the j-th node in the optimal subgraph corresponding to the i-th target node obtained by the L-th layer graph neural network, n is the number of nodes in the optimal subgraph corresponding to the i-th target node, and R(.) is the readout operation, for example, it can be max, sum, or mean operation, that is, finding the maximum value / sum / mean of the feature vectors of all nodes in the optimal subgraph.

[0053] In this embodiment of the present invention, the graph neural network is trained using the following loss function:

[0054]

[0055] in, Represents a set of labeled training nodes, the first item (i.e. -log(y τ ·σ(MLP(z τ )))) is the cross entropy loss function, the second term (i.e. ||Θ||2) is the L2 norm of the graph neural network parameter Θ, y τ Represents the given τth node Tags, z τ Represents the τth node The vector representation of the corresponding sub-graph feature (obtained according to the aforementioned graph pooling processing method), λ is the weight parameter, MLP is the multi-layer perceptron, and σ is the normalization function (such as the softmax function).

[0056] In an embodiment of the present invention, a graph neural network can be trained using a portion of the reserved labeled node data. Specifically, it is trained by extracting subgraphs from these nodes, learning representations, and then classifying them. During training, the order used to extract node subgraphs can be set by the user according to needs or experience, because no matter how many orders the subgraph of neighbor nodes is, the representation finally learned by the graph neural network is still a vector.

[0057] The following application scenario is used as an example. As previously mentioned, when performing fraud detection tasks in financial transaction networks, the target node classification method in graph-structured data can be viewed as a potential fraud account detection method. In step 1, account information and transaction data between accounts in the financial transaction network are obtained, and financial accounts are used as nodes. The attribute characteristics of the nodes can be extracted from the relevant information of the financial accounts. The transaction behaviors between financial accounts constitute the edges between nodes, thereby forming graph-structured data, which is a homogeneous attribute graph. Then, through step 2, an initial subgraph set corresponding to the set of target nodes to be classified is formed. After that, through step 3, the optimal order corresponding to each target node and the optimal number of layers of the graph neural network are obtained. Finally, through step 4, each target account is classified as a normal account or a fraudulent account. Therefore, the fraudulent account can be determined based on the classification results.

[0058] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, a target node classification method in graph structure data provided by an embodiment of the present invention is described in detail below with reference to a specific embodiment.

[0059] 1. Problem definition.

[0060] For a given undirected graph in is a set of nodes, ε=(e1,e2,…,e M ) edge set, is the characteristic matrix of the nodes in the graph, that is, each node has a d-dimensional characteristic vector, is the symbol of a real number set. Given a set of target nodes There are c' nodes in total, and the goal is to predict the corresponding labels for these c' nodes in is a set of node labels, Represents a label set, where y i represents the i-th label, and p is the total number of label types.

[0061] 2. Initial subgraph embedding.

[0062] For the target node set In the initialization phase, a set of neighbor nodes with a fixed order is selected for each node. For example, the order can be set to 1, that is, the first-order neighbors of the target node are selected to construct a subgraph. By taking the node in the target node set V as the center node and extracting the neighbor nodes with a fixed order as the center node, a set of initial subgraphs can be obtained. for Get the initial vector representation corresponding to the subgraph:

[0063]

[0064] Among them, i∈{1,2,…,c}, Representing a subgraph The initial vector representation (embedded vector representation), V i′ Representing a subgraph The node set, h j Representing a subgraph Midpoint v j The initial vector representation of the subgraph obtained at this time is actually the set of feature vectors of the nodes in the subgraph.

[0065] 3. Search for the optimal order and number of network layers based on reinforcement learning.

[0066] In order to classify the subgraph set obtained in the second part, it is necessary to design a suitable graph neural network architecture. Some of the best-performing graph neural network architectures are currently used as benchmark models, including graph convolutional neural networks (GCNs), graph attention neural networks (GATs), and a network model for sampling and aggregating neighbor nodes (GraphSAGE). In specific applications, the choice can be based on the characteristics of the graph structure in the application scenario. For example, when the graph is small and the feature information is not rich, GCN can be used as the benchmark model; when the graph is not large and the feature information is rich, GAT can be used as the benchmark model; when the graph is large, GraphSAGE can be used as the benchmark model. To achieve the above goals, two key parameters need to be determined: one is the optimal order k selected when constructing the optimal subgraph. The order value is pre-set in the initial stage (that is, the fixed order value mentioned in the second part), but it may not be optimal for specific tasks. The other is the specific number of graph neural network layers. By stacking the optimal number of graph neural network layers L, better classification results can be achieved. The selection of k and L is determined by reinforcement learning methods. By inputting the initial vector representation of the subgraph obtained in the second part into the reinforcement learning method, the optimal k strategy π1 and the optimal number of network layers L strategy π2 are obtained in the given search space. Among them, the search space mainly refers to the neighbor order of sampling the target node when constructing the subgraph and the number of layers of the graph neural network. For example, the neighbor order is pre-set to [1,2] and the number of layers of the graph neural network is [1,3]. Of course, the specific range value can be set according to the actual situation.

[0067] The above process of learning the optimal policies π1 and π2 can be formally described as a Markov decision process (MDP):

[0068] State Space: Transform the state s in time step t into t ∈S is defined as The initial vector representation of the subgraph extracted from t=0,…,T-1, where T is the total training step length of each round.

[0069] Action Space: The action at time step t consists of two sub-actions Composition: Reinforcement learning agent performs actions To get the value of the neighbor order, perform the action To obtain the value of the number of layers of the graph neural network. Therefore, the overall sequence of MDP can be described as That is, starting from the initial state, the agent will receive a reward feedback each time it performs an action, and then the state transition will occur to the next state. In order to generate this sequence, the agent will The number of connections between the extracted subgraph and other subgraphs related to the target node within k hops is used to form a probability distribution P, and the next state s is obtained based on P. t+1 Specifically, for the current time step t, the number of connections (i.e., the number of edges) between each other subgraph and the subgraph corresponding to the current time step t accounts for the proportion of the number of connections between the subgraph corresponding to the current time step t and the external nodes as the transition probability P i , and then according to the transition probability P i Select a subgraph from all other subgraphs and use the initial subgraph representation vector of the selected subgraph as the state s in the next time step t+1 t+1 .

[0070] Reward Function: This function determines whether the representation vector of the subgraph at time step t is good enough by judging whether the model can achieve better accuracy. To this end, we first define the metric for accuracy improvement:

[0071]

[0072] Where b is an integer greater than 1, which is a hyperparameter that reflects the window size of the historical time steps involved in the comparison; In state s t Next, perform action a t Finally, the obtained order value is combined with the value of the number of layers of the graph neural network to verify the set of labeled nodes The accuracy of classifying the nodes in In state s t Next, perform action a t After receiving the reward feedback, is the accuracy of node classification on the validation set at time step i, Represents the average accuracy in the last b time steps. Intuitively, the training strategy is to find a positive increasing To ensure the continuous improvement of classification accuracy.

[0073] Termination: Model-free deep reinforcement learning is used to solve the MDP problem (i.e., to find the optimal order and number of layers). The basic idea is to learn to take the optimal action through exploration. Due to the limited number of target nodes, the current round ends once the agent makes a decision for each target. The DQN algorithm is primarily used to reduce the number of states in the search space. More specifically, DQN approximates the state-action value of the Bellman optimality equation:

[0074]

[0075] Among them, a t and a′ both indicate action; represents the action-value function, Indicates the state s corresponding to the current time step t t Next, perform action a t The state action value after γ represents the discount factor of future cumulative returns. The action a′ in the expression is to make the action value function The action when the maximum value is reached, As a whole, it covers the cumulative state-action value from the next time step t+1 to the maximum time step T.

[0076] In the embodiment of the present invention, a greedy strategy (∈-greedy) is used to select Action a t , and obtain the policy function π:

[0077]

[0078] In simple terms, when making action decisions, a random action is selected with the probability of the greedy strategy parameter ∈ (∈ is a small positive number, set by the user according to the actual situation) (when the randomly generated random value is less than ∈, a random action is selected) and used as the final action, and the state s is selected with a probability of 1-∈. t The action-value function under The maximum action, greedy strategy allows the algorithm to better balance the relationship between "exploration" and "exploitation".

[0079] Based on the above introduction, during the training phase, at the current time step t, after each action is executed, the aforementioned The reward feedback is calculated by the formula, and then Calculate the state action value and finally decide the final action a through the strategy function π t , forming a record of the current time step t Continuously iterate and obtain records of each time step, use all records to train the agent, optimize the agent's network parameters until convergence (when training reaches the specified number of epochs and the test accuracy reaches the peak); after training, the optimal neighbor order corresponding to each target node and the optimal number of layers of the graph neural network are obtained. At the same time, within each time step, the corresponding order and number of layers are determined based on the final action, and a new subgraph of the corresponding order is extracted from the graph structure data with the target node corresponding to the corresponding time step as the center, and then the new subgraph and the corresponding number of layers are stored in the cache. When the amount of data in the cache reaches the set threshold, the cached subgraph is used to train the graph neural network of the corresponding number of layers. After training, the cache corresponding to the graph neural network of the corresponding number of layers is cleared. The graph neural network trained here will be used in the subsequent reward feedback calculation process.

[0080] Since constructing a graph neural network model independently for each target node to be classified will greatly increase the number of training parameters, thereby significantly increasing the training time, in order to solve the above problems, the present invention adopts a parameter sharing and embedding cache mechanism. Among them: 1) The parameter sharing mechanism includes: in the initialization model stage, a layer number L is determined max Then, according to the output of the agent at each time step, the specified layers are repeatedly extracted from the basic graph neural network model to construct a graph neural network model with the corresponding number of layers, thereby avoiding the increase in training cost caused by additional parameters. For example, initialize a layer with 5 layers (L max =5) Basic Graph Neural Network, and then take out several layers from the basic graph neural network in order according to the number of layers given by the agent's action. If the action given by the agent is to take 3 layers, then take the first 3 layers from the basic graph neural network, but it is necessary to ensure that the optimal number of layers corresponding to each target node does not exceed L max 2) The embedding cache mechanism includes: caching the initial vector representation of the new subgraph (that is, the set of all node features in the new subgraph) as a batch of data, and performing To reduce unnecessary embedding transfer operations. When the number of cached batch data reaches the preset batch size, the second sub-action will be executed The number of layers obtained and the cached embeddings are used to trigger model reconstruction and training. Once the graph neural network training is completed, the cache space is cleared to ensure that the cache can be refilled later.

[0081] The relationship between agent training and graph neural network training can be understood as follows: the graph neural network is trained on a labeled training node set, and then the graph neural network classification accuracy ACC is calculated on a labeled verification node set to obtain reward feedback, thereby guiding the training of the agent. The actions given by the agent may adjust the number of layers of the graph neural network, leading to the reconstruction and training of the graph neural network model.

[0082] 4. Image pooling and classification.

[0083] The third step above allows us to search for the optimal order k and number of layers L for acquiring the subgraph embedding for each target node. This allows us to construct the optimal subgraph structure for the target node and design the optimal graph neural network architecture for classification. However, before this, graph pooling is required on the optimal subgraph. Compared to the subsequent vector representation of the subgraph, which is a collection of feature vectors of nodes within the subgraph, graph pooling directly yields a vector representation describing the optimal subgraph's features.

[0084] The specific approach of graph pooling is to aggregate the feature vector representations of the nodes in the optimal subgraph into a feature vector of a super node, which is used to represent the feature vector of the optimal subgraph. The vector representation of the optimal subgraph feature corresponding to the i-th target node is described as:

[0085]

[0086] Among them, z i Represents the vector representation of the optimal subgraph feature corresponding to the i-th target node, The feature vector representation of the j-th node in the optimal subgraph corresponding to the ith target node obtained by the L-th layer (i.e., the optimal number of layers corresponding to the ith target node) of the graph neural network, n is the number of nodes in the optimal subgraph corresponding to the ith target node, and R(.) is a readout operation (READOUT operation), for example, it can be a max, sum, or mean operation, that is, finding the maximum value / sum / mean of the feature vectors of all nodes in the optimal subgraph.

[0087] In the embodiment of the present invention, the vector representation of the obtained optimal subgraph feature is directly classified, and the classification result is the classification result of the target node corresponding to the optimal subgraph feature.

[0088] Of course, graph neural networks need to be trained, and the loss function during training is expressed as:

[0089]

[0090] in, represents a set of labeled training nodes, the first term represents the cross entropy loss function, ||Θ||2 is the L2 norm of the graph neural network model parameters, zτ The vector representation of the subgraph feature corresponding to the τth node (obtained according to the aforementioned graph pooling method), λ is the weight parameter, MLP is a multi-layer perceptron, and the final embedding vector z i The dimension is reduced to the number of classification categories, thus achieving the purpose of classification.

[0091] In an embodiment of the present invention, when the optimal number of layers corresponding to different target nodes is the same, the same graph neural network is used, and each graph neural network with the optimal number of layers is trained using the above-mentioned loss function; in order to improve training efficiency, during training, each graph neural network is trained in order from low to high optimal number of layers, and the graph neural network with the lowest optimal number of layers is trained first. Based on the trained graph neural network with the lowest optimal number of layers, the graph neural network with the second lowest optimal number of layers is trained until the training of all graph neural networks is completed.

[0092] like Figure 2 As shown, a schematic diagram of the model structure of the target node classification method in the graph structure data provided by the present invention is shown; from left to right, it contains three larger dotted boxes, the leftmost dotted box provides an example of graph structure data, and the leftmost dotted box uses different filling patterns to distinguish neighbors of different orders and other nodes; the middle dotted box provides an overall architecture for implementing the aforementioned third part of the solution, the upper half is used to search for the optimal number of layers of the graph neural network (reinforcement learning search GNN architecture module), and the lower half is used to search for the optimal order of the subgraph (reinforcement learning subgraph construction module); in the rightmost dotted box, the reinforcement learning process, graph pooling and classification process are shown from left to right. Considering the space problem and the reinforcement learning process part has defined subgraphs of different orders, the neighbor nodes of different orders are not filled.

[0093] Figure 3 The basic process of the above method of the present invention is demonstrated, in which: extracting a subgraph for the target node and obtaining a vector representation corresponds to the aforementioned second part; updating the subgraph extraction strategy based on the reinforcement learning algorithm, classifying the subgraphs by designing the GNN structure through reinforcement learning, and whether the subgraph extraction strategy and the GNN design strategy are optimal correspond to the aforementioned third part; the output subgraph extraction method and the GNN design method correspond to the aforementioned fourth part.

[0094] The above solutions provided by the embodiments of the present invention can be generally described as the process shown in Table 1 below.

[0095] Table 1: Description of the overall process of the present invention

[0096]

[0097] (1) First, for the given graph structure data Initialize Lmax Layer GNN model, reinforcement learning agent strategy π1, π2, memory cache GNN Cache

[0098] (2) Randomly select a target node during the training phase and extract its k init Constructing subgraphs from neighboring nodes And use formula (1) to obtain the vector representation corresponding to the subgraph And use it as the initial state s0 of the agent in the reinforcement learning algorithm;

[0099] (3) In each round of training, an action pair (i.e., the final action selected by the policy function) is selected through a greedy strategy to obtain the neighbor order and the number of layers of the graph neural network l, and then a new subgraph of the selected target node is re-extracted based on the obtained neighbor order. and will and through The obtained l is saved to the cache In. When middle The number reaches the preset threshold B D When , stack l layers of graph neural network models, and use batch processing method to train the stacked graph neural network model by optimizing the loss function defined in the previous article. After training, Clear About Then the reward function is calculated to obtain reward feedback, and the reward function is calculated according to the action. The probability distribution P formed by the number of connections between the extracted subgraph and other subgraphs related to the nodes within the k-th order of the target node is used to select the next subgraph, and then Save to Save to Finally, the strategies π1, π2 are optimized by formula (3).

[0100] (5) After the training is completed, the optimal order k and optimal number of layers L corresponding to each target node derived from the obtained optimal strategy are used to retrain the graph neural network with each optimal number of layers; the subgraph of the optimal order corresponding to each target node is processed by graph pooling and input into the trained graph neural network with the corresponding optimal number of layers for classification.

[0101] In the above solution provided by the embodiment of the present invention, the classification task of the target node is converted into the classification of the subgraph constructed with the target node as the center. The subgraph classification task is completed using the designed graph neural network model, and the construction of the subgraph and the design of the graph neural network model architecture are determined by the reinforcement learning method. It has the following main advantages: 1) When classifying the target node, the feature information of the node and part of the structural information of the graph are fully utilized, which is conducive to improving the accuracy of node classification. 2) The graph neural network structure designed for classification based on the reinforcement learning algorithm does not require a large amount of prior knowledge, greatly reducing the workload of designing the model, and can also ensure the accuracy of node classification.

[0102] Through the description of the above embodiments, those skilled in the art will clearly understand that the above embodiments can be implemented through software or by using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) and includes a number of instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute the methods described in the various embodiments of the present invention.

[0103] Example 2

[0104] The present invention also provides a target node classification system in graph structure data, which is mainly implemented based on the method provided in the above embodiment, such as Figure 4 As shown, the system mainly includes:

[0105] A data acquisition unit, used to acquire graph structure data consisting of a node set and an edge set;

[0106] a subgraph initialization unit for extracting neighbor nodes of a corresponding fixed order from the graph structure data with each target node as the center according to a set fixed order for the target node set to be classified, thereby forming an initial subgraph set corresponding to the target node set to be classified;

[0107] An optimal strategy search unit is used to use reinforcement learning methods combined with the initial subgraph set to search for the optimal strategy in the state space, obtain the optimal order corresponding to each target node and the optimal number of layers of the graph neural network, and each state in the state space is determined by using the features of all nodes in a single subgraph;

[0108] The node classification unit is used to extract neighbor nodes of the corresponding optimal order from the graph structure data with the corresponding target node as the center according to the corresponding optimal order for each target node to construct an optimal subgraph, and use the graph neural network with the corresponding optimal number of layers that has been trained to obtain a vector representation describing the characteristics of the optimal subgraph, and perform classification to obtain the category of each target node.

[0109] Those skilled in the art will clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0110] Example 3

[0111] The present invention also provides a processing device, such as Figure 5 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the aforementioned embodiment.

[0112] Furthermore, the processing device further includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0113] In the embodiment of the present invention, the specific types of the memory, input device, and output device are not limited; for example:

[0114] The input device can be a touch screen, image acquisition device, physical button or mouse;

[0115] The output device may be a display terminal;

[0116] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.

[0117] Example 4

[0118] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.

[0119] In the embodiments of the present invention, the computer-readable storage medium may be provided in the aforementioned processing device, for example, as a memory in the processing device. Alternatively, the computer-readable storage medium may be a USB flash drive, a removable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0120] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for classifying target nodes in graph structure data, characterized in that: include: Obtain graph structure data consisting of a node set and an edge set; Each node feature in the graph structure data is a feature vector extracted based on node information, and the node information includes: text information and / or image information of the node; For the target node set to be classified, extracting neighbor nodes of corresponding fixed orders from the graph structure data with each target node as the center according to a set fixed order to construct a subgraph, thereby forming an initial subgraph set corresponding to the target node set to be classified; Using reinforcement learning methods combined with an initial set of subgraphs, we search for the optimal strategy in the state space to obtain the optimal order and number of layers of the graph neural network for each target node. Each state in the state space is determined by the features of all nodes in a single subgraph. For each target node, the optimal subgraph is constructed by extracting the neighboring nodes of the corresponding optimal order from the graph structure data with the corresponding target node as the center according to the corresponding optimal order, and the vector representation describing the characteristics of the optimal subgraph is obtained by using the graph neural network with the corresponding optimal number of layers trained, and classification is performed to obtain the category of each target node. The method of using reinforcement learning to combine with the initial subgraph set to search for the optimal strategy in the state space and obtain the corresponding optimal order and the optimal number of layers of the graph neural network includes: Using the initial representation vectors of all subgraphs in the initial subgraph set as input, the agent is trained using reinforcement learning to obtain the optimal decision, that is, to determine the optimal subgraph order corresponding to the target node and the optimal number of layers of the graph neural network; The training process is as follows: a subgraph is randomly selected at the initial time step, and its initial representation vector is used as the state at the initial time step. After that, the transition probability is calculated based on the connection relationship between the subgraph corresponding to the previous time step and other subgraphs. The corresponding subgraph at the next time step is determined according to the transition probability, and the initial subgraph representation vector of the corresponding subgraph is used as the state of the next time step; under the state of the current time step, different actions are selected from the action space, and the corresponding reward feedback is obtained after executing different actions and the corresponding state action value is calculated. The final action is selected in combination with the strategy function, and the state, final action, reward value and the state of the next time step are combined. The state of the time step forms the record data of the current time step; the record data of multiple time steps are used to train the intelligent agent until convergence, thereby completing the training of the intelligent agent; wherein each action includes a sub-action for selecting an order value and a sub-action for selecting a layer value of the graph neural network; at the same time, in each time step, the corresponding order and number of layers are determined based on the final action, and a new subgraph of the corresponding order is extracted from the graph structure data with the target node corresponding to the corresponding time step as the center, and then the new subgraph and the corresponding number of layers are stored in the cache, and the new subgraph is used to train the graph neural network of the corresponding number of layers, and applied to the subsequent reward feedback calculation process; The method of obtaining a vector representation describing the optimal subgraph features by using a graph neural network having a corresponding optimal number of layers through training includes: The vector representation of the optimal subgraph feature corresponding to the i-th target node is described as: Among them, L represents the optimal number of layers corresponding to the i-th target node, z i Represents the vector representation of the optimal subgraph feature corresponding to the i-th target node, represents the feature vector representation of the jth node in the optimal subgraph corresponding to the i-th target node obtained by the L-th layer graph neural network, n is the number of nodes in the optimal subgraph corresponding to the i-th target node, and R(.) is the read operation; The loss function during graph neural network training is expressed as: in, Represents a set of labeled training nodes, y τ Represents the given τth node Tags, z τ It represents the vector representation of the subgraph feature corresponding to the τth node, ||Θ||2 is the L2 norm of the graph neural network parameter Θ, λ is the weight parameter, MLP is the multi-layer perceptron, and σ is the normalization function.

2. A target node classification method in graph structure data according to claim 1, characterized in that: The final action selection method is: Each action a in the current time step t t By binary action Composition, by executing the first sub-action Get the value of the order by executing the second sub-action Get the value of the number of layers of the graph neural network and select different sub-actions from the given action space With sub-actions After executing them separately in the state of the current time step, the reward feedback of each action is determined according to the classification accuracy of the subgraph corresponding to the current time step t, and the state-action value obtained after executing each action at the current time step t is determined in combination with the cumulative state-action value from the given next time step t+1 to the maximum time step T. The strategy function is used to judge whether the current time step t meets the set conditions. If so, an action is randomly selected as the final action of the current time step t. If not, the action corresponding to the maximum state-action value is selected as the final action of the current time step t.

3. The target node classification method in graph structure data according to claim 1, characterized in that: Parameter sharing and embedding caching mechanisms are used in the implementation of reinforcement learning methods; The parameter sharing mechanism includes: initializing the model to determine a layer number L max The basic graph neural network of the agent is constructed by repeatedly extracting the specified layers from the basic graph neural network model in sequence according to the output of the agent at each time step to construct a graph neural network model with the corresponding number of layers; wherein the optimal number of layers corresponding to each target node does not exceed L max ; The embedded cache mechanism includes: caching the vector representation of the new subgraph as a batch of data. When the number of cached batch data reaches the preset batch size, the reconstruction and training of the graph neural network is triggered using the corresponding number of layers and the corresponding vector representation of the new subgraph. After the training is completed, the cache space is cleared; the vector representation of the new subgraph is a set of all node features in the new subgraph.

4. A target node classification system in graph structure data, characterized in that: The method according to any one of claims 1 to 3 is implemented, and the system comprises: A data acquisition unit, used to acquire graph structure data consisting of a node set and an edge set; a subgraph initialization unit for extracting neighbor nodes of a corresponding fixed order from the graph structure data with each target node as the center according to a set fixed order for the target node set to be classified, thereby forming an initial subgraph set corresponding to the target node set to be classified; An optimal strategy search unit, which uses reinforcement learning methods combined with an initial subgraph set to search for the optimal strategy in the state space, obtaining the optimal order and the optimal number of layers of the graph neural network corresponding to each target node. Each state in the state space is determined using the features of all nodes in a single subgraph. The node classification unit is used to extract neighbor nodes of the corresponding optimal order from the graph structure data with the corresponding target node as the center according to the corresponding optimal order for each target node to construct an optimal subgraph, and use the graph neural network with the corresponding optimal number of layers that has been trained to obtain a vector representation describing the characteristics of the optimal subgraph, and perform classification to obtain the category of each target node.

5. A processing device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

6. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images