Training method of graph neural network model for community search and community search method

Abstract

The invention provides a community search graph neural network model training method and a community search method, and the method comprises the steps: inputting training graph data into an encoder of a pre-constructed graph neural network model, and outputting the initial node representation of each labeled node in the training graph data by the encoder; based on the initial node representation of each annotation node, taking optimization of context perception and global perception of the encoder as a target, training the encoder until a first training ending condition is met, and obtaining a trained encoder; freezing parameters of the trained encoder, acquiring intermediate node representations of labeling nodes by utilizing the trained encoder, inputting the intermediate node representations into a prompt generator, generating prompt vectors of the labeling nodes by the prompt generator, constructing a prediction loss function according to the prompt vectors, training the prompt generator by utilizing the prediction loss function, and outputting the trained encoder. And obtaining a trained prompt generator until a second training ending condition is met. According to the invention, the accuracy and efficiency of community search can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a training method for a graph neural network model for community search and a community search method. Background Art

[0002] Community search, a fundamental problem in graph data mining, has attracted extensive research interest due to its great potential in various real-world applications, such as recommender systems and fraud detection. Unlike community detection, which aims to partition a graph into distinct groups, community search focuses on identifying a compact subgraph that depends on a specific query node.

[0003] Current community search methods can be divided into two main categories: rule-based and learning-based. Rule-based algorithms utilize specific graph connectivity metrics—such as k-core, k-truss, k-clique, and k-ECC—along with graph traversal techniques to identify community members. While these algorithms exhibit strong performance, their reliance on predefined rules and parameters limits their flexibility, particularly when community structures vary significantly across query nodes and different types of networks.

[0004] In addition, searching for subgraphs that meet the conditions through graph traversal is a time-consuming process, which greatly affects the overall efficiency of the model. To address these problems, some studies have proposed learning-based methods that use graph neural networks to embed implicit patterns in community examples into node representations. Figure 1 As shown, these methods usually adopt a two-stage framework: (1) train a community score calculation model and determine the optimal threshold through a set of query nodes and example communities; (2) identify the community of the query node based on the score and threshold generated by the trained model.

[0005] Existing learning-based methods can be characterized by how the model is trained. Specifically, early methods directly used the query node and all the nodes in the corresponding real community to train the model in a supervised learning manner. However, these methods are highly dependent on a large number of community examples for model training, which faces challenges when labeled data is limited. To address this issue, recent researchers have introduced semi-supervised learning or unsupervised learning methods that use a small amount of labeled information or the intrinsic topological features of the graph to train the GNN encoder. Despite demonstrating good generalization capabilities, fixed embedding-based methods have difficulty capturing specific knowledge related to the community search task, and there is still much room for performance improvement.

[0006] How to comprehensively extract community knowledge from the topological structure of the graph and effectively transfer this knowledge to downstream community search tasks is a technical difficulty that needs to be overcome urgently and is crucial to improving the accuracy and efficiency of community search. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a training method for a graph neural network model for community search and a community search method, so as to comprehensively extract community knowledge from the topological structure of the graph and effectively transfer the knowledge to downstream community search tasks, thereby improving the accuracy and efficiency of community search.

[0008] In a first aspect, the present invention provides a method for training a graph neural network model for community search, the method comprising the following steps: The training graph data is input into the encoder of a pre-built graph neural network model, and the encoder outputs the initial node representation of each labeled node in the training graph data; wherein the training graph data is composed of multiple labeled nodes, multiple undirected edges connecting the labeled nodes, and the basic attributes of each labeled node. The graph neural network model includes an encoder, a hint generator, and a decoder. The encoder is used to determine the node representation of each node based on the topological structure of the graph data and the basic attributes of each node in the graph data. The hint generator is used to generate a hint vector for each node based on the cohesive characteristics of the graph data. The decoder is used to determine the community search result of the query node object based on the node representation and hint vector corresponding to the query node object. The node representation is a neighbor-aware representation. Based on the initial node representation of each labeled node, the encoder is trained with the goal of optimizing its contextual awareness and global awareness until a first training end condition is met, thereby obtaining a trained encoder. Optimizing contextual awareness includes aligning node representations with their corresponding context subgraphs and minimizing the distance between adjacent nodes in the embedding space. Optimizing global awareness includes maximizing the similarity between local node representations and the global summary and maximizing the difference between the community subgraph encoding rate and the overall graph encoding. Freeze the parameters of the trained encoder, use the trained encoder to obtain the intermediate node representation of each labeled node, input the intermediate node representation into the prompt generator, and have the prompt generator generate a prompt vector for each labeled node. Based on the prompt vector, a prediction loss function is constructed, and the prompt generator is trained using the prediction loss function until the second training end condition is met to obtain a trained prompt generator; the prediction loss function is used to measure the quality of the prompt vector generated by the prompt generator.

[0009] Optionally, based on the node representation of each labeled node, the encoder is trained to optimize its contextual awareness and global awareness, including: The first loss function is constructed with the goal of aligning node representation with its corresponding context subgraph. ; The second loss function is constructed with the second goal of minimizing the distance between adjacent nodes in the embedding space ; The third loss function is constructed with the third goal of maximizing the similarity between local node representation and global summary. ; The fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding ; By calculating the formula

[0010] Get the comprehensive loss function ;in, Represents a hyperparameter used to control the trade-off between different loss functions; Using comprehensive loss function Perform backpropagation on the encoder.

[0011] Optionally, the first loss function is constructed with the goal of achieving alignment between node representations and their corresponding context subgraphs. ,include: Get each annotation node Hop neighbor collection; from Obtain the context graph representation of each labeled node in the hop neighbor set; Construct the first loss function based on contrastive learning and context graph representation ; The first loss function The expression is:

[0012]

[0013]

[0014] in, represents the set of labeled nodes in the training graph data, Represents two different annotation nodes in the annotation node set, represents the cosine similarity, represents the central node, Represents a label node The context graph representation of represents the temperature hyperparameter, Indicates the number of hops, , Represents a label node of Hopping neighborhood features, represents the learnable weight matrix, , Represents the dimension of the feature, Indicates multi-headed attention, Representation layer normalization, , represents the number of attention layers, Represents a label node Neighbor perception representation, Represents a label node The neighbor token sequence of , represents a learnable hyperparameter, express Hop neighborhood matrix, , , , , Represents a label node There is no connection between them. Represents two different annotation nodes in the annotation node set, Represents a label node Connected between, Represents the node features of the labeled node, , Represents the basic properties of the node, represents position embedding, , Indicates the total number of nodes.

[0015] Optional, second loss function The expression is: .

[0016] Optional, third loss function The expression is:

[0017]

[0018]

[0019]

[0020] in, Indicates the number of sampling points, represents the expected value of the positive sample, Represents the score of the labeled node, which is used to quantify the local node and global graph summary The mutual information size between represents the expected value of negative samples, Indicates the The features of negative samples, represents the learnable weight matrix, represents the mean pooling operation, Represents negative samples, which are generated by the graph encoder based on the perturbed features after the original feature matrix is ​​perturbed by the destruction function.

[0021] Optionally, a fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding rate ,include: According to the node representation of each labeled node in the training graph data, the whole graph encoding rate corresponding to the training graph data is calculated; the encoding rate is used to measure the compactness of the node embedding; the expression of the whole graph encoding rate is ,in, represents the whole image coding rate, Represents the node representation of each labeled node in the training graph data, represents the identity matrix, represents the reconstruction error.

[0022] Get the adjacency matrix of each labeled node and calculate the community coding rate corresponding to the labeled node based on the adjacency matrix; the expression of the community coding rate is ,in, represents the community coding rate, represents the average degree of the labeled nodes in the training graph data, Represents a label node The member matrix of , Represents a label node The adjacent indicator vector of ; The rate reduction index is calculated based on the whole-image coding rate and the community coding rate. The rate reduction index is used to measure the difference between the whole-image coding rate and the community coding rate. The expression of the rate reduction index is: ; Construct the fourth loss function based on the rate reduction indicator .

[0023] Optional, fourth loss function The expression is: ;in, represents the hyperparameter, Used to control the compactness of node embedding, Used to balance the whole-graph encoding rate and the community encoding rate.

[0024] Optionally, the hint generator generates a hint vector for each node based on the cohesive features of the graph data, including: Clustering multiple labeled nodes to obtain multiple clusters; each cluster includes at least one labeled node, and the labeled nodes in different clusters are different from each other; Calculate the cluster center vector of each cluster, use the cluster center vector as the initial prompt vector of each labeled node in the cluster, and generate personalized prompts based on the initial prompt vector; the expression of personalized prompts is:

[0025] in, Represents a label node The prompt vector, represents the number of initial hint vectors, Indicates the Initial hint vector at the labeled node The exponential normalized weight of Indicates the An initial hint vector, Indicates the Initial hint vector at the annotation node The initial weight of By calculating the formula

[0026]

[0027] Get the node representation of the training graph data based on cohesion ; Represents the transformation function, which includes two linear layers, a batch normalization layer and a ReLU activation function. represents the parameters of the frozen trained encoder, Represents the node features after integrating personalized prompts, represents the intermediate node representation; According to the node representation , calculate the membership prediction probability between each labeled node; the membership prediction probability is used to characterize the prediction probability of being in the same community, and the expression of the membership prediction probability is:

[0028] in, Represents a given query node Mark nodes The community score of Represents the one-hot vector corresponding to the query node object, which is used to indicate the labeled nodes that belong to the same community as the query node object and the labeled nodes that do not belong to the same community. Represents a label node The cohesive node representation of Construct a prediction loss function based on the member prediction probability.

[0029] Optionally, the prediction loss function is expressed as:

[0030] in, represents the prediction score of the labeled node, represents the true labels of the pre-annotated annotation nodes.

[0031] In a second aspect, the present invention provides a community search method, which adopts the model trained by the above method, uses the model to perform community search, and obtains community search results.

[0032] The beneficial effects of the present invention are: The training method of the graph neural network model for community search provided by the present invention can model the local similarity between adjacent nodes by aligning the node representation with its corresponding context subgraph and minimizing the distance between adjacent nodes in the embedding space during the training process, and learn potential community knowledge from the graph structure information, so that the graph neural network model captures rich community knowledge, which is beneficial to improving the accuracy of community search; by maximizing the similarity between local node representation and global summary and maximizing the difference between community subgraph encoding rate and total graph encoding during the training process, it can capture long-range dependencies between nodes and learn discriminative node representations to supplement task-specific knowledge, which is beneficial to improving the accuracy of community search; at the same time, by freezing the parameters of the trained encoder and only tuning the prompt vector, the model sensitivity and memory bias caused by full parameter fine-tuning can be reduced, the model performance can be improved, and thus the efficiency of community search can be improved; in addition, when downstream label data is scarce or the distribution of test data is inconsistent with that of training data, the negative transfer problem caused by the target difference between pre-training and downstream tasks is solved, which is beneficial to improving the accuracy and efficiency of community search. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the process of traditional learning-based community search algorithm; Figure 2 This is a flowchart of a method for training a graph neural network model for community search in an embodiment of the present invention; Figure 3 This is a structural diagram of a graph neural network model for community search in an embodiment of the present invention; Figure 4 This is a comparison chart of the effects of the graph neural network model for community search provided by the present invention and other methods in an embodiment of the present invention; Figure 5aThis is a comparison chart of the training efficiency of the community search graph neural network model provided by the present invention and other methods in the comparative embodiment of the present invention; Figure 5b This is a comparison chart of the inference efficiency of the graph neural network model for community search provided by the present invention and other methods in the comparative embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to comprehensively extract community knowledge from the topological structure of the graph and effectively transfer the knowledge to the downstream community search task, thereby improving the accuracy and efficiency of community search, the present invention discloses a training method for a graph neural network model for community search and a community search method. The training method for the graph neural network model for community search can model the local similarity between adjacent nodes by aligning the node representation with its corresponding context subgraph and minimizing the distance between adjacent nodes in the embedding space during the training process, and learn the potential community knowledge from the graph structure information, so that the graph neural network model captures rich community knowledge, which is beneficial to improving the accuracy of community search; by maximizing the local node The similarity between representation and global summary, maximizing the difference between community subgraph encoding rate and total graph encoding, can capture long-range dependencies between nodes and learn discriminative node representations, supplement task-specific knowledge, and help improve the accuracy of community search; at the same time, by freezing the parameters of the trained encoder and only tuning the prompt vector, the model sensitivity and memory bias caused by full parameter fine-tuning can be reduced, the performance of the model can be improved, and thus the efficiency of community search can be improved; in addition, when downstream label data is scarce or the distribution of test data is inconsistent with that of training data, the negative transfer problem caused by the target difference between pre-training and downstream tasks is solved, significantly improving the accuracy and efficiency of community search.

[0035] The following describes the training method of the graph neural network model for community search provided by the present invention in conjunction with specific embodiments.

[0036] like Figure 2 As shown, the training method of the graph neural network model for community search in an embodiment of the present invention includes the following steps: In step 21, the training graph data is input into the encoder of the pre-built graph neural network model, and the encoder outputs the initial node representation of each labeled node in the training graph data.

[0037] It should be understood that graph data is a data organization based on a graph structure, used to represent entities (nodes) and the relationships (edges) between them. Nodes represent entities or objects, such as users, products, and locations. Nodes can have attributes (such as the user's age or the product's name); edges represent the relationships between nodes, for example, user A follows user B, and product C belongs to category D. Edges can be directed (such as a following relationship) or undirected (such as a friend relationship), and can also have weights (such as interaction frequency).

[0038] In the embodiment of the present invention, the training graph data is composed of a plurality of labeled nodes, a plurality of undirected edges connecting the labeled nodes, and the basic attributes of each labeled node. Specifically, the graph data can be represented as ,in, Represents a collection of nodes, represents the edge set, Represents the basic attribute feature matrix of the node. The goal of community search is to find a subgraph (i.e., a community) related to the query node, such that the nodes in the community have dense internal connections.

[0039] In one feasible embodiment, fraud detection requires a community search of known suspicious accounts (query nodes) in the transaction network to quickly locate high-risk communities associated with these accounts and identify fraud rings. In this embodiment, the training graph data is a labeled transaction network graph, with labeled nodes representing user accounts labeled with risk labels (risk labels indicate fraud risk or non-fraud risk, with risk information obtained and labeled from a database of solved cases). Undirected edges between labeled nodes represent transaction behavior between user accounts, and basic node attributes include registration information and transaction frequency.

[0040] In another feasible embodiment, to recommend product advertisements, a community search is performed on interested users (query nodes) in the user's social network to quickly locate user communities close to the interested users, thereby effectively recommending product advertisements. In this embodiment, the training graph data is annotated user social networks, with annotated nodes representing user accounts labeled with interest tags (interest tags indicate interest or disinterest, and interest information can be obtained and labeled from public social databases). Undirected edges between annotated nodes represent social behaviors (likes, recommendations, and favorites) between user accounts, and basic node attributes include age, gender, etc.

[0041] In addition, in other embodiments, the training method of the graph neural network model for community search disclosed in the present invention can also be used for searching disease transmission paths, building local area networks, etc.

[0042] like Figure 3 As shown, the community search graph neural network model in the embodiment of the present invention includes an encoder 301, a hint generator 302, and a decoder 303. The encoder 301 is used to determine the node representation of each node based on the topological structure of the graph data and the basic attributes of each node in the graph data. The hint generator 302 is used to generate a hint vector for each node based on the cohesive characteristics of the graph data. The decoder 303 is used to determine the community search result of the query node object based on the node representation and hint vector corresponding to the query node object.

[0043] It should be understood that the community search results represent nodes in the graph that are in the same community as the query node and nodes that are not in the same community.

[0044] The following describes a process in which an encoder determines an initial node representation of each node based on the topological structure of graph data and basic attributes of each node in the graph data in an embodiment of the present invention.

[0045] Specifically, the encoder is configured as follows: The eigenvectors of the graph’s Laplacian matrix are used to generate position embeddings for each node (this process is conventional for those skilled in the art and will not be described in detail here), and then the equation , get the node characteristics of each node ,in, represent the basic attribute and position embedding respectively.

[0046] Then, the graph convolution operator is used to calculate the neighborhood matrix of each node with different hop counts, and the neighbor token sequence of each node is assigned (representing the set of neighbor nodes of the node at different hop counts). For example, by calculating the formula Get the node Hop neighborhood matrix, where , For nodes , can be obtained from the set Extract the relevant token sequence and represent it as .

[0047] Afterwards, the nodes are aligned through a linear layer. Preprocess the sequence matrix: ;in, is a learnable weight matrix.

[0048] Finally, the neighbor token sequence is fed into a In the transformer of the layer, the node is calculated Neighbor-aware representation :

[0049]

[0050] in, and Represent multi-head self-attention and layer normalization respectively. Represents a position-wise feedforward network consisting of two linear layers and a Activation function composition. represents the central node, and the set Representation node The neighbor token of .

[0051] In an embodiment of the present invention, the neighbor-aware representation of the labeled node is used as the initial node representation to perform community search.

[0052] In another feasible embodiment, considering the difference in intimacy between a node and its neighboring nodes, a cohesion index can be used to determine the hyperparameters of each node. .

[0053] The cohesion measures the ratio of the border edges connecting the subgraph to the external nodes (the edges connecting the subgraph to the external nodes are those whose one end is in the subgraph node and the other end is in the external node, which are called border edges. The larger the ratio, the tighter the connection between the subgraph and the external nodes). In order to maintain a higher ratio of unique information of the nodes, in this embodiment, the formula is calculated.

[0054] Get the node around Cohesion of jumping subgraphs .in, Representing a subgraph The supplementary image of represents the number of edges between internal and external nodes of the subgraph, is the sum of the degrees of all nodes in the subgraph. The number of hops can be determined by grid search The value of .

[0055] Step 22 : Based on the initial node representation of each labeled node, the encoder is trained with the goal of optimizing the context awareness and global awareness of the encoder until a first training end condition is met, thereby obtaining a trained encoder.

[0056] Research has shown that the communities of most query nodes significantly overlap with their neighboring nodes. For example, in fraud detection, a single account may belong to multiple fraud communities simultaneously; different fraud communities share a large number of shared intermediate nodes or edge accounts; and fraudulent accounts form dense subgraphs through frequent transactions, with incidental connections to legitimate accounts blurring community boundaries. This phenomenon can lead to inaccurate community search results. For example, a legitimate account could be misidentified as a fraudulent account due to an inadvertent transaction with a fraudulent account and thus be included in the fraudulent account community search results.

[0057] To address situations where there is significant overlap between the query node's community and its neighboring nodes, in embodiments of the present invention, the training of the graph neural network model includes context-aware optimization, specifically encompassing two optimization objectives: aligning node representations with their corresponding context subgraphs, and minimizing the distance between adjacent nodes in the embedding space. The core idea is that connected nodes should be close to each other in the embedding space, while unconnected nodes should be far apart; nodes should be similar to their context while remaining dissimilar to the contexts of other nodes in the embedding space.

[0058] Specifically, in the embodiment of the present invention, the first loss function is constructed with the first goal of achieving alignment between node representation and its corresponding context subgraph. ; With minimizing the distance between adjacent nodes in the embedding space as the second goal, construct the second loss function , according to the first loss function and the second loss function Using the CNN to train the encoder can effectively solve the problem of reduced community search accuracy caused by the above overlap.

[0059] In a feasible embodiment, the first loss function is constructed with the first goal of achieving alignment between node representation and its corresponding context subgraph. The process is: Get each annotation node Hop neighbor collection; from Obtain the context graph representation of each labeled node in the hop neighbor set; Construct the first loss function based on contrastive learning and context graph representation ; The first loss function The expression is:

[0060]

[0061]

[0062] in, represents the set of labeled nodes in the training graph data, Represents two different annotation nodes in the annotation node set, represents the cosine similarity, represents the central node, Represents a label node The context graph representation of represents the temperature hyperparameter, Indicates the number of hops, , Represents a label node of Hopping neighborhood features, represents the learnable weight matrix, , Represents the dimension of the feature, Indicates multi-headed attention, Representation layer normalization, , represents the number of attention layers, Represents a label node Neighbor perception representation, Represents a label node The neighbor token sequence of , represents a learnable hyperparameter, express Hop neighborhood matrix, , , , , Represents a label node There is no connection between them. Represents two different annotation nodes in the annotation node set, Represents a label node Connected between, Represents the node features of the labeled node, , Represents the basic properties of the node, represents position embedding, , Indicates the total number of nodes.

[0063] The second loss function constructed The expression is: .

[0064] However, context-aware optimization mainly focuses on modeling the proximity of adjacent nodes, but fails to model the global distribution of node representations. For example: although two nodes are far apart, they may share similar local structural features and therefore may belong to the same community. Taking fraud detection as an example, fraudsters may manipulate some "clean" users to cover up their fraudulent behavior. This relationship is often not a direct contact between local adjacent nodes, which will lead to missed fraud detection. On the other hand, in addition to capturing the relationship between long-distance nodes, learning discriminative representations for nodes in different communities is crucial to improving the performance of community search. Therefore, the present invention performs global-aware optimization, including maximizing the similarity between local node representations and global summaries, and maximizing the difference between community subgraph encoding rate and total graph encoding.

[0065] Specifically, the third loss function is constructed with the third goal of maximizing the similarity between the local node representation and the global summary. ; The fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding , based on the third loss function And the fourth loss function Train the encoder of the graph neural network model.

[0066] Among them, the third loss function The expression is:

[0067]

[0068]

[0069]

[0070] in, Indicates the number of sampling points, represents the expected value of the positive sample, Represents the score of the labeled node, which is used to quantify the local node and global graph summary The mutual information size between represents the expected value of negative samples, Indicates the The features of negative samples, represents the learnable weight matrix, represents the mean pooling operation, Represents negative samples, which are generated by the graph encoder based on the perturbed features after the original feature matrix is ​​perturbed by the destruction function.

[0071] The fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding The process includes: According to the node representation of each labeled node in the training graph data, the whole graph encoding rate corresponding to the training graph data is calculated; the encoding rate is used to measure the compactness of the node embedding; the expression of the whole graph encoding rate is ,in, represents the whole image coding rate, Represents the node representation of each labeled node in the training graph data, represents the identity matrix, represents the reconstruction error.

[0072] Get the adjacency matrix of each labeled node and calculate the community coding rate corresponding to the labeled node based on the adjacency matrix; the expression of the community coding rate is ,in, represents the community coding rate, represents the average degree of the labeled nodes in the training graph data, Represents a label node The member matrix of , Represents a label node The adjacent indicator vector of ; The rate reduction index is calculated based on the whole-image coding rate and the community coding rate. The rate reduction index is used to measure the difference between the whole-image coding rate and the community coding rate. The expression of the rate reduction index is: ; Construct the fourth loss function based on the rate reduction indicator . , the fourth loss function The expression is: ;in, represents the hyperparameter, Used to control the compactness of node embedding, Used to balance the whole-graph encoding rate and the community encoding rate.

[0073] In order to extract community knowledge from the comprehensive topological structure of the graph, after constructing the first loss function, the second loss function, the third loss function, and the fourth loss function, the present invention calculates the formula

[0074] Get the comprehensive loss function ;in, Represents a hyperparameter that controls the trade-off between different loss functions.

[0075] Then use the comprehensive loss function The encoder is back-propagated until the first training end condition is met to obtain the trained encoder.

[0076] It should be noted that the first training end condition varies in different actual needs.

[0077] In a feasible embodiment, the first training end condition is configured as a comprehensive loss value corresponding to the comprehensive loss function being less than a preset comprehensive loss threshold.

[0078] In another feasible embodiment, the first training end condition is configured as the number of updates of the encoder parameters reaching a preset number of updates.

[0079] Step 23: Freeze the parameters of the trained encoder, use the trained encoder to obtain the intermediate node representation of each labeled node, input the intermediate node representation into the prompt generator, and have the prompt generator generate a prompt vector for each labeled node. A prediction loss function is constructed based on the prompt vector, and the prompt generator is trained using the prediction loss function until the second training end condition is met, thereby obtaining a trained prompt generator.

[0080] The above prediction loss function is used to measure the quality of the prompt vector generated by the prompt generator.

[0081] In the traditional pre-training and fine-tuning paradigm, graph neural network (GNN) models obtain good initial weights through unsupervised graph structure learning, and then re-adjust the model parameters based on supervised downstream tasks. However, the goal gap between the pre-training and fine-tuning stages often leads to suboptimal performance on downstream tasks. Furthermore, when labeled data is scarce, this technique is susceptible to overfitting.

[0082] To address the above issues, this paper introduces a hint tuning method, which freezes the parameters of the pre-trained graph encoder and only tunes a small number of learnable hint markers.

[0083] Specifically, the hint generator generates a hint vector for each node based on the cohesive features of the graph data, including: Clustering multiple labeled nodes to obtain multiple clusters; each cluster includes at least one labeled node, and the labeled nodes in different clusters are different from each other; Calculate the cluster center vector of each cluster, use the cluster center vector as the initial prompt vector of each labeled node in the cluster, and generate personalized prompts based on the initial prompt vector; the expression of personalized prompts is:

[0084] in, Represents a label node The prompt vector, represents the number of initial hint vectors, Indicates the Initial hint vector at the labeled node The exponential normalized weight of Indicates the An initial hint vector, Indicates the Initial hint vector at the annotation node The initial weight of By calculating the formula

[0085]

[0086] Get the node representation of the training graph data based on cohesion ; Represents the transformation function, which includes two linear layers, a batch normalization layer and a ReLU activation function. represents the parameters of the frozen trained encoder, Represents the node features after integrating personalized prompts, represents the intermediate node representation; According to the node representation , calculate the membership prediction probability between each labeled node; the membership prediction probability is used to characterize the prediction probability of being in the same community, and the expression of the membership prediction probability is:

[0087] in, Represents a given query node Mark nodes The community score of Represents the one-hot vector corresponding to the query node object, which is used to indicate the labeled nodes that belong to the same community as the query node object and the labeled nodes that do not belong to the same community. Represents a label node The cohesive node representation.

[0088] Construct a prediction loss function based on the member prediction probability.

[0089] In a feasible embodiment, the expression of the prediction loss function is:

[0090] in, represents the prediction score of the labeled node, represents the true labels of the pre-annotated annotation nodes.

[0091] In one embodiment of the present invention, the second training end condition is configured as a prediction loss value corresponding to the prediction loss function being less than a preset prediction loss threshold.

[0092] To evaluate the effectiveness of the training method for the graph neural network model for community search provided by the present invention, we focus on the community search task under two main settings: the inductive setting and the hybrid setting. In the inductive setting, all communities are divided equally into two groups. One group is used to generate training and validation samples, and the other group is retained as test samples. This setting aims to evaluate the model's ability to generalize to previously unseen communities. In the hybrid setting, the communities are also divided into two groups. The training and validation sets are from the first group, while the test set consists of all communities. This configuration better reflects real-world scenarios, where the communities to be predicted may or may not have been encountered during training.

[0093] The performance of baseline models and our model on the community search task is evaluated on multiple datasets and settings. Figure 4 The results of F1 score are shown. The present invention is GPP-CS.

[0094] like Figure 4 As shown, the method provided by the present invention generally outperforms traditional rule-based methods in inductive and hybrid settings. This highlights the effectiveness of learning community patterns directly from training data. Turning to learning-based models, it is observed that Transzero-GS consistently performs best on most datasets. In the inductive setting, Transzero-GS improves on QDGNN by 25.19% and 39.34% on the Cora and Citeseer datasets, respectively. In addition, COCLEP shows an average improvement of 12.28% compared to QD-GNN. These results indicate that unsupervised and semi-supervised methods can better generalize their performance to settings containing unseen communities than end-to-end supervised learning methods.

[0095] Next, the performance of the GPP-CS provided by the present invention was compared in depth with the baseline model. Overall, the model provided by the present invention consistently outperformed the baseline model on all datasets, including inductive and hybrid settings. Specifically, in the inductive setting, GPP-CS improved by an average of 11.82% on the best baseline Transzero-GS. This is because the context-level and global-level training strategies of the model provided by the present invention enable it to better capture the underlying community structure, thereby improving its generalization performance. In addition, this can be attributed to the fact that the hint tuning module is able to capture task-specific knowledge from the pre-trained model using the provided community samples. In the hybrid setting, GPP-CS showed a significant improvement of 16.99%-36.17% compared to the baseline model. These results emphasize the wide applicability of the GPP-CS provided by the present invention in a variety of scenarios.

[0096] In another embodiment, the performance of GPP-CS and typical learning-based baseline methods in terms of training and inference efficiency is also analyzed. Figure 5a As shown, the training efficiency of the baseline models shows that Transzero-GS outperforms all other models, while COCLE performs the worst. Specifically, Transzero-GS achieves an average acceleration of 192.45 times compared to COCLE. This difference may be attributed to the hypergraph sampling and multi-layer graph convolution modules adopted by COCLE. In contrast, COCLEP is a variant of COCLE that uses graph partitioning technology to reduce computational overhead, significantly reducing the running time on large-scale graphs. In comparison, the performance of the model provided by the present invention is between Transzero-GS and COCLEP. Compared with COCLE, the model provided by the present invention achieves an average acceleration of 29.93 times. There are two main reasons why the training efficiency of the model provided by the present invention is lower than that of Transzero-GS: first, the global perception pre-training module introduces additional sampling and computation; second, the model provided by the present invention involves additional overhead related to prompt tuning during the training process.

[0097] In terms of inference efficiency, Figure 5b As shown, the model provided by the present invention provides comparable performance on multiple datasets, achieving an average speedup of 32.53 times compared to COCLEP and a speedup of 4.24 times compared to Transzero-GS. It is worth noting that the performance improvement of GPP-CS on larger datasets is particularly significant, achieving 36.02 times and 38.46 times speedups compared to COCLEP on the Dblp and Cs datasets, respectively. These improvements are attributed to the lighter-weight graph encoder in the model provided by the present invention, which generates node embeddings and directly calculates community labels without traversing the entire graph. In addition, these advantages highlight the great potential of the model provided by the present invention in real-world applications, where inference speed is crucial to improving user experience.

[0098] In summary, the training method of the graph neural network model for community search provided by the present invention can model the local similarity between adjacent nodes by aligning the node representation with its corresponding context subgraph and minimizing the distance between adjacent nodes in the embedding space during the training process, and learn potential community knowledge from the graph structure information, so that the graph neural network model captures rich community knowledge, which is beneficial to improving the accuracy of community search; by maximizing the similarity between local node representation and global summary and maximizing the difference between community subgraph encoding rate and total graph encoding during the training process, it can capture long-range dependencies between nodes and learn discriminative node representations to supplement task-specific knowledge, which is beneficial to improving the accuracy of community search; at the same time, by freezing the parameters of the trained encoder and only tuning the prompt vector, the model sensitivity and memory bias caused by full parameter fine-tuning can be reduced, the performance of the model can be improved, and thus the efficiency of community search can be improved; in addition, when downstream label data is scarce or the distribution of test data is inconsistent with that of training data, the negative transfer problem caused by the target difference between pre-training and downstream tasks is solved, and the accuracy and efficiency of community search are significantly improved.

[0099] The present invention also discloses a community search method, which adopts a model trained by the above training method, uses the model to perform community search, and obtains community search results.

[0100] The community search method disclosed in the present invention utilizes the model trained by the above-mentioned training method and uses the model to perform community search to obtain community search results, thereby improving the accuracy and efficiency of community search.

[0101] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0102] The one or more embodiments of this application are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this application should be included in the scope of protection of this application.

Claims

1. A training method for a graph neural network model for community search, characterized in that: include: Input the training graph data into the encoder of a pre-built graph neural network model, and the encoder outputs the initial node representation of each labeled node in the training graph data; wherein the training graph data is composed of multiple labeled nodes, multiple undirected edges connecting the labeled nodes, and basic attributes of each labeled node, and the graph neural network model includes an encoder, a hint generator, and a decoder, the encoder is used to determine the node representation of each node based on the topological structure of the graph data and the basic attributes of each node in the graph data, the hint generator is used to generate a hint vector for each node based on the cohesive characteristics of the graph data, and the decoder is used to determine the community search result of the query node object based on the node representation and hint vector corresponding to the query node object, and the node representation is a neighbor-aware representation; Based on the initial node representations of the labeled nodes, the encoder is trained with the goal of optimizing contextual awareness and global awareness of the encoder until a first training end condition is satisfied, thereby obtaining a trained encoder; wherein optimizing contextual awareness includes aligning the node representation with its corresponding context subgraph and minimizing the distance between adjacent nodes in the embedding space; optimizing global awareness includes maximizing the similarity between the local node representation and the global summary and maximizing the difference between the community subgraph encoding rate and the total graph encoding; Freeze the parameters of the trained encoder, use the trained encoder to obtain the intermediate node representation of each labeled node, input the intermediate node representation into the prompt generator, generate a prompt vector for each labeled node by the prompt generator, construct a prediction loss function based on the prompt vector, and train the prompt generator using the prediction loss function until a second training end condition is met, thereby obtaining a trained prompt generator; the prediction loss function is used to measure the quality of the prompt vector generated by the prompt generator.

2. The training method according to claim 1, characterized in that The step of training the encoder based on the node representation of each labeled node with the goal of optimizing the context awareness and global awareness of the encoder comprises: The first loss function is constructed with the goal of aligning node representation with its corresponding context subgraph. ; The second loss function is constructed with the second goal of minimizing the distance between adjacent nodes in the embedding space ; The third loss function is constructed with the third goal of maximizing the similarity between local node representation and global summary. ; The fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding ; By calculating the formula Get the comprehensive loss function ;in, Represents a hyperparameter used to control the trade-off between different loss functions; Using the comprehensive loss function Backpropagation is performed on the encoder.

3. The training method according to claim 2, characterized in that The first loss function is constructed with the first goal of aligning the node representation with its corresponding context subgraph. ,include: Get each of the labeled nodes Hop neighbor collection; From the said Obtaining a context graph representation of each of the labeled nodes from a hopping neighbor set; Based on contrastive learning and the context graph representation, the first loss function is constructed ; The first loss function The expression is: in, represents the set of labeled nodes in the training graph data, represents two different annotation nodes in the annotation node set, represents the cosine similarity, represents the central node, Represents a label node The context graph representation of represents the temperature hyperparameter, Indicates the number of hops, , Represents a label node of Hopping neighborhood features, represents the learnable weight matrix, , Represents the dimension of the feature, Indicates multi-headed attention, Representation layer normalization, , represents the number of attention layers, Represents a label node Neighbor perception representation, Represents a label node The neighbor token sequence of , represents a learnable hyperparameter, express Hop neighborhood matrix, , , , , Represents a label node There is no connection between them. represents two different annotation nodes in the annotation node set, Represents a label node Connected between, represents the node characteristics of the labeled node, , Represents the basic properties of the node, represents position embedding, , Indicates the total number of nodes.

4. The training method according to claim 3, characterized in that The second loss function The expression is: 。 5. The training method according to claim 4, characterized in that The third loss function The expression is: in, Indicates the number of sampling points, represents the expected value of the positive sample, Represents the score of the labeled node, which is used to quantify the local node and global graph summary The mutual information size between represents the expected value of negative samples, Indicates the The characteristics of negative samples, represents the learnable weight matrix, represents the mean pooling operation, Represents negative samples, which are generated by the graph encoder based on the perturbed features after the original feature matrix is ​​perturbed by the destruction function.

6. The training method according to claim 5, characterized in that The fourth loss function is constructed with the fourth goal of maximizing the difference between the community subgraph encoding rate and the total graph encoding. ,include: According to the node representation of each labeled node in the training graph data, the whole graph encoding rate corresponding to the training graph data is calculated; the encoding rate is used to measure the compactness of node embedding; the expression of the whole graph encoding rate is ,in, represents the whole picture coding rate, represents the node representation of each labeled node in the training graph data, represents the identity matrix, represents the reconstruction error; Obtain the adjacency matrix of each labeled node, and calculate the community coding rate corresponding to the labeled node based on the adjacency matrix; the expression of the community coding rate is ,in, represents the community coding rate, represents the average degree of the labeled nodes in the training graph data, Represents a label node The member matrix of , Represents a label node The adjacent indicator vector of ; A rate reduction index is calculated based on the entire image coding rate and the community coding rate. The rate reduction index is used to measure the difference between the entire image coding rate and the community coding rate. The expression of the rate reduction index is: ； Constructing the fourth loss function according to the rate reduction index .

7. The training method according to claim 6, characterized in that The fourth loss function The expression is: ;in, represents the hyperparameter, Used to control the compactness of node embedding, Used to balance the entire image encoding rate and the community encoding rate.

8. The training method according to claim 7, characterized in that: The prompt generator generates a prompt vector for each node according to the cohesive feature of the graph data, including: Clustering the multiple labeled nodes to obtain multiple clusters; the clusters include at least one labeled node, and the labeled nodes in different clusters are different; Calculate the cluster center vector of each cluster, use the cluster center vector as the initial prompt vector of each labeled node in the cluster, and generate personalized prompts based on the initial prompt vector; the expression of the personalized prompt is: in, Represents a label node The prompt vector, represents the number of initial hint vectors, Indicates the Initial hint vector at the labeled node The exponential normalized weight of Indicates the An initial hint vector, Indicates the Initial hint vector at the annotation node The initial weight of By calculating the formula Obtaining the node representation of the training graph data based on cohesion ; represents a transformation function, which includes two linear layers, a batch normalization layer and a ReLU activation function, represents the frozen parameters of the trained encoder, Represents the node features after integrating personalized prompts, represents the intermediate node representation; According to the node representation , calculate the membership prediction probability between each labeled node; the membership prediction probability is used to characterize the prediction probability of being in the same community, and the expression of the membership prediction probability is: in, Represents a given query node Mark nodes The community score of Represents the one-hot vector corresponding to the query node object, which is used to indicate the labeled nodes that belong to the same community as the query node object and the labeled nodes that do not belong to the same community. Represents a label node The cohesive node representation of The prediction loss function is constructed according to the member prediction probability.

9. The training method according to claim 8, characterized in that The expression of the prediction loss function is: in, represents the prediction score of the labeled node, represents the true labels of the pre-annotated annotation nodes.

10. A community search method, characterized in that: The community search method adopts a model trained by any one of the training methods of claims 1-9, and uses the model to perform community search to obtain community search results.

Images