Graph neural network pre-training method and system based on community structure generality

Abstract

The invention relates to the technical field of artificial intelligence graph neural networks (GNN), in particular to a graph neural network pre-training method and system based on community structure generality. The pre-training method comprises the following steps: preprocessing an input graph to form a graph G (V, A) comprising a node set V and an edge set A; community structure detection is carried out on the graph G (V, A) to identify the community structure of the graph G (V, A), and a community is highly identified by adopting a modularity-based Louvain method or a label propagation LPA method; the invention provides a graph neural network pre-training method based on community structure generality. The problems that in the prior art, large-scale graph data processing efficiency is low, and dependence on labeled data is high can be effectively solved. Through preprocessing and community structure detection on an input graph, a modularity optimized Louvain method or a label propagation LPA method is adopted, and communities can be efficiently identified and divided, so that local and global structure information of the graph is reserved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence graph neural network (GNN) technology, and specifically to a graph neural network pre-training method and system based on community structure commonality. Background Art

[0002] Graph neural networks (GNNs) have shown strong capabilities in processing graph-structured data and are widely used in fields such as network analysis, biochemistry, information diffusion, and recommendation systems. However, compared with the fields of computer vision and natural language processing, the pre-training technology of graph neural networks is still relatively backward, usually showing limited performance improvements and high computational costs. This challenge stems from the uniqueness of graph data, especially its non-Euclidean structure and semantic variations across datasets, which hinder the direct migration of pre-training strategies.

[0003] In the fields of computer vision and natural language processing, pre-training technology has made significant progress. For example, models such as GPT in the field of natural language processing can capture rich contextual information through large-scale pre-training, thereby performing well in downstream tasks. However, these pre-training techniques cannot be directly applied to graph data because graph data has complex topological structures and heterogeneity, requiring specially designed pre-training strategies. The technical problem raised by this is: how to design a pre-training method based on the commonality of community structure, make full use of the node similarity within the community and the heterogeneity across communities, and improve the effect and efficiency of graph neural networks, which has become a hot spot and difficulty in current research. Summary of the invention

[0004] The present invention provides a graph neural network pre-training method and system based on community structure commonalities to solve the technical problem of how to design a pre-training method based on community structure commonalities, make full use of the node similarity within the community and the heterogeneity across communities, and improve the effect and efficiency of the graph neural network.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: On the one hand, a graph neural network pre-training method based on community structure commonality is provided, and the pre-training method comprises the following steps: Preprocess the input graph to form a graph G(V,A) containing a node set V and an edge set A; Perform community structure detection on the graph G(V,A) to identify the community structure of the graph G(V,A), and use the Louvain method based on modularity or the LPA method of label propagation to highly identify the community; A community sampler is used to extract representative subgraphs from the identified communities to increase sampling accuracy. Construct a graph encoder, optimize the parameters of the graph encoder through contrastive learning, apply the InfoNCE loss function in contrastive learning, and configure a graph convolutional network (GCN) or a graph attention network (GAT) for representation learning; The parameters of the pre-trained graph encoder are applied to the downstream node classification task to output the node classification results.

[0006] Furthermore, the community structure detection of the graph G(V,A) is performed using the Lou-vain method, which includes the following steps: In the initialization phase, each node is individually assigned to a different community, and each node constitutes an initial community; In the local movement phase, for any node i, the modularity change ΔQ after it moves to the community where the neighbor node j is located is calculated. If ΔQ>0, the node i is moved to the community where the node j is located. In the merging phase, the generated communities are treated as single nodes to construct a new graph and the above steps are repeated; The evaluation formula is used to evaluate the community division effect. The evaluation formula is: , where A ij represents the edge connection status between node i and node j, k i and j i is the degree of the node, M is the total number of edges in the graph, and δ(c i ,c j ) is the Kronecker delta function, which takes the value of 1 only when nodes i and j belong to the same community.

[0007] Furthermore, extracting representative subgraphs from the identified communities using a community sampler involves the following steps: The node with the highest degree in each community is selected as the starting point to ensure that the initial features of the subgraph contain the key nodes of the community; Use Monte Carlo Tree Search (MCTS) to gradually select neighbor nodes with the highest connection efficiency to the current subgraph, and add the neighbor nodes to the subgraph for node expansion; Regularly evaluate the representativeness of the subgraph during the expansion process, and adjust the sampling strategy by calculating the proportion of core nodes covered in the community; Multiple exploration paths are simulated in Monte Carlo Tree Search (MCTS) to find the best node set, and the sampling scheme is updated according to the exploration path to determine the sample quality of the subgraph.

[0008] Furthermore, the steps of building a graph encoder include: Feature initialization: assigning initial features to nodes, including node labels and / or degree vectors; Graph convolution transfer, in the graph convolution network (GCN), the standardized adjacency matrix is ​​used for feature transfer and aggregation. The calculation formula for feature transfer is: ,in, is an adjacency matrix containing self-loops, is the diagonal matrix of node degree, σ is the activation function, H (l) is the representation of the lth layer, W (l) is the learnable weight matrix of the lth layer; Through the multi-head attention mechanism in the graph attention network (GAT), the attention coefficient between nodes is calculated to achieve accurate feature weighted aggregation. The calculation formula of the attention coefficient is: , where a is the self-learning attention vector, ∥ represents the vector connection operation, and a T is the transpose of the attention vector a, N(i) is the set of neighbor nodes of node i, k is a node in N(i), and Wh i Indicates that the feature vector h of node i is i Through the linear transformation of the weight matrix W, a new eigenvector Wh is obtained. k Indicates that the feature vector h of node k is k Through the weight matrix W, a linear transformation is performed to obtain another new eigenvector.

[0009] Furthermore, extracting representative subgraphs from the identified communities using a community sampler also includes the following steps: Introducing a node activity index to prioritize nodes with high activity and dense connections to improve the representativeness of the subgraph; When adding new nodes, check and filter out edges or isolated low-connectivity nodes to maintain the structural integrity of the subgraph; When the sampling strategy does not achieve the expected effect, the backtracking mechanism is enabled to resample some community nodes to ensure the quality of the final subgraph.

[0010] Furthermore, the steps of applying the InfoNCE loss function in contrastive learning include: Generate corresponding positive and negative samples for each target sample. Positive samples come from the same community, and negative samples are randomly sampled from other communities. InfoNCE loss is used to calculate the similarity difference between positive samples and negative samples, which is used to maximize the similarity of positive samples and minimize the similarity of negative samples. The similarity difference is calculated as follows: , where sim represents the similarity function, using dot product or cosine similarity, z i is the representation of the target sample, z j+ and z j - Represent positive samples and negative samples respectively; The gradient descent method is used to update and adjust the parameters of the graph encoder according to the loss value to ensure that different communities in the embedding space are distinguishable.

[0011] Furthermore, the steps of applying the parameters of the pre-trained graph encoder to the downstream node classification task include: Design a soft classifier based on a fully connected layer, input node embedding features, output node category probabilities, and use them to build a classifier; The cross entropy loss function is used to train the classifier and minimize the error between the actual node category and the predicted result, which is used to train and optimize the classifier. The classifier performance is evaluated by the accuracy and F1 score of the model on the validation set, and the parameters are tuned to evaluate the classifier for classification accuracy.

[0012] Furthermore, the optimization of the InfoNCE loss function adopts an enhanced negative sample mining strategy to identify and use negative samples with high similarity to positive samples to improve the contrastive learning effect.

[0013] Furthermore, configurable hyperparameter settings are utilized in the construction of the graph encoder and node classification, allowing users to fine-tune the learning rate, number of network layers, and activation function type to adapt to the detailed characteristics of different datasets.

[0014] On the other hand, a graph neural network pre-training system based on community structure commonality is provided, which is used to implement the above graph neural network pre-training method based on community structure commonality, and the pre-training system includes: The community detection module is used to perform community detection on the input graph to identify the community structure; Community sampling module, used to sample representative subgraphs and optimize subgraphs; The graph encoding module is used to optimize the parameters of the graph encoder through contrastive learning; A downstream application module for applying the pre-trained graph encoder to node classification tasks; The graph encoding module supports scalable network structures, allowing users to customize the number of layers and node feature dimensions of the graph neural network to adapt to different graph structure learning needs.

[0015] The beneficial effects of the present invention are: The present invention proposes a graph neural network pre-training method based on the commonality of community structure, which can effectively solve the problems of low efficiency in large-scale graph data processing and strong dependence on labeled data in the prior art. By preprocessing the input graph and detecting the community structure, the Louvain method for modularity optimization or the LPA method for label propagation can be used to efficiently identify and divide the community, thereby retaining the local and global structural information of the graph. The introduction of the community sampler makes it possible to extract representative subgraphs from the community. By optimizing the sampling process using strategies such as Monte Carlo Tree Search (MCTS), the sampling accuracy and representativeness of the subgraphs are improved.

[0016] Furthermore, in the process of building the graph encoder, the parameters of the graph encoder were optimized by comparing the learning strategy and applying the InfoNCE loss function, making the model more stable in embedding learning. The graph convolutional network (GCN) and graph attention network (GAT) were used for feature representation learning to ensure that the similarity features of nodes within the community can be fully learned and perform well in the node classification task. This pre-training method not only reduces the need for a large amount of labeled data, but also improves the adaptability of the model under heterogeneous graph structures, and ultimately improves the efficiency of graph neural networks for node classification tasks in practical applications.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a neural network pre-training method in one embodiment of the present invention; Figure 2 FIG. 1 is an architecture diagram of a neural network pre-training in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The present invention provides the following preferred embodiments: Example

[0021] In order to solve the problem of insufficient extraction of graph structure information in the process of pre-training of large-scale graph data by graph neural networks, this embodiment provides a graph neural network pre-training method based on community structure commonality, which describes in detail the entire process of input graph pre-processing, community structure detection, representative subgraph sampling, graph encoder construction and downstream node classification to improve the application efficiency and prediction accuracy of the model in large-scale and complex networks, such as Figure 1 and Figure 2 As shown, the method comprises the following steps: S100, preprocessing the input graph to form a graph G(V,A) including a node set V and an edge set A.

[0022] S200, performing community structure detection on the graph G(V,A) to identify the community structure of the graph G(V,A), and using the modularity-based Louvain method or the label propagation LPA method to highly identify the community.

[0023] S300, using a community sampler to extract representative subgraphs from the identified communities to increase the accuracy of sampling.

[0024] S400. Build a graph encoder, optimize the parameters of the graph encoder through contrastive learning, apply the InfoNCE loss function in contrastive learning, and configure a graph convolutional network (GCN) or a graph attention network (GAT) for representation learning.

[0025] S500: Apply the parameters of the pre-trained graph encoder to the downstream node classification task to output the node classification result.

[0026] It is important to understand that the preprocessing of the input graph aims to structure the raw data into a node set V and an edge set A suitable for graph neural network operations, forming a graph G(V,A). In this process, a variety of data cleaning and formatting techniques can be used to make the data meet the requirements of the graph model, including removing noisy nodes and edges, and ensuring the connectivity and balance of the graph.

[0027] Furthermore, community structure detection is the basis of the entire pre-training method. This step applies the modularity (Louvain) method or the label propagation (LPA) method to identify community features in the graph. The modularity method divides communities by maximizing the edge density within the community. Specifically, each node is independently regarded as a community when the system is initialized. During the community merging process, the change in modularity after different nodes join the adjacent community is continuously evaluated, and the node affiliation is adjusted accordingly, so that the modularity gradually tends to the optimal. It should be understood that this method can effectively detect multi-level community structures and obtain community division results through iterative optimization. The label propagation method uses the frequent exchange of information between nodes to achieve community discovery, which is suitable for the rapid division of large-scale networks.

[0028] Furthermore, this embodiment comprehensively considers the stability and representativeness of the community structure, and uses a community sampler to strategically extract subgraphs from the identified community. The heuristic Monte Carlo Tree Search (MCTS) technique is used here to simulate and evaluate several paths within the community, and select neighbor nodes with the best connection efficiency. In the process of subgraph selection, effective coverage of key nodes and edges is guaranteed, and possible unexpected structural deviations are further corrected by adjusting the sampling strategy. It can be understood that the representativeness of the subgraph lies not only in covering key community nodes, but also in its efficient compression as a feature of the entire graph.

[0029] Furthermore, in the construction phase of the graph encoder, by configuring a graph convolutional network (GCN) or a graph attention network (GAT), this embodiment realizes efficient embedding of graph node features. It should be understood that the graph convolutional network (GCN) is used to extract and propagate node features through a normalized convolution operator, and the node features are aggregated using the adjacency matrix and the node degree matrix; while the graph attention network (GAT) evaluates the weights of different features between nodes through a multi-head attention mechanism to achieve weighted concentration of important information. Through these two configurations, the encoder can not only grasp the local graph structure information, but also obtain the complex relationship between nodes. At the same time, under the contrastive learning framework, the InfoNCE loss function is applied as the target to optimize the parameters of graph embedding. InfoNCE focuses on adjusting the embedding distance between positive and negative sample pairs, and enhances the expressiveness of community features by maximizing the representation similarity between nodes in the same community.

[0030] Furthermore, the parameters are applied to the downstream node classification task through the pre-training results of the graph encoder. At this point, the node features extracted by the graph encoder can be directly used as input for the construction and optimization of the classification model. Through the refined learning of node features, the model can accurately output the category information corresponding to the node. At this stage, the effect and quality of the encoder pre-training are reflected by observing the performance of the model on the validation set (such as accuracy and F1 score).

[0031] The benefit of this embodiment is that by introducing the steps of community commonality and subgraph extraction, the graph neural network can better understand and utilize the potential community information in the graph structure during the pre-training stage. Targeted feature learning not only improves the encoder's adaptability on large-scale graphs, but also effectively supports the execution of downstream classification tasks and improves performance. Through such a design, the model can be flexibly applied to graph datasets of different scales and complexities. Example

[0032] In order to solve the problem of effectively identifying community structures in large-scale graph data, this embodiment further refines the application of the Louvain method in community detection, and ensures the accuracy and stability of community division by describing in detail the processing of each step in the initialization, local movement and merging stages.

[0033] It is important to understand that in the initialization phase, each node is individually assigned to a different community, and each node is considered an initial community. This phase lays the foundation for subsequent community detection, ensuring that the microstructure in the graph can be adjusted very flexibly in the initial state. This method of initial node allocation can increase flexibility and provide sufficient space for the gradual optimization of modularity.

[0034] Furthermore, in the local movement phase, the algorithm performs a detailed check on each node i and calculates the modularity change ΔQ after it moves to the community where its neighbor node j is located. It can be understood that the positive or negative value of ΔQ directly indicates the value of the movement for the optimization of the overall community structure. If ΔQ>0, it means that the movement can enhance the internal connection strength between communities, thereby moving node i to the community where node j is located. In this process, attention should be paid to the accuracy of the calculation and the precise adjustment of the pace to explore as many node movement schemes as possible.

[0035] Furthermore, in the merging phase, the generated community is treated as a single node to reconstruct a new graph. This mechanism is repeated multiple times to gradually stabilize the community structure. It can be understood that the repeated operations in this phase are to converge to the local optimal community division, so as to achieve a long-term stable state through multiple global merging.

[0036] The community division effect is evaluated using the modularity evaluation formula, which is: , where A ij represents the edge connection status between node i and node j, k i and j i is the degree of the node, M is the total number of edges in the graph, and δ(c i ,c j ) is the Kronecker delta function, which takes the value of 1 only when nodes i and j belong to the same community. It should be understood that this formula not only considers the direct connection between nodes, but also improves the compactness within the community by maximizing the modularity of the entire graph.

[0037] It is understandable that this embodiment may also adopt the modularity Q calculation formula of the Louvain method: , where n c is the number of communities, l c is the number of edges in the community, d cis the sum of the degrees of all nodes in the community, and M is the total number of edges in the graph.

[0038] In addition, during implementation, the introduction of additional features can be considered, such as adjusting the selection priority of mobile nodes or dynamically updating the influence of node connections to further optimize the identification process of community structure. This will involve calculating the distance or weight between nodes, thereby changing the calculation method of modularity or the mobility strategy.

[0039] This embodiment not only provides specific steps for community detection, but also ensures the precision of community division in complex graphs, while reducing computational complexity and retaining the integrity of graph data and the coherence of community features. This method shows high efficiency and reliability when processing multi-level large networks, and provides effective path support for the analysis and processing of related graph data. Example

[0040] In order to solve the problem of how to extract representative subgraphs in graph data processing, this embodiment further optimizes the strategy used by the community sampler in the process of extracting subgraphs, aiming to improve the representativeness and contextual relevance of subgraphs. By reasonably designing the selection of starting nodes, using Monte Carlo Tree Search (MCTS) to expand subgraphs, and evaluating and adjusting sampling strategies, it is ensured that the extracted subgraphs can accurately and efficiently reflect the structural characteristics of the original community.

[0041] Specifically, in the initial stage of subgraph extraction, the node with the highest degree in each community is selected as the starting point. This strategy is based on the node degree as an indicator of the activity or importance of the potential community, thereby ensuring that the selected starting node has sufficient representativeness and connection breadth. It should be understood that by giving priority to the degree, it is possible to ensure that the initial features of the subgraph contain information about key nodes, thereby increasing the effectiveness and accuracy of subsequent expansion.

[0042] Furthermore, Monte Carlo Tree Search (MCTS) is applied to select neighbor nodes with the highest connection benefits to the current subgraph for node expansion. In this search strategy, the edge weights, connection density, and potential information propagation paths between adjacent nodes are considered when evaluating the connection benefits. It can be understood that MCTS makes the subgraph extraction process not only directional but also capable of dynamic adjustment by simulating multiple possible paths and selecting the expansion scheme with the maximum benefit. During the sampling process, the best nodes are continuously selected to join the subgraph to ensure that the extracted subgraph has high connectivity and representativeness in the original community structure.

[0043] Furthermore, during the expansion process, the representativeness of the subgraph is regularly evaluated, and the sampling strategy is dynamically adjusted by calculating the proportion of the subgraph covering the core nodes of the community to avoid deviation from the community center or missing key connections. It should be understood that this process involves continuous monitoring of the integrity of the subgraph structure to ensure that the node selection strategy can be corrected in time to obtain a better subgraph structure and maximize information concentration and expression.

[0044] Furthermore, through the simulation of multiple exploration paths in MCTS, we can find a node set with better coverage and matching by exploring the breadth of potential paths, and then update the sampling plan based on this information. This feedback mechanism allows the method to ensure sample quality in a more complex community structure, and through adaptive plan adjustment, it can achieve the best configuration between balancing sampling cost and expression quality.

[0045] Through this embodiment, the subgraph extraction method not only enhances the ability to capture community internal information and avoids information loss and redundancy, but also adapts to graph structures of different scales and complexities, effectively improving the accuracy of expressing community structures. This design shows high adaptability in practical applications and can provide high-quality input data for subsequent graph neural network tasks. Example

[0046] In order to solve the problem of effectively extracting node features and improving representation capabilities in graph neural networks, this embodiment further refines the construction process of the graph encoder by describing in detail multiple stages of feature initialization, graph convolution transfer, and graph attention network to maximize the model's capture and utilization of graph structure information.

[0047] In the feature initialization phase, initial feature assignment is performed on the nodes. The basis for feature assignment may include the node's label and / or degree vector. It should be understood that through the initial feature assignment, the node obtains a set of basic feature vectors, which not only contain basic connection information, but may also reflect the category or functional characteristics of the node. The initial feature assignment provides the necessary prerequisite for subsequent feature transfer and aggregation, and ensures that the model has a unified starting point during the calculation process.

[0048] Furthermore, in the graph convolution transfer stage, the embodiment adopts the feature transfer and aggregation mechanism in the graph convolution network (GCN). The standardized adjacency matrix used here is used to play a role in feature transfer, and its calculation formula is: ,in, is an adjacency matrix containing self-loops, is the diagonal matrix of node degree, σ is the activation function, H (l) is the representation of the lth layer, W (l)is the learnable weight matrix of the lth layer. It can be understood that through this normalization process, the model can effectively propagate and aggregate node features within the neighborhood, thereby more accurately capturing the structure and characteristics of the local group. The normalization operation ensures the stability of feature propagation and the uniformity of the numerical range, so that the model still maintains the effectiveness of information in the deep structure.

[0049] On this basis, the model's feature aggregation capability is enhanced through the graph attention network (GAT). The multi-head attention mechanism implements the calculation of the attention coefficient between nodes, thereby achieving accurate feature weighted aggregation. Its calculation formula is: , where a is the self-learning attention vector, ∥ represents the vector connection operation, and a T is the transpose of the attention vector a, N(i) is the set of neighbor nodes of node i, k is a node in N(i), and Wh i Indicates that the feature vector h of node i is i Through the linear transformation of the weight matrix W, a new eigenvector Wh is obtained. k Indicates that the feature vector h of node k is k Through the weight matrix W, a linear transformation is performed to obtain another new feature vector. Through the above mechanism, the model can automatically assign feature weights, thereby achieving a high degree of attention to important nodes. When dealing with noisy data or low-quality connections, this method effectively alleviates the problems of information dilution and error propagation.

[0050] Furthermore, the propagation process of the graph attention network (GAT) is as follows: , where the attention coefficient α ij Determines the contribution of node j to node i when aggregating features.

[0051] Through this embodiment, the graph encoder demonstrates higher feature representation capabilities and flexibility, and can adapt to graph structure applications of different types and scales. Under the multi-level construction of feature initialization, convolution aggregation, and attention weighting, the network model's understanding and utilization of complex graph data has been improved, providing reliable technical support for subsequent graph data analysis based on the commonality of community structures. Example

[0052] In order to solve the problem of how to efficiently extract representative subgraphs from the community structure, this embodiment further optimizes the refinement strategy of the community sampler by introducing node activity indicators, node screening mechanisms, and backtracking resampling steps, aiming to improve the structural integrity and expression richness of the subgraph.

[0053] Specifically, in the initial stage of subgraph construction, the sampler introduces a node activity indicator. This indicator can be derived from the node's historical activity record, connection strength, or the frequency of key events it has participated in. It should be understood that by giving priority to nodes with high activity and deep connection density, the sampler can enhance the representativeness of the subgraph for the original community. These active nodes usually play a key role in information dissemination and social interaction, and other nodes connected to them often have a high correlation. Therefore, selecting these nodes as the center of the subgraph can better reflect the dynamic structural characteristics of the community.

[0054] Furthermore, during the subgraph expansion process, the sampler will carefully check and filter out marginal or isolated low-connectivity nodes when adding new nodes. It is understandable that the operation at this stage is aimed at maintaining the structural integrity of the subgraph. Too many low-connectivity nodes may cause the subgraph to become sparse and lack coherence, affecting the model's ability to understand the overall structure of the community. Through such a screening mechanism, it is ensured that each newly added node is an effective supplement to the existing subgraph structure, avoiding the introduction of noise data and invalid information.

[0055] Furthermore, when the sampling strategy does not achieve the expected results, that is, when the subgraph fails to fully reflect the characteristics of the target community, the sampler enables the backtracking mechanism for resampling. This mechanism allows local adjustment and optimization of the subgraph selection process. It should be understood that the backtracking process is like the backtracking algorithm in the search problem. By revoking previous undesirable choices and exploring other possible node combinations, the quality of the final subgraph is effectively improved. This mechanism provides the sampler with adaptive capabilities, which can dynamically correct the poor structure caused by early selection bias and achieve more accurate community expression.

[0056] The benefit of this embodiment is that, by combining node activity indicators, connectivity screening and backtracking resampling strategies, the sampler not only successfully improves the representativeness and integrity of the subgraph, avoids information loss, but also ensures the structural consistency and expression potential of the subgraph. These characteristics play a role in guaranteeing the subsequent application of graph neural networks in community structure analysis, enabling them to more accurately simulate real interactions and relationship networks within the community. Through the application of the optimized sampler, the overall solution has taken a step forward in completeness, practicality and efficiency, providing a solid foundation for various practical application scenarios for processing complex network structures. Example

[0057] In order to solve the problem of how to effectively use community structure information to improve the discrimination of the embedding space during the pre-training of the graph neural network, this embodiment further refines the strategy used in contrastive learning, especially in the application of the InfoNCE loss function. By generating positive and negative sample sets corresponding to the target samples, calculating the similarity difference and updating the parameters of the graph encoder, accurate community discrimination can be achieved in the embedding space.

[0058] In this embodiment, in the initial stage of contrastive learning, corresponding positive and negative samples are generated for each target sample. The selection of positive samples is based on the attribute that they belong to the same community as the target sample. This selection strategy ensures that the similar structures and characteristics shared by the positive and target samples are reflected in the feature vector space. Negative samples are randomly sampled from other different communities. The purpose of introducing these samples is to form a contrast scale in the model and further clarify the boundaries between different communities. It should be understood that this sample generation strategy not only enhances the expression of cohesion within the community, but also improves the ability to demarcate across community categories.

[0059] Furthermore, in the process of contrastive learning, the InfoNCE loss function helps optimize the model by calculating the similarity difference between positive samples and negative samples. Its calculation method is: , where sim represents the similarity function, using dot product or cosine similarity, z i is the representation of the target sample, z j + and z j - Represent positive samples and negative samples respectively. It can be understood that by maximizing the similarity of positive samples and minimizing the similarity of negative samples, this loss function can effectively bring the representations of samples in the same community closer and pull the representations of samples in different communities farther apart, which is conducive to constructing a clear embedding space.

[0060] Furthermore, after the loss function is calculated, the parameters of the graph encoder are updated and adjusted using the gradient descent method. This process optimizes the model parameters in an iterative manner based on the feedback information of the loss value. It should be understood that the gradient descent method continuously adjusts the weight distribution within the model by reducing the loss, so that different communities show clear boundaries in the embedding space. Through optimization iterations, the node representations within the community tend to be consistent, while the node representations of different communities show sufficient differences, which enhances the generalization ability and classification performance of the model.

[0061] Through the practice of this embodiment, the use of community commonality for comparative learning effectively improves the ability of the graph neural network to distinguish between different communities and enhances its stability and accuracy when processing large-scale complex networks. This optimization strategy supports more accurate capture of information hidden in the community structure, thus providing a solid foundation for subsequent analysis and application. Example

[0062] In order to solve the problem of how to efficiently apply the graph encoder parameters obtained by pre-training to downstream node classification tasks after the graph neural network is pre-trained, this embodiment makes further refinements and optimizations in the process of designing and optimizing the node classifier. By introducing a soft classifier based on a fully connected layer, applying a cross entropy loss function, and evaluating the classifier performance through accuracy and F1 score on the validation set, efficient and accurate node classification is ensured in downstream tasks.

[0063] like Figure 2 As shown in the figure, in the specific implementation, a soft classifier based on a fully connected layer is first designed. The classifier receives the node embedding features output by the graph encoder, and then obtains the probability distribution of the node category through a series of fully connected layers with nonlinear mapping. This process requires the selection of appropriate activation functions, such as ReLU or Softmax, to ensure that the classifier can capture the subtle differences in node embedding, and then build a robust classification framework. It should be noted that Figure 2 The Softmax activation function is shown, but it should not be used as a limitation of this embodiment. The classification model is consistent with the graph encoder in the pre-training task, and the nodes are classified after passing through the fully connected layer. Classification results The calculation formula is as follows: , where Wc is the parameter, Hc is the output of the graph encoder, and Bc is the bias term. The classification loss function is: , where v represents a node in the graph, V is the set of all nodes in the graph, i represents the index of the category, M is the total number of categories, and Y i,v is the true label of node v in the i-th category. For multi-classification problems, Y is usually a one-hot encoded vector in which only one element is 1 and the rest are 0. i,v ) represents the natural logarithm of the predicted probability of node v in the i-th category, λ is the regularization parameter, and θ is the parameter of the model.

[0064] It is important to understand that the design of the fully connected layer aims to fully exploit the node features captured in the pre-training phase, which provide rich contextual information to guide classification decisions based on the commonalities of community structures.

[0065] Furthermore, the cross entropy loss function is applied to train the classifier in order to minimize the error between the true category of the node and the predicted result. The cross entropy loss function is selected for its effectiveness in dealing with multi-category classification problems. It can quantify the difference between the predicted probability distribution and the true distribution, thereby guiding the adjustment of the model weights. Through back-propagation optimization, the weight parameters of the fully connected layer are iteratively updated to gradually reduce the classification error. It can be understood that through this optimization strategy, the classifier can fit the training data more accurately, thereby improving its ability to generalize to unknown data.

[0066] Further, in terms of the performance evaluation of the classifier, the accuracy and F1 score on the validation set are used to comprehensively measure the performance of the classifier. The accuracy provides an intuitive evaluation of the overall prediction accuracy, while the F1 score takes into account the precision and recall rate, which is particularly suitable for evaluating model performance in class-imbalanced datasets. It should be understood that this evaluation method can not only identify the strengths and weaknesses of the model, but also provide a basis for further hyperparameter adjustment. If the verification result is not ideal, the embodiment will adjust key parameters such as the model architecture or learning rate to seek further performance improvement.

[0067] The benefit of this embodiment is that through accurate feature extraction, optimized training strategy and comprehensive performance evaluation, the practicality and efficiency of the graph encoder in downstream node classification tasks are ensured. Using the rich feature space constructed in the pre-training stage, the classifier can maintain excellent classification performance on different types of nodes. This optimization process provides solid theoretical and technical support for a wide range of application scenarios including social network analysis, recommendation systems, and biological network research, and realizes seamless conversion from graph structure to node category information. Example

[0068] In order to solve the problem of insufficient comparative learning effect when using the InfoNCE loss function for graph neural network pre-training, this embodiment further optimizes the negative sample selection strategy by introducing an enhanced negative sample mining strategy to identify and use negative samples with high similarity to positive samples, thereby improving the effect of comparative learning.

[0069] In this embodiment, the first thing that needs to be understood is that the selection of negative samples affects the effect of the InfoNCE loss function. Ordinary negative sample selection may lead to insufficient distinction of negative samples in the contrastive learning process, thereby affecting model performance. Therefore, this embodiment mines out negative samples that have a high similarity to positive samples by analyzing the similarity distribution in the sample space. This enhancement strategy is not a simple random sampling, but uses cluster analysis or distance measurement to screen potential "difficult negative samples". These samples are more effective in constructing a difficult decision boundary because they have similar characteristics to positive samples.

[0070] Furthermore, the negative samples optimized by this strategy can stimulate the network to perform detailed representation learning, allowing the model to better identify subtle feature differences. This process improves the ability to discriminate information by dynamically adjusting the sample weights in the loss function. It can be understood that the enhanced negative sample mining strategy accurately guides the gradient calculation, making each parameter update more targeted, thereby effectively improving the generalization ability of the model.

[0071] Through the method of this embodiment, negative samples are not only the opposite of positive samples, but also part of the enhanced learning paradigm, which provides a more solid competitiveness for subsequent pre-trained models in a variety of downstream task environments. The improved training strategy can improve the expressiveness of the model in network node recognition and enhance the application efficiency in complex graph data. Example

[0072] To address the need for flexibility in graph encoder construction and node classification of graph neural networks for different data sets, this embodiment further optimizes the hyperparameter configurability of the model, allowing users to adjust the learning rate, number of network layers, and activation function type according to specific application scenarios to adapt to diverse feature requirements.

[0073] In this embodiment, the graph encoder design is based on modularity, and users can adjust multiple parameters through a configuration interface. For example, the choice of learning rate directly affects the convergence speed and stability of gradient descent. Users can flexibly set the appropriate learning rate according to the size and complexity of the data set to achieve better training results. It should be understood that there is often a balance between model complexity and performance, and the user's freedom to choose the number of network layers allows this balance to be adjusted to cope with graph data of different sizes and structures.

[0074] Furthermore, the choice of activation function type determines the nonlinear expression ability of each layer of the network. Different activation functions, such as ReLU, Leaky ReLU, or Sigmoid, can be selected to handle nonlinear features or smooth outputs to match specific data feature distributions. This flexibility ensures the model's strong adaptability to diverse data sets and enhances its performance in various tasks.

[0075] Through the configurable hyperparameter settings provided in this embodiment, users can respond to changing data requirements with greater agility in node classification tasks. This customized solution not only improves the ease of use of the model, but also reduces the time cost of development and tuning, providing support for complex large-scale graph data analysis and applications. Example

[0076] In order to address the requirements for adaptability and efficiency of diversified graph data in the pre-training process of graph neural networks based on community structures, this embodiment provides a graph neural network pre-training system based on the commonalities of community structures, and refines and optimizes the modules of the pre-training system, providing a flexible and high-performance solution for complex graph data through the high degree of system integration.

[0077] Specifically, in the overall architecture of the system, the first is the community detection module, which performs detailed community detection on the input graph to identify and extract community structure information in the graph. The identification of community structure is a key step in the pre-training of graph neural networks, because the relationship between communities characterizes the aggregation of features in a specific group. By applying advanced graph clustering algorithms, this module can efficiently distinguish densely interconnected groups of nodes, thereby providing an accurate basis for subsequent sampling and encoding processes. It should be understood that the accuracy of community detection directly affects the representativeness of the subgraph and is the cornerstone of the performance of the entire system.

[0078] Furthermore, the community sampling module ensures accurate reproduction of the structural features of the graph by sampling representative subgraphs from the detected communities. Sampling not only focuses on the selection of nodes within the community, but also optimizes the sampled subgraphs to preserve information density and diversity. In addition, the module applies subgraph optimization techniques to reduce unnecessary computational burden and improve sampling efficiency. This strategy improves the robustness of the pre-training method by obtaining consistent feature representations under different graph structures.

[0079] Furthermore, the graph encoding module optimizes the parameters of the graph encoder through comparative learning to capture the common characteristics of the community in the graph. This module provides flexible scalability in model design, allowing users to customize the number of layers and node feature dimensions of the graph neural network according to specific learning tasks. By setting the appropriate number of encoding layers and feature space, this module can generate an embedded representation that fully reflects the complex relationship between nodes in the context of the existing community structure, allowing the model to demonstrate extremely high accuracy and generalization capabilities in downstream tasks.

[0080] Furthermore, the downstream application module is responsible for applying the pre-trained and optimized graph encoder to the node classification task. This module not only uses classification technology, but also integrates evaluation indicators and model tuning functions to ensure the maximum performance of the classifier. By accurately decoding the node embedding, the module successfully establishes a mapping from complex graph features to clear node categories, demonstrating superior recognition of new patterns on unknown data sets.

[0081] Through the systematic construction of this embodiment, a full-process solution from graph structure recognition to node feature extraction is realized. This integrated pre-training system shows excellent applicability in graph data of different scales and types, and can be dynamically adjusted to adapt to specific task requirements. Its modular structure not only provides users with a high degree of flexibility and customization possibilities, but also significantly improves the overall efficiency and effectiveness of graph neural networks in various downstream tasks through seamless collaboration between modules. In complex application scenarios such as social network analysis and bioinformatics research, the use of this system provides a new technical path for the intelligent processing of graph data.

[0082] The above embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A graph neural network pre-training method based on community structure commonality, characterized in that: The pre-training method comprises the following steps: Preprocess the input graph to form a graph G(V,A) containing a node set V and an edge set A; Performing community structure detection on the graph G(V, A) to identify the community structure of the graph G(V, A), and using a modularity-based Louvain method or a label propagation LPA method to highly identify the community; A community sampler is used to extract a representative subgraph from the identified community to increase the accuracy of sampling; Constructing a graph encoder, optimizing parameters of the graph encoder by contrastive learning, applying an InfoNCE loss function in the contrastive learning, and configuring a graph convolutional network (GCN) or a graph attention network (GAT) for representation learning; The pre-trained parameters of the graph encoder are applied to a downstream node classification task to output a node classification result.

2. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: The community structure detection of the graph G(V,A) adopts the Lou-vain method, which includes the following steps: In the initialization phase, each of the nodes is individually assigned to a different community, and each of the nodes constitutes an initial community; In the local movement stage, for any node i, the modularity change ΔQ after it moves to the community where the neighbor node j is located is calculated. If ΔQ>0, the node i is moved to the community where the node j is located; In the merging phase, the generated communities are treated as a single node to construct a new graph, and the above steps are repeated; The community division effect is evaluated using an evaluation formula, which is: , where A ij represents the edge connection status between node i and node j, k i and j i is the degree of the node, M is the total number of edges in the graph, and δ(c i ,c j ) is the Kronecker delta function, which takes the value of 1 only when the node i and the node j belong to the same community.

3. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: The step of extracting a representative subgraph from the identified community using a community sampler comprises the following steps: Selecting the node with the highest degree in each community as a starting point to ensure that the initial features of the subgraph contain key nodes of the community; Using Monte Carlo Tree Search (MCTS), gradually selecting neighbor nodes with the highest connection efficiency to the current subgraph, and adding the neighbor nodes to the subgraph for node expansion; The representativeness of the subgraph is regularly evaluated during the expansion process, and the sampling strategy is adjusted by calculating the proportion of core nodes in the covered community; In the Monte Carlo Tree Search (MCTS), multiple exploration paths are simulated to find the best node set, and a sampling scheme is updated according to the exploration paths to determine the sample quality of the subgraph.

4. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: The step of constructing a graph encoder comprises: Feature initialization, performing initial feature assignment on the node, wherein the assignment includes the label and / or degree vector of the node; Graph convolution transfer, in the graph convolutional network (GCN), a standardized adjacency matrix is ​​used for feature transfer and aggregation. The calculation formula of the feature transfer is: ,in, is an adjacency matrix containing self-loops, is the diagonal matrix of node degree, σ is the activation function, H (l) is the representation of the lth layer, W (l) is the learnable weight matrix of the lth layer; Through the multi-head attention mechanism in the graph attention network (GAT), the attention coefficient between nodes is calculated to achieve accurate feature weighted aggregation. The calculation formula of the attention coefficient is: , where a is the self-learning attention vector, ∥ represents the vector connection operation, and a T is the transpose of the attention vector a, N(i) is the set of neighbor nodes of node i, k is a node in N(i), and Wh i Indicates that the feature vector h of node i is i Through the linear transformation of the weight matrix W, a new eigenvector Wh is obtained. k Indicates that the feature vector h of node k is k Through the weight matrix W, a linear transformation is performed to obtain another new eigenvector.

5. The graph neural network pre-training method based on community structure commonality according to claim 3, characterized in that: The step of extracting a representative subgraph from the identified community using a community sampler further comprises the following steps: Introducing a node activity index to give priority to nodes with high activity and deep connection density to improve the representativeness of the subgraph; When adding new nodes, check and filter out edges or isolated low-connectivity nodes to maintain the structural integrity of the subgraph; When the sampling strategy does not achieve the expected effect, a backtracking mechanism is enabled to resample some community nodes to ensure the quality of the final subgraph.

6. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: The step of applying the InfoNCE loss function in the contrastive learning comprises: Generate corresponding positive samples and negative samples for each target sample, where the positive samples come from the same community and the negative samples are randomly sampled from other communities; The InfoNCE loss is used to calculate the similarity difference between the positive sample and the negative sample, so as to maximize the similarity of the positive sample and minimize the similarity of the negative sample. The similarity difference is calculated as follows: , where sim represents the similarity function, using dot product or cosine similarity, z i is the representation of the target sample, z j + and z j - Represent positive samples and negative samples respectively; The parameters of the graph encoder are updated and adjusted according to the loss value using the gradient descent method to ensure that different communities in the embedding space are distinguishable.

7. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: The step of applying the parameters of the graph encoder obtained by pre-training to the downstream node classification task comprises: Design a soft classifier based on a fully connected layer, input node embedding features, output node category probabilities, and use them to build a classifier; The cross entropy loss function is applied to train the classifier to minimize the error between the actual node category and the predicted result, so as to train and optimize the classifier; The classifier performance was evaluated by the accuracy and F1 score of the model on the validation set, and parameter tuning was performed to evaluate the classification accuracy of the classifier.

8. The graph neural network pre-training method based on community structure commonality according to claim 6, characterized in that: The optimization of the InfoNCE loss function adopts an enhanced negative sample mining strategy to identify and use the negative samples that have a high similarity with the positive samples to improve the contrastive learning effect.

9. The graph neural network pre-training method based on community structure commonality according to claim 1, characterized in that: Configurable hyperparameter settings are utilized in the construction of the graph encoder and node classification, allowing users to fine-tune the learning rate, number of network layers, and activation function type to adapt to the detailed characteristics of different datasets.

10. A graph neural network pre-training system based on community structure commonality, characterized in that: The pre-training system is used to implement the graph neural network pre-training method based on community structure commonality according to any one of claims 1 to 9, and the pre-training system includes: The community detection module is used to perform community detection on the input graph to identify the community structure; A community sampling module for sampling representative subgraphs and optimizing said subgraphs; A graph encoding module, used for optimizing the parameters of the graph encoder by contrastive learning; A downstream application module for applying the pre-trained graph encoder to a node classification task; The graph encoding module supports a scalable network structure, allowing users to customize the number of layers and node feature dimensions of the graph neural network to adapt to different graph structure learning needs.

Images