Large-scale graph data-oriented clustering method and system

Abstract

The invention provides a large-scale graph data-oriented clustering method, which belongs to the field of artificial intelligence and big data processing, and comprises the following steps: acquiring graph data, and carrying out standardized processing; randomly sampling seed nodes, calculating the transition probability of each jump to a neighbor node, and combining all high-frequency nodes accessed by the random walk according to the random walk to obtain a set; calculating the transition probability of each node skipping to a neighbor node, carrying out random walk, counting the total number of times of the nodes appearing in the same walk sequence, and calculating a local structure connection affinity matrix; calculating a connection confidence coefficient weight, and training the graph neural network; inputting all nodes of the whole graph to be clustered into the trained graph neural network; the invention further provides a clustering system. The problems of semantic drift, relatively low robustness and relatively high calculation overhead are solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and big data processing technology, and in particular to clustering methods and systems for large-scale graph data. Background Technology

[0002] With the rapid development of information technology, graph data has become an important tool for describing complex systems in the real world, and is widely used in social networks, citation networks, bioinformatics, and the Industrial Internet of Things. Graph clustering, as a fundamental task for analyzing graph data, aims to divide the nodes in the graph into several unrelated clusters, such that nodes within a cluster are tightly connected and have similar attributes, while nodes between clusters are sparsely connected.

[0003] In recent years, deep clustering methods based on graph neural networks have attracted widespread attention due to their powerful feature extraction capabilities. For example, Chinese invention patent application CN116450911A, entitled "An Attribute Graph Clustering Method and System," integrates fuzzy clustering objectives into graph convolutional networks to complete attribute graph clustering tasks. Existing graph clustering techniques typically utilize contrastive learning frameworks, constructing positive and negative sample pairs through data augmentation or random sampling to encourage the model to learn discriminative node representations. However, existing sampling-based or augmentation-based graph clustering methods still suffer from the following drawbacks when processing large-scale complex graph data:

[0004] (1) The sampling strategy lacks semantic constraints and is prone to semantic drift: Most existing random walk sampling strategies only jump based on the topological structure of the graph. In graph data with heterogeneity, pure structural sampling is prone to classifying nodes with completely different features into the same sequence, introducing noise signals and causing the cluster boundaries to be blurred.

[0005] (2) The loss function lacks discriminative power: Existing contrastive learning frameworks usually treat all sample pairs equally. However, the reliability of connections in graph structures varies. Ignoring the differences in connection strength and forcing the model to bring strong and weak connections closer together with equal weights can lead to overfitting when dealing with blurred edge nodes, reducing the robustness of the algorithm.

[0006] (3) Scalability bottleneck of large-scale data: Traditional spectral clustering or graph neural network methods based on the whole graph usually require eigenvalue decomposition or matrix multiplication of the adjacency matrix of the whole graph. Its space and time complexity are usually proportional to the square or even the cube of the number of nodes. When the number of graph nodes reaches millions, this computational overhead is beyond the capacity of existing hardware, making it impossible for the algorithm to be implemented on large-scale datasets.

[0007] Therefore, there is an urgent need for a scalable large-scale graph data clustering method that can effectively integrate attributes and structures to solve the heterogeneity problem, and can adaptively adjust learning weights based on connectivity confidence. Summary of the Invention

[0008] The technical problem to be solved by this invention is: how to solve the problems of semantic drift, low robustness and large computational cost of existing sampling or augmentation-based graph clustering methods when processing large-scale complex graph data.

[0009] This invention solves the above-mentioned technical problems through the following technical solution: a clustering method for large-scale graph data, comprising: Obtain the topological structure information and node attribute feature matrix of the graph data, and perform standardization processing on the node attribute feature matrix to obtain the standardized feature matrix. Randomly sample multiple nodes from the entire graph's node set as seed nodes. Calculate each seed node Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node Starting with a random walk, we visit high-frequency nodes to obtain local community sets. ; Calculate the local community set Each node Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times they appear in the same walk sequence. and Calculate the local structural connectivity affinity matrix based on the total number of iterations; Based on the local structural connection affinity matrix, the connection confidence weight of any node pair in the local community set is calculated. The connection confidence weight is introduced into the contrastive loss to obtain the weighted contrastive loss. The nodes in the local community set and the local subgraph structure are input into the graph neural network. The graph neural network is trained based on the weighted contrastive loss to obtain the trained graph neural network. Input all nodes of the graph to be clustered into the trained graph neural network, obtain the global embedding representation matrix, and output the clustering result after dividing the global embedding representation matrix.

[0010] This invention first performs attribute-guided random walk sampling on graph data to construct a mini-batch node set containing rich semantic information—a local community set. The method effectively filters out heterogeneous noise, and the size of the local community set is much smaller than the overall graph size, resulting in low computational overhead. Next, a walk is performed again within the generated mini-batch node set to collect co-occurrence information and calculate the structural connection affinity matrix within the local area. Subsequently, this affinity matrix is ​​used to calculate the connection confidence weights, which are then introduced into the contrastive learning loss function of the graph neural network for dynamic weighted training of sample pairs with different reliability. Finally, the trained graph neural network is used to calculate the embedding representation of all nodes in the graph, and the final clustering result is obtained by partitioning the global embedding representation matrix. The attribute-guided sampling strategy filters out structurally connected but semantically conflicting nodes at the data input stage, preventing semantic drift and effectively suppressing heterogeneous noise, ensuring the purity of the training data. The confidence weighting mechanism enables the model to distinguish between core connections and edge connections, giving higher-reliability samples greater learning weights, thus improving clustering robustness.

[0011] Preferably, each seed node Jump to neighbor node transition probability for:

[0012] in, For nodes The set of neighboring nodes, hour, , hour, , Represents a node Attributes and characteristics of neighboring nodes Deviation between attribute characteristics, To control the hyperparameters of attribute sensitivity, This is the normalization constant.

[0013] This invention introduces a feature distance constraint in the random walk starting from each seed node. The transition probability is corrected by using the attribute feature distance between nodes. The walk path tends to visit homogeneous nodes with similar attributes to the seed node, ensuring that the generated local community set is a high-purity set.

[0014] Preferably, the method for determining high-frequency nodes is: statistically analyzing data from the seed node. The total number of times each node is visited in all the random walk sequences is used to sort the nodes from highest to lowest total number of visits. The top K nodes are selected as high-frequency nodes, where K is a preset positive integer.

[0015] Preferably, the elements in the local structural connection affinity matrix are nodes. With nodes The frequency of observations occurring in the actual walk path minus the frequency of nodes in the random case. With nodes Simultaneous occurrence of theoretical background frequency. This allows for accurate measurement of the salience of the connection.

[0016] Preferably, the formula for calculating the elements in the local structural connectivity affinity matrix is ​​as follows:

[0017] in, Represents a node With nodes Connection affinity, , , Represents a node With nodes The total number of times they appear in the same walk sequence. Represents a node The total number of times it appears in all walk sequences in the current batch. Represents a node The total number of times it appears in all walk sequences in the current batch. This represents the sum of the lengths of all walk sequences generated within the current batch.

[0018] Preferably, when the node With nodes Connection affinity At that time, node With nodes Being identified as a positive sample pair indicates a significant structural correlation between the two, when the node With nodes Connection affinity At that time, node With nodes Being identified as a negative sample pair indicates that the association between the two is weaker than the random level or that they are each other's noise.

[0019] Preferably, a monotonic mapping function is used to map the absolute values ​​of the elements in the local structural connection affinity matrix to weights, thus obtaining the weights for any node pair. Connection confidence weights .

[0020] Preferably, when the monotonic mapping function is a linear normalized function, any node pair Connection confidence weights The calculation method is as follows:

[0021] in, Represents the local structural connectivity affinity matrix Middle elements The absolute value, , These represent the local structural connectivity affinity matrices of the current batch. The maximum and minimum absolute values.

[0022] Preferred weighted comparison loss for:

[0023] in, , These represent nodes in the local community set. ,node ,node The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Represents the set of positive samples. This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, , Represents a node With nodes Connection confidence weights Represents a node With nodes Connection confidence weights This refers to the temperature parameter.

[0024] This invention introduces connection confidence weights into the contrastive loss function. By dynamically weighting both the bringing-in effect of positive sample pairs and the repulsion effect of negative sample pairs, higher learning weights can be assigned to high-reliability samples, which helps improve clustering robustness and enhances the model's adaptability to complex graph structures.

[0025] This invention also provides a clustering system for large-scale graph data, comprising: The graph data preprocessing module is used to obtain the topological structure information and node attribute feature matrix of the graph data, and to standardize the node attribute feature matrix to obtain the standardized feature matrix. The batch construction module based on attribute-guided walk is used to randomly sample multiple nodes from the full graph node set as seed nodes. Calculate each seed node Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node Starting with a random walk, we visit high-frequency nodes to obtain local community sets. ; The affinity matrix calculation module based on batch walks is used to calculate local community sets. Each node Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times they appear in the same walk sequence. and Calculate the local structural connectivity affinity matrix based on the total number of iterations; The confidence-weighted contrastive learning module is used to calculate the connection confidence weight of any node pair in the local community set based on the connection affinity matrix of the local structure. The connection confidence weight is introduced into the contrastive loss to obtain the weighted contrastive loss. The nodes in the local community set and the local subgraph structure are input into the graph neural network, and the graph neural network is trained based on the weighted contrastive loss to obtain the trained graph neural network. The inference module is used to input all nodes of the full graph to be clustered into the trained graph neural network, obtain the global embedding representation matrix, and output the clustering result after dividing the global embedding representation matrix. Attached Figure Description

[0026] Figure 1 This is a flowchart of the clustering method for large-scale graph data provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a clustering system for large-scale graph data provided in Embodiment 2 of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Example 1 like Figure 1As shown, this embodiment provides a clustering method for large-scale graph data. The graph data processed in this invention is a large-scale network with clear physical / business meaning, such as social networks, protein-protein interaction networks, or transportation networks. Based on this type of data, the following steps are performed: Step 1: Obtain the topological structure information and node attribute feature matrix of the graph data, and standardize the node attribute feature matrix to obtain the standardized feature matrix.

[0029] Suppose the input graph data ,in, Represents the set of nodes in the entire graph. Represents the set of edges in the entire graph. Represents the node attribute feature matrix. , Indicates the number of nodes. For feature dimensions.

[0030] To eliminate differences in the units and value ranges of different attribute features and prevent features with larger values ​​from dominating subsequent distance calculations, this invention modifies the node attribute feature matrix. The standardization process is performed to obtain the standardized feature matrix. Standardized feature matrix Middle elements The calculation formula is: (1) in, Represents the standardized feature matrix The Middle The node at the th The values ​​in each feature dimension Represents the node attribute feature matrix The Middle The node at the th The original values ​​in each feature dimension Represents the node attribute feature matrix The Middle The mean of all node values ​​across each feature dimension. Represents the node attribute feature matrix The Middle The standard deviation of all node values ​​across each feature dimension. (This refers to the node attribute feature matrix.) After standardization, the standardized feature matrix Each feature column satisfies a distribution with a mean of 0 and a standard deviation of 1, providing a unified metric space for subsequent Euclidean distance-based sampling.

[0031] Step 2: Randomly sample from the entire graph node set One node serves as the seed node. , Multiple seed nodes Constructing a seed node set ,Right now .

[0032] For each seed node An attribute-guided random walk is performed to explore potential homogeneous neighborhoods for each seed node. The specific method for performing an attribute-guided random walk is as follows: For each seed node... Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node By starting a random walk and visiting high-frequency nodes, a local community set is obtained.

[0033] Each seed node Jump to neighbor node transition probability The calculation method is as follows: (2) in, Represents a node The set of neighboring nodes, Represents a node Belongs to node The set of neighboring nodes, i.e., nodes For nodes The neighboring nodes, when the node Belongs to node When the set of neighboring nodes is obtained, i.e., the node... For nodes When choosing a neighboring node, select the function. Equal to 1, when node Not a node When the set of neighboring nodes is obtained, i.e., the node... Not a node When choosing a neighboring node, select the function. It equals 0. Represents a node The corresponding standardized feature matrix, Representing neighboring nodes The corresponding standardized feature matrix, The calculation is for nodes The corresponding standardized feature matrix and neighbor nodes The squared Euclidean distance of the corresponding standardized feature matrix represents the node. Attributes and characteristics of neighboring nodes Deviation between attribute characteristics, To control the hyperparameters of attribute sensitivity, This is the normalization constant.

[0034] High-frequency nodes refer to the K nodes with the highest total number of visits in each walk sequence, where K is a preset positive integer. This filters out marginal nodes that are only visited occasionally, retaining core nodes that are tightly coupled with the seed node. This invention introduces feature distance constraints into the random walks starting from each seed node. The walk path tends to visit homogeneous nodes with similar attributes to the seed node, ensuring that the generated local community set is a high-purity set. The local community set is dynamically generated using local walks, containing rich semantic information. The size of the local community set is much smaller than the size of the entire graph. The local community set is a small batch of nodes, which avoids loading the entire graph into GPU memory.

[0035] The attribute-guided sampling strategy filters out structurally connected but semantically conflicting nodes at the data input end, preventing semantic drift and effectively suppressing mismatched noise, thus ensuring the purity of the training data.

[0036] Step 3: Calculate the value of each node in the local community set. Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times a structure appears in the same walk sequence is used to calculate the local structural connectivity matrix.

[0037] Each node Jump to neighbor node transition probability The calculation method is the same as that of formula (2), that is: (3) Among them, nodes It belongs to a local community collection. Represents a node The set of neighboring nodes, Represents a node Belongs to node The set of neighboring nodes, i.e., nodes For nodes The neighboring nodes, when the node Belongs to node When the set of neighboring nodes is obtained, i.e., the node... For nodes When choosing a neighboring node, select the function. Equal to 1, when node Not a node When the set of neighboring nodes is obtained, i.e., the node... Not a node When choosing a neighboring node, select the function. It equals 0. Represents a node The corresponding standardized feature matrix, Representing neighboring nodes The corresponding standardized feature matrix, The calculation is for nodes The corresponding standardized feature matrix and neighbor nodes The squared Euclidean distance of the corresponding standardized feature matrix represents the node. Attributes and characteristics of neighboring nodes Deviation between attribute characteristics, To control the hyperparameters of attribute sensitivity, This is the normalization constant.

[0038] Step 3 will extract each node from the local community set. The starting range of the walk is limited to the local community set generated in step 2, by visiting each node in the local community set. Perform the attribute-guided random walk again and count the nodes. With other nodes in the local community set Co-occurrence of nodes and , This represents a local community set. This invention counts nodes. With nodes The total number of times a node appears in the same walk sequence, as long as the node With nodes If they appear in the same walk sequence, it is considered a co-occurrence.

[0039] The elements in the local structural connection affinity matrix are nodes. With nodes The frequency of observations occurring in the actual walk path minus the frequency of nodes in the random case. With nodes Simultaneous occurrence of theoretical background frequency. Definition. Local structural connectivity affinity matrix The dimension is To accurately measure the significance of the connection, the observed co-occurrence frequency is subtracted from the random background frequency for calculation. elements in : (4) in, Represents a node With nodes Connection affinity, when At that time, node With nodes Being classified as a positive sample pair indicates a significant structural association between the two. At that time, node With nodes A pair of samples is considered a negative sample if the correlation between them is weaker than random or if they are each other's noise. Node ,node This indicates that only the relationships between nodes within the current batch (local community set) are calculated. Represents a node With nodes The total number of times they appear in the same walk sequence. Represents a node The total number of times it appears in all walk sequences in the current batch (local community set). Represents a node The total number of times it appears in all walk sequences in the current batch (local community set). This represents the sum of the lengths of all walk sequences generated in the current batch, i.e., the total number of walks.

[0040] The first term to the right of the equal sign in formula (4) Reflects the nodes With nodes The second term refers to the observed frequency along the actual walking path. Reflects the nodes under random conditions With nodes Simultaneously occurring theoretical background frequency.

[0041] Step 4: Connecting affinity matrices based on local structure Calculate the connection confidence weights of any pair of nodes in the local community set. Introducing connection confidence weights into the contrastive loss The graph neural network (GNN) is trained by inputting nodes from the local community set and the local subgraph structure into the encoder. The goal is to minimize the weighted contrastive loss. Mini-batch stochastic gradient descent is used to train the network, and the trained GNN is obtained when the weighted contrastive loss is minimized. This invention employs existing deep graph neural networks, such as GraphSAGE (Graph Sample and Aggregated) or Graph Convolutional Network (GCN), by inputting nodes from the local community set and the local subgraph structure into the encoder of the graph neural network. This allows us to obtain low-dimensional embedding representations of nodes. These are the parameters for the encoder.

[0042] This invention uses a monotonic mapping function to connect the local structure to the affinity matrix. The absolute value of each element is mapped to a weight, resulting in a pair of nodes. Connection confidence weights The monotonic mapping function can be a linear normalization function, a sigmoid function, a logarithmic function, or a piecewise function. This embodiment uses a linear normalization function. Connect the local structures to the affinity matrix The absolute value of an element is mapped to an interval. The specific calculation formula is as follows: (5) in, Represents the local structural connectivity affinity matrix Middle elements The absolute value, , These represent the local structural connectivity affinity matrices of the current batch. Maximum and minimum absolute values. Calculated connection confidence weights. The larger the value, the more likely it is to be a node. With nodes The higher the confidence level of the connection relationship between them.

[0043] Introducing connection confidence weights into the contrastive loss function The effect of bringing positive sample pairs closer together and the effect of repelling negative sample pairs are dynamically weighted simultaneously to obtain a mini-batch weighted contrastive loss. : (6) in, Nodes in a local community set The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Nodes in a local community set The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Nodes in a local community set The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Represents the set of positive samples, and the nodes in the set of positive samples. With nodes Connection affinity Greater than 0, This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, , Represents a node With nodes Connection confidence weights Represents a node With nodes Connection confidence weights This refers to the temperature parameter.

[0044] The confidence weighting mechanism enables the model to distinguish between core connections and edge connections, giving higher learning weights to high-reliability samples, which helps improve clustering robustness and enhances the model's adaptability to complex graph structures.

[0045] Step 5: After the graph neural network has been trained and converged, a trained graph neural network is obtained. All nodes of the entire graph to be clustered are input into the trained graph neural network to obtain the global embedding representation matrix. The K-Means algorithm is used to partition the global embedding representation matrix, and the clustering label of each node is output, thus obtaining the clustering result.

[0046] This invention first performs attribute-guided random walk sampling on graph data, using the attribute feature distance between nodes to correct the transition probability and construct a mini-batch node set containing rich semantic information, effectively filtering out mismatch noise. Next, a walk is performed again within the generated mini-batch node set to collect co-occurrence information and calculate the structural connectivity affinity matrix within a local range. Subsequently, this affinity matrix is ​​used to calculate the connectivity confidence weights and introduced into the contrastive learning loss function of the graph neural network, dynamically weighting training on sample pairs with different reliability. Finally, the trained graph encoder is used to calculate the embedding representation of all nodes in the graph, and the final clustering result is output through a clustering algorithm (such as K-Means). This invention effectively solves the semantic drift problem caused by mismatch in large-scale graph clustering and significantly improves the robustness of the graph neural network to edge nodes through confidence weighting. The entire algorithm is based on sampling and mini-batch training, avoiding full-graph matrix operations and enabling efficient processing of large-scale graph data.

[0047] Example 2 See Figure 2 This embodiment provides a clustering system for large-scale graph data, including: The graph data preprocessing module is used to obtain the topological structure information and node attribute feature matrix of the graph data, and to standardize the node attribute feature matrix to obtain the standardized feature matrix.

[0048] The batch construction module based on attribute-guided walk is used to randomly sample multiple nodes from the full graph node set as seed nodes. Calculate each seed node Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node Starting with a random walk, we visit high-frequency nodes to obtain local community sets. .

[0049] Each seed node Jump to neighbor node transition probability for:

[0050] in, For nodes The set of neighboring nodes, hour, , hour, , Represents a node Attributes and characteristics of neighboring nodes Deviation between attribute characteristics, To control the hyperparameters of attribute sensitivity, This is the normalization constant.

[0051] High-frequency nodes are the K nodes that are visited the most in each walk sequence. High-frequency nodes are determined by counting the number of visits from the seed node. The total number of times each node is visited in all the random walk sequences is used to sort the nodes from highest to lowest total number of visits. The top K nodes are selected as high-frequency nodes, where K is a preset positive integer.

[0052] The affinity matrix calculation module based on batch walks is used to calculate local community sets. Each node Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times they appear in the same walk sequence. and The local structural connectivity affinity matrix is ​​calculated based on the total number of iterations.

[0053] The elements in the local structural connection affinity matrix are nodes. With nodes The frequency of observations occurring in the actual walk path minus the frequency of nodes in the random case. With nodes Simultaneously occurring theoretical background frequency.

[0054] The formula for calculating the elements in the local structural connectivity affinity matrix is:

[0055] in, Represents a node With nodes Connection affinity, , , Represents a node With nodes The total number of times they appear in the same walk sequence. Represents a node The total number of times it appears in all walk sequences in the current batch. Represents a node The total number of times it appears in all walk sequences in the current batch. This represents the sum of the lengths of all walk sequences generated within the current batch. When a node... With nodes Connection affinity At that time, node With nodes Being identified as a positive sample pair indicates a significant structural correlation between the two, when the node With nodes Connection affinity At that time, node With nodes Being identified as a negative sample pair indicates that the association between the two is weaker than the random level or that they are each other's noise.

[0056] The confidence-weighted contrastive learning module is used to calculate the connection confidence weight of any pair of nodes in the local community set based on the connection affinity matrix of the local structure. The connection confidence weight is introduced into the contrastive loss to obtain the weighted contrastive loss. The nodes in the local community set and the local subgraph structure are input into the graph neural network, and the graph neural network is trained based on the weighted contrastive loss to obtain the trained graph neural network.

[0057] This invention employs a monotonic mapping function to map the absolute values ​​of elements in the local structural connection affinity matrix to weights, thereby obtaining arbitrary node pairs. Connection confidence weights .

[0058] When the monotonic mapping function is a linear normalized function, any node pair Connection confidence weights The calculation method is as follows:

[0059] in, Represents the local structural connectivity affinity matrix Middle elements The absolute value, , These represent the local structural connectivity affinity matrices of the current batch. The maximum and minimum absolute values.

[0060] Weighted comparison loss for:

[0061] in, , These represent nodes in the local community set. ,node ,node The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Represents the set of positive samples. This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, , Represents a node With nodes Connection confidence weights Represents a node With nodes Connection confidence weights This refers to the temperature parameter.

[0062] The inference module is used to input all nodes of the full graph to be clustered into the trained graph neural network, obtain the global embedding representation matrix, and output the clustering result after dividing the global embedding representation matrix.

[0063] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A clustering method for large-scale graph data, characterized by: include: Obtain the topological structure information and node attribute feature matrix of the graph data, and perform standardization processing on the node attribute feature matrix to obtain the standardized feature matrix. Randomly sample multiple nodes from the entire graph's node set as seed nodes. Calculate each seed node Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node Starting with a random walk, we visit high-frequency nodes to obtain local community sets. ; Calculate the local community set Each node Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times they appear in the same walk sequence. and Calculate the local structural connectivity affinity matrix based on the total number of iterations; Based on the local structural connection affinity matrix, the connection confidence weight of any node pair in the local community set is calculated. The connection confidence weight is introduced into the contrastive loss to obtain the weighted contrastive loss. The nodes in the local community set and the local subgraph structure are input into the graph neural network. The graph neural network is trained based on the weighted contrastive loss to obtain the trained graph neural network. Input all nodes of the graph to be clustered into the trained graph neural network, obtain the global embedding representation matrix, and output the clustering result after dividing the global embedding representation matrix.

2. The clustering method for large-scale graph data according to claim 1, characterized in that: Each seed node Jump to neighbor node transition probability for: in, For nodes The set of neighboring nodes, hour, , hour, , Represents a node Attributes and characteristics of neighboring nodes Deviation between attribute characteristics, To control the hyperparameters of attribute sensitivity, This is the normalization constant.

3. The clustering method for large-scale graph data according to claim 1, characterized in that: The high-frequency nodes are determined by: statistically analyzing the seed nodes. The total number of times each node is visited in all the random walk sequences is used to sort the nodes from highest to lowest total number of visits. The top K nodes are selected as high-frequency nodes, where K is a preset positive integer.

4. The clustering method for large-scale graph data according to claim 1, characterized in that: The elements in the local structural connection affinity matrix are nodes. With nodes The frequency of observations occurring in the actual walk path minus the frequency of nodes in the random case. With nodes Simultaneously occurring theoretical background frequency.

5. The clustering method for large-scale graph data according to claim 1, characterized in that: The formula for calculating the elements in the local structural connectivity affinity matrix is: in, Represents a node With nodes Connection affinity, , , Represents a node With nodes The total number of times they appear in the same walk sequence. Represents a node The total number of times it appears in all walk sequences in the current batch. Represents a node The total number of times it appears in all walk sequences in the current batch. This represents the sum of the lengths of all walk sequences generated within the current batch.

6. The clustering method for large-scale graph data according to claim 5, characterized in that: When node With nodes Connection affinity At that time, node With nodes Being identified as a positive sample pair indicates a significant structural correlation between the two, when the node With nodes Connection affinity At that time, node With nodes Being identified as a negative sample pair indicates that the association between the two is weaker than the random level or that they are each other's noise.

7. The clustering method for large-scale graph data according to claim 1, characterized in that: A monotonic mapping function is used to map the absolute values ​​of the elements in the local structural connection affinity matrix to weights, thus obtaining the weights for any node pair. Connection confidence weight .

8. The clustering method for large-scale graph data according to claim 7, characterized in that: When the monotonic mapping function is a linear normalized function, any node pair Connection confidence weights The calculation method is as follows: in, Represents the local structural connectivity affinity matrix medium elements The absolute value, , These represent the local structural connectivity affinity matrices of the current batch. The maximum and minimum absolute values.

9. The clustering method for large-scale graph data according to claim 1, characterized in that: Weighted comparison loss for: in, , These represent nodes in the local community set. ,node ,node The low-dimensional embedding representation is obtained by inputting the local subgraph structure into the graph neural network. Represents the set of positive samples. This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, This represents the computation of low-dimensional embedding representations. With low-dimensional embedding representation cosine similarity, , Represents a node With nodes Connection confidence weights Represents a node With nodes Connection confidence weights This refers to the temperature parameter.

10. A clustering system for large-scale graph data, characterized by: include: The graph data preprocessing module is used to obtain the topological structure information and node attribute feature matrix of the graph data, and to standardize the node attribute feature matrix to obtain the standardized feature matrix. The batch construction module based on attribute-guided walk is used to randomly sample multiple nodes from the full graph node set as seed nodes. Calculate each seed node Jump to neighbor node transition probability From each seed node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, merging all seed nodes. And from each seed node Starting with a random walk, we visit high-frequency nodes to obtain local community sets. ; The affinity matrix calculation module based on batch walks is used to calculate local community sets. Each node Jump to neighbor node transition probability From each node Departure, according to transition probability Perform a random walk of fixed length and number of iterations, and count the nodes. With nodes The total number of times they appear in the same walk sequence. and Calculate the local structural connectivity affinity matrix based on the total number of iterations; The confidence-weighted contrastive learning module is used to calculate the connection confidence weight of any node pair in the local community set based on the connection affinity matrix of the local structure. The connection confidence weight is introduced into the contrastive loss to obtain the weighted contrastive loss. The nodes in the local community set and the local subgraph structure are input into the graph neural network, and the graph neural network is trained based on the weighted contrastive loss to obtain the trained graph neural network. The inference module is used to input all nodes of the full graph to be clustered into the trained graph neural network, obtain the global embedding representation matrix, and output the clustering result after dividing the global embedding representation matrix.

Images