A friend recommendation method based on cross-scale graph contrastive learning

Abstract

This invention provides a friend recommendation method based on cross-scale graph contrastive learning, belonging to the field of network science and technology. The method specifically includes: acquiring user data and modeling the user data to construct graph structure data; performing data augmentation on the modeled graph structure data using a random sampling augmentation method; constructing a graph convolutional neural network consisting of two stacked graph convolutional layers as an encoder, using the encoder to encode the node vectors of the data-augmented view to obtain the vector representation of the nodes; mapping the node vector representations to the graph vector representations using a readout function; constructing positive and negative sample pairs respectively, and using information-maximizing nonparametric mutual information estimation as the loss function to update the parameters of the encoder and readout function to obtain the trained encoder and readout function; and using the trained encoder and readout function to perform friend recommendations for a given user, solving the problem of ignoring the overall structural information of the graph in graph contrastive learning.

Description

Technical Field

[0001] This invention relates to the field of network science and technology, and in particular to a friend recommendation method based on cross-scale graph comparison learning. Background Technology

[0002] Social networks have experienced tremendous growth and widespread adoption in recent years, becoming an important part of people's daily lives. With the rapid growth of social networks, people increasingly need to find and build new social relationships online. Friend recommendation systems, as a key function of social networks, can help users discover potential friends with similar interests, providing a richer and more meaningful social experience.

[0003] However, traditional friend recommendation systems often face challenges such as difficulty in accurately capturing user interests, the cold start problem, and privacy protection. In recent years, graph representation learning, as a powerful machine learning technique, has made significant progress in the field of friend recommendation to address these issues. Graph representation learning maps topological graphs to a low-dimensional vector space while preserving the original graph structure and supporting graph inference. Traditional graph analysis methods suffer from high computational and space costs and struggle to capture the highly nonlinear structural features of networks. This has driven the emergence of graph neural network methods, which are highly advantageous in handling irregular non-Euclidean data. Graph neural networks incorporate the structural characteristics of graph data into their algorithmic models and utilize the structural features between sample instances to effectively and fully represent the relationships between different samples, thereby maximizing the utilization of the structural characteristics of real-world graphs.

[0004] In real life, data loss due to difficulties in collecting user information is very common, and labels, a crucial piece of information, are among the frequently missing data. The lack of labels leads to training difficulties in existing graph neural network (Graph Neural Network) methods. This is because the loss function for Graph Neural Network models relies on labels during training, typically predicting the label of a node based on its learned node representation vector. When true labels are missing, the label-dependent loss function cannot guide model training. Contrastive learning, however, can guide model training by bringing positive node pairs closer together and pushing negative node pairs apart, without requiring label information. Currently, the application of contrastive learning in Graph Neural Networks is still in its early stages, and due to the complex structural characteristics of graphs, its application faces many challenges: 1) The selection of data augmentation methods: Current augmentation methods often discard nodes, edges, or mask node attribute information. However, blindly applying augmentation methods to different tasks can lead to severe information loss in the graph and introduce excessive bias, reducing the model's performance in downstream tasks. 2) Current graph contrastive learning methods mostly train by comparing vectors between nodes, failing to fully utilize the crucial information of the graph's global semantic structure.

[0005] Currently, most work on graph contrastive learning is based on labeled supervised learning, ignoring the rich structural and attribute information inherent in the underlying graph data. Furthermore, labeling graphs is costly, preventing most existing methods from being applied to real-world data. With the success of contrastive learning in multiple fields, researching its application to graph-structured data is of great significance. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention proposes a friend recommendation method based on cross-scale graph comparison learning, which solves the problem of ignoring the overall structural information of the graph in graph comparison learning.

[0007] This invention proposes a friend recommendation method based on cross-scale graph contrastive learning, which includes the following steps:

[0008] Step 1: Acquire user data and model the user data to construct a graph structure data;

[0009] Step 2: Use random sampling augmentation to augment the modeled graph structure data to obtain the augmented view;

[0010] Step 3: Construct a graph convolutional neural network consisting of two stacked graph convolutional layers as an encoder. Use the encoder to encode the node vectors of the obtained view to obtain the vector representation of the node.

[0011] Step 4: Use the read function to map the vector representation of the nodes into the vector representation of the graph;

[0012] Step 5: Construct positive and negative sample pairs based on the vector representations of the nodes and the vector representation of the graph, and use information maximization nonparametric mutual information estimation as the loss function to update the parameters of the encoder and readout function to obtain the trained encoder and readout function;

[0013] Step 6: Use the trained encoder and readout function to make friend recommendations for a given user;

[0014] The graph structure data in step 1 includes: nodes, node vectors, and edges between nodes: where the nodes represent users, the node vectors represent user data, and the edges between nodes represent the friend relationships between users. If two users have a friend relationship, then there is an edge between the corresponding two nodes.

[0015] The data augmentation method described in step 2 is as follows: Construct a set of augmentation functions based on the augmentation functions corresponding to existing data augmentation methods, denoted as T = (t1, t2, ..., t...). k ), where t1 represents the first enhancement function; t2 represents the second enhancement function; t k Let t represent the k-th enhancement function; perform two random samplings on T to obtain two different views, where each random sampling selects two enhancement functions t from T. m and t n We will perform data augmentation on the modeled graph structure data, and then combine the augmented graph structure data into a single view. The augmented graph structure data will be denoted as follows: and Where G represents the modeled graph structure data; This indicates the use of the enhancement function t m Graph structure data after data augmentation; This indicates the use of the enhancement function t n Graph structure data after data augmentation;

[0016] The graph convolutional layer determines the adjacency matrix A of the nodes based on the input view, and uses the adjacency matrix A and the feature matrix H of the layer. (l) The feature information of a node is propagated to its neighboring nodes, and then the weight matrix W is used. (l) The feature matrix H of the next layer is obtained by using the activation function σ. (l+1) ;

[0017] The process by which the graph convolutional layer obtains the feature matrix of the next layer is shown in the following equation:

[0018]

[0019] in It is a degree matrix; It is an adjacency matrix with self-loops, and Where I N H is the identity matrix; (l +1) H is the feature matrix of the next layer; (l) W is the feature matrix constructed by the convolutional layer of the current graph based on the node vectors of the nodes. (l) σ is the independent weight matrix of the convolutional layer in this graph; σ is the activation function.

[0020] The method for constructing positive and negative sample pairs in step 5 is as follows: For a node B in the graph structure data, select the corresponding node B′ from the data-augmented view, and use the vector representation of node B, the vector representation of node B′, and the vector representation of the graph of the view where node B′ is located to form a positive sample pair; For node B, arbitrarily select several nodes from the graph structure data and the data-augmented view to form several negative sample pairs, wherein each negative sample pair includes: the vector representation of node B, the vector representation of any selected node, and the vector representation of the graph of the view where node B′ is located;

[0021] The loss function described in step 5 is expressed as:

[0022]

[0023] in z is the loss function; i and z j These are the projection vectors generated after data augmentation for the i-th and j-th nodes in the view, respectively; 1 [k≠i] is an indicator function, and the function value of the indicator is 1 when k≠i, and 0 otherwise; s() is a function that measures the similarity between two embedding vectors; The temperature coefficient in the loss function;

[0024] The method for obtaining the trained encoder and readout function in step 5 is as follows: the similarity between positive sample pairs and between a positive sample pair and several negative sample pairs are calculated using the loss function. The training of the encoder and readout function is completed by maximizing the similarity between positive sample pairs and minimizing the similarity between positive sample pairs and negative sample pairs.

[0025] Step 6 further includes:

[0026] Step 6.1: For a given user, randomly select several users to construct a set of users to be recommended.

[0027] Step 6.2: Determine whether there are any users in the set of users to be recommended who are not in the graph structure data constructed in Step 1; if so, proceed to Step 6.3; otherwise, proceed to Steps 2-5 to update the vector representation of the node corresponding to the user for the user who is not in the graph structure data, and then proceed to Step 6.3.

[0028] Step 6.3: Use the similarity calculation function to calculate the similarity between the given user and each user in the set of users to be recommended. Sort the calculation results from largest to smallest and select the top S users with the highest similarity as friends for this recommendation.

[0029] The beneficial effects of adopting the above technical solution are as follows:

[0030] The present invention provides a novel machine learning method, namely a self-supervised learning method. This learning method does not rely on manual labeling. It designs agent tasks and uses the input data itself as supervision to learn from rich unlabeled data, thereby obtaining more generalizable graph embeddings that can be applied to downstream tasks.

[0031] Unlike existing methods that rely on comparative learning on samples, this invention does not simply use one or two augmentation methods for data augmentation. Instead, it continuously utilizes random sampling augmentation methods during training to explore augmentation methods that adapt to the input graph, reducing the loss of important information in the graph and mitigating the problem of performance degradation caused by blindly using inappropriate augmentation methods for different downstream tasks. Furthermore, this invention does not only train the model by comparing node-level information but also uses a readout function to further encode the node vectors encoded by the graph encoder to obtain graph-level vector representations. The model is then trained by comparing node-level and graph-level information, incorporating global semantic structure information of the graph into the model during training, thereby improving the performance of downstream tasks.

[0032] This invention addresses the issue of information loss in graphs and the neglect of global semantic structure information caused by blindly using augmentation methods for isomorphic networks. Attached Figure Description

[0033] Figure 1 This is a flowchart of a friend recommendation method based on cross-scale graph contrastive learning in this embodiment;

[0034] Figure 2 This is a schematic diagram illustrating the principle of a friend recommendation method based on cross-scale graph contrastive learning in this embodiment. Detailed Implementation

[0035] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0036] In this embodiment, since most real-world data in recent years can be conveniently represented graphically, such as social networks, biomolecular networks, and communication networks, therefore... Figure 1 Graph representation has always been a hot topic in academic research. The learned graph representation can be used to conveniently perform downstream graph tasks, which can be broadly categorized into four types: node classification, link prediction, clustering, and visualization.

[0037] This embodiment presents a friend recommendation method based on cross-scale graph contrastive learning, such as... Figure 1 As shown, it includes the following steps:

[0038] Step 1: Obtain user data and model the user data to construct a graph structure data.

[0039] In this implementation, user data is obtained using the publicly available Cora dataset, Citeseer dataset, and Pubmed dataset.

[0040] The graph structure data includes: nodes, node vectors, and edges between nodes: where nodes represent users, node vectors represent user data, and edges between nodes represent friend relationships between users. If two users have a friend relationship, then there is an edge between the corresponding two nodes.

[0041] Step 2: Use random sampling augmentation to augment the modeled graph structure data to obtain the augmented view;

[0042] The data augmentation method is as follows: Construct an augmentation function set based on the augmentation functions corresponding to existing data augmentation methods, denoted as T = (t1, t2, ..., t...). k ), where t1 represents the first enhancement function; t2 represents the second enhancement function; t k Let t represent the k-th enhancement function; perform two random samplings on T to obtain two different views, where each random sampling selects two enhancement functions t from T. m and t n We will perform data augmentation on the modeled graph structure data, and then combine the augmented graph structure data into a single view. The augmented graph structure data will be denoted as follows: and Where G represents the modeled graph structure data; This indicates the use of the enhancement function tm Graph structure data after data augmentation; This indicates the use of the enhancement function t n Graph structure data after data augmentation.

[0043] In this embodiment, an augmentation function set is constructed by selecting augmentation functions corresponding to four commonly used data augmentation methods, including: two structural augmentation methods, namely Edge Removing (ER) and Node Dropping (ND), and two feature augmentation methods, namely Feature Masking (FM) and Feature Shuffling (FS); denoted as T = (t1, t2, t3, t4). First, two methods are randomly sampled from the augmentation function set for data augmentation to obtain one view. Then, two more methods are randomly selected from the augmentation function set for data augmentation to obtain another view, resulting in two views in total. These two views are then input into two dedicated graph neural networks to learn node representations. Based on the learned features, a readout function is used to obtain the graph representation. Finally, a contrastive loss function is designed based on InfoNCE to optimize the graph representation, aiming to make consistent representations in the two views closer together and inconsistent representations farther apart.

[0044] Besides the four commonly used data augmentation methods mentioned above, there are other data augmentation methods, as shown in the diagram. Existing data augmentation methods can be combined arbitrarily, but usually need to be selected based on the downstream task. This implementation method uses random sampling to select augmentation methods for combination, which can save the best results during training, thus avoiding manual selection of augmentation methods. Existing data augmentation combination methods usually select two or three data augmentation methods, but this requires manual selection of appropriate augmentation method combinations based on the downstream task. This requires experience or continuous experimentation. Excessive use of data augmentation methods can damage the graph structure data.

[0045] Step 3: Construct a graph convolutional neural network consisting of two stacked graph convolutional layers as an encoder. Use the encoder to encode the node vectors of the obtained view to obtain the vector representation of the node.

[0046] The graph convolutional layer determines the adjacency matrix A of the nodes based on the input view, and uses the adjacency matrix A and the feature matrix H of the layer. (l) The feature information of a node is propagated to its neighboring nodes, and then the weight matrix W is used. (l) The feature matrix H of the next layer is obtained by using the activation function σ. (l+1) .

[0047] In this embodiment, such as Figure 2As shown, the graph convolutional neural network is constructed by stacking layers, and the structure of each graph convolutional layer is the same. A graph convolutional neural network (GCN) composed of two stacked graph convolutional layers is constructed as an encoder. The encoding is to fully explore the deep information contained in the graph, which is beneficial to downstream tasks. The specific steps are the next segment and formula (1) and the two segments after it. In this way, the high-order nonlinear relationship contained in the graph can be explored to improve the accuracy of friend recommendation.

[0048] The graph convolutional layer is shown in the following formula:

[0049]

[0050] in It is a degree matrix; It is an adjacency matrix with self-loops, and Where I N H is the identity matrix; (l +1) H is the feature matrix of the next layer; (l) W is the feature matrix constructed by the convolutional layer of the current graph based on the node vectors of the nodes. (l) σ is the independent weight matrix of the convolutional layer in this graph; σ is the activation function, and in this embodiment, the activation function is the ReLU function.

[0051] In this embodiment, in order to preserve the node's own characteristic information during information propagation, the adjacency matrix A and the identity matrix I are... N Add them together to construct an adjacency matrix with self-loops. And for the adjacency matrix Perform normalization operation This is to maintain the feature matrix H during information transmission. (l) The original distribution is preserved to prevent large differences in feature distribution between nodes with excessively high degree and nodes with low degree.

[0052] Step 4: Use the read function to map the vector representation of the nodes into the vector representation of the graph;

[0053] The readout function is expressed as follows:

[0054]

[0055] Where R(H) is the graph-level vector representation obtained by aggregating the node vector information of all nodes in the graph; R() is the readout function; σ is the feature vector of node i in the data-enhanced view; N is the number of nodes in the graph structure data; σ is the activation function, and in this embodiment, the activation function is the sigmoid function.

[0056] Step 5: Construct positive and negative sample pairs based on the vector representations of the nodes and the vector representation of the graph, and use Information Maximizing Nonparametric Exploration (InfoNCE) as the loss function to update the parameters of the encoder and readout function to obtain the trained encoder and readout function.

[0057] The method for constructing positive and negative sample pairs is as follows: For a node B in the graph structure data, select the corresponding node B′ from the data-augmented view, and use the vector representation of node B, the vector representation of node B′, and the vector representation of the graph of the view where node B′ is located to form a positive sample pair; For node B, arbitrarily select several nodes from the graph structure data and the data-augmented view to form several negative sample pairs, wherein each negative sample pair includes: the vector representation of node B, the vector representation of any selected node, and the vector representation of the graph of the view where node B′ is located.

[0058] The loss function is expressed as:

[0059]

[0060] in z is the loss function; i and z j These are the projection vectors generated after data augmentation for the i-th and j-th nodes in the view, respectively; 1 [k≠i] is an indicator function, and the function value of the indicator is 1 when k≠i, and 0 otherwise; s() is a function that measures the similarity between two embedded vectors. In this embodiment, the cosine similarity function is used. The temperature coefficient in the loss function directly affects the shape of the probability distribution in the loss function.

[0061] The method for obtaining the trained encoder and readout function is as follows: the similarity between positive sample pairs and between a positive sample pair and several negative sample pairs are calculated using the loss function. The encoder and readout function are trained by maximizing the similarity between positive sample pairs and minimizing the similarity between positive and negative sample pairs.

[0062] In this embodiment, the methods described above for calculating the similarity between positive sample pairs and between positive and negative sample pairs make the node vector representations of two similar users more similar, which is beneficial for friend recommendations. Since the temperature coefficient in the loss function generally does not have a fixed default value, and the temperature coefficient... Controlling the penalty intensity for difficult-to-distinguish negative samples determines the degree of attention the contrastive loss pays to them. A larger temperature coefficient results in lower attention, reducing the difference between positive and negative samples, which may prevent the proposed method from learning sufficiently discriminative features. Conversely, a smaller temperature coefficient results in higher attention. Therefore, choosing an appropriate temperature coefficient is crucial in contrastive learning.

[0063] Step 6: Use the trained encoder and readout function to recommend friends to a given user.

[0064] Step 6.1: For a given user, randomly select several users to construct a set of users to be recommended.

[0065] Step 6.2: Determine whether there are any users in the set of users to be recommended who are not in the graph structure data constructed in Step 1; if so, proceed to Step 6.3; otherwise, proceed to Steps 2-5 to update the vector representation of the node corresponding to the user for the user who is not in the graph structure data, and then proceed to Step 6.3.

[0066] Step 6.3: Use the similarity calculation function to calculate the similarity between the given user and each user in the set of users to be recommended. Sort the calculation results from largest to smallest and select the top S users with the highest similarity as friends for this recommendation.

[0067] In this embodiment, for a given user, 100 users are randomly selected to construct a set of users to be recommended. It is then determined whether there are any users in the set of users to be recommended who are not in the graph structure data constructed in step 1. If so, step 6.3 is executed. Otherwise, steps 2-5 are executed for users who are not in the graph structure data to update the vector representation of the node corresponding to the user, and then step 6.3 is executed again. Then, the similarity calculation function is used to calculate the similarity of the nodes, and the similarity is sorted from largest to smallest according to the calculation results. The top 10 users are selected as friends for this recommendation.

[0068] The similarity calculation function is expressed as:

[0069] sim(a,b)=a·b (4)

[0070] Where sim() represents the similarity calculation function; a is the vector representation of the node corresponding to a given user; and b is the vector representation of the node corresponding to any user in the set of users to be recommended.

[0071] In this embodiment, a node classification comparison experiment was conducted on three commonly used public datasets, Cora, Citeseer, and Pubmed, against eight graph representation learning algorithms: MLP, CHEBYSHEV, GCN, GAT, DeepWalk, DGI, GAE, and MVGRL. The accuracy of the classification results was calculated, and the experimental results are shown in Table 1. According to the results in Table 1, it can be seen that the node classification accuracy of the method of this invention is significantly better than other related methods.

[0072] Table 1. Accuracy Comparison of Node Classification Results

[0073]

[0074] Where X represents the vector representation of a node, Y represents the node label, and A represents the adjacency matrix; CS_GCL(LL) indicates that the method of the present invention does not use the vector representation of the graph, that is, it does not use the readout function, but only uses the encoder; CS_GCL(LG) indicates that the method of the present invention uses the vector representation of the graph, that is, it uses the encoder and the readout function.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A friend recommendation method based on cross-scale graph contrastive learning, characterized in that, The method includes the following steps: Step 1: Acquire user data and model the user data to construct a graph structure data; Step 2: Use random sampling augmentation to augment the modeled graph structure data to obtain the augmented view; The data augmentation method is as follows: Construct a set of augmentation functions based on the augmentation functions corresponding to existing data augmentation methods, denoted as... ,in This represents the first enhancement function; This represents the second enhancement function; This represents the k-th enhancement function; Two random samples are taken from T to obtain two different views, where each random sample selects two enhancement functions from T. and We will perform data augmentation on the modeled graph structure data, and then combine the augmented graph structure data into a single view. The augmented graph structure data will be denoted as follows: and ;in This represents the graph structure data after modeling. Indicates the use of enhancement functions Graph structure data after data augmentation; Indicates the use of enhancement functions Graph structure data after data augmentation; Step 3: Construct a graph convolutional neural network consisting of two stacked graph convolutional layers as an encoder. Use the encoder to encode the node vectors of the obtained view to obtain the vector representation of the node. The graph convolutional layer determines the adjacency matrix of nodes based on the input view. Using adjacency matrix With the feature matrix of this layer The feature information of a node is propagated to its neighboring nodes, and then the weight matrix is ​​used. and activation function Obtain the feature matrix of the next layer ; The process by which the graph convolutional layer obtains the feature matrix of the next layer is shown in the following equation: ； in It is a degree matrix; It is an adjacency matrix with self-loops, and ;in It is the identity matrix; This is the feature matrix of the next layer; This is the feature matrix constructed by the convolutional layer of the current graph based on the node vectors of the nodes; These are the independent weight matrices of the convolutional layers in this graph; For activation functions; Step 4: Use the read function to map the vector representation of the nodes into the vector representation of the graph; Step 5: Construct positive and negative sample pairs based on the vector representations of the nodes and the vector representations of the graph, and use information maximization nonparametric mutual information estimation as the loss function to update the parameters of the encoder and readout function to obtain the trained encoder and readout function. The method for constructing positive and negative sample pairs is as follows: for a certain node in the graph structure data... Select the corresponding node from the data-enhanced view. Utilizing nodes Vector representation, nodes Vector representation and nodes The vector representations of the graph in the view form positive sample pairs; for nodes Several nodes are arbitrarily selected from the graph structure data and the data-augmented view to form several negative sample pairs, where each negative sample pair includes: nodes The vector representation of a node, the vector representation of any selected node, and the node The vector representation of the graph in the view; Step 6: Use the trained encoder and readout function to recommend friends to a given user.

2. The friend recommendation method based on cross-scale graph contrastive learning according to claim 1, characterized in that, The graph structure data in step 1 includes: nodes, node vectors, and edges between nodes: where nodes represent users, node vectors represent user data, and edges between nodes represent friend relationships between users. If two users have a friend relationship, then there is an edge between the corresponding two nodes.

3. The friend recommendation method based on cross-scale graph contrastive learning according to claim 2, characterized in that, The loss function described in step 5 is expressed as: ； in The loss function; and These are the projection vectors generated after data augmentation for the i-th and j-th nodes in the view, respectively. For an indicator function, and when The function value of this indicator is 1 when it is active, and 0 otherwise. A function to measure the similarity between two embedded vectors; This represents the temperature coefficient in the loss function.

4. The friend recommendation method based on cross-scale graph contrastive learning according to claim 3, characterized in that, The method for obtaining the trained encoder and readout function in step 5 is as follows: the similarity between positive sample pairs and between a positive sample pair and several negative sample pairs are calculated using the loss function. The training of the encoder and readout function is completed by maximizing the similarity between positive sample pairs and minimizing the similarity between positive sample pairs and negative sample pairs.

5. The friend recommendation method based on cross-scale graph contrastive learning according to claim 1, characterized in that, Step 6 further includes: Step 6.1: For a given user, randomly select several users to construct a set of users to be recommended; Step 6.2: Determine whether there are any users in the set of users to be recommended who are not in the graph structure data constructed in Step 1; if so, proceed to Step 6.3; otherwise, proceed to Steps 2-5 to update the vector representation of the node corresponding to the user for the user who is not in the graph structure data, and then proceed to Step 6.

3. Step 6.3: Use the similarity calculation function to calculate the similarity between the given user and each user in the set of users to be recommended. Sort the calculation results from largest to smallest and select the top S users with the highest similarity as friends for this recommendation.

Images