A news recommendation method based on contrastive learning and hypergraph enhancement

Abstract

The present invention relates to a news recommendation method based on contrastive learning enhanced by hypergraph, comprising S1, constructing a user intention-guided graph attention news recommendation model; S2, constructing a hypergraph-enhanced contrastive learning news recommendation model; and S3, calculating overall loss. The present invention benefits from the intention interaction learning model, HGCLRec, which models the user's intention trend when selecting news and has a better overall dependence on high-order neighbor information; the present invention uses a hypergraph structure contrastive learning model, and HGCLRec achieves data expansion of sparse user news interactions through multi-view contrastive learning, making more efficient use of the graph structure. The HGCLRec of the present invention performs better than methods based on traditional neural networks, thanks to the rich contrastive learning objectives. The user's intention and representation are jointly modeled through the hidden layer of the hypergraph neural network, enriching the node representation and thus improving the recommendation effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of personalized news recommendation, and in particular relates to a news recommendation method based on comparative learning of hypergraph enhancement. Background Art

[0002] The ever-increasing number of news and articles on many news platforms, such as Google News and Microsoft News, makes it difficult for users to quickly find content they are interested in. To meet these needs, personalized news recommendations have emerged. Personalized news recommendations refer to providing users with personalized news recommendations based on their interests, preferences, and behaviors. Personalized news recommendation services typically analyze user behavioral data, browsing history, and media usage to provide users with personalized news services based on their interests and preferences, helping users understand their interests and needs and select news sources that suit them. Furthermore, personalized news recommendations can help users understand the similarities and differences between different media outlets and their mutual influence, thereby better understanding the coverage and perspectives of different media outlets on the same topic.

[0003] Personalized news recommendation services typically require the use of several algorithms and techniques, including cluster analysis, text mining, and machine learning. Recommendation algorithms are the core of news recommendation systems and play a crucial role in effectively performing the task. Researching better recommendation algorithms not only helps find information that is of greater interest and significance to users, but also improves the operation of news recommendation platforms and increases user engagement. In recent years, graph convolutional neural networks (GCNNs), due to their outstanding performance in learning from graph data, have rapidly developed and are widely used in many fields involving graph data. In news recommendation systems, the vast majority of information is inherently graph-structured. Due to the advantages of GCNNs in representation learning and their ability to explicitly encode high-level signals from user interactions through multi-layer networks, their application in news recommendation systems has rapidly grown. Graph neural networks are used to mine valuable information from the massive amounts of data required for news recommendation. Graph mining helps people discover patterns and relationships in data, leading to a better understanding of the data and enabling a variety of applications.

[0004] Current news recommendation methods typically learn user interests from the news they have clicked on historically. However, users often have different intentions when interacting with news, and this approach does not model these intentions in a fine-grained manner, resulting in poor recommendation results. There is an urgent need to enhance data features and extract more comprehensive semantic representations from news text through effective encoder design. Furthermore, graph neural networks have become a common method in the field of news recommendation due to their advantage in modeling graph-structured data. However, existing news recommendation methods based on graph neural networks often produce oversmoothing effects when learning high-order neighbor information, which limits the expression of high-order collaborative signals, affects user personalized representation, and leads to reduced recommendation effectiveness. There is an urgent need to build efficient graph structures and improve effective graph neural networks to mine more complex data relationships. Therefore, designing personalized news recommendation systems based on data features and efficient graph structures based on graph neural networks is of great significance.

[0005] This paper focuses on news recommendation based on graph neural networks. After investigating and analyzing existing news recommendation methods, it is found that:

[0006] 1. Current GNN-based methods ignore different user intentions when interacting with news, which can more accurately model user interests. Users may choose to interact with news for various reasons, and hidden intentions will guide users' news choices.

[0007] 2. For graph neural network applications, deeper embedding propagation layers can also lead to oversmoothing. With the addition of graph propagation layers and repeated use of Laplace smoothing, over-fusion of the interaction graph can homogenize the features of different nodes, making different user embedding representations more difficult to distinguish. This limits the expression of high-level collaborative relationships and causes oversmoothing.

[0008] Therefore, fine-grained user preferences and differentiable representations are crucial. Summary of the Invention

[0009] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a news recommendation method based on comparative learning of hypergraph enhancement.

[0010] The present invention solves the technical problem by the following technical solutions:

[0011] A news recommendation method based on contrastive learning of hypergraph enhancement, characterized in that the method comprises the following steps:

[0012] S1. Build a user intent-guided graph attention news recommendation model. This model can continuously learn richer high-order collaborative signals by stacking attention layers and propagation layers based on intent-based interactive learning. A single-layer network consists of an intent-based attention mechanism and a propagation layer. Based on the attention layer in traditional graph attention networks, its basic function is to aggregate information from neighboring nodes. By embedding the user's intent on the topic, it can effectively utilize the data and obtain collaborative signals. The propagation layer propagates the aggregated information to the nodes in the next layer.

[0013] S2. Build a hypergraph-enhanced contrastive learning news recommendation model. This model applies a hypergraph neural network to effectively utilize graph structures, designs a hyperedge aggregation mechanism to handle data correlations during representation learning, and better represent relationships between underlying data. Furthermore, through contrastive learning, the intent interaction view and the hypergraph structure view are jointly trained as two contrasting views. This allows for more informative intent representations and more discriminative hypergraph data structure representations, enriching user representations and improving recommendations.

[0014] S3. Overall loss calculation: To combine the representations of the intent interaction learning module and the hypergraph structure learning module, they are applied to contrastive learning from two views. By combining the intent interaction learning embedding and the hypergraph structure learning representation, the representations of semantically similar positive sample pairs are made closer in the representation space, and the representation distance of semantically dissimilar negative sample pairs is made farther, so as to achieve a clustering-like effect, achieve mutual collaboration and supervision between the intent interaction learning view and the hypergraph structure learning view, and thus realize data enhancement of users and news.

[0015] Furthermore, the step S1 is specifically as follows:

[0016] The news set {i1, i2, i3} that user u1 has interacted with is constructed with an intent node that integrates news topics, which explicitly expresses the user u1’s attention scores on different topics. A more fine-grained interest modeling at the intent level is achieved through the graph attention network. Different weight ratios are assigned to each intent p∈P, and attention weights are assigned to topics extracted from each news article, completing the interaction and combination of different intents at a more fine-grained level. The attention layer (Intent-basedAttention) based on intent interaction learning is expressed as shown in formula (1):

[0017]

[0018] Where: t i is the ID embedding of topic t, which is assigned an attention score α(t,u) to quantify its importance, as shown in formula (2):

[0019]

[0020] User embedding represents the embedding representation of aggregated historical news. The left-normalized form of the user’s aggregated neighbors at layer l is shown in formula (3):

[0021]

[0022] The symmetric normalized form is shown in formula (4):

[0023]

[0024] By integrating the two normalization methods, we can effectively utilize the two normalization methods while reducing the influence of nodes with high popularity, and obtain formula (5):

[0025]

[0026] After L layers of propagation, the node can receive messages propagated from its L-hop neighbors, average all representations, and combine the embeddings of different layers with weighted sums to capture the user and news representations with self-connected graph convolutions, and obtain the final embedding representation of user u and news i. The calculation method is shown in formula (6):

[0027]

[0028] The inner product of the user and news representation is used to predict the score of the user adopting the news. The specific calculation method is shown in formula (7):

[0029]

[0030] The objective function uses the Bayesian personalized ranking BPR loss function, and the training strategy objective function is set as shown in formula (8):

[0031]

[0032] Furthermore, the step S2 is specifically as follows:

[0033] The establishment of a hypergraph is based on the establishment of hyperedges, which is in turn based on the relationships between nodes. The K-nearest neighbor algorithm and the K-means clustering algorithm are combined to construct a dynamic hypergraph to take advantage of local and global structures. The K-means algorithm is used to cluster the entire feature graph to obtain the initial hyperedge c. The distance from the node to the cluster center is calculated using the Euclidean distance of the K-nearest neighbor algorithm. The nearest topK nodes are connected to the hyperedge, and the input feature embedding is used to construct the hypergraph structure. Therefore, the hyperedge set will be dynamically adjusted as the feature embedding network is trained and optimized. In this way, a hypergraph structure for modeling high-order data relationships is obtained.

[0034] Aggregate the original node information to the hyperedge node, propagate the node layer to the hyperedge layer a through a layer of network, and the hyperedge obtains the information of the nodes. The calculation method is shown in formula (9):

[0035]

[0036] The propagation process is to put the hyperedge information a back to the original node χ. The calculation method is shown in formula (10):

[0037]

[0038] Finally, the nonlinear activation function is used to obtain the representation of the node in the next layer after passing through the hypergraph convolutional network. The calculation method is shown in formula (11):

[0039] χ (l+1) =σ(χ′) (11)

[0040] The hypergraph adjacency matrix H and node features are input into the hypergraph convolutional network to obtain the node output representation of users and news. Based on the above introduction, the convolution operation can be simplified to the form of formula (12):

[0041]

[0042] Furthermore, the step S3 is specifically as follows:

[0043] Positive and negative sample pairs are given by different colors. The (z, χ) representations of the same node obtained by different methods are regarded as positive sample pairs, and the nodes of different users or news are regarded as negative sample pairs. The HGCLRec model generates positive and negative sample pairs for comparison. The contrastive learning objective is used to minimize the distance between positive sample pairs and maximize the distance between negative sample pairs. The calculation method is shown in formula (13):

[0044]

[0045] The hypergraph structure module is used for joint training of contrastive learning loss, and a multi-task learning strategy is used to combine the traditional ranking loss and the proposed contrastive loss. The specific definition is shown in formula (14):

[0046]

[0047] The advantages and beneficial effects of the present invention are:

[0048] This invention utilizes a hypergraph-enhanced comparative learning news recommendation method. Benefiting from an intention interaction learning model, HGCLRec models user intention trends when selecting news, demonstrating a better overall reliance on high-order neighbor information. Through a hypergraph structure comparative learning model, HGCLRec leverages multi-view (from regular graphs to hypergraphs) comparative learning to augment sparse user news interaction data and more effectively utilize graph structures. HGCLRec consistently outperforms traditional neural network-based methods, benefiting from a rich set of comparative learning objectives. The hidden layer of the hypergraph neural network jointly models user intentions and representations, enriching node representations and thus improving recommendation effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the various intentions of users clicking on news in the news recommendation scenario of the present invention.

[0050] Figure 2 This is a model diagram of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below through specific examples. The following examples are only illustrative and not restrictive, and the scope of protection of the present invention cannot be limited thereto.

[0052] A news recommendation method based on contrastive learning of hypergraph enhancement, characterized in that the method comprises the following steps:

[0053] S1. Build a user intent-guided graph attention news recommendation model. This model can continuously learn richer high-order collaborative signals by stacking attention layers and propagation layers based on intent-based interactive learning. A single-layer network consists of an intent-based attention mechanism and a propagation layer. Based on the attention layer in traditional graph attention networks, its basic function is to aggregate information from neighboring nodes. By embedding the user's intent on the topic, it can effectively utilize the data and obtain collaborative signals. The propagation layer propagates the aggregated information to the nodes in the next layer.

[0054] The news set {i1, i2, i3} that user u1 has interacted with is constructed with an intent node that integrates news topics, which explicitly expresses the user u1’s attention scores on different topics. A more fine-grained interest modeling at the intent level is achieved through the graph attention network. Different weight ratios are assigned to each intent p∈P, and attention weights are assigned to topics extracted from each news article, completing the interaction and combination of different intents at a more fine-grained level. The attention layer (Intent-basedAttention) based on intent interaction learning is expressed as shown in formula (1):

[0055]

[0056] Where: t i is the ID embedding of topic t, which is assigned an attention score α(t,u) to quantify its importance, as shown in formula (2):

[0057]

[0058] User embedding represents the embedding representation of aggregated historical news. The left-normalized form of the user’s aggregated neighbors at layer l is shown in formula (3):

[0059]

[0060] The symmetric normalized form is shown in formula (4):

[0061]

[0062] By integrating the two normalization methods, we can effectively utilize the two normalization methods while reducing the influence of nodes with high popularity, and obtain formula (5):

[0063]

[0064] After L layers of propagation, the node can receive messages propagated from its L-hop neighbors, average all representations, and combine the embeddings of different layers with weighted sums to capture the user and news representations with self-connected graph convolutions, and obtain the final embedding representation of user u and news i. The calculation method is shown in formula (6):

[0065]

[0066] The inner product of the user and news representation is used to predict the score of the user adopting the news. The specific calculation method is shown in formula (7):

[0067]

[0068] The objective function uses the Bayesian personalized ranking BPR loss function, and the training strategy objective function is set as shown in formula (8):

[0069]

[0070] S2. Build a hypergraph-enhanced contrastive learning news recommendation model. This model applies a hypergraph neural network to effectively utilize graph structures, designs a hyperedge aggregation mechanism to handle data correlations during representation learning, and better represent relationships between underlying data. Furthermore, through contrastive learning, the intent interaction view and the hypergraph structure view are jointly trained as two contrasting views. This allows for more informative intent representations and more discriminative hypergraph data structure representations, enriching user representations and improving recommendations.

[0071] The establishment of a hypergraph is based on the establishment of hyperedges, which is in turn based on the relationships between nodes. The K-nearest neighbor algorithm and the K-means clustering algorithm are combined to construct a dynamic hypergraph to take advantage of local and global structures. The K-means algorithm is used to cluster the entire feature graph to obtain the initial hyperedge c. The distance from the node to the cluster center is calculated using the Euclidean distance of the K-nearest neighbor algorithm. The nearest topK nodes are connected to the hyperedge, and the input feature embedding is used to construct the hypergraph structure. Therefore, the hyperedge set will be dynamically adjusted as the feature embedding network is trained and optimized. In this way, a hypergraph structure for modeling high-order data relationships is obtained.

[0072] Aggregate the original node information to the hyperedge node, propagate the node layer to the hyperedge layer a through a layer of network, and the hyperedge obtains the information of the nodes. The calculation method is shown in formula (9):

[0073]

[0074] The propagation process is to put the hyperedge information a back to the original node χ. The calculation method is shown in formula (10):

[0075]

[0076] Finally, the nonlinear activation function is used to obtain the representation of the node in the next layer after passing through the hypergraph convolutional network. The calculation method is shown in formula (11):

[0077] χ (l+1) =σ(χ′) (11)

[0078] The hypergraph adjacency matrix H and node features are input into the hypergraph convolutional network to obtain the node output representation of users and news. Based on the above introduction, the convolution operation can be simplified to the form of formula (12):

[0079]

[0080] S3. Overall loss calculation: To combine the representations of the intent interaction learning module and the hypergraph structure learning module, we apply them to contrastive learning from two perspectives. By combining the intent interaction learning embedding and the hypergraph structure learning representation, the representations of semantically similar positive sample pairs are closer in the representation space, while the representations of semantically dissimilar negative sample pairs are farther apart, achieving a clustering-like effect. This allows the intent interaction learning view and the hypergraph structure learning view to collaborate and supervise each other, thereby achieving data enhancement for users and news.

[0081] Positive and negative sample pairs are given by different colors. The (z, χ) representations of the same node obtained by different methods are regarded as positive sample pairs, and the nodes of different users or news are regarded as negative sample pairs. The HGCLRec model generates positive and negative sample pairs for comparison. The contrastive learning objective is used to minimize the distance between positive sample pairs and maximize the distance between negative sample pairs. The calculation method is shown in formula (13):

[0082]

[0083] The hypergraph structure module is used for joint training of contrastive learning loss, and a multi-task learning strategy is used to combine the traditional ranking loss and the proposed contrastive loss. The specific definition is shown in formula (14):

[0084]

[0085] Extensive experiments are conducted on two real-world datasets, MIND and Feeds, to evaluate the effectiveness of HGCLRec. The datasets are shown in Table 1.

[0086] Table 1

[0087]

[0088] The test samples selected for the experiment follow a general strategy, ranking all item sets with which the user has not interacted. HGCLRec was implemented in PyTorch on a Titan Rtx GPU. Parameters were fine-tuned through model training, and recommendations were obtained based on the best validation results to date. Recommendation performance was evaluated using four ranking metrics: Area Under the Curve (AUC), Mean Response Rate (MRR), nDCG@5, and nDCG@10. Pretrained Glove and TransE entity embeddings were used for initialization, with the embedding size set to 64. Parameters were initialized using the default Xavier distribution, with the batch size set to 4096. Hyperparameters included the number of convolutional layers to 3, the dropout rate to 0.5, the learning rate to 0.001, the number of clusters to 1000, λ1 to the range [1e-10, e-6], and τ to the range [0.01, 1]. Parameters were selected and adjusted based on the validation set results.

[0089] To demonstrate the effectiveness of the HGCLRec model, we experimentally implemented the proposed model and compared it with all RecBole news recommendation models. RecBole is a unified open-source framework for developing and replicating recommendation algorithms. Table 2 reports the average performance of all models for AUC, MRR, nDCG@5, and NDCG@10, demonstrating the superior performance of HGCLRec. The top-performing method in each column is indicated in bold.

[0090] Table 2

[0091]

[0092] Experimental results show that the proposed HGCLRec method consistently outperforms other comparison models across all evaluation metrics on both datasets. Compared to the second-place NCL model, HGCLRec achieves 2.34% and 0.66% improvements on nDCG@10 on the MIND and Feeds datasets, respectively. HGCLRec also significantly outperforms the baseline model in AUC, MRR, and nDCG@5.

[0093] The experimental results verify the advantages of the HGCLRec method, which can be summarized as follows: (1) Thanks to the intention interaction learning model, HGCLRec models the user's intention trend when selecting news and has a better overall dependence on high-order neighbor information. (2) Through the hypergraph structure contrast learning model, HGCLRec achieves data expansion of sparse user news interactions through multi-view (from ordinary graph to hypergraph) contrast learning, making more effective use of the graph structure. It can also be seen that the proposed HGCLRec always performs better than the traditional neural network-based method, thanks to the rich contrast learning objectives. The user's intention and representation are jointly modeled through the hidden layer of the hypergraph neural network, enriching the node representation and thus improving the recommendation effect.

[0094] Although the embodiments and drawings of the present invention are disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.

Claims

1. A news recommendation method based on contrastive learning and hypergraph enhancement, characterized by: The steps of the method are: S1. Build a user intent-guided graph attention news recommendation model. This model continuously learns richer high-order collaborative signals by stacking attention layers and propagation layers based on intent-based interactive learning. A single-layer network consists of an intent-based attention mechanism and a propagation layer. Based on the attention layer in traditional graph attention networks, its basic function is to aggregate information from neighboring nodes. By embedding the user's intent on the topic, it effectively utilizes the data and obtains collaborative signals. The propagation layer propagates the aggregated information to the nodes in the next layer. S2. Build a hypergraph-enhanced contrastive learning news recommendation model. This model applies a hypergraph neural network to effectively utilize graph structures, designs a hyperedge aggregation mechanism to handle data correlations during representation learning, and better represent relationships between underlying data. Furthermore, through contrastive learning, the intent interaction view and the hypergraph structure view are jointly trained as two contrasting views. This allows for more informative intent representations and more discriminative hypergraph data structure representations, enriching user representations and improving recommendations. S3. Overall loss calculation: To combine the representations of the intent interaction learning module and the hypergraph structure learning module, we apply them to contrastive learning from two perspectives. By combining the intent interaction learning embedding and the hypergraph structure learning representation, we make the representations of semantically similar positive sample pairs closer in the representation space, and the representation distance of semantically dissimilar negative sample pairs farther apart, achieving a clustering-like effect. This allows the intent interaction learning view and the hypergraph structure learning view to collaborate and supervise each other, thereby achieving data enhancement for users and news. The step S1 is specifically as follows: The news set {i1, i2, i3} that user u1 has interacted with is constructed with an intent node that integrates news topics, which explicitly expresses the attention scores of user u1 on different topics. A more fine-grained intent-level interest modeling is achieved through the graph attention network. Different weight ratios are assigned to each intent p∈P, and attention weights are assigned to topics extracted from each news article, completing the interaction and combination of different intents at a more fine-grained level. The attention layer representation based on intent interaction learning is shown in formula (1): Where: t i is the ID embedding of topic t, which is assigned an attention score α(t,u) to quantify its importance, as shown in formula (2): User embedding represents the embedding representation of aggregated historical news. The left-normalized form of the user’s aggregated neighbors at layer l is shown in formula (3): The symmetric normalized form is shown in formula (4): By integrating the two normalization methods, we can effectively utilize the two normalization methods while reducing the influence of nodes with high popularity, and obtain formula (5): After L layers of propagation, the node can receive messages propagated from its L-hop neighbors, average all representations, combine the embeddings of different layers with weighted sums to capture the user and news representations with self-connected graph convolutions, and obtain the final embedding representation of user u and news i. The calculation method is shown in formula (6): The inner product of the user and news representation is used to predict the score of the user adopting the news. The specific calculation method is shown in formula (7): The objective function uses the Bayesian personalized ranking BPR loss function, and the training strategy objective function is set as shown in formula (8): The step S2 is specifically as follows: The establishment of a hypergraph is based on the establishment of hyperedges, which is in turn based on the relationships between nodes. The K-nearest neighbor algorithm and the K-means clustering algorithm are combined to construct a dynamic hypergraph to take advantage of local and global structures. The K-means algorithm is used to cluster the entire feature graph to obtain the initial hyperedge c. The distance from the node to the cluster center is calculated using the Euclidean distance of the K-nearest neighbor algorithm. The nearest topK nodes are connected to the hyperedge, and the input feature embedding is used to construct the hypergraph structure. Therefore, the hyperedge set will be dynamically adjusted as the feature embedding network training is optimized. In this way, a hypergraph structure for modeling high-order data relationships is obtained. Aggregate the original node information to the hyperedge node, propagate the node layer to the hyperedge layer a through a layer of network, and the hyperedge obtains the information of the nodes. The calculation method is shown in formula (9): Propagation is to put the recovered hyperedge information a back to the original node χ. The calculation method is shown in formula (10): Finally, the nonlinear activation function is used to obtain the representation of the node in the next layer after passing through the hypergraph convolutional network. The calculation method is shown in formula (11): x (l+1) =σ(χ′) (11) The hypergraph adjacency matrix H and node features are input into the hypergraph convolutional network to obtain the node output representation of users and news. Through the above introduction, the convolution operation is simplified to the form of formula (12): The step S3 is specifically as follows: Positive and negative sample pairs are given by different colors. The (z, χ) representations of the same node obtained by different methods are regarded as positive sample pairs, and the nodes of different users or news are regarded as negative sample pairs. The HGCLRec model generates positive and negative sample pairs for comparison. The contrastive learning objective is used to minimize the distance between positive sample pairs and maximize the distance between negative sample pairs. The calculation method is shown in formula (13): The hypergraph structure module is used for joint training of contrastive learning loss, and a multi-task learning strategy is used to combine the traditional ranking loss and the proposed contrastive loss. The specific definition is shown in formula (14):

Images