Personalized article recommendation system and recommendation method based on dynamic circle layer information

Abstract

The application discloses a kind of personalized article recommendation system based on dynamic circle layer information, including time series neighborhood encoder module and circle layer information contrast learning module.Wherein, time series neighborhood encoder module is responsible for encoding the time series behavior and circle layer structure characteristics of user.Circle layer information contrast learning module is responsible for encoding the circle layer information of user.The article recommendation system based on dynamic circle layer information and its recommendation method of the application, the time series bipartite graph of user-product interaction is established based on user historical behavior data, the circle layer where user is found by the method of contrast learning, by fusing the personal interest and circle layer interest characteristics of user, model can utilize circle layer characteristics to supplement personal interest characteristics when user personal historical behavior is lack, to alleviate cold start problem.In addition, since circle layer characteristics include the behavior of multiple users, it is less affected by the noise in single user behavior.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a personalized article recommendation system and method. Background Technology

[0002] Recommendation systems are a crucial technology in today's internet applications, designed to help users discover content of interest and improve the efficiency of content delivery. Article recommendation systems aim to recommend articles to users based on their individual reading interests. Currently, article recommendation systems are widely used on content platforms such as Toutiao, Zhihu, and Xiaohongshu. Existing recommendation systems primarily model users' reading interests based on their historical clicks and reading history. Based on user behavior modeling techniques, existing recommendation systems are mainly divided into traditional methods and deep learning-based methods.

[0003] Traditional product recommendation methods primarily rely on static data, such as user preferences and product static features. These methods model user interests using techniques like collaborative filtering, content-based filtering, and matrix factorization. Collaborative filtering typically generates recommendations based on user history and similar user behavior. Content-based filtering matches product features with user preferences. Matrix factorization methods directly embed user / product IDs into a vector and use inner products to model user-product interactions. However, these methods fail to effectively represent collaborative information. Furthermore, they struggle to handle temporal information and the dynamic nature of user needs. Product timeliness and the evolution of user interests are often overlooked; these methods fail to effectively capture temporal information and the dynamic changes in user needs.

[0004] In recent years, deep learning-based methods, especially graph neural networks (GNNs), have attracted widespread attention in product recommendation systems. GNNs construct interaction graphs between users and products while simultaneously modeling feature information from the graph structure. This allows the model to effectively encode users' latent interests through deep neural networks, thus providing more accurate recommendations. However, current personalized recommendation methods based on graph neural networks suffer from two main problems: 1) Many new users lack historical reading behavior, making it difficult to model personal interests using graph neural networks. 2) User click behavior contains a lot of noise, which can interfere with the modeling of graph neural networks.

[0005] Therefore, it is necessary to design a more effective personalized article recommendation method. Summary of the Invention

[0006] To address the aforementioned problems, this invention aims to propose an article recommendation system and method based on dynamic circle information. This system and method construct a temporal bipartite graph of user-product interactions based on users' historical behavior data. Through contrastive learning, it identifies the circle to which a user belongs. By fusing user's personal interests and circle interest features, the model can supplement personal interest features with circle features when personal historical behavior is lacking, thus alleviating the cold start problem. Furthermore, since circle features include the behavior of multiple users, they are less affected by noise in the behavior of a single user.

[0007] This invention is specifically achieved through the following technical solution:

[0008] An article recommendation system based on dynamic concentric circle information, the system comprising: a temporal neighborhood encoder module and a concentric circle information comparison learning module; wherein,

[0009] The temporal neighborhood encoder module consists of a self-attention-based user behavior encoder and a temporal encoding function. The temporal neighborhood encoder module is used to encode the user's temporal behavior and hierarchical structure features. Specifically, for a user at time t, its most recent reading behavior is first sampled, then the temporal features of the reading behavior are encoded using the temporal encoding function. Next, the information from the reading behavior is aggregated through a self-attention mechanism to obtain a neighborhood representation of the user's personal interests based on the user's historical reading behavior.

[0010] The circle information contrastive learning module is used to encode the circle information of the user; specifically, it includes: firstly, sampling positive and negative samples for the user, where positive samples include users in the same circle and negative samples include users in different circles; then, maximizing the mutual information between the user and the positive sample user representations and minimizing the mutual information between the user and the negative sample user representations through contrastive learning; finally, using the vector representing the circle information output by the time encoder... And represent the neighborhood of personal interests. The vector representing the layer information (hereinafter referred to as layer representation) Concatenate the data to obtain the final representation of user u. The Used to predict the articles the user will read;

[0011] Among them, an optimization objective function based on click behavior prediction is used to... Optimize.

[0012] As a further improvement of the present invention, an optimization objective function based on click behavior prediction is used to... The optimization specifically includes:

[0013] The article clicked by user u at time t is u. pos The articles that were not clicked were u neg , then u and u pos and u neg The probability p that an edge is formed between them pos p neg They are respectively:

[0014]

[0015] Where ||·||2 represents the element-wise squaring operation of the vector. For u pos The final expression, For u neg The final representation is defined as follows: the binary cross-entropy loss function is defined as the objective function for click behavior prediction:

[0016] L _p =-log(p pos )-log(1-p neg ),

[0017] The goal of the loss function based on binary cross-entropy is to make p pos p is close to 1. neg It is close to 0.

[0018] As a further improvement of the present invention, the sampling of positive and negative samples for the user specifically includes: considering other users connected to user u through a triangle as candidate positive samples, which means users who have clicked on at least two articles with u; if there are no neighbors in the same triangle as u, then ordinary 2-hop neighbors are selected as candidate positive samples; negative samples are randomly sampled from users outside the candidate positive samples.

[0019] As a further improvement of the present invention, the step of maximizing the mutual information between the user and the positive sample user representation and minimizing the mutual information between the user and the negative sample user representation through contrastive learning specifically includes:

[0020] Assuming l was sampled p positive samples and l n Given a negative sample node, define u p,i and u n,j Let i and j be the i-th positive sample and j-th negative sample of node u, respectively. For the sampled positive and negative sample nodes, the neighborhood structure and temporal information are encoded using a temporal neighborhood encoder:

[0021]

[0022] Among them, Enc c For a time-series neighborhood encoder, The neighborhood representation of positive samples. The neighborhood representation of the negative sample. and These are the initial features for positive and negative samples, respectively. The initial features of the previously read articles v are N(u) p,i (t) represents all pairs of u before time t. p,i The set of historical article nodes that have been interacted with, N(u n,i (t) represents all pairs of u before time t. n,j A collection of historical article nodes with which there has been interaction;

[0023] Based on the layer representation, predict u and positive sample nodes u p,i and negative sample node u n,j Probability within the same sphere:

[0024]

[0025] To maximize mutual information with positive sample users and minimize mutual information with negative sample users, a contrastive learning loss function is defined as follows:

[0026]

[0027] As a further improvement of the present invention, wherein Enc c It shares parameters with the time-series neighborhood encoder module.

[0028] As a further improvement of the present invention, the user u is represented by a concentric circle. The average of the neighborhood representations of all nodes output by the temporal neighborhood encoder module is directly taken.

[0029] As a further improvement of the present invention, the objective function is the sum of the loss function based on click behavior prediction and the loss function based on circle comparison learning:

[0030] L = L p +L c ,

[0031] The ADAM optimizer is used to optimize according to the objective function.

[0032] This invention also provides an article recommendation method based on dynamic layer information, the method comprising:

[0033] For a user at time t, firstly, their most recent reading behavior is sampled. Then, the temporal features of the reading behavior are encoded using a temporal encoding function. Next, the information of the reading behavior is aggregated through a self-attention mechanism to obtain a neighborhood representation of personal interests based on the user u's historical reading behavior.

[0034] For each user, positive and negative samples are collected. Positive samples include users within the same social circle, while negative samples include users outside the same social circle. Then, contrastive learning is used to maximize the mutual information between the user and the positive sample user representations, and minimize the mutual information between the user and the negative sample user representations. Finally, the vector representing the social circle information output by the temporal encoder is used... And represent the neighborhood of personal interests. and circle representation Concatenate the data to obtain the final representation of user u. The Used to predict the articles the user will read;

[0035] Among them, an optimization objective function based on click behavior prediction is used to... Optimize.

[0036] The beneficial effects of this invention are as follows: The article recommendation system and method based on dynamic circle information of this invention establish a time-series bipartite graph of user-product interaction based on user historical behavior data, find the circle to which the user belongs through contrastive learning, and by integrating the user's personal interests and circle interest features, the model can supplement personal interest features with circle features when the user's personal historical behavior is lacking, thereby alleviating the cold start problem. Furthermore, since the circle features include the behavior of multiple users, they are less affected by noise in the behavior of a single user. Attached Figure Description

[0037] Figure 1 This is a diagram illustrating the overall framework of the article recommendation system based on dynamic layer information of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] The article recommendation system based on dynamic layer information of this invention is attached. Figure 1 As shown, it mainly consists of two parts: a temporal neighborhood encoder module and a layer information comparison and learning module.

[0040] The temporal neighborhood encoder module is responsible for encoding the user's temporal behavior and hierarchical structure features. For a user at time t, its most recent reading behavior is first sampled, then the temporal features of these reading behaviors are encoded using a temporal encoding function, and finally, the information from these reading behaviors is aggregated through a self-attention mechanism.

[0041] The circle information contrastive learning module is responsible for encoding the user's circle information. First, it samples a user's positive samples (users in the same circle) and negative samples (users in different circles). Then, through contrastive learning, it maximizes the mutual information with the positive sample user representations and minimizes the mutual information with the negative sample user representations. Finally, the user vector output by the temporal encoder and the vector representing circle features are concatenated to predict the user's next article to read. The following section first gives the definition of article recommendation, and then details these two modules.

[0042] Given a set G = (U, D, H, X), where U is the set of user nodes and D is the set of articles, It is a record of the articles a user reads, representing the user's... i In t i I read the article at all times. i Let X be the initial feature matrix of users and articles. Given G, the goal of article recommendation is to obtain a learnable function that maps user nodes and article nodes to the embedding space and predicts the probability that any user will read a certain article at time t.

[0043] The temporal neighborhood encoder module consists of a self-attention-based user behavior encoder and a temporal encoding function. The self-attention-based user behavior encoder encodes the user's historical behavior information, while the temporal encoding function models the temporal features generated by the user's historical behavior. Specifically, if N(u,t) represents the set of all historical article nodes that have interacted with u before time t, the calculation process of the temporal user behavior encoder is shown in equation (1):

[0044]

[0045] Where, x u and x v The initial features are those of user u and the articles v they have read in their history. The feature that article v contains time-series information. This represents the personal interest representation obtained based on user u's historical reading behavior. `Enc(·)` is the encoder that encodes the structural features of its neighbors, `TempFunc(·)` is the function that models the temporal features of its historical reading behavior, `t'` is the time user u reads article `v`, and `t` is the current time. A self-attention mechanism is used because of its good expressive power. Specifically, the initial features of user u and the features of its historically read articles are first concatenated:

[0046] Then it is fed into a self-attention mechanism to learn interest by studying historical reading records:

[0047] Q = W q Xu,n K = W k X u,n V = W v X u,n (4)

[0048]

[0049] Where Q is the query matrix in the self-attention mechanism, K is the key matrix in the self-attention mechanism, V is the value matrix in the self-attention mechanism, and W... k W q and W v These are the learnable parameter matrices, Att is the abbreviation for the self-attention function, softmax() is the softmax normalization function, and d is the number of columns in the K matrix.

[0050] Final decision The last column of the output matrix, i.e., the vector corresponding to position u, serves as the user's personal interest representation.

[0051] The `TempFunc()` function encodes the time interval between the historical reading behavior at time `t'` and the current time `t`. Here, trigonometric functions are used to encode the time interval. Specifically, for the article `v` read by user `u` at time `t'`, it is mapped to a 2D... t The formula for dimensional time encoding is:

[0052]

[0053] Where Δt=t-t', Let be a learnable vector. Combining equations (1) and (2), the temporal user behavior encoder can be uniformly written as:

[0054]

[0055] The circle information comparison learning module aims to learn information about the user's circle, thereby enhancing the user's interest representation through circle information. To effectively learn the dynamic representation of the user's circle, this invention proposes a comparison learning framework.

[0056] The framework mainly consists of the following two parts: (1) For node u at time t, sample the positive and negative samples of its sphere, where the positive samples are users belonging to the same sphere as u, and the negative samples are users not in the same sphere as u. (2) Through contrastive learning, make the representation of user u as similar as possible to that of the positive sample users, and as dissimilar as possible to that of the negative sample users.

[0057] Sampling users belonging to the same sphere as user u requires considering both structural and temporal information while avoiding excessive computational overhead. Triangular connectivity is a significant structural feature within spheres, and many studies utilize triangles to mine spheres in graphs (see Non-Patent Literature 1). Therefore, other users connected to user u through triangles are considered candidate positive samples, meaning users who have clicked on at least two articles with u. If no neighbor exists within the same triangle as u, ordinary two-hop neighbors are selected as candidate positive samples. Negative samples are randomly sampled from users outside the candidate positive samples. Considering the high overhead of enumerating all triangles when u has many neighbors, a limit is set on the number of candidate positive samples; the algorithm can exit the positive sample search when the number of positive samples reaches the limit.

[0058] Assuming the above sampling algorithm samples l p positive samples and l n Given a negative sample node, define u p,i and u n,j Let i and j be the i-th positive sample and j-th negative sample of node u, respectively. For the sampled positive and negative sample nodes, the temporal neighborhood encoder defined in equations (1) and (2) is used to encode their neighborhood structure and temporal information:

[0059]

[0060] Among them, Enc c By Enc t To save computational overhead, Enc c It shares parameters with the temporal neighborhood encoder. The layer representation does not use the representation corresponding to the center node, but instead uses the average of the representations of all input nodes. This allows the layer representation to better reflect neighborhood features rather than the features of the nodes themselves. The layer representation of node u is also obtained through Enc... c The calculation yielded:

[0061]

[0062] Since the node representation of u has already been calculated once in equation (4), the layer representation of u can be directly taken as the average of all node representations output in equation (4).

[0063] Finally, the neighborhood representation and circle representation The components are concatenated to obtain the final representation of node u:

[0064]

[0065] The model optimization objectives are divided into two parts: the first part is the optimization objective based on click behavior prediction, and the second part is the optimization objective based on circle information comparison learning.

[0066] First, we introduce the objective function based on click behavior prediction. Assume that at time t, the document clicked by user u is u. pos The unclicked document is u neg , then u and u pos and u neg The probability p that an edge is formed between them pos p neg They are respectively:

[0067]

[0068] Where ||·||² represents the element-wise squaring operation of the vector, and MLP is an abbreviation for Multi-layer Perceptron. The goal of the loss function based on binary cross-entropy is to make p pos p is close to 1. neg It is close to 0.

[0069] The following binary cross-entropy loss function is defined as the objective function for click behavior prediction:

[0070]

[0071] The following describes the optimization objective based on concentric circle information comparison learning. First, it predicts u and positive sample nodes u based on the concentric circle representation. p,i and negative sample node u n,j Probability within the same sphere:

[0072]

[0073] To maximize mutual information with positive sample users and minimize mutual information with negative sample users, referring to non-patent literature 2, this invention designs a contrastive learning loss function based on DGI:

[0074]

[0075] Finally, the optimization objective function (or total loss function) is the sum of the loss function based on click behavior prediction and the loss function based on circle-based comparative learning:

[0076] L = L p +L c (17)

[0077] Finally, the ADAM optimizer is used to optimize the model based on the loss function, and the trained model can be obtained.

[0078] Non-patent literature 1: Yin H, BensonA R, Leskovec J, et al. Local higher-order graphclustering[C] / / Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. Halifax, NS, Canada, August 13-17, 2017. New York: ACM, 2017: 555-564.

[0079] Non-patent literature 2: P,Fedus W,Hamilton WL,et al.Deep graph infomax[C] / / Proceedings of the 7th International conference on learning representations ICLR 2019.New Orleans,Louisiana,United States,May 6-May 9,2019.

[0080] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention. For those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, and these should all be considered as falling under the protection of the present invention.

Claims

1. A personalized article recommendation system based on dynamic circle layer information, characterized in that, The system comprises a time neighborhood encoder module and a circle layer information contrast learning module; wherein the time neighborhood encoder module is composed of a user behavior encoder based on self-attention and a time encoding function; the time neighborhood encoder module is used to encode the time behavior and circle layer structure features of the user, specifically comprising: for a user at time t, first sampling the nearest reading behavior, then using the time encoding function to encode the time sequence features of the reading behavior, then aggregating the information of the reading behavior through the self-attention mechanism to obtain the neighborhood representation about personal interest based on the historical reading behavior of the user u The circle layer information contrast learning module is configured to encode the circle layer information of the user, specifically including: first, sampling positive samples and negative samples of the user, wherein the positive samples include users in the same circle layer, and the negative samples include users not in the same circle layer; then, maximizing mutual information between the user and the positive sample user representation and minimizing mutual information between the user and the negative sample user representation through contrast learning; finally, outputting a vector representing the circle layer information by using a time sequence encoder and the neighborhood representation of the personal interest and the vector representing the circle layer information of the user u to obtain the final representation of the user u The for predicting an article to be read by the user; wherein the vector representing the circle layer information of the user u directly takes the average of the neighborhood representations of all nodes output by the time sequence neighborhood encoder module Wherein, the optimization objective function based on click behavior prediction is used to optimize the ​ The sampling of the positive sample and the negative sample for the user specifically comprises: regarding other users connected with the user u through a triangle as candidate positive samples, which means that the user who has clicked at least two articles with u is a candidate positive sample, and if there is no neighbor of u in the same triangle, a common 2-hop neighbor is selected as a candidate positive sample; the negative sample is randomly sampled from users other than the candidate positive samples. The optimization objective function is a sum of a click behavior prediction-based loss function and a circle layer contrast learning loss function: L = L p + L c , An ADAM optimizer is used to optimize according to the optimization objective function.

2. The system of claim 1, wherein, Optimizing the objective function based on click behavior prediction includes: Specifically includes: At time t, the article clicked by user u is u pos And the article not clicked by user u is u neg Then, u and u pos Form an edge between u and u neg The probability p pos , p neg Respectively: Where ||·||2 represents the element-wise squaring operation of a vector, and MLP stands for Multilayer Perceptron. For u pos The final expression, For u neg The final representation is defined as follows: the binary cross-entropy loss function is defined as the objective function for click behavior prediction: L p = log(p pos )- log(1-p neg ), The goal of the loss function based on binary cross-entropy is to make p pos close to 1 so that p neg close to 0.

3. The system of claim 1, wherein, The maximization of the mutual information between the user and the positive sample user representation and the minimization of the mutual information between the user and the negative sample user representation through contrast learning specifically comprise: Suppose l p positive samples and l n negative samples are sampled, define u p,i and u n,j as the i-th positive sample and the j-th negative sample of node u, respectively; for the sampled positive and negative sample nodes, the temporal neighborhood encoder is used to encode the neighborhood structure and temporal information thereof: where Enc c is a temporal neighborhood encoder, is a neighborhood representation of positive samples, is a neighborhood representation of negative samples, and are initial features of positive and negative samples, respectively, x v is an initial feature of a history read article v, N(u p,i ,t) is a set of all history article nodes that have interacted with u p,i before t time, N(u n,j ,t) is a set of all history article nodes that have interacted with u n,j before t time; predicting u based on a vector representing circle layer information and positive sample nodes u p,i and negative sample nodes u n,j probability within the same circle layer: In order to maximize the mutual information with the positive sample user and minimize the mutual information with the negative sample user, a contrast learning loss function is defined:

4. The system of claim 3, wherein, Enc c Parameters are shared with the temporal neighborhood encoder module. 5.A method for article recommendation based on dynamic circle layer information, characterized in that, The method is based on the article recommendation system according to any one of claims 1-4, and the method comprises: For a user at time t, first sample its recent reading behavior, then encode the time sequence feature of the reading behavior by using time encoding function, then aggregate the information of the reading behavior by self-attention mechanism, get the neighborhood representation about personal interest based on the history reading behavior of user u The positive sample of the user is sampled, and the positive sample includes users in the same circle layer, and the negative sample includes users not in the same circle layer; then, mutual information between the user and the positive sample user representation is maximized through contrast learning, and mutual information between the user and the negative sample user representation is minimized; finally, a vector representing circle layer information output by a time sequence encoder is used and a neighborhood of personal interests is represented and a vector representing circle layer information are spliced to obtain a final representation of the user u The are used to predict an article to be read by the user. Wherein, the optimization objective function based on click behavior prediction is used to optimize the ​

Images