Personalized article recommendation system based on dynamic circle layer information and recommendation method thereof

Abstract

The invention discloses a personalized article recommendation system based on dynamic circle layer information. The personalized article recommendation system comprises a time sequence neighborhood encoder module and a circle layer information comparison learning module. Wherein the time sequence neighborhood encoder module is responsible for encoding time sequence behaviors and ring layer structure characteristics of users. And the circle layer information comparison learning module is responsible for coding the circle layer information of the user. According to the article recommendation system based on the dynamic circle layer information and the recommendation method thereof, a user-product interaction time sequence bipartite graph is established based on user historical behavior data, the circle layer where the user is located is found through a comparative learning method, and by fusing personal interests of the user and circle layer interest characteristics, the user experience is improved. The model can utilize the circle layer features to supplement the personal interest features when the user is lack of personal historical behaviors so as to relieve the cold boot problem. In addition, due to the fact that the ring layer features comprise behaviors of a plurality of users, the influence of noise in the behavior of a single user is smaller.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a personalized article recommendation system and a recommendation method thereof. Background Art

[0002] Recommendation system is one of the important technologies in today's Internet applications. It aims to help users find interesting content and improve the efficiency of push. The article recommendation system aims to recommend articles of interest to users based on their personal reading interests. At present, article recommendation systems have been widely used on content platforms such as Toutiao, Zhihu and Xiaohongshu. Existing recommendation systems mainly model users' reading interests through their historical clicks and readings. Based on the technology of modeling user behavior, existing recommendation systems are mainly divided into traditional methods and deep learning-based methods.

[0003] Traditional product recommendation methods are mainly based on static data, such as users' personal preferences and static features of products. These methods model user interests through techniques such as collaborative filtering, content-based filtering, and matrix decomposition. Collaborative filtering methods usually generate recommendations based on user historical behavior and the behavior of similar users. Content-based filtering matches based on product features and user preferences. Matrix decomposition-based methods directly embed user / product IDs into a vector and use inner products to model user-product interactions. However, these methods cannot effectively represent collaborative information. In addition, they have difficulty in handling the dynamics of temporal information and user needs. The timeliness of products and the evolution of user interests are often ignored, and these methods fail to effectively capture the information of time and the dynamic changes of user needs.

[0004] In recent years, deep learning-based methods, especially graph neural networks (GNNs), have attracted widespread attention in product recommendation systems. GNNs construct an interaction graph between users and products while modeling the feature information of the graph structure information. This enables the model to effectively encode the user's potential interests through deep neural networks, thereby providing more accurate recommendations. However, there are two problems with the current personalized recommendation methods based on graph neural networks. 1) Many new users lack historical reading behavior, so it is difficult to model personal interests through graph neural network methods. 2) There is a lot of noise in the user's click behavior, which will interfere with the modeling of graph neural networks.

[0005] Therefore, it is necessary to design a more effective personalized article recommendation method. Summary of the invention

[0006] In view of the above problems, the present invention aims to propose an article recommendation system and a recommendation method based on dynamic circle information. The article recommendation system and the recommendation method establish a temporal bipartite graph of user-product interaction based on user historical behavior data, find the user's circle through comparative learning, and integrate the user's personal interests and circle interest features. The model can use circle features to supplement personal interest features when the user's personal historical behavior is lacking, so as to alleviate the cold start problem. In addition, since the circle features include the behaviors of multiple users, they are less affected by the noise in the behavior of a single user.

[0007] The present invention is specifically implemented through the following technical solutions:

[0008] An article recommendation system based on dynamic circle information, the system comprising: a temporal neighborhood encoder module and a circle information comparison learning module; wherein,

[0009] The temporal neighborhood encoder module consists of a user behavior encoder based on self-attention and a time encoding function; the temporal neighborhood encoder module is used to encode the user's temporal behavior and circle structure characteristics, specifically including: for a user at time t, first sample his recent reading behavior, then use the time encoding function to encode the temporal characteristics of the reading behavior, then aggregate the information of the reading behavior through the self-attention mechanism to obtain a neighborhood representation of personal interests based on the historical reading behavior of user u

[0010] The circle information contrast learning module is used to encode the circle information of the user; specifically, it includes: firstly sampling the positive sample and negative sample of the user, the positive sample includes the users in the same circle, and the negative sample includes the users in different circles; then, by contrast learning, the mutual information between the user and the positive sample user is minimized, and the mutual information between the user and the negative sample user is maximized; finally, the vector representing the circle information output by the temporal encoder is used And the neighborhood of personal interest is represented by and the vector representing the circle information (hereinafter referred to as circle representation) Concatenate to get the final representation of user u Said For predicting the articles that the user is about to read;

[0011] Among them, the optimization objective function based on click behavior prediction is used to Make optimizations.

[0012] As a further improvement of the present invention, an optimization objective function based on click behavior prediction is used to The optimizations include:

[0013] At time t, the article clicked by user u is u pos , and the unclicked articles are u neg , then u and u pos and u neg The probability p of forming an edge between pos 、p neg They are:

[0014]

[0015] where ||·|| 2 It is a vector element-by-element square operation. for u pos The final expression of for u neg The final representation of , defines the following binary cross entropy loss function as the objective function based on click behavior prediction:

[0016]

[0017] The goal of the binary cross entropy loss function is to make p pos is close to 1 so that p neg Close to 0.

[0018] As a further improvement of the present invention, the sampling of positive samples and negative samples for the user specifically includes: considering other users connected with user u through a triangle as candidate positive samples, which means users who have clicked on at least two articles with u. If there are no neighbors in the same triangle as u, then ordinary 2-hop neighbors are selected as candidate positive samples; negative samples are randomly sampled from users other than the candidate positive samples.

[0019] As a further improvement of the present invention, wherein minimizing the mutual information between the user and the positive sample user representation and maximizing the mutual information between the user and the negative sample user representation through contrastive learning specifically includes:

[0020] Assume that the sample l p positive samples and l n negative sample nodes, define u p,i and u n,j are the i-th positive sample and j-th negative sample of node u respectively; for the sampled positive and negative sample nodes, the temporal neighborhood encoder is used to encode their neighborhood structure and temporal information:

[0021]

[0022] Among them, Enc c is a temporal neighborhood encoder, is the neighborhood representation of the positive sample, is the neighborhood representation of negative samples, and are the initial features of positive and negative samples, respectively, x v are the initial features of the articles v that have been read historically, For all the p,i A collection of historical article nodes that have interacted. For all the n,j A collection of historical article nodes that have interacted;

[0023] Predict u and positive sample node u based on circle representation p,i And the negative sample node u n,j Probability of being in the same circle:

[0024]

[0025] In order to maximize the mutual information with positive sample users and minimize the mutual information with negative sample users, the contrastive learning loss function is defined:

[0026]

[0027] As a further improvement of the present invention, wherein Enc c Shares parameters with the temporal neighborhood encoder module.

[0028] As a further improvement of the present invention, the circle of user u is represented by Directly take the average of the neighborhood representations of all nodes output by the temporal neighborhood encoder module.

[0029] As a further improvement of the present invention, the optimization objective function is the sum of the loss function based on click behavior prediction and the loss function based on circle comparison learning:

[0030]

[0031] The ADAM optimizer is used to perform optimization according to the optimization objective function.

[0032] The present invention also provides an article recommendation method based on dynamic circle information, the method comprising:

[0033] For a user at time t, first sample his recent reading behavior, then use the time encoding function to encode the temporal characteristics of the reading behavior, and then aggregate the information of the reading behavior through the self-attention mechanism to obtain a neighborhood representation of personal interests based on the historical reading behavior of user u

[0034] For the user, a positive sample and a negative sample are sampled, wherein the positive sample includes users in the same circle, and the negative sample includes users in different circles; then, the mutual information between the user and the positive sample user is minimized through contrastive learning, and the mutual information between the user and the negative sample user is maximized; finally, the vector representing the circle information output by the temporal encoder is used And the neighborhood of personal interest is represented by and circle representation Concatenate to get the final representation of user u Said For predicting the articles that the user is about to read;

[0035] Among them, the optimization objective function based on click behavior prediction is used to Make optimizations.

[0036] The beneficial effects of the present invention are as follows: the article recommendation system based on dynamic circle information and its recommendation method of the present invention establish a time-series bipartite graph of user-product interaction based on user historical behavior data, find the user's circle through comparative learning, and by integrating the user's personal interests and circle interest features, the model can use circle features to supplement personal interest features when the user's personal historical behavior is lacking, so as to alleviate the cold start problem. In addition, since the circle features include the behaviors of multiple users, they are less affected by the noise in the behavior of a single user. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an overall framework diagram of the article recommendation system based on dynamic circle information of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0039] The article recommendation system based on dynamic circle information of the present invention is shown in the attached Figure 1 As shown in the figure, it mainly includes two parts: the temporal neighborhood encoder module and the circle information comparison learning module.

[0040] The temporal neighborhood encoder module is responsible for encoding the user's temporal behavior and circle structure characteristics. For a user at time t, first sample his recent reading behavior, then use the time encoding function to encode the temporal characteristics of these reading behaviors, and then aggregate the information of these reading behaviors through the self-attention mechanism.

[0041] The circle information contrast learning module is responsible for encoding the user's circle information. First, it samples the positive samples (users in the same circle) and negative samples (users in different circles) of a user. Then, through contrast learning, it minimizes the mutual information with the positive sample user representation and maximizes the mutual information with the negative sample user representation. Finally, the user vector output by the temporal encoder is concatenated with the vector representing the circle features to predict the next article that the user will read. The following first gives the definition of article recommendation, and then introduces these two modules in detail.

[0042] Given a set G = (U, D, H, X), where U is the set of user nodes and D is the set of articles, It is the record of the user reading the article, representing user u i In t i Read the article at the time i , X is the initial feature matrix of users and articles. Given G, the purpose of article recommendation is to obtain a learnable function that maps user nodes and article nodes to the embedding space and predicts the probability of any user reading an article at time t.

[0043] The temporal neighborhood encoder module consists of a self-attention-based user behavior encoder and a temporal encoding function. The self-attention-based user behavior encoder encodes the user's historical behavior information, while the temporal encoding function models the temporal features generated by the user's historical behavior. Specifically, if represents the set of all historical article nodes that have interacted with u before time t, then the calculation process of the temporal user behavior encoder is shown in formula (1):

[0044]

[0045]

[0046] Among them, x u and x v are the initial features of user u and the articles v he has read in the past, is the feature of article v containing temporal information, represents the personal interest representation based on the historical reading behavior of user u, Enc(·) is the encoder that encodes the neighbor structure features, TempFunc(·) is the function that models the temporal features of historical reading behavior, t′ is the time when user u read article v, and t is the current time. The self-attention mechanism is used because it has good expression ability. Specifically, the initial features of user u and the features of the articles he has read in the past are first concatenated:

[0047]

[0048] This is then fed into the self-attention mechanism to learn interests by learning historical reading records:

[0049] Q=W q X u,n ,K=W k X u,n ,V=W v X u,n (4)

[0050]

[0051] Among them, Q is the query matrix in the self-attention mechanism, K is the Key matrix in the self-attention mechanism, V is the value matrix in the self-attention mechanism, and W k , W q and W v are learnable parameter matrices, Att is the abbreviation of self-attention function (Attention), softmax() is the softmax normalization function, and d is the number of columns in the K matrix.

[0052] Final The last column of the output matrix, i.e., the vector corresponding to the position of u, is used as the user's personal interest representation.

[0053] The function TempFunc(·) encodes the time interval between the time t′ corresponding to the historical reading behavior and the current time t. Here, trigonometric functions are used to encode the time interval. Specifically, for the article v read by user u at time t′, map it to the 2d t The formula for dimensional time encoding is:

[0054]

[0055] Where, Δt=t-t', is a learnable vector. By combining equations (1) and (2), the temporal user behavior encoder can be written uniformly as:

[0056]

[0057] The circle information comparison learning module aims to learn the information of the user's circle, and then enhance the user's interest expression through the circle information. In order to effectively learn the dynamic expression of the user's circle, the present invention proposes a comparison learning framework.

[0058] The framework mainly consists of the following two parts: (1) For node u at time t, sample positive and negative samples of its circle, where positive samples are users in the same circle as u, and negative samples are users in a different circle from u. (2) Through contrastive learning, the representation of user u is made as similar as possible to that of the positive sample users, while as dissimilar as possible to that of the negative sample users.

[0059] Sampling users who belong to the same circle as u requires considering both structural and temporal information, while not causing excessive computational overhead. Triangular connectivity is a notable structural feature in circles, and many studies have used triangles to mine circles in graphs. Please refer to non-patent document 1. Therefore, other users who are connected to user u through triangles are regarded as candidate positive samples, which means users who have clicked on at least two articles with u. If there are no neighbors in the same triangle as u, an ordinary 2-hop neighbor is selected as a candidate positive sample. Negative samples are randomly sampled from users other than candidate positive samples. Considering that when u has many neighbors, the cost of enumerating all triangles is high, a limit is set for the number of candidate positive samples. When the number of positive samples reaches the limit, the algorithm for searching for positive samples can be exited.

[0060] Assume that the above sampling algorithm samples l p positive samples and l n negative sample nodes, define u p,i and u n,j are the i-th positive sample and j-th negative sample of node u respectively. For the sampled positive and negative sample nodes, the temporal neighborhood encoder defined in equations (1) and (2) is used to encode their neighborhood structure and temporal information:

[0061]

[0062] Among them, Enc c By Enc t To save computational overhead, Enc c The circle representation does not use the representation corresponding to the central node, but uses the average of all input node representations as the circle representation. The advantage of this is that the circle representation better reflects the neighborhood characteristics rather than the characteristics of the node itself. The circle representation of node u is also obtained by Enc c The calculation results are:

[0063]

[0064] Since the node representation of u has been calculated once in formula (4), the circle representation of u can be directly taken as the average of all node representations output in formula (4).

[0065] Finally, the neighborhood is represented by and circle representation Concatenate to get the final representation of node u:

[0066]

[0067] The model optimization goal is divided into two parts. The first part is the optimization goal based on click behavior prediction, and the second part is the optimization goal based on circle information comparative learning.

[0068] First, we introduce the objective function based on click behavior prediction. Assume that the document clicked by user u at time t is u pos , and the unclicked document is u neg , then u and u pos and u neg The probability p of forming an edge between pos , p neg They are:

[0069]

[0070] where ||·|| 2 is the vector element-by-element square operation, and MLP is the abbreviation of Multi-layer Perceptron. The goal of the loss function based on binary cross entropy is to make p pos is close to 1 so that p neg Close to 0.

[0071] The following binary cross entropy loss function is defined as the objective function based on click behavior prediction:

[0072]

[0073] The following introduces the optimization goal based on circle information contrast learning. First, based on the circle representation prediction u and the positive sample node u p,i And the negative sample node u n,j Probability of being in the same circle:

[0074]

[0075] In order to maximize the mutual information with positive sample users and minimize the mutual information with negative sample users, referring to non-patent document 2, the present invention designs a contrastive learning loss function based on DGI:

[0076]

[0077] Finally, the optimization objective function (or total loss function) is the sum of the loss function based on click behavior prediction and the loss function based on circle comparison learning:

[0078]

[0079] Finally, use the ADAM optimizer to optimize according to the loss function to obtain the trained model.

[0080] Non-patent literature 1: Yin H, Benson AR, Leskovec J, et al. Local higher-order graphclustering[C] / / Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. Halifax, NS, Canada, August 13-17, 2017. New York: ACM, 2017: 555-564.

[0081] Non-patent document 2: P,Fedus W,Hamilton WL,et al.Deep graph infomax[C] / / Proceedings of the 7th International conference on learning representations ICLR 2019.New Orleans,Louisiana,United States,May 6-May 9,2019.

[0082] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention. For ordinary technicians in this field, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A personalized article recommendation system based on dynamic circle information, characterized in that: The system includes: a temporal neighborhood encoder module and a circle information comparison learning module; wherein the temporal neighborhood encoder module is composed of a user behavior encoder based on self-attention and a time encoding function; the temporal neighborhood encoder module is used to encode the user's temporal behavior and circle structure characteristics, specifically including: for a user at time t, first sampling his most recent reading behavior, then using the time encoding function to encode the temporal characteristics of the reading behavior, and then aggregating the information of the reading behavior through the self-attention mechanism to obtain a neighborhood representation of personal interests based on the historical reading behavior of user u The circle information contrast learning module is used to encode the circle information of the user; specifically, it includes: firstly sampling the positive sample and negative sample of the user, the positive sample includes the users in the same circle, and the negative sample includes the users in different circles; then, by contrast learning, the mutual information between the user and the positive sample user is minimized, and the mutual information between the user and the negative sample user is maximized; finally, the vector representing the circle information output by the temporal encoder is used And the neighborhood of personal interest is represented by and the vector representing the circle information Concatenate to get the final representation of user u Said For predicting the articles that the user is about to read; Among them, the optimization objective function based on click behavior prediction is used to Make optimizations.

2. The system according to claim 1, wherein: The optimization objective function based on click behavior prediction is used to The optimization includes: At time t, the article clicked by user u is u pos , and the unclicked articles are u neg , then u and u pos and u neg The probability p of forming an edge between pos 、p neg They are: Where ||·||2 is the vector element-by-element square operation, MLP is a multi-layer perceptron, for u pos The final expression of for u neg The final representation of , defines the following binary cross entropy loss function as the objective function based on click behavior prediction: The goal of the binary cross entropy loss function is to make p pos is close to 1 so that p neg Close to 0.

3. The system according to claim 1, wherein: The sampling of positive samples and negative samples for the user specifically includes: considering other users connected with user u through a triangle as candidate positive samples, which means users who have clicked on at least two articles with u; if there is no neighbor in the same triangle as u, then selecting an ordinary 2-hop neighbor as a candidate positive sample; negative samples are randomly sampled from users other than the candidate positive samples.

4. The system according to claim 1, wherein: The step of minimizing the mutual information between the user and the positive sample user representation and maximizing the mutual information between the user and the negative sample user representation by contrastive learning specifically includes: Assume that the sample l p positive samples and l n negative sample nodes, define u p,i and u n,j are the i-th positive sample and j-th negative sample of node u respectively; for the sampled positive and negative sample nodes, the temporal neighborhood encoder is used to encode their neighborhood structure and temporal information: Among them, Enc c is a temporal neighborhood encoder, is the neighborhood representation of the positive sample, is the neighborhood representation of negative samples, and are the initial features of positive and negative samples, respectively, x v are the initial features of the articles v that have been read historically, For all the p,i A collection of historical article nodes that have interacted. For all the n,j A collection of historical article nodes that have interacted; Predict u and positive sample node u based on the vector representing circle information p,i And the negative sample node u n,j Probability of being in the same circle: In order to maximize the mutual information with positive sample users and minimize the mutual information with negative sample users, the contrastive learning loss function is defined:

5. The system according to claim 4, wherein: Enc c Shares parameters with the temporal neighborhood encoder module.

6. The system according to claim 1, wherein: The vector representing the circle information of the user u Directly take the average of the neighborhood representations of all nodes output by the temporal neighborhood encoder module.

7. The system according to claim 1, wherein: The optimization objective function is the sum of the loss function based on click behavior prediction and the loss function based on circle comparison learning: The ADAM optimizer is used to perform optimization according to the optimization objective function.

8. An article recommendation method based on dynamic circle information, characterized in that: The method is based on the article recommendation system according to any one of claims 1 to 7, and the method comprises: For a user at time t, first sample his recent reading behavior, then use the time encoding function to encode the temporal characteristics of the reading behavior, and then aggregate the information of the reading behavior through the self-attention mechanism to obtain a neighborhood representation of personal interests based on the historical reading behavior of user u For the user, a positive sample and a negative sample are sampled, wherein the positive sample includes users in the same circle, and the negative sample includes users in different circles; then, the mutual information between the user and the positive sample user is minimized through contrastive learning, and the mutual information between the user and the negative sample user is maximized; finally, the vector representing the circle information output by the temporal encoder is used And the neighborhood of personal interest is represented by and the vector representing the circle information Concatenate to get the final representation of user u Said For predicting the articles that the user is about to read; Among them, the optimization objective function based on click behavior prediction is used to Make optimizations.

Images