A diversified news recommendation method based on determinant point process

Abstract

The application discloses a diversified news recommendation method based on a determinant point process, and comprises the following steps: training a news encoder and a user encoder through a news title text and user historical reading; obtaining a representation vector of a user u via the user encoder; obtaining a list of representation vectors of candidate news via the news encoder; calculating the similarity between each two of the candidate news to obtain a similarity matrix S; obtaining a score vector r of each news in the candidate news list; introducing a weighting factor alpha to form a kernel matrix L'; iterating a greedy algorithm for the kernel matrix L' n times, returning a news y with the maximum total revenue in the candidate news list in each iteration; and iterating n times until a target news subset with a size of n is formed as the output result of the method. u m u i Compared with other comparative methods, the accuracy of the application is in the same order of magnitude, and the diversity is obviously improved.​​​

Description

Technical Field

[0001] This invention belongs to the technical field of news recommendation and machine learning research. It proposes a diversified news recommendation method based on a matrix-based point process, which effectively improves recommendation diversity while ensuring recommendation accuracy. Background Technology

[0002] In response to the significant problem of information overload in the information age, recommender systems and algorithms have the ability to uncover users' hidden interests and filter content that users may be interested in, thereby improving the efficiency of various application platforms. Traditional recommendation methods, deep learning methods, and various hybrid recommendation methods have all achieved good results in recommender systems. However, existing recommendation methods typically attempt to build models that accurately predict the interaction between users and items. The most accurate recommendations may overly favor popular items and are likely to recommend known items, which is not necessarily the most useful outcome for users.

[0003] Therefore, beyond accuracy, diversity has gradually become an important goal for recommender systems. Firstly, diverse recommendations can broaden users' horizons, uncover hidden interests, avoid recommending known items, and prevent duplicate recommendations, thus providing users with truly meaningful information. Secondly, diverse recommender systems can mitigate phenomena such as the "information cocoon" effect and the "echo chamber effect." Focusing on the diversity of news recommendations and using diverse recommendation algorithms can achieve a certain "breaking free" effect at the algorithmic level, avoiding the risks of rigid individual thinking and social group division, and using technology to assume social responsibility.

[0004] Existing diversification recommendation algorithms can be broadly categorized into three types: post-processing diversification strategies, supervised learning-based diversification strategies, and determinant-based diversification strategies. Determinant-based diversification strategies are efficient algorithms that elegantly combine the objectives of diversity and relevance, featuring clear structure, high computational efficiency, and strong scalability when combined with various cutting-edge recommendation models such as knowledge graphs and reinforcement learning. However, existing methods do not adequately integrate various optimization strategies for accelerating determinant-based diversification inference with cutting-edge personalized recommendation methods. Therefore, using determinant-based diversification methods to enhance the diversity of news recommendations is necessary. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the shortcomings of the single evaluation of accuracy in the existing technology, and to provide a diversified news recommendation method based on a matrix point process, which improves the diversity of news recommendations while ensuring the accuracy of news recommendations.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is: a diversified news recommendation method based on a matrix-based point process, the specific steps of which are as follows:

[0007] A diversified news recommendation method based on a deterministic point process includes the following steps:

[0008] Step S1: Construct an attention-based news encoder, including a text encoding layer, a word-level multi-head self-attention network layer, and a word-level attention network layer; the text encoding layer takes the title text of the input news g as a word sequence of length M [w1, w2, ..., w M After training the BERT text encoder, the output semantic vector sequence [e1, e2, ..., e] is obtained. M The word-level multi-head self-attention network layer has h self-attention heads, and the i-th... th Multi-head representation of a word The calculation method for multi-head representation is as follows:

[0009]

[0010]

[0011] in, It is the i-th th The word passes through the kth word. th The attention head and the jth th The relative importance of each word learned. and All are the kth th The parameters to be learned for the attention head; the i-th attention head in the word-level attention network. th Attention weights for each word The calculation steps are as follows:

[0012]

[0013]

[0014] Where V w v w These are all parameters to be learned in the network, q w It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated after softmax normalization. The final output of the news encoder is a weighted sum of the multi-head representations of each word. f t Let be the representation vector of news item t;

[0015] Step S2: Based on the news encoder in Step S1, construct the user encoder, including a news-level multi-head self-attention network layer and a news-level attention network layer; process the user's historical reading list R. uThe project is input one by one into the news encoder to obtain a news representation sequence of length M [r1, r2, ..., r M The news-level multi-head self-attention network layer has h self-attention heads, the i-th head... th Multiple sources indicated that the news... The calculation method for multi-head representation is as follows:

[0016]

[0017]

[0018] in, It is the i-th th News item via the kth th The attention head and the jth th The relative importance learned from each news item and All are the kth th The parameters to be learned for the i-th attention head; in a news-level attention network... th Attention weights for each word The calculation steps are as follows:

[0019]

[0020]

[0021] Where V n v n These are all parameters to be learned in the network, q n It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated using softmax normalization. The final output of the user encoder is the weighted sum of the above steps: P u Let be the representation vector of user u;

[0022] Step S3: During the training phase, the encoding of user u is represented as P. u The encoding of news t is f t For each positive sample, K=4 negative samples are randomly selected. Predicting whether a user clicks on a news article is treated as an independent prediction task. The inner product is used to calculate the user's rating for K+1=5 candidate news articles. The scores are regularized using the softmax function, and the click probability of a positive sample is... The training loss function is the sum of the negative log probabilities of all positive samples. Where S is the set of positive samples; the Adam gradient backpropagation optimization algorithm is used to train and learn the parameters to be learned in the news encoder and user encoder;

[0023] Step S4: Given a news recommendation task, there is a user u's historical news reading list and a candidate news list C of length m. u Input the historical news reading list into the user encoder trained in steps S1 to S3 to obtain the user feature representation vector P. u The candidate news titles are input one by one into the news encoder obtained in steps S1 to S3 to obtain a list of news feature representation vectors [f1, f2, ..., f m ];

[0024] Step S5: Calculate the pairwise similarity of news items in the candidate news list to obtain the similarity matrix S, where <f i ,f j >=S ij Calculate the relevance between the user and all news items in the candidate news list to obtain the user's rating vector r for each news item in the candidate news list. u The i-th element in the vector has <P u ,f i >=r i ;

[0025] Step S6: The diagonal vector is r u The diagonal matrix is ​​Diag(r) u Construct a kernel matrix L for the determinant point process algorithm. L is written in Gram matrix form as shown below, where B i B j For intermediate representation objects in the calculation:

[0026] L ij = i B j >= <r i f i ,r j f j >=r i r j <f i ,f j >=Diag(r u )·S·Diag(r u )

[0027] Suppose we start from candidate news list C u Select a subset Y of length n from the kernel matrix L, and construct a submatrix L from the kernel matrix L according to the corresponding positions of the elements in subset Y. Y Construct a sub-similarity matrix S from the similarity matrix S. Y From the rating vector r u Construct sub-score vector r Y The logarithmic determinant of the submatrix is ​​calculated as follows:​

[0028]

[0029] The former term of this log determinant measures the relevance score of the item subset, and the latter term measures the diversity score of the item subset; therefore, the log determinant log det(L corresponding to the submatrix of the measurement subset Y Y ) takes into account both the relevance and the diversity degree of the measurement subset;

[0030] Step S7: Introduce an adjustment trade-off factor θ∈[0,1] that balances relevance and diversity, and the score vector is updated to exp(αr u ), the kernel matrix is updated as L'=Diag(exp(αr u ))·S·Diag(exp(αr u ));

[0031] Step S8: Obtain a news subset of length n from the candidate news list C u , which requires iterating the following algorithm n times. Each iteration of the algorithm returns one piece of news y with the maximum total revenue from the candidate news list to be selected i , and the iteration is performed n times until a target subset of size n is formed The optimization problem to be processed in each iteration is

[0032] The specific method is as follows:

[0033] (1) Input the candidate news set Z, the kernel matrix L, and the size of the target selected news set is n;

[0034] (2) Initialize the intermediate coefficients for calculation:

[0035] (3) Initialize the output target set: select the first item to join the target set Y g ={j};

[0036] (4) Determine whether the selection task has been completed: determine whether #Y g <n, if not, jump to step (7), and if yes, proceed to the next step;

[0037] (5) Traverse all unselected items i∈Z\Y in order g , and update the coefficient e each time i =(L ji -<c j ,c i >) / d i ,

[0038] (6) Update the target set Y g =Y g ∪{j}, proceed to step (4);

[0039] (7) The algorithm ends and returns the target news set Y. g .

[0040] The advantages of this invention compared to the prior art are:

[0041] (1) This invention trains an attention-based news encoder and a user encoder using news headline text and the user's historical news reading data. The representation vector of user u via the user encoder is P. u The list of representation vectors for candidate news items obtained through the news encoder is [f1, f2, ..., f m Calculate the pairwise similarity of each news item in the candidate news list to obtain the similarity matrix S. Calculate the relevance between the user and all news items in the candidate news list to obtain the user's rating vector r for each news item in the candidate news list. u By introducing a tradeoff factor α to adjust for correlation and diversity, a kernel matrix L' = Diag(exp(αr) is formed. u ))·S·Diag(exp(αr u For the kernel matrix L', an iterative greedy algorithm is used n times. In each iteration, the algorithm returns the news item y with the highest total return from the list of candidate news items to be selected. i Iterate n times until a target news subset of size n is formed. As an output of the method, this invention achieves accuracy on the same order of magnitude and significantly improves diversity compared to other comparative methods.

[0042] (2) In the research field of diversified news recommendation, the two-stage method of applying post-processing strategy cannot guarantee the combined effect, and the one-stage method of applying supervised learning needs to find the most suitable objective function according to the requirements. However, the diversified strategy based on the determinant point process proposed in this invention is an efficient algorithm that elegantly combines the two objectives of diversity and relevance and has strong scalability. It is easy to combine with knowledge graphs and reinforcement learning and other cutting-edge recommendation models in the later stage. Attached Figure Description

[0043] Figure 1 This is a diagram of the overall architecture of the present invention;

[0044] Figure 2 Structural diagram of the news encoder of this invention;

[0045] Figure 3 The user encoder structure diagram of the present invention;

[0046] Figure 4The flowchart of the determinant point process algorithm of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] This invention is a diverse news recommendation method based on a matrix-based point process, with the model structure as follows: Figure 1 As shown, this invention trains a news encoder and a user encoder based on a multi-head attention mechanism, fully utilizing the attention information of news at the word level and the attention information of users at the news level. Then, user representations and news representations are applied to construct a kernel matrix, enabling this invention to achieve the same order of magnitude accuracy and significantly improve diversity compared to existing news recommendation methods.

[0049] Step S1: Construct an attention-based news encoder, with the model structure as follows: Figure 2 As shown, it includes a text encoding layer, a word-level multi-head self-attention network layer, and a word-level attention network layer. The text encoding layer takes the longest 30 words from the headline text of the input news g and uses them as a word sequence of length M [w1, w2, ..., w M After training the BERT text encoder, the output is a low-dimensional semantic vector sequence [e1, e2, ..., e...]. M A word-level multi-head self-attention network layer has h self-attention heads, the i-th... th Multi-head representation of a word The calculation method for multi-head representation is as follows:

[0050]

[0051]

[0052] in, It is the i-th th The word passes through the kth word. th The attention head and the jth th The relative importance of each word learned. and All are the kth th The parameters to be learned for the i-th attention head in a word-level attention network. th Attention weights for each word The calculation steps are as follows

[0053]

[0054]

[0055] Where V w v w These are all parameters to be learned in the network, qw It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated using softmax normalization. The final output of the news encoder is a weighted sum of the multi-head representations of each word: f t Let be the representation vector of news t.

[0056] Step S2: Based on the news encoder from Step S1, construct the user encoder. The model structure is as follows: Figure 3 As shown, this includes a news-level multi-head self-attention network layer and a news-level attention network layer. The user's historical reading list R... u The project is input one by one into the news encoder to obtain a news representation sequence of length M [r1, r2, ..., r M The news-level multi-head self-attention network layer has h self-attention heads, the i-th head... th Multiple sources indicated that the news... The calculation method for multi-head representation is as follows:

[0057]

[0058]

[0059] in, It is the i-th th News item via the kth th The attention head and the jth th The relative importance learned from each news item and All are the kth th The parameters to be learned for the i-th attention head in a news-level attention network. th Attention weights for each word The calculation steps are as follows:

[0060]

[0061]

[0062] Where V n v n These are all parameters to be learned in the network, q n It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated using softmax normalization. The final output of the user encoder is the weighted sum of the above steps: P u Let be the representation vector of user u.

[0063] Step S3: During the training phase, the encoding of user u is represented as P. u The encoding of news t is f t For each user, a maximum of 50 clicked news articles are randomly sampled. For each positive sample, K=4 negative samples are randomly sampled. Predicting whether a user clicks a news article is treated as an independent prediction task. The inner product is used to calculate the user's rating for K+1=5 candidate news articles. The scores are regularized using the softmax function, and the click probability of a positive sample is... The training loss function is the sum of the negative log probabilities of all positive samples. Here is the set of positive samples. The Adam gradient backpropagation optimization algorithm is used to train and learn the parameters to be learned in the news encoder and user encoder.

[0064] Step S4: Given a news recommendation task, there is a user u's historical news reading list and a candidate news list C of length m. u Input the historical news reading list into the user encoder trained in steps S1 to S3 to obtain the user feature representation vector P. u The candidate news titles are input one by one into the news encoder obtained in steps S1 to S3 to obtain a list of news feature representation vectors [f1, f2, ..., f m ].

[0065] Step S5: Calculate the pairwise similarity of news items in the candidate news list to obtain the similarity matrix S, where <f i ,f j >=S ij Calculate the relevance between the user and all news items in the candidate news list to obtain the user's rating vector r for each news item in the candidate news list. u ,have <P u ,f i >=r i .

[0066] Step S6: The diagonal vector is r u The diagonal matrix is ​​Diag(r) u Construct a kernel matrix L for the determinant point process algorithm. L is written in Gram matrix form as shown below, where B i B j For intermediate representation objects in the calculation:

[0067] L ij = i B j >= <r i f i ,r i f j >=r​i r j <f i ,f j >=Diag(r u )•S•Diag(r u )

[0068] Assuming from candidate project C u Select a subset Y of length n from the kernel matrix L, and construct a submatrix L from the kernel matrix L according to the corresponding positions of the elements in subset Y. Y Construct a sub-similarity matrix S from the similarity matrix S. Y From the rating vector r u Construct sub-score vector r Y The logarithmic determinant of the submatrix is ​​calculated as follows:

[0069]

[0070] The first term of the logarithmic determinant measures the relevance score of the item subset, and the second term measures the diversity score of the item subset. Therefore, the logarithmic determinant logdet(L) of the submatrix corresponding to subset Y is... Y It can take into account both the relevance and diversity of the subset.

[0071] Step S7: Introduce an adjusted tradeoff factor θ∈[0,1] to balance relevance and diversity, and update the score vector to exp(αr u ), The kernel matrix is ​​updated to L' = Diag(exp(αr) u ))·S·Diag(exp(αr u ))

[0072] Step S8: From the candidate news list C u To obtain a subset of news items of length n, the following algorithm needs to be iterated n times. The algorithm flowchart is as follows: Figure 4 As shown. In each iteration, the algorithm returns the news item y with the highest total return from the list of candidate news items to be selected. i Iterate n times until a target subset of size n is formed. The optimization problem to be addressed in each iteration is: All browsing log data of each user are aggregated together to form the user's historical features (50 randomly sampled clicked news items are used as historical features) and a candidate item list to form a recommendation task. n=10 and the weighting factor θ=0.8 is taken.

[0073] For the accuracy objective, the evaluation metrics used are AUC, mean reciprocal ranking (MRR), normalized cumulative loss gain (NDCG@5), and NDCG@10. For the diversity objective, the evaluation metrics used are the average in-list distance (ILAD) and the minimum in-list distance (ILMD), where S... ij This is the similarity value between pairs of items i and j, calculated using the cosine similarity of news representation vectors:

[0074]

[0075]

[0076] Table 1. Recommendation results of traditional news recommendation methods and this invention.

[0077]

[0078] Note: The table above shows the performance of the proposed method and the control model on the Microsoft real news dataset MIND-small in the United States. A brief introduction to the benchmark model follows.

[0079] 1. DeepFM is a model that combines factorization machines and deep neural networks. Here, we use the structure of LibFM (a classic recommendation algorithm based on matrix factorization).

[0080] 2. DKN learns news representations and user representations from news headlines using convolutional neural networks, and predicts recommendation results based on the similarity between historical news and candidate news.

[0081] 3. GRU uses a denoising autoencoder to learn news representations and a GRU network to learn user representations.

[0082] 4. NPA uses a personalized attention network to learn news and user representation.

[0083] 5. NRSM uses multi-head self-attention and additive attention mechanisms to learn news representations.

[0084] 6. LSTUR uses convolutional neural networks and attention mechanisms to learn news representations, randomly samples user browsing history as long-term user interests, and uses a GRU network to learn user representations from browsing history.

[0085] 7. MRR is the maximum marginal relevance model, which uses the weighted sum of item relevance and list similarity as the objective function to maximize it.

[0086] The experimental results show that the proposed diversified news recommendation method based on determinant-based point processes, for the MIND-small dataset, demonstrates better accuracy and significantly improved diversity compared to traditional algorithms that prioritize accuracy. Furthermore, compared to some diversified recommendation algorithms that do not use the determinant-based point process model (such as MRR), the proposed method also performs better in both accuracy and diversity.

[0087] The parts of this invention not described in detail are well-known in the art. The above embodiments are only used to illustrate the technical solutions of this invention and are not intended to limit the scope of the specific implementation. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of this invention as defined and determined by the claims. All inventions utilizing the concept of this invention are within the scope of protection.

Claims

1. A diversified news recommendation method based on a matrix-based point process, characterized in that, Includes the following steps: Step S1: Construct an attention-based news encoder, including a text encoding layer, a word-level multi-head self-attention network layer, and a word-level attention network layer; Step S2: Based on the news encoder in Step S1, construct the user encoder, including a news-level multi-head self-attention network layer and a news-level attention network layer; Step S3: During the training phase, the encoding of user u is represented as P. u The encoding of news t is f t For each positive sample, K=4 negative samples are randomly selected. Predicting whether a user clicks on a news article is treated as an independent prediction task. The inner product is used to calculate the user's rating for K+1=5 candidate news articles. The scores are regularized using the softmax function, and the click probability of a positive sample is... The training loss function is the sum of the negative log probabilities of all positive samples. Where S is the set of positive samples; the Adam gradient backpropagation optimization algorithm is used to train and learn the parameters to be learned in the news encoder and user encoder; Step S4: Given a news recommendation task, there is a user u's historical news reading list and a candidate news list C of length m. u Input the historical news reading list into the user encoder trained in steps S1 to S3 to obtain the user feature representation vector P. u The candidate news titles are input one by one into the news encoder obtained in steps S1 to S3 to obtain a list of news feature representation vectors [f1, f2, ..., f m ]; Step S5: Calculate the pairwise similarity of news items in the candidate news list to obtain the similarity matrix S, where <f i f j >=Si j Calculate the relevance between the user and all news items in the candidate news list to obtain the user's rating vector r for each news item in the candidate news list. u The i-th element in the vector has <P u f i >=r i ; Step S6: The diagonal vector is r u The diagonal matrix is ​​Diag(r) u Construct a kernel matrix L for the determinant point process algorithm. L is written in Gram matrix form as shown below, where B i B j For intermediate representation objects in the calculation: L ij =<B i ,B j >=<r i f i ,r j f j >=r i r j <f i ,f j >=Diag(r u )·S·Diag(r u ) Suppose we start from candidate news list C u Select a subset Y of length n from the kernel matrix L, and construct a submatrix L from the kernel matrix L according to the corresponding positions of the elements in subset Y. Y Construct a sub-similarity matrix S from the similarity matrix S. Y From the rating vector r u Construct sub-score vector r Y The logarithmic determinant of the submatrix is ​​calculated as follows: The first term of this logarithmic determinant measures the relevance score of the item subset, and the second term measures the diversity score of the item subset; therefore, the logarithmic determinant of the submatrix corresponding to subset Y is log det(L Y It takes into account both the relevance and diversity of the subsets. Step S7: Introduce an adjusted tradeoff factor θ∈[0,1] to balance relevance and diversity, and update the score vector to exp(αr u ), The kernel matrix is ​​updated to L′=Diag(exp(αr) u ))·S·Diag(exp(αr u )); Step S8: From the candidate news list C u To obtain a subset of news items of length n, the following algorithm needs to be iterated n times. In each iteration, the algorithm returns the news item y with the highest total return from the list of candidate news items to be selected. i Iterate n times until a target subset of size n is formed. The optimization problem to be addressed in each iteration is: The specific method is as follows: (1) Input candidate news set Z, kernel matrix L, and target selected news set size n. (2) Initialize the intermediate coefficients for calculation: c i =[], (3) Initialize the output target set: Select the first item Add to target set Y g ={j}; (4) Determine whether the selection task has been completed: determine whether there is #Y g <n, if not, jump to step (7), and if yes, proceed to the next step; (5) Iterate through all unselected items i∈Z\Y in order. g Each time the coefficient e is updated i =(L ji - <c j c i >) / d i c i =[c i e i ], (6) Update the target set Y g =Y g ∪{j}, proceed to step (4); (7) The algorithm ends and returns the target news set Y. g .

2. The method according to claim 1, characterized in that, The specific implementation method of step 1 is as follows: the text encoding layer takes the title text of the input news g as a word sequence of length M [w1, w2, ..., w...]. M After training the BERT text encoder, the output is a semantic vector sequence [e1, e2, ..., e...]. M The word-level multi-head self-attention network layer has h self-attention heads, and the i-th... th Multi-head representation of a word The calculation method for multi-head representation is as follows: in, It is the i-th th The word passes through the kth word. th The attention head and the jth th The relative importance of each word learned. and All are the kth th The parameters to be learned for the attention head; the i-th attention head in the word-level attention network. th Attention weights for each word The calculation steps are as follows: Where V w v w These are all parameters to be learned in the network, g w It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated using softmax normalization; the final output of the news encoder is a weighted sum of the multi-head representations of each word. f t Let be the representation vector of news t.

3. The method according to claim 2, characterized in that, The specific implementation method of step 2 is to retrieve the user's historical reading list R. u The project is input one by one into the news encoder to obtain a news representation sequence [r1, r2, ..., r] of length M. M The news-level multi-head self-attention network layer has h self-attention heads, the i-th head... th Multiple sources indicated that the news article... The calculation method for multi-head representation is as follows: in, It is the i-th th News item via the kth th The attention head and the jth th The relative importance learned from each news item and All are the kth th The parameters to be learned for the i-th attention head; in a news-level attention network... th Attention weights for each word The calculation steps are as follows: Where V n v n These are all parameters to be learned in the network, q n It is a query vector. It is an intermediate value in the calculation process. These are the final attention weights calculated using softmax normalization. The final output of the user encoder is the weighted sum of the above steps: P u Let be the representation vector of user u.

Images