A news recommendation method based on user interest modeling

Abstract

The application discloses a kind of personalized news recommendation method based on user interest modeling, including the following steps: obtaining news text data, pre-processing text data, deleting stop words, vectorization and the like operation;Vectorized news data is transported to text feature extractor for feature extraction, and news features are obtained;Build a news recommendation model based on user interest, combine residual graph convolution network module with user hot interest module, and output user click probability;Model training, model training is based on cross-entropy loss function training;The click prediction probability of a series of candidate news input is sorted, and the top K news set as the recommended result is output.The application is used for the personalized news recommendation of user, and can effectively improve the effectiveness of recommendation.

Description

Technical Field

[0001] This invention relates to the field of news recommendation, and more specifically to a deep learning-based news recommendation method based on user reading preferences. Background Technology

[0002] Personalized recommendation systems generate recommendation lists for users based on their behavior (such as search history and reading history). Traditional news recommendation methods can be broadly categorized into content-based filtering and collaborative filtering. While both methods can achieve news recommendation functionality to some extent, they both have inherent limitations, resulting in unsatisfactory actual recommendation performance and failing to meet real-world usage needs.

[0003] Content-based algorithms were initially the most popular and efficient methods, and were therefore widely used at the outset of any recommender system. In content-based filtering recommendation methods, news items that a user has not yet read are recommended by analyzing the attributes of news items the user has previously been interested in. These systems do not consider user ratings of these items. In content-based algorithms, the similarity calculation method directly affects the accuracy of the recommendation results. Several widely used similarity calculation methods include: ① Euclidean distance: One of the most common methods to measure the similarity between two vectors by calculating the sum of the squared distances of each element in the two vectors. ② Cosine similarity: The main idea of ​​this method is to measure the similarity between two vectors by calculating the cosine of the angle between them. ③ Pearson similarity: The Pearson correlation coefficient reflects the degree of linear correlation between two vectors. ④ Jaccard similarity: Jaccard similarity is often used to compare the similarity and differences between two finite sample sets. Compared with other collaborative filtering systems, content-based recommender systems have many advantages, such as user independence, transparency, and the ability to handle new items in the cold start problem. However, content-based methods also have some shortcomings, such as limited content analysis, overly specialized or lacking rating data from new users, and insufficient accuracy for certain specific problems.

[0004] In collaborative filtering, user ratings of news items are taken into account when making recommendations. Users with similar ratings are called neighbors and are used to predict ratings for unrated news items. The accuracy of collaborative filtering-based recommendation methods relies on effective modeling developed using different algorithms (such as SVM, Naive Bayes, decision trees, clustering, etc.). However, in real-world scenarios, many commercial recommendation systems use very sparse datasets because users typically don't rate every item, but only a small number of items.

[0005] In recent years, graph neural network models built on user-news interaction have modeled users and news as a bipartite graph structure, updated user interests with news embeddings, and calculated the similarity between candidate news and user interests to predict user click preferences. Such models have achieved excellent results in news recommendation tasks, but are still not suitable for sparse environments. Summary of the Invention

[0006] The technical problem this invention aims to solve is that, in sparse environments with only a few news items in interactive datasets, models built solely using interactive behaviors suffer from severe accuracy issues. This invention provides a deep learning-based news recommendation method that addresses the cold-start problem caused by uneven and sparse data distribution in existing collaborative filtering and content-based news recommendation methods.

[0007] The technical solution adopted by this invention to solve the above-mentioned technical problems is a news recommendation method based on user interest modeling, comprising the following steps:

[0008] Step 1: Preprocess the news data according to the required news attribute information to construct a news dataset; in the news dataset used for training, if there is interaction between the user and the news, it is marked as a positive sample, indicating that the user has read the news; otherwise, it is marked as a negative sample.

[0009] Step 2: Construction and training of the news recommendation model:

[0010] The news recommendation model is trained using deep learning data from the news dataset. When the preset stopping training condition is met, the trained news recommendation network model is obtained.

[0011] The news recommendation network model is used to output the probability of a user clicking on candidate news items. It includes a feature extraction module, a residual graphical convolutional network (GCN), a user hot interest module, and a fully connected neural network. The feature extraction module includes a click sequence feature extractor and a news text feature extractor, used to extract recent click sequence features and news text features, respectively. The residual GCN receives news text features and outputs news embeddings and long-term user interest features. The user hot interest module receives recent click sequence features and outputs user hot interest features. The fully connected neural network combines the user's long-term interest features with the user hot interest features, and the combined result serves as the user's final interest representation.

[0012] Step 3: The news recommendation model outputs the predicted user clicks:

[0013] Given a user and candidate news items, the candidate news items are filtered and preprocessed based on a preset input data format to obtain news data. The news data is then input into a trained news recommendation model. The news recommendation model outputs the user's final interest representation and news embedding. The user's final interest representation is used as the user embedding. The probability value of the user clicking on the candidate news item is calculated using the user embedding and the news embedding. Finally, candidate news items are selected and recommended to the user based on the probability value.

[0014] The beneficial effects of this invention are that, starting from user interests, it effectively utilizes the feature information of news text and the user interest representation generated during the user-news interaction process for personalized news recommendation tasks, thereby making up for the shortcomings of existing methods that only utilize interaction records, improving the accuracy of news recommendation, and ultimately realizing personalized news recommendation for users; by extracting user hot interests to supplement user features, the performance of news recommendation is further improved. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the principle of personalized news recommendation in this invention.

[0016] Figure 2 This is a diagram showing the overall structure of the model of the present invention;

[0017] Figure 3 This is a schematic diagram of the training process of the present invention. Detailed Implementation

[0018] Deep learning-based news recommendation methods are implemented using a constructed news recommendation network model. This model is used for click prediction, outputting the probability that a user will click on candidate news items. The design idea of ​​the news recommendation network model based on user interest modeling is as follows: by utilizing the length of the news text and the user's dwell time, the influence weight of the news on the user's interest is calculated. Then, the user interest vector based on the news features is obtained by multiplying the news features with the corresponding weight values ​​and combining the feature vectors.

[0019] The news recommendation network model based on user interest modeling includes a feature extraction module, a residual graph convolutional network (GCN) based on user-news interaction, a user hot interest module based on users' recent reading interests, and a fully connected neural network.

[0020] The feature extraction module includes a click sequence feature extractor and a news text feature extractor, used to extract recent click sequence features and news text features, respectively. The recent click sequence features are input to the user's hot interest module, and the news text features are input to the residual GCN. The residual GCN is used to capture long-term stable user interest features based on the user's interaction with news, i.e., the user's news reading history and news feature vectors. Furthermore, news in the residual GCN is clustered according to topic categories through topic extraction, which can alleviate the data sparsity problem caused by a lack of interaction in some news articles to some extent.

[0021] The feature vector output by the residual GCN, as a long-term interest feature, is combined with the user hot interest feature calculated from the user hot interest module through a fully connected neural network. The result of the combination of these two feature vectors is output by the fully connected neural network as the user's final interest representation and participates in subsequent news click prediction calculations.

[0022] The specific implementation steps are as follows:

[0023] Step 1: Preprocess the news data according to the required news attribute information to construct a news dataset;

[0024] As required, the implementation example uses the real-world online news dataset Adressa for experiments. Specifically, the model training uses the Adressa-10week news click log, a news click log dataset with a ten-week time span. Initially, the network model is built primarily based on user-news interactions and news attribute information. In the news dataset used for training, interactions between users and news articles are marked as positive samples, indicating that the user has read the news; negative samples indicate that the news has not been seen by the user.

[0025] Step 2: News Recommendation Model Training:

[0026] The news recommendation model is trained using deep learning data from the news dataset. When the preset stopping training condition is met, the trained news recommendation network model is obtained.

[0027] Step 3: The news recommendation model based on user interests outputs the predicted results of user clicks:

[0028] Given a user and candidate news items, the candidate news items are filtered and preprocessed based on a preset input data format to obtain news data. The news data is then input into a news text feature extractor to extract the text features of the candidate news items as input to a graph convolutional network.

[0029] The residual GCN receives the news feature vectors extracted by the news text extractor and calculates the long-term and stable interest features of the user. The long-term interest features are also used as the input for adjusting the candidate news data. The user hot interest module receives the click sequence features extracted by the click sequence extractor from the user's recent news reading records, and calculates and outputs the user hot interest vector.

[0030] Finally, the fully connected neural network combines the long-term interest and the hot interest of the user to obtain the final interest representation. The final interest representation is used as the user embedding, and the candidate news data as the news embedding calculates the click probability through a deep neural network layer for click prediction.

[0031] Specifically, the news data screened from the Adressa-10week dataset has a summary text part that contains the main relevant entities and their corresponding entity types in the news content.

[0032] See Figure 1 For a given user and candidate news, the following steps are used to predict whether the user will click on the candidate news:

[0033] Step S1 filters the data in the dataset, selects the start and end times of the reading session event, user ID, news ID, timestamp, news title and news summary, and news article length to construct the dataset. At the same time, the interaction data in the first eight weeks of the dataset is used to construct the graph, the data in the ninth week is used to generate the training data, and 20% of the data in the last week is used for the verification experiment, and 80% is used for testing.

[0034] One piece of user data constructed corresponds to a news sequence vector composed of news IDs, while one piece of news data contains several parts: the user stay time calculated from the start and end times of the reading session event, the news title word sequence, the news summary entity word sequence, and the number of words in the news text.

[0035] Step S2 preprocesses the input news data according to the preset rules:

[0036] First, filter the stop words, that is, filter the stop words with no effective semantics, such as "ba", "ne", etc. Removing stop words reduces the redundancy of the training data, and at the same time filters out meaningless entities and entity types in the news brief, such as emotions, locations, and acronyms; then perform encoding conversion on the processed corpus, and initially convert the words into vector mode using one-hot encoding; finally, take the news text vector as the preprocessed news data output.

[0037] Step S3 calculates the news feature vector in vector form using the text feature extractor CNN. The news feature vector will be used as candidate news data for news embedding in the final click probability prediction, and will also be output to the residual GCN.

[0038] The text feature extractor is trained using the news text vectors obtained in step 201 as input data. This feature extractor is a network module based on two parallel convolutional neural networks (CNNs), which learn headline-level and summary-level representations of the news article, respectively, using the headline and summary vectors as input. The concatenation of these two representations is considered the final feature representation of the news article. The text feature extractor uses two CNNs in parallel to process the headline embedding T and summary embedding G, with each CNN having its own weight parameters. Each CNN in the text feature extractor consists of a convolutional layer and a max-pooling layer. Since the convolutional layers require a fixed-length input, special characters are used to pad the end of the original input for all headline inputs shorter than 10 characters and summary document inputs shorter than 80 characters. The dimensions of word embeddings and entity type embeddings are both set to K = 50, and the dimensions of news embeddings, user embeddings, and topic embeddings are all set to D = 128.

[0039] Step S4: The residual GCN of the news recommendation model receives the news feature vector obtained in step S3 and outputs the news embedding and the user's long-term stable interest features. The user hot interest module receives the click sequence feature extractor, which extracts the click sequence features from the user's recent news reading records, calculates and outputs the user hot interest vector. A fully connected neural network combines the user's long-term interests and user hot interests to obtain the final interest representation. The final interest representation, as the user embedding and as the news embedding, is used by a deep neural network layer to calculate the probability value of the user clicking on candidate news.

[0040] Step S5 compares and sorts a series of candidate news articles based on the probability values ​​obtained in step S4, and finally selects the top K news articles to recommend to the user.

[0041] In this specific embodiment, the personalized news recommendation model method based on user interests includes two parts: training and prediction. The training part mainly obtains a trained (i.e., well-trained) news recommendation model. In step S2, news text features are obtained through a set text feature extractor, which is obtained from the training part. After obtaining the well-trained news recommendation model, the candidate news sequence to be predicted is input to obtain the predicted click probability of the user on the candidate news and the news recommendation result. The prediction part is to execute the above steps S2-S5 based on the trained news recommendation model.

[0042] During training, an absolute error-based loss function and the Adam optimizer are used. After the text feature extractor is trained, the one-hot code vector of any news text is input, and its output vector is used as the news text feature vector, which is then used as the input to the neural network models of the subsequent two modules. The input to the residual GCN module is the text features of all news articles processed by the text feature extractor, as well as the user-news interaction relationships. The input to the user hot interest module is the text features of the news sequence recently clicked by the user, processed by the text feature extractor.

[0043] Those skilled in the art will understand that the news recommendation model is built as follows:

[0044] Suppose there exist S users U = {u1, u2, ..., u...} S For Q news items I = {d1, d2, ..., d...} Q Click history of}. User interaction matrix Y∈R S×Q It is defined based on the user's implicit feedback, where y u,d =1 indicates that user u clicked on news d; otherwise, y = 1. u,d =0. Furthermore, from the recent click history, the recent click sequence s of user u can be obtained. u ={d u,1 ,d u,2 ,…,d u,n}, where d u,j ∈I represents the j-th news item recently clicked by user u. Furthermore, for each news item d, its topic distribution T can be obtained using the LDA model. d ={T d,i}, i = 1, 2, ..., k, Where T d,i Let z represent the probability that news d belongs to the i-th topic, and then select the topic category with the highest probability to establish the association between news d and topic z.

[0045] In this embodiment, a news recommendation model is built using Python. The model consists of three parts: a feature extraction module, a residual GCN module, and a user interest module. The model takes the news title and summary as input features. Each news title T contains a series of words T = [w1, w2, ..., w...]. m The news summary contains a series of entities E = {e1, e2, ..., e}. n} and its type C = {c1, c2, ..., c n}, where c j It is the j-th entity e j The type.

[0046] In the news information extraction stage, the title is represented as T = [w1, w2, ..., w m] T The summary representation is G = [e1, f(c1), e2, f(c2), ..., e n ,f(c n )] T Each input corresponds to one of the two parallel CNNs in the feature extraction module. f(c j ) = W c c j It is a transformation function. It is a trainable matrix, where k1 is the entity embedding dimension and k2 is the entity type embedding dimension. The outputs of the two CNNs are shown below. and The concatenation of these features forms the final feature representation of the news text, d:

[0047]

[0048] In the above formula, d∈R D D is the text embedding dimension, f c It is a tight layer, and Connecting them together forms the final expression of the characteristics of the news text.

[0049] The residual GCN module computes the news text feature representation d received from its neighbors by calculating the linear average combination of all its sampled neighborhoods:

[0050]

[0051] In the above formula, N(d) represents the uniform sampling of 20 neighboring nodes with a fixed size for each news embedding node, where d represents the news text features. u and z are the feature vectors associated with the user and news topic of the news embedding node d. u and z are randomly initialized during training, while the news embedding node d is initialized using the feature embeddings obtained from the text information extractor mentioned above. W u ∈R D×D and W z ∈R D×D These are trainable weight matrices for users and topics, respectively. News embedding updates. The calculation is as follows:

[0052]

[0053] In the above formula, σ1 and σ2 are nonlinear activation functions ReLU, W i ∈R D×D and b i ∈R D×D These are the weight matrix and bias of the i-th layer of the GCN, respectively, where X represents the node vector from the initial input, used as a residual term in node updates. Finally, the candidate news embeddings are calculated.

[0054] Targeting users' long-term interest characteristics l Similarly, news is aggregated from a user's neighboring news nodes. However, it's necessary to consider the weighted impact of different news items on user interest. Here, a user reading speed metric, v, is used to represent the degree of interest a user has in a particular news item, v = d. len / t. d len Let ν be the number of words in the news article, and t be the user's dwell time. Simultaneously, these reading speeds are normalized to obtain ν′, which is then multiplied by the corresponding news features. Finally, these features are summed to obtain the user's long-term interest feature u. l The calculation is as follows:

[0055]

[0056] Where, d i and ν′ i This represents the news characteristics and normalized reading speed of the i-th item in the corresponding user's click record.

[0057] For the user's hot interests module, this module accepts the user's recent click sequence features processed by the feature extraction module. This invention uses a Transformer network to embed the user's short-term interaction sequence. Due to its characteristics, the Transformer network considers the sequential relationship between news items in the embedding. Furthermore, a neural network model based on an attention mechanism can also guide the model to focus on features that have a significant impact on the user's final choice. Assume that user u's latest clicked news item is {d1, d2, ..., d...}. l The multi-head attention mechanism is used to model the different effects of recently clicked news items on candidate news items d, and the user's hot interest embedding u is calculated. s :

[0058]

[0059] Where u c It is the user's current content-level interest embedding, α j It is the user clicking on the news d j The influence weights of (j = 1, ..., l) on candidate news d are represented by the long-term user embedding vector u. l and short-term user embedding vector u s After being concatenated, the data is transformed into K=50 dimensions through a fully connected network, and the final user embedding u=W[u] is calculated. l ;u s Finally, the user embedding and candidate news embedding are combined. Comparison, calculated using cosine similarity. To predict the probability of user u clicking on news d:

[0060]

[0061] Model structure as follows Figure 2 As shown, the news recommendation model consists of a text feature extraction module, a residual GCN module, and a user hot interest module.

[0062] The text feature extraction module first converts the news text vector obtained in step 1 into a text feature vector using a text feature extractor (that is, it converts each text word vector in the news text into a text feature vector using a text feature extractor). Figure 2 The CNN component in the code is the text feature extractor. This module uses two parallel CNNs to extract the headline and summary features of the news article, and then concatenates the two features to form the basic news features for use by subsequent modules.

[0063] The residual GCN module consists of a GCN-based graph network module. The GCN module stacks three layers of GCN network to extract long-term user interest features and news features.

[0064] The user hot interest module consists of a Transformer-based sequence model, which comprises six basic encoders and decoders used to extract short-term user interest features from recent behavior sequences.

[0065] Those skilled in the art will understand that the specific parameter design process for training a news recommendation model is as follows:

[0066] 1) The model training process is as follows: Figure 3 As shown. The number of recent news clicks by the selected user is set to 10. For LDA, the number of topics is set to 20. In GCN, nodes sample a fixed number of neighboring users and neighboring news documents, with the number of user sampling nodes and news sampling nodes set to 10 and 30, respectively. The training data (i.e., the fusion vector time series corresponding to the network topics used for training) uses a batch size of 256. In each training iteration, the loss function value of the model is calculated. Based on the loss function value, the model parameters are adjusted using the Adam optimization algorithm. Model training terminates when the current loss function value is higher than the previous loss function value. User and topic embeddings are randomly initialized using a Gaussian distribution with a mean of 0 and a standard deviation of 0.1. The Adam algorithm is used to optimize the parameters with a learning rate of 0.0001. The L2 regularization coefficient is set to 0.005, and dropout is set to 0.5.

[0067] To train the model in this invention, positive samples are selected from existing click-through history, and an equal number of negative samples are selected from never-before-seen click behaviors. The training samples are represented as X = (u, x, y), where x is the candidate news item that y is used to predict whether it will be clicked, and y is the predicted click probability. Each positive sample y = 1, and each negative sample y = 0. Each input sample has its own estimated probability. This indicates whether a user will click on news item x. The loss function used in training the news recommendation model in this application is the cross-entropy loss function, calculated as follows;

[0068]

[0069] In the above formula, δ+ represents the positive sample dataset, δ- represents the negative sample dataset, ||W||2 is the L2 regularization of all trainable parameters in the network, and λ is the penalty coefficient.

[0070] 2) Sorting recommendation.

[0071] The news recommendation model outputs the probability P = {p1, p2, ..., p...} of a series of candidate news items that the user clicks on. l Based on this probability score, these candidate news articles are sorted, and the top K candidate news articles are selected as the recommendation results to be displayed to the user. K can be an integer greater than 1 and less than the length of the candidate news article sequence.

[0072] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A news recommendation method based on user interest modeling, characterized in that, Includes the following steps: Step 1: Preprocess the news data according to the required news attribute information to construct a news dataset; in the news dataset used for training, if there is interaction between the user and the news, it is marked as a positive sample, indicating that the user has read the news; otherwise, it is marked as a negative sample. Step 2: Construction and training of the news recommendation model: The news recommendation model is trained using deep learning data from the news dataset. When the preset stopping training condition is met, the trained news recommendation network model is obtained. The news recommendation network model is used to output the probability of a user clicking on candidate news items. It includes a feature extraction module, a residual graphical convolutional network (GCN), a user hot interest module, and a fully connected neural network. The feature extraction module includes a click sequence feature extractor and a news text feature extractor, used to extract recent click sequence features and news text features, respectively. The residual GCN receives news text features and outputs news embeddings and long-term user interest features. The user hot interest module receives recent click sequence features and outputs user hot interest features. The fully connected neural network combines the user's long-term interest features with the user hot interest features, and the combined result serves as the user's final interest representation. Targeting users' long-term interest characteristics It is aggregated from the user's neighboring news nodes, and uses a user reading speed metric v to represent the degree of interest a user has in a particular news item. ; For the number of words in the news article, User dwell time; At the same time, these reading speeds were normalized to obtain Then, multiply these features by the corresponding news features, and finally add them together to obtain the user's long-term interest features. The calculation is as follows: in, and This represents the news features and normalized reading speed of the i-th item in the corresponding user's click record; For the user hot interest module, a Transformer network is used to embed short-term user interaction sequences. A multi-head attention mechanism is used to model the different effects of the user's most recently clicked news on candidate news d, and the user hot interest embedding is calculated. : in It is the user's current content-level interest embedding. It is the user clicking on the news The weight of influence on candidate news items ,common News item; embedding long-term users into vectors and short-term user embedding vectors After being concatenated, the data is transformed into K=50 dimensions through a fully connected network, and the final user embedding is calculated. Finally, embed the user Embedded with candidate news Comparison, through cosine similarity calculate To come to users Click News probability prediction : ； Step 3: The news recommendation model outputs the predicted user clicks: Given a user and candidate news items, the candidate news items are filtered and preprocessed based on a preset input data format to obtain news data. The news data is then input into a trained news recommendation model. The news recommendation model outputs the user's final interest representation and news embedding. The user's final interest representation is used as the user embedding. The probability value of the user clicking on the candidate news item is calculated using the user embedding and the news embedding. Finally, candidate news items are selected and recommended to the user based on the probability value.

2. The method as described in claim 1, characterized in that, The preprocessing includes filtering stop words and meaningless entities and entity types in the news summary. The filtered corpus is then encoded and converted using one-hot encoding to initially convert words into vector patterns. Finally, the news text vectors are used as the preprocessed news data.

3. The method as described in claim 1, characterized in that, The news data includes user dwell time calculated from the start and end times of the reading session, word sequences in the news title, word sequences in the news summary entity, and words in the news body.

Images