A news recommendation method based on text semantic mining

Abstract

This invention is a news recommendation method based on text semantic mining. The method includes the following steps: obtaining news item information and user reading history log information from a database; establishing a word embedding matrix for the news item titles using pre-trained word vectors; obtaining news title embedding representation vectors through feature extraction; performing topic modeling on the news text content based on a neural topic model to obtain news content topic embedding vectors, forming the final news feature representation; analyzing behavioral data in the user's reading history logs and extracting user feature representations based on the user's reading records; and introducing a time decay function into the model to generate a top N candidate news items for recommendation. This invention analyzes the word-level and topic features of news items using a bidirectional recurrent neural network and a neural topic model, effectively mining rich semantic information in news texts, more accurately representing news item features, and improving recommendation performance.

Description

Technical Field

[0001] This invention belongs to the field of news recommendation technology, specifically relating to a news recommendation method based on text semantic mining. Background Technology

[0002] The ever-growing World Wide Web (WWW) has gradually changed the way people find and read news, shifting from traditional print media to online portals. To alleviate information overload, recommender systems are widely used in modern online services, helping users quickly find relevant content they need. A single user often has multiple interests, reflected in the different news items they browse. Meanwhile, important semantic features of news are implicit in text segments of varying granularity, and news content is filled with different types of thematic information, making it crucial for learning accurate user representations of news and news recommendations. However, existing news recommendation methods often overlook the fact that fine-grained modeling of news items can enhance recommendation effectiveness.

[0003] CN2014104033786 discloses a news recommendation system. The data relevance analysis in this system constructs a new personalized news recommendation hypergraph model by mining the inherent relationships between phrases. In the hypergraph model, nodes represent phrases, edges represent certain inherent connections between phrases, and the weight of the edges is used to represent the contribution of this connection. However, this news recommendation system relies more on historical data to obtain high-quality customers and recommends newly published news or news without sufficient access records to users, and the news atlas effect is not good.

[0004] CN201510242541X discloses a news recommendation method based on user interests. This method can quickly and efficiently store and process news data by utilizing interactive user terminals and recommendation systems. However, this recommendation method has a long data processing time, and due to the rapid updates of news, the computational overhead of calculating news similarity is very large. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a news recommendation method based on text semantic mining. It utilizes a bidirectional recurrent neural network to extract word-level semantic information from news titles, obtaining a news title representation vector. Then, a neural topic model is used to model the topic of the news text content, obtaining topic semantic information, which is concatenated with the title embedding representation vector to jointly represent the news item features. An attention network is used to assign weights to different interests in the user's historical reading records, obtaining a user-side feature representation vector. A time decay function is introduced into the model, and a scoring function is used to calculate the user-side feature representation and the feature representation of the target news item, generating news items that better match the user's reading interests, thus achieving news recommendation. Specifically, the method includes the following steps:

[0006] S10. Retrieve news item information and user reading history log information from the database, specifically including the following steps:

[0007] S11. Retrieve news item information from the database, including news number, news title, news content, and publication date and timestamp;

[0008] S12. Retrieve the news items that the user has viewed from the user database, including the user ID, the news ID, and the user's reading timestamp;

[0009] S13. The acquired information is organized and preprocessed to obtain the training set and the test set.

[0010] S20. Construct a word embedding matrix for the news article titles using pre-trained word vectors. Specifically, the title of each news article is represented as a matrix composed of word embeddings. This maps the n words in the title to a d-dimensional vector, focusing on the meaning of each word. The word embedding transforms the news title from a sequence of words into a semantic vector matrix X. 1：m = [x1, x2, ..., x n ].

[0011] S30. Use a bidirectional recurrent neural network to extract features from the news headline word embedding matrix to obtain the news headline embedding vector;

[0012] Specifically, the steps include the following:

[0013] S31. Extract headline features from news articles using a bidirectional recurrent neural network to capture contextual information of word sequences:

[0014] i t =σ(W i x t +U i h t-1 +b i )

[0015] f t =σ(W f x t +U f h t-1 +b f )

[0016]

[0017] o t =σ(W o x t +U o h t-1 +b o )

[0018]

[0019]

[0020] Where σ(·) is the activation function, i t It is an input gate, f t It's the Gate of Oblivion, o t It's an output gate. represents the candidate information that needs to be updated to the current cell state at time t, and b is the bias vector;

[0021] S32. Learn the title sequence using forward propagation and backward propagation respectively, and calculate the hidden state H:

[0022]

[0023] S33. Based on an attention network, extract more information features from the news headline, select keywords from the headline, and obtain the feature representation vector v of the news headline by weighting and summing the contextual representations of the words using attention weights. t as follows:

[0024]

[0025] S40. The neural topic model is activated to perform topic modeling on the news text content, obtaining news content topic embedding vectors. The news headline embedding vector and the news content topic embedding vector are then combined to form the final news feature representation vector. The method for topic modeling of news text content includes the following steps:

[0026] S41. Based on the variational autoencoder framework, it learns latent topics through an encoder-decoder approach. Let x be the bag-of-words representation of a given news text, where v is the vocabulary. In the encoder, u = f u (x), logσ=f σ (x), where u and σ are prior parameters of the topic model distribution parameterized in the decoder network, f u f σ It is a linear transformation with the ReLU activation function;

[0027] S42. Generate document topics using a decoder, and plot the topic distribution using the Gaussian softmax function, i.e., z ~ N(μ, σ). 2 ), θ = softmax(z)

[0028] Where z is the latent topic variable, σ is the subject distribution, and k is the predefined number of topics, through... Learn to predict words The probability, Similar to the topic word distribution matrix in LDA topic model, Let p represent the relevance between the i-th word and the j-th topic. Each word is extracted from p to reconstruct the input x, and then intermediate parameters w0 and θ are used to construct a topic representation, as follows:

[0029]

[0030]

[0031] in Represents a set of topic representations with predefined d dimensions. It is a linear transformation with ReLU activation function. It is a weighted sum of each topic representation, considered as the overall topic representation of the news item;

[0032] S43. The final news feature representation vector v is formed by combining the news headline embedding vector and the news content theme embedding vector: v = Concat(v t v c ).

[0033] S50. Analyze the behavioral data in the user's reading history log, and extract the user preference feature representation vector on the user side based on the user's reading record;

[0034] The following method analyzes user reading history logs and extracts user feature representation vectors based on user reading records on the user side:

[0035] An attention network is used to compare the news articles a user has read with the target news article to obtain the user's final embedding vector. The user u's historical reading records are represented as {d1, d2, ... d}. n}, its embedding can be represented as: {v1, v2, ... v n An attention network is used to assign different weights to news items clicked by users to learn the user's different interests in each news item. For a target news item, the similarity between the embedding representation of news items read by the user and the target news item is calculated, and the embedding representation of news items in the user's historical reading record is calculated as the user's representation vector of the target news item, where the interest weight of each news item read by the user is determined by the similarity. In the attention mechanism, the query is the target news item, and the user's historical reading news items are the keys and values. The obtained user feature representation is as follows:

[0036] v u =Attention(q, k, v)

[0037] =softmax(qk) T )v.

[0038] S60. Introduce the time decay factor into the model and use the scoring function to calculate the similarity between the user preference feature representation vector and the news feature representation vector, generating the top N recommended news candidate sets.

[0039] The time decay function here is defined as:

[0040]

[0041] Where λ is a parameter that needs to be adjusted during training to control the decay rate of the news, and t and t0 represent the reading time and the publication time of the news at a certain moment. The top N candidate news items are retrieved based on the scoring function, which is as follows:

[0042]

[0043] Where N is the predefined number of items to be retrieved during the matching phase.

[0044] The beneficial effects of this invention are:

[0045] (1) This invention uses bidirectional recurrent neural networks and neural topic models to learn news headlines and news content at different granularities, which can effectively extract word-level semantic information and topic semantic information of news items, and enrich the feature representation of news items.

[0046] (2) This invention uses attention networks to simulate the influence of news items in the user's historical reading records on the target news, thereby representing the user's different interests and improving the accuracy of user preferences.

[0047] (3) The present invention introduces a time decay function, which to a certain extent satisfies the timeliness of news recommendation and enables real-time news recommendation. Attached Figure Description

[0048] Figure 1 This is an overall flowchart of an embodiment of the present invention. Detailed Implementation

[0049] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0050] like Figure 1 As shown, this invention is a news recommendation method based on text semantic mining, specifically including the following steps:

[0051] S10. Retrieve news item information and user reading history log information from the database.

[0052] First, retrieve news item information from the database, including news number, news title, news content, and publication date and timestamp.

[0053] Then, retrieve the news items that the user has viewed from the user database, including the user ID, news ID, and user reading timestamp.

[0054] Finally, the acquired information is organized and preprocessed to obtain the training set and the test set.

[0055] S20. Construct a word embedding matrix for the headlines of news items using pre-trained word vectors.

[0056] The title of each news article can be represented as a matrix of word embeddings, that is, mapping the n words in the title to a d-dimensional vector. The focus is on the meaning of the words. Word embedding transforms the news title from a sequence of words into a semantic vector matrix X. 1：n = [x1, x2, ..., x n Word embeddings can be any pre-trained word embedding model, such as fastText, word2Vec, or GloVe. This example uses word vectors pre-trained by the fastText model, and the dimension d of the word vectors is 100.

[0057] S30. Use a bidirectional recurrent neural network to extract features from the news headline word embedding matrix to obtain the news headline embedding vector;

[0058] First, the headline features of the news are extracted based on a bidirectional recurrent neural network to capture the contextual information of the word sequence:

[0059] i t =σ(W i x t +U i h t-1 +b i )

[0060] f t =σ(W f x t +U f h t-1 +b f )

[0061]

[0062] o t =σ(W o x t +U o h t-1 +b o )

[0063]

[0064]

[0065] Where σ(·) is the activation function, i t It is an input gate, f t It's the Gate of Oblivion, o t It's an output gate. represents the candidate information that needs to be updated to the current cell state at time t, and b is the bias vector;

[0066] Then, the title sequence is learned using forward and backward propagation respectively, and the hidden state H is calculated:

[0067]

[0068] Finally, based on the attention network, more information features are extracted from the news headlines, and keywords in the headlines are selected. The contextual representations of the words are weighted and summed using attention weights to obtain the feature representation vector v of the news headline. t as follows:

[0069]

[0070] S40. Activate the neural topic model to perform topic modeling on the news text content to obtain the news content topic embedding vector, and combine the news title embedding vector and the news content topic embedding vector to form the final news feature representation vector.

[0071] S41. Based on the variational autoencoder framework, it learns latent topics through an encoder-decoder approach. Let x be the bag-of-words representation of a given news text, where v is the vocabulary. In the encoder, u = f u (x), logσ=f σ (x), where u and σ are prior parameters of the topic model distribution parameterized in the decoder network, f u f σ It is a linear transformation with the ReLU activation function;

[0072] S42. Generate document topics using a decoder, and plot the topic distribution using the Gaussian softmax function, i.e., z ~ N(μ, σ). 2 ), θ = sofmax(z)

[0073] Where z is the latent topic variable, σ is the subject distribution, and k is the predefined number of topics, through... Learn to predict words The probability, Similar to the topic word distribution matrix in LDA topic model, This represents the relevance between the i-th word and the j-th topic. Each word is extracted from p to reconstruct the input x, and intermediate parameters are further used. To construct a topic representation using θ, the representation is as follows:

[0074]

[0075]

[0076] in Represents a set of topic representations with predefined d dimensions. It is a linear transformation with ReLU activation function. It is a weighted sum of each topic representation, which can be regarded as the overall topic representation of the news item;

[0077] S43. The final news feature representation vector v is formed by combining the news headline embedding vector and the news content theme embedding vector: v = Concat(v t v c ).

[0078] S50. Analyze the behavioral data in the user's reading history log and extract user preference features on the user side based on the user's reading records;

[0079] An attention network is used to compare the news articles a user has read with the target news article to obtain the user's final embedding vector. The user u's historical reading record is represented as {d1, d2, ... d}. n}, its embedding can be represented as: {v1, v2, ..., v n The attention network is used to assign different weights to the news items clicked by the user in order to learn the user's different interests in each news item. For a target news item, the similarity between the embedding representation of the news items read by the user and the target news item is calculated, and the embedding representation of the news items in the user's reading history is calculated as the user's representation vector of the target news item. The interest weight of each news item read by the user is determined by the similarity. In the attention mechanism, the query is the target news item, and the user's reading history of news items are the keys and values. The obtained user preference feature representation is as follows:

[0080] v u =Attention(q, k, v)

[0081] =softmax(qk) T )v.

[0082] S60. Introduce the time decay factor into the model and use the scoring function to calculate the similarity between the user preference feature representation vector and the news feature representation vector, generating the top N recommended news candidate sets.

[0083] The time decay function is defined as:

[0084]

[0085] Where λ is a parameter that needs to be adjusted during training to control the decay rate of the news, and t and t0 represent the reading time and the publication time of the news at a certain moment. The top N candidate news items are retrieved based on the scoring function, which is as follows:

[0086]

[0087] Where N is the predefined number of items to be retrieved during the matching phase; in this example, the predefined number of N is 50.

[0088] This invention analyzes word-level and topic features of news items using bidirectional recurrent neural networks and neural topic models, which can effectively mine rich semantic information in news texts, more accurately represent the features of news items, and improve the recommendation effect.

[0089] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A news recommendation method based on text semantic mining, characterized in that: The recommended method includes the following steps: S10. Retrieve news item information and user reading history log information from the database; S20. Construct a word embedding matrix for the headlines of news items using pre-trained word vectors; S30. Use a bidirectional recurrent neural network to extract features from the news headline word embedding matrix to obtain the news headline embedding vector; S40. Activate the neural topic model to perform topic modeling on the news text content to obtain the news content topic embedding vector, and combine the news title embedding vector and the news content topic embedding vector to form the final news feature representation vector; S50. Analyze the behavioral data in the user's reading history log, and extract the user preference feature representation vector on the user side based on the user's reading record; S60. Introduce a time decay factor into the model and use a scoring function to calculate the similarity between the user preference feature representation vector and the news feature representation vector, generating a top N recommended news candidate set, where: Step S30, which uses a bidirectional recurrent neural network to extract features from the news headline word embedding matrix to obtain the news headline embedding vector, specifically includes the following steps: S31. Extract headline features from news articles using a bidirectional recurrent neural network to capture contextual information of word sequences: i t =σ(W i x t +U i h t-1 +b i ) f t =σ(W f x t +U f h t-1 +b f ) the t =σ(W o x t +U o h t-1 +b o ) Where σ(·) is the activation function, i t It is an input gate, f t It's the Gate of Oblivion, o t It's an output gate. represents the candidate information that needs to be updated to the current cell state at time t, and b is the bias vector; S32. Learn the title sequence using forward propagation and backward propagation respectively, and calculate the hidden state H: S33. Based on an attention network, extract more information features from the news headline, select keywords from the headline, and obtain the feature representation vector v of the news headline by weighting and summing the contextual representations of the words using attention weights. t as follows: The method for performing topic modeling on news text content in step S40 includes the following steps: S41. Based on the variational autoencoder framework, it learns latent topics through an encoder-decoder approach. Let x be the bag-of-words representation of a given news text, where v is the vocabulary. In the encoder, u = f u (x), logσ=f σ (x), where u and σ are prior parameters of the topic model distribution parameterized in the decoder network, f u f σ It is a linear transformation with the ReLU activation function; S42. Generate document topics using a decoder, and plot the topic distribution using the Gaussian softmax function, i.e., z ~ N(μ, σ). 2 ), θ = softmax(z) Where z is a latent topic variable, and k is the predefined number of topics, through... Learn to predict words The probability, Similar to the topic word distribution matrix in LDA topic model, This represents the relevance between the i-th word and the j-th topic. Each word is extracted from p to reconstruct the input x, and intermediate parameters are further used. To construct a topic representation using θ, the representation is as follows: in Represents a set of topic representations with predefined d dimensions. It is a linear transformation with ReLU activation function. It is a weighted sum of each topic representation, considered as the overall topic representation of the news item; S43. The final news feature representation vector v is formed by combining the news headline embedding vector and the news content theme embedding vector: v = Concat(v t v c ).

2. The news recommendation method based on text semantic mining according to claim 1, characterized in that: The method for analyzing behavioral data in the user's reading history log in step S50 and extracting user preference feature representation vectors based on the user's reading records on the user side includes the following steps: S51. Use an attention network to compare the news that the user has read with the target news to obtain the user's final embedding vector. Represent the user u's historical reading record as {d1, d2, ... d}. n }, its embedding representation is: {v1,v2,...,v n }; S52. An attention network is used to assign different weights to the news items clicked by the user in order to learn the user’s different interests in each news item. For a target news item, the similarity between the news embedding representation that the user has read and the target news item is calculated, and the news item embedding representation in the user’s historical reading record is calculated as the user representation vector of the target news item. The interest weight of each news item read by the user is determined by the similarity. S53. In the attention mechanism, the query is the target news, and the user's historical news reading items are the keys and values. The obtained user preference features are represented as follows: v u =Attention(q,k,v) =softmax(qk T )v。 3. The news recommendation method based on text semantic mining according to claim 1, characterized in that: The time decay function in step S60 is defined as follows: Where λ is a parameter that needs to be adjusted during training to control the decay rate of the news, and t and t0 represent the reading time and the publication time of the news at a certain moment. The top N candidate news items are retrieved based on the scoring function, which is as follows: Where N is the predefined number of items to be retrieved during the matching phase.

4. The news recommendation method based on text semantic mining according to claim 1, characterized in that: Step S10 specifically includes the following steps: S11. Retrieve news item information from the database, including news number, news title, news content, and publication date and timestamp; S12. Retrieve the news items that the user has viewed from the user database, including the user ID, the news ID, and the user's reading timestamp; S13. The acquired information is organized and preprocessed to obtain the training set and the test set.

5. The news recommendation method based on text semantic mining according to claim 1, characterized in that: The method for establishing the news title word embedding matrix in step S20 is as follows: the title of each news article is represented as a matrix composed of word embeddings, that is, mapping the n words in the title to a d-dimensional vector. The focus is on the meaning of the words. Word embedding transforms the news title from a sequence of words into a semantic vector matrix X. 1：n = [x1, x2, ..., x n ].

Images