News recommendation method and system based on attention mechanism and user multi-interest modeling

Abstract

The application relates to a news recommendation method and system based on an attention mechanism and user multi-interest modeling. The method first represents candidate news and words in a user historical clicked news sequence with vectors respectively; connects the candidate news and the word sequence of the user historical reading news in a word level neural network, calculates the weight of each word by using an attention mechanism, and obtains a word feature vector o w ; takes the words of a news title as input, calculates a news vector matrix of each news by using the attention mechanism, inputs the news into a news level neural network, and outputs a news feature vector o n after passing through a news level attention mechanism layer and a news level feature fusion layer; selects a category frequently browsed by the user, adds the attention mechanism, and obtains a news category feature vector o v . Finally, o w , o n and o v are connected, and the final click probability is calculated through an activation function. The method provided by the application improves the accuracy of personalized news recommendation.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and news recommendation, specifically to a news recommendation method and system based on attention mechanisms and user multi-interest modeling. Background Technology

[0002] With the development of internet technology, traditional newspapers are declining, while online news consumption is growing rapidly. As early as 2012, users visiting news portals had already become a major part of overall internet traffic. However, sparse user base, rapidly increasing number of news items, rapidly diminishing value of those items, and dynamic shifts in user preferences are making it increasingly difficult for users of various news platforms to select news that interests them. News recommendations can leverage user history and behavior to uncover user interests, helping them find news that resonates with them, thus alleviating information overload and improving the user reading experience.

[0003] Learning from the accurate expressions of news and users are two core tasks of news recommendation. Early personalized news recommendation technologies can be mainly divided into three categories: content-based, collaborative filtering-based, and hybrid recommendation. Each of these has its own problems, such as limited user configuration, missing textual information, and difficulty in modeling. Compared to traditional recommendation technologies, deep learning has advantages such as nonlinear transformation, representation learning, and sequence modeling. Furthermore, with the development of deep learning technology, natural language processing, graph convolutional neural networks, and other techniques have also been introduced into the field of personalized news recommendation.

[0004] Currently, most news recommendation methods learn user vectors using users' historical reading sequences and interact with candidate news articles to obtain the model's output. However, treating news articles as a whole may overlook some low-dimensional matching signals between user interests and candidate news articles, such as lexical-level relationships. Furthermore, merging different news representations into a single user vector can introduce significant biases, hindering user modeling. Therefore, how to accurately personalize news recommendations has become a pressing issue. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a news recommendation method and system based on attention mechanisms and user multi-interest modeling.

[0006] The technical solution of this invention is: a news recommendation method based on attention mechanism and user multi-interest modeling, comprising:

[0007] Step S1: Use word vectors pre-trained from the text corpus to represent the words in the news headlines, and use a normal distribution to randomly generate the news category vector matrix E. vert and its subclass vector matrix E sub_vert ;

[0008] Step S2: Construct an attention-based news recommendation network, including: a vocabulary-level neural network, a news modeling network, a news-level neural network, and a category-level neural network; firstly, construct a user vocabulary sequence w using candidate news titles and historical news titles, then input the user vocabulary sequence into the vocabulary-level neural network for feature extraction, obtaining a matrix composed of w. And the final word feature vector o w ;

[0009] Step S3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Inputting the news modeling network yields a news headline vector matrix. Will After applying the attention mechanism, the representation vector matrix X of the news headline is obtained. t ; X t With E vert and E vert The concatenation yields the news vector matrix E. n ;

[0010] Step S4: E n The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector o. n ;

[0011] Step S5: Select the C news categories that the user browses most frequently and their corresponding news vector matrices E. v Input the category-level neural network to obtain the final representation E of the user for each frequently browsed news category. o E o After inputting the category-level feature fusion layer, the output is the news category feature vector o. v ;

[0012] Step S6: Place o w o n and o v The concatenation operation yields vector o, which is then passed through an activation function to obtain the final prediction result. Meanwhile, a loss function is constructed for training the attention-based news recommendation network.

[0013] Compared with the prior art, the present invention has the following advantages:

[0014] 1. This invention discloses a news recommendation method based on attention mechanisms and user multi-interest modeling. It integrates user selection information on news vocabulary and categories, introducing vocabulary-level and category-level neural networks in addition to the news-level network, incorporating user preferences and product features at different levels. In the vocabulary-level neural network, this invention connects candidate news articles with the user's historical news sequences, calculating weights for each word in the sequence to obtain the final vocabulary-level output. In the category-level neural network, this invention selects the most frequently viewed news categories based on the number of news articles the user has historically viewed, adds an attention mechanism, calculates weights for each category, and thus obtains the final category-level output. Finally, these are connected to the output of the news-level neural network, processed by an activation function to obtain the final classification prediction result, and a loss function is constructed to train the model. Compared with other basic models, this invention significantly improves accuracy.

[0015] 2. Existing news recommendation neural network models often neglect user selection information at the vocabulary and category levels. This invention constructs a neural network model that integrates user selection information at both the vocabulary and category levels, thereby solving this problem. Specifically, in addition to the news-level neural network, this invention adds vocabulary-level and category-level neural networks to collaboratively represent users and news information, enhancing the value of the information and significantly improving the accuracy compared to traditional recommendation methods. Attached Figure Description

[0016] Figure 1 This is a flowchart of a news recommendation method based on attention mechanism and user multi-interest modeling in an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram of the structure of the vocabulary-level neural network in an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of the structure of the news-level neural network in an embodiment of the present invention;

[0019] Figure 4 This is a schematic diagram of the structure of the category-level neural network in an embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of the structure of a news recommendation network based on an attention mechanism in an embodiment of the present invention;

[0021] Figure 6 This is a structural block diagram of a news recommendation method system based on attention mechanism and user multi-interest modeling in an embodiment of the present invention. Detailed Implementation

[0022] This invention provides a news recommendation method based on attention mechanism and user multi-interest modeling, which makes full use of users' selection information at the vocabulary and category levels, and improves the accuracy of personalized news recommendation by combining attention mechanism.

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0024] Example 1

[0025] like Figure 1 As shown in the figure, an embodiment of the present invention provides a news recommendation method based on attention mechanism and user multi-interest modeling, which includes the following steps:

[0026] Step S1: Use word vectors pre-trained from the text corpus to represent the words in the news headlines, and use a normal distribution to randomly generate the news category vector matrix E. vert and its subclass vector matrix E sub_vert ;

[0027] Step S2: Construct an attention-based news recommendation network, including: a vocabulary-level neural network, a news modeling network, a news-level neural network, and a category-level neural network; first, construct a user vocabulary sequence w using candidate news titles and historical news titles, then input the user vocabulary sequence into the vocabulary-level neural network for feature extraction, obtaining a matrix composed of w. And the final word feature vector o w ;

[0028] Step S3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Inputting the news modeling network yields a news headline vector matrix. Will After applying the attention mechanism, the representation vector matrix X of the news headline is obtained. t ; X t With E vert and E vert The concatenation yields the news vector matrix E. n ;

[0029] Step S4: E n The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector o. n ;

[0030] Step S5: Select the C news categories that the user browses most frequently and their corresponding news vector matrices E. v Input the category-level neural network to obtain the final representation E of the user for each frequently browsed news category. o E oAfter inputting the category-level feature fusion layer, the output is the news category feature vector o. v ;

[0031] Step S6: Place o w o n and o v The concatenation operation yields vector o, which is then passed through an activation function to obtain the final prediction result. Meanwhile, a loss function is constructed for training an attention-based news recommendation network.

[0032] In one embodiment, step S1 above involves using word vectors pre-trained from text corpus to represent the words in the news headline, and randomly generating the news category vector matrix E using a normal distribution. vert and its subclass vector matrix E sub_vert Specifically, it includes:

[0033] This invention uses 200-dimensional word vectors pre-trained from text corpora to represent words in news headlines, and randomly generates 100-dimensional news category vector matrices E using a normal distribution. vert and subclass vector matrix E sub_vert .

[0034] In one embodiment, step S2 above, which involves constructing an attention-based news recommendation network, includes: a vocabulary-level neural network, a news modeling network, a news-level neural network, and a category-level neural network. First, a user vocabulary sequence w is constructed using candidate news titles and historical news titles. This user vocabulary sequence is then input into the vocabulary-level neural network for feature extraction, resulting in a matrix composed of w. And the final word feature vector o w Specifically, it includes:

[0035] Step S21: The word sequence connection layer constructs the user word sequence w based on the input candidate news titles and the historical news titles that the user has viewed, as shown in formula (1):

[0036]

[0037] Where w is the user's word sequence, w can and [CLS] represents the sequence of titles and words for candidate news and historical news that the user has viewed, respectively. [CLS] represents the semantic representation of the entire input word sequence. [SEP] and [NEWS_SEP] are the segments between the candidate news and historical news sequences.

[0038] The words in different news headlines are connected by [CLS], [SEP], and [NEWS_SEP], and then converted into word vectors and input into a word-level neural network.

[0039] Step S22: The vocabulary-level attention mechanism layer and feature extraction layer are shown in formulas (2) to (5):

[0040]

[0041]

[0042]

[0043]

[0044] in, The word vector matrix consists of the user's vocabulary sequence w input to the attention mechanism layer. Q represents the weight matrix to be trained in the vocabulary-level neural network. w ,K w V w They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication;

[0045] Q represents w Dimensions Z is the output vector generated by the i-th head in the multi-head self-attention mechanism. w This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism;

[0046] The specific calculation method of the softmax function in formula (3) is shown in formula (4);

[0047] o w This represents the lexical feature vector matrix output by the lexical-level neural network.

[0048] like Figure 2 The diagram shows the structure of a word-level neural network.

[0049] In one embodiment, step S3 above: Inputting the news modeling network yields a news headline vector matrix. Will After applying the attention mechanism, the representation vector matrix X of the news headline is obtained. t ; X t With E vert and E vert The concatenation yields the news vector matrix E. n Specifically, it includes:

[0050] Word vector matrix Input the news modeling network, extract the text features of the news headlines, and compare them with the news category vector matrix E. vert Connect E and its subclass vector matrix vert The vector matrix E of the news is obtained. n :

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] in, E is the word vector matrix. pos Encode the position of words in news headlines. The news headline vector matrix is ​​used as input to the attention mechanism layer. Q represents the weight matrix to be trained in the news modeling network. t ,K t V t They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication;

[0057] Q represents t Dimensions Z is the output vector generated by the i-th head in the multi-head self-attention mechanism. t This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism;

[0058] X t A vector matrix representing news headlines;

[0059] E vert and E vert The news category vector matrix and the news subcategory vector matrix represent the vectors of news categories and subcategories, respectively, and are related to X. t The concatenation yields the news vector matrix E. n .

[0060] In one embodiment, step S4 above: E n The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector o. n Specifically, it includes:

[0061] The news vector matrix E n The input is a news-level neural network, which includes a news-level attention mechanism layer and a news-level feature fusion layer, and finally outputs a news feature vector o. n :

[0062]

[0063]

[0064]

[0065] F n =X n T Z n (14)

[0066]

[0067] in, Q is the weight matrix to be trained in a news-level neural network. n ,K n V n E n and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication;

[0068] Q represents n Dimensions Z is the output vector generated by the i-th head in the multi-head self-attention mechanism. n This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism;

[0069] X represents the weights and biases of the linear layer, respectively. n The weight matrix representing the proportion of news items viewed in the user's history in the final representation of the sequence;

[0070] F n The final representation of the news sequence includes: F n_his The final representation of the sequence of news items viewed in the user's history and F n_can The final representation of the candidate news, F. n_can ;

[0071] Indicates the Hadamardi product; o n This is the news feature vector, which is the final output of the news-level neural network.

[0072] like Figure 3The diagram shown is a schematic of the structure of a news-level neural network.

[0073] In one embodiment, step S5 above: Select the C news categories that the user browses most frequently and their corresponding news vector matrix E v Input the category-level neural network to obtain the final representation E of the user for each frequently browsed news category. o E o After inputting the category-level feature fusion layer, the output is the news category feature vector o. v Specifically, it includes:

[0074] Step S51: Select the C news categories that the user browses most frequently and their corresponding news vector matrices E. v Input the category-level neural network to obtain the final representation E of the user for each frequently browsed news category. o :

[0075]

[0076]

[0077]

[0078]

[0079] E o =X v T (E v +Z v (20)

[0080] Among them, h len Let C represent the number of news items viewed by the user in the past, and let C represent the number of news categories selected by the user. The news categories are sorted according to the number of news categories viewed in the user's past history, and the C most frequently viewed news categories and their corresponding news vectors are selected to construct the frequently viewed news vector matrix E. v ;

[0081] Q is the weight matrix to be trained in a category-level neural network. v ,K v V v E v and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication;

[0082] Q represents v The dimension of any query vector in the vector. Z is the output generated by the i-th head in the multi-head self-attention mechanism.v This represents the output matrix obtained by concatenating the outputs of all heads in the multi-head self-attention mechanism;

[0083] X represents the weights and biases of a linear layer in a category-level neural network, respectively. v This indicates the weight of each news item in each category in the final representation of that category, E. o The final representation of the C news categories most frequently viewed by users;

[0084] Step S52: E o The input category-level feature fusion layer yields the user's final representation F on the neural network at the category level. o ;

[0085] R = softmax(W) c E o +b c ) (twenty one)

[0086] F o =R T E o (twenty two)

[0087]

[0088] Among them, W c ,b c These represent the weights and biases of the linear layer, respectively.

[0089] R represents the weights of each category vector; F o The final representation of the user on the neural network at the category level;

[0090] o v This is the news category feature vector that is ultimately output by the category-level neural network.

[0091] like Figure 4 The diagram shown is a schematic of the structure of a category-level neural network.

[0092] In one embodiment, step S6 above: o w o n and o v The concatenation operation yields vector o, which is then passed through an activation function to obtain the final prediction result. Simultaneously, a loss function is constructed for training the attention-based news recommendation network, specifically including:

[0093]

[0094]

[0095] Among them, W y Indicates calculation The weight matrix, b y Indicates calculation The bias vector, It is a prediction of the probability of news clicks;

[0096] Meanwhile, the attention-based news recommendation network is trained according to the loss function, as shown in Equation (26):

[0097]

[0098] Where, p y This represents the actual click probability of news articles. It is the predicted probability of news clicks.

[0099] The Adam optimization algorithm, also known as the Adam gradient backpropagation optimization algorithm, optimizes the word vector parameters, attention weight matrix parameters, and other parameters mentioned in the above steps in the attention-based news recommendation network during the training process, and finally obtains the optimized news recommendation result based on the attention mechanism.

[0100] like Figure 5 The diagram shows a schematic representation of the structure of a news recommendation network based on an attention mechanism.

[0101] The performance of the attention-based news recommendation network of this invention was compared with that of five control models on the Microsoft MIND real news dataset in the United States. The results are shown in Table 1.

[0102] Table 1 Comparison of recommendation results between traditional news recommendation methods and the method of this invention.

[0103]

[0104] The control models used are as follows:

[0105] 1. libFM stands for Factorization Machine Library, a classic recommendation algorithm based on matrix factorization proposed by Steffen Rendle in 2010.

[0106] 2. DSSM is a deep neural network semantic model that uses a deep neural network structure to sort a set of documents for a given query as follows, transforms the input query and documents into a common semantic space, and then calculates cosine similarity for recommendation.

[0107] 3. DeepFM combines deep neural networks with matrix factorization, combining the advantages of breadth and depth models. It jointly trains matrix factorization models and deep neural network models, while learning low-order and high-order feature combinations, achieving good results in click-through rate prediction scenarios.

[0108] 4. DKN integrates knowledge graphs with neural networks for news recommendation. It uses a convolutional neural network that integrates knowledge to combine the semantic representation of news with the knowledge representation to form a new vector representation. Then, it establishes an attention mechanism from the user's news click history to candidate news, and selects news with higher scores to recommend to the user.

[0109] 5. GRU is an improved model of recurrent neural network. Recurrent neural network models have been studied and applied in the field of natural language processing for a long time. Based on the idea of ​​LSTM gated unit, GRU merges the input gate and forget gate into the update gate. While solving the problems of long memory and gradient vanishing in recurrent neural networks, it reduces the number of parameters, thus gaining widespread attention and application.

[0110] As can be seen from the comparison results in Table 1, the news recommendation network model based on attention mechanism and user multi-interest modeling proposed in this invention shows improvements in some metrics compared to previous models for the MIND dataset, both the vocabulary-level neural network and the category-level neural network. After simultaneously applying vocabulary, news, and category-level neural networks, optimal performance was achieved in metrics such as AUC (the probability that a predicted positive example is ranked higher than a negative example), MRR (the average reciprocal rank of recommended items), and NDCG (normalized cumulative gain considering the ranking order of recommended items, where @n indicates that only the first n news items in the ranking list are considered for normalized cumulative gain). This demonstrates the improvement in the performance of the news-level neural network after integrating the vocabulary-level and category-level neural networks, with AUC showing a 2-3% improvement compared to the previous GRU model. In summary, the news recommendation method based on attention mechanism and user multi-interest modeling adopted in this invention can effectively improve the accuracy of the model by integrating multi-dimensional user selection information.

[0111] This invention discloses a news recommendation method based on attention mechanisms and user multi-interest modeling. It integrates user selection information on news vocabulary and categories, introducing vocabulary-level and category-level neural networks in addition to the news-level network, incorporating user preferences and product features at different levels. In the vocabulary-level neural network, the invention connects candidate news articles with the user's historical news sequences, calculating weights for each word in the sequence to obtain the final vocabulary-level output. In the category-level neural network, the invention selects the most frequently viewed news categories based on the number of news articles the user has historically viewed, adds an attention mechanism, calculates weights for each category, and obtains the final category-level output. Finally, these are connected to the output of the news-level neural network, processed by an activation function to obtain the final classification prediction result, and a loss function is constructed to train the model. Compared with other basic models, this invention significantly improves accuracy.

[0112] Existing news recommendation neural network models often neglect user selection information at the vocabulary and category levels. This invention constructs a neural network model that integrates user selection information at both the vocabulary and category levels, thereby solving this problem. Specifically, in addition to the news-level neural network, this invention adds vocabulary-level and category-level neural networks to collaboratively represent users and news information, thereby enhancing the value of the information and significantly improving the accuracy compared to traditional recommendation methods.

[0113] Example 2

[0114] like Figure 6 As shown, this embodiment of the invention provides a news recommendation method system based on attention mechanism and user multi-interest modeling, including the following modules:

[0115] The module 71 for generating news category and subcategory vector matrices is used to represent words in news headlines using word vectors pre-trained from text corpora, and to randomly generate news category vector matrices E using a normal distribution. vert and its subclass vector matrix E sub_vert ;

[0116] A vocabulary-level neural network module 72 is constructed to build a user vocabulary sequence w using candidate news titles and historical news titles. The user vocabulary sequence is then input into the vocabulary-level neural network for feature extraction, resulting in a matrix composed of w. And the final word feature vector o w ;

[0117] Construct a news modeling network module 73, used to... Inputting the news modeling network yields a news headline vector matrix. Will After applying the attention mechanism, the representation vector matrix X of the news headline is obtained. t ; X t With E vert and E vert The concatenation yields the news vector matrix E. n ;

[0118] Construct a news-level neural network module 74 to use E n The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector o. n ;

[0119] A category-level neural network module 75 is constructed to select the C news categories most frequently viewed by users and their corresponding news vector matrices E. v Input the category-level neural network to obtain the final representation E of the user for each frequently browsed news category. o E o After inputting the category-level feature fusion layer, the output is the news category feature vector o. v ;

[0120] Personalized news recommendation module 76, used to recommend news items to users. w o n and o v The concatenation operation yields vector o, which is then passed through an activation function to obtain the final prediction result. Meanwhile, a loss function is constructed for training an attention-based news recommendation network.

[0121] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A news recommendation method based on attention mechanisms and user multi-interest modeling, characterized in that, include: Step S1: Use word vectors pre-trained from the text corpus to represent the words in the news headlines, and use a normal distribution to randomly generate news category vector matrices. and its subclass vector matrix ; Step S2: Construct an attention-based news recommendation network, including: a vocabulary-level neural network, a news modeling network, a news-level neural network, and a category-level neural network; First, a user vocabulary sequence is constructed using candidate news headlines and historical news headlines. The user's word sequence is input into the word-level neural network for feature extraction, resulting in... The matrix formed And the final word feature vector ; Step S3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full Inputting the news modeling network yields a news headline vector matrix. ,Will The attention mechanism yields the representation vector matrix of the news headlines. ;Will and and The concatenation yields the news vector matrix. ; Step S4: The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector. ; Step S5: Select the pages most frequently browsed by the user. Each news category and its corresponding news vector matrix Input the category-level neural network to obtain the final representation of the user for each frequently browsed news category. ;Will After inputting the category-level feature fusion layer, the output is a news category feature vector. ; Step S6: [The sentence is incomplete and requires more context to be translated accurately.] , and Connect to obtain a vector and will After the activation function, the final prediction result is obtained. Simultaneously, a loss function is constructed for training the attention-based news recommendation network.

2. The news recommendation method based on attention mechanism and user multi-interest modeling according to claim 1, characterized in that, Step S2: Constructing an attention-based news recommendation network, including: a vocabulary-level neural network, a news modeling network, a news-level neural network, and a category-level neural network; firstly, constructing a user vocabulary sequence using candidate news titles and historical news titles. The user's word sequence is input into the word-level neural network for feature extraction, resulting in... The matrix formed And the final word feature vector Specifically, it includes: Step S21: The word sequence connection layer constructs a user vocabulary sequence based on the input candidate news titles and the historical news titles that the user has previously viewed. As shown in formula (1): in, For user word sequences, and These represent the word sequences of the candidate news items and the historical news items that the user has viewed, respectively. The semantic representation of the entire input word sequence. To separate candidate news from historical news sequences; Step S22: The vocabulary-level attention mechanism layer and the feature extraction layer are shown in formulas (2) to (5): in, The user vocabulary sequence is input to the attention mechanism layer. The resulting word vector matrix This is the weight matrix to be trained in the vocabulary-level neural network; They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication; express Dimensions The first in the multi-head self-attention mechanism The output vector generated by each head. This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism; The specific calculation method of the softmax function in formula (3) is shown in formula (4); This represents the lexical feature vector matrix output by the lexical-level neural network.

3. The news recommendation method based on attention mechanism and user multi-interest modeling according to claim 2, characterized in that, Step S3: ... Inputting the news modeling network yields a news headline vector matrix. ,Will The attention mechanism yields the representation vector matrix of the news headlines. ;Will and and The concatenation yields the news vector matrix. Specifically, it includes: The word vector matrix Input the news modeling network, extract the text features of the news headlines, and compare them with the news category vector matrix. and its subclass vector matrix connection To obtain the vector matrix of the news : in, The word vector matrix, Encode the position of words in news headlines. The news headline vector matrix is ​​used as input to the attention mechanism layer. The weight matrix to be trained in the news modeling network. They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication; express Dimensions The first in the multi-head self-attention mechanism The output vector generated by each head. This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism; The vector matrix representing news headlines; and The news category vector matrix and the news subcategory vector matrix represent the vectors of news categories and subcategories, respectively. The concatenation yields the news vector matrix. .

4. The news recommendation method based on attention mechanism and user multi-interest modeling according to claim 1, characterized in that, Step S4: ... The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector. Specifically, it includes: The news vector matrix The news-level neural network is input into the news-level neural network, which includes a news-level attention mechanism layer and a news-level feature fusion layer, and finally outputs a news feature vector. : in, This refers to the weight matrix to be trained in the news-level neural network. They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication; express Dimensions The first in the multi-head self-attention mechanism The output vector generated by each head. This represents the output vector matrix obtained by concatenating the output vectors of all heads in the multi-head self-attention mechanism; These represent the weights and biases of the linear layer, respectively. The weight matrix representing the proportion of news items viewed in the user's history in the final representation of the sequence; The final representation of the news sequence includes: The final representation of the sequence of news items viewed in the user's history and The final representation of the candidate news. ; It represents the Hadamardi (or Hadama) stack; The news feature vector is the final output of the news-level neural network.

5. The news recommendation method based on attention mechanism and user multi-interest modeling according to claim 1, characterized in that, Step S5: Select the browsers most frequently visited by the user. Each news category and its corresponding news vector matrix Input the category-level neural network to obtain the final representation of the user for each frequently browsed news category. ;Will After inputting the category-level feature fusion layer, the output is a news category feature vector. Specifically, it includes: Step S51: Select the page most frequently viewed by the user Each news category and its corresponding news vector matrix Input the category-level neural network to obtain the final representation of the user for each frequently browsed news category. : in, This indicates the number of news articles a user has viewed in the past. This represents the number of news categories selected by the user. The news categories are sorted according to the number of times the user has viewed them historically, and the categories most frequently viewed by the user are selected. Construct a vector matrix of frequently viewed news categories and their corresponding news vectors. ; This represents the weight matrix to be trained in a category-level neural network. They represent and The query vector matrix, key vector matrix, and value vector matrix generated by multiplication; express The dimension of any query vector in the vector. This is the first in the multi-head self-attention mechanism. The output generated by the size This represents the output matrix obtained by concatenating the outputs of all heads in the multi-head self-attention mechanism; These represent the weights and biases of a linear layer in a category-level neural network, respectively. This indicates the weight of each news item in each category in the final representation of that category. The most frequently browsed by users The final representation of each news category; Step S52: ... Input the category-level feature fusion layer to obtain the user's final representation on the neural network at the category level. ; in, These represent the weights and biases of the linear layer, respectively. These are the weights of the vectors for each category; The final representation of the user on the neural network at the category level; This is the news category feature vector that is ultimately output by the category-level neural network.

6. The news recommendation method based on attention mechanism and user multi-interest modeling according to claim 1, characterized in that, Step S6: ... , and Connect to obtain a vector and will After the activation function, the final prediction result is obtained. Simultaneously, a loss function is constructed for training the attention-based news recommendation network, specifically including: in, Indicates calculation The weight matrix, Indicates calculation The bias vector, It is a prediction of the probability of news clicks; Meanwhile, the attention-based news recommendation network is trained according to the loss function, as shown in formula (26): in, This represents the actual click probability of news articles. It is the predicted probability of news clicks.

7. A news recommendation method system based on attention mechanism and user multi-interest modeling, characterized in that, Includes the following modules: The module generates news category and subcategory vector matrices, which uses word vectors pre-trained from text corpora to represent words in news headlines, and randomly generates news category vector matrices using a normal distribution. and its subclass vector matrix ; A vocabulary-level neural network module is constructed to build user vocabulary sequences using candidate news headlines and historical news headlines. The user's word sequence is input into the word-level neural network for feature extraction, resulting in... The matrix formed And the final word feature vector ; Construct a news modeling network module to use for Inputting the news modeling network yields a news headline vector matrix. ,Will The attention mechanism yields the representation vector matrix of the news headlines. ;Will and and The concatenation yields the news vector matrix. ; Build a news-level neural network module to... The input is a news-level neural network, which passes through a news-level attention mechanism layer and a news-level feature fusion layer, and outputs a news feature vector. ; Build a category-level neural network module to select the categories most frequently visited by users. Each news category and its corresponding news vector matrix Input the category-level neural network to obtain the final representation of the user for each frequently browsed news category. ;Will After inputting the category-level feature fusion layer, the output is a news category feature vector. ; Personalized news recommendation module, used to... , and Connect to obtain a vector and will After the activation function, the final prediction result is obtained. Simultaneously, a loss function is constructed for training the attention-based news recommendation network.

Images