Personalized news recommendation method and system based on user global interest migration perception

Abstract

The present invention discloses a personalized news recommendation method and system based on user global interest migration perception. The method first extracts news headline text and news headline entity text from news data, constructs user sequences, click-through relationship networks between users and news, and sequential relationship networks between news items; then calculates a multi-perspective news content representation; then calculates a user content interest representation; then constructs a global news migration graph, convolves the click relationships between users and news items, and the migration relationships between news items; and then fuses these two relationships to obtain a user migration interest representation; finally, the user content interest representation and migration interest representation are combined to construct a final news recommendation model based on user global interest migration perception, thereby achieving personalized recommendations. The system uses web interaction technology to visualize data analysis and recommendation results. The present invention can effectively improve the accuracy of personalized recommendations and is highly robust.

Description

Technical Field

[0001] The present invention relates to a personalized news recommendation method, and in particular to a personalized news recommendation method and system based on user global interest migration perception. Background Art

[0002] The rapid development of online news platforms has enabled users to share, search, and browse news information in a timely manner. This has greatly enriched people's information needs, but has also led to the problem of information overload. Recommendation systems, which use information filtering technology to recommend information that users may be interested in, have become an effective solution to this problem.

[0003] News recommendation systems can be roughly divided into two categories: collaborative filtering-based news recommendation and content-based news recommendation. Based on the collaborative filtering-based and content-based approaches, many new branches have emerged. Based on the problems they address, they are mainly divided into privacy-preserving recommendation systems, knowledge-aware recommendation systems, sequence-aware recommendation systems, and explainable recommendation systems. Due to the special nature of anonymous browsing, news usually stores user browsing records in the form of session sequences, making it difficult to obtain users' long-term browsing information. Therefore, the importance of sequence-based recommendation in news recommendation is particularly significant. In 2019, researchers such as Wang systematically summarized the classification of sequential recommendation systems. Sequential recommendation systems are mainly divided into three categories according to their development technology: traditional sequential recommendation systems, sequential recommendation systems based on latent semantic models, and recommendation systems based on deep learning.

[0004] Traditional sequential recommendation systems are mainly divided into those based on sequential pattern mining and those based on Markov chain models. Sequential pattern mining methods primarily mine frequent patterns from sequence data and then use these patterns to guide subsequent recommendations. While straightforward, sequential pattern mining often generates a large number of redundant patterns, adding unnecessary time and space overhead. Another significant drawback is that, due to frequency constraints, it often misses infrequently occurring patterns and items, limiting recommendations to popular items. Markov chain-based recommendation systems sequentially model the transitions between user-item interactions to predict the next interaction. Due to the Markov property, which assumes that the current interaction depends only on one or more recent interactions, they can only capture short-term dependencies and ignore long-term ones. Furthermore, they can only capture point-wise dependencies and ignore inter-set dependencies. Traditional sequential recommendation models leverage their natural advantage in modeling the sequential correlations between user-item interactions in a sequence. While intuitive and simple, they suffer from numerous drawbacks.

[0005] Sequential recommendation based on latent semantic models primarily uses factorization machines to learn the latent representation of each user or item. Unlike collaborative filtering methods, the matrix or tensor that needs to be factorized is composed of interactions, rather than the rating matrix used in collaborative filtering. Sequential recommendation based on latent semantic models is susceptible to data sparsity, making it difficult to achieve ideal recommendation results. Furthermore, the limited linear representation capabilities of the factorization machine and the inclusion of useless feature cross-combinations can significantly impact recommendation results. In recent years, with the development of deep learning, some researchers have combined deep learning methods with factorization machines and applied them to sequential recommendation, improving their linear representation capabilities. However, this results in complex model design and high computational complexity.

[0006] Sequential recommendation systems based on deep learning primarily use two representative techniques: recurrent neural networks (RNNs) and graph neural networks (GNNs) to model the interactions between users and items, as well as between them. RNNs have advantages in handling long-term dependencies in sequences, but they still struggle to model very long sequences. Furthermore, RNNs struggle to model high-order dependencies within sequences and accurately represent user preferences when the amount of news a user browses is sparse. In recent years, research examining the interactions between users and news from a graph perspective has become increasingly popular. Researchers typically construct the user-item interaction sequence as a directed graph, with each item as a node and each sequence as a path. Graph neural networks are then used to learn item representations based on this constructed graph structure, and then user historical information is combined to model the user. Other researchers have also divided the user-item interaction sequence into time slices based on temporal information. Items within the same time slice are connected using hyperedges to construct a hypergraph. These hypergraph convolutions are then used to learn item representations, leveraging the existing item representations to dynamically represent users. By transforming the relationships between sequences to construct a graph-relational data structure and elevating first-order sequence relationships to higher-order graph relationships, complex dependencies between item contexts can be more easily modeled. For sequences with relatively few historical news views, contextual information from the news sequence can be used to supplement this information, alleviating the difficulty in modeling user preferences when the user's historical news views are sparse. However, existing research typically only considers the click-through relationship between users and news, ignoring the inherent connections within the global news viewing sequence and, consequently, overlooking the global shift in user interests. This leads to incomplete descriptions of user interests and fails to meet user needs.

[0007] How to implement a recommendation algorithm that combines comprehensive content mining with user global interest transition awareness is a pressing issue in news recommendation systems. Based on the aforementioned research background, this paper addresses the issues of insufficient global information aggregation and news representation in news recommendation systems by proposing a personalized news recommendation system based on user global interest transition awareness. This user global interest transition awareness is defined as: from a global perspective, it mines the interactive connections between user click behaviors and news, while also considering the sequential transition connections between news sequences to model high-order relationships between users and news. Summary of the Invention

[0008] Purpose of the invention: To address the deficiencies of the prior art, the present invention provides a personalized news recommendation method and system based on user global interest migration perception.

[0009] Technical solution: To achieve the above-mentioned purpose, the present invention adopts the following technical solution:

[0010] A personalized news recommendation method based on user global interest migration perception, the method comprising the following steps:

[0011] (1) Preprocess the data of users’ historical browsing news and candidate news, and construct a global news-to-news sequential relationship network and a click relationship network between users and news;

[0012] (2) Calculation of candidate news content representation

[0013] The attention mechanism is used to calculate the self-attention representation of the title text and entities of the candidate news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is also calculated. The representation of the candidate news content is obtained by multi-perspective fusion calculation by combining the representations of the two perspectives and the cross-attention representation between the two perspectives.

[0014] (3) Calculation of User Content Interest Representation

[0015] The attention mechanism is used to calculate the self-attention representation of the title text and entities of the user's historical browsing news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is also calculated. The representations of the two perspectives are combined with the cross-attention representation between the two perspectives to perform multi-perspective fusion calculation to obtain the content representation of the user's historical browsing news. The content representation of the user's historical browsing news is then aggregated based on the attention mechanism to obtain the representation of the user's content interest.

[0016] (4) Calculation of User Migration Interest Representation

[0017] The global news-to-news sequential relationship network and the click relationship network between users and news constructed in step (1) are integrated to construct a global news migration graph. The user node representation and different neighbor news node representations in the global news migration graph are used as inputs. A two-layer migration-aware graph attention network is used to perform information aggregation and learning. Finally, the user migration interest representation, news representation, and migration representation are obtained. The migration representation includes a propagation representation and an influence representation.

[0018] (5) Joint recommendation model

[0019] Combine the candidate news content representation obtained in step (2), the user content interest representation obtained in step (3), the user migration interest representation, the news representation, and the migration representation obtained in step (4), build a joint recommendation module and perform news recommendation, calculate the similarity between the user content interest representation obtained in step (3) and the candidate news content representation obtained in step (2), and obtain the user content interest score, calculate the similarity between the user migration interest representation and the candidate news representation and the migration representation, and obtain the user migration interest score; perform a weighted sum of the user content interest score and the user migration interest score to obtain the final user's interaction probability with the candidate news, and finally return a list of top-k recommended news in the candidate news set according to the sorting of the interaction probability;

[0020] (6) System function display.

[0021] The personalized news recommendation method based on user global interest migration perception, the specific method of step (1) is to extract the news title text field and the title entity field from the user's historical browsing news and candidate news, obtain the corresponding news title word vector and entity vector as the initial vector representation through the entity vector trained on the Glove word vector; construct each user's historical browsing news sequence according to the time sequence of the user browsing news, and simultaneously construct a click relationship network between the user and the historical browsing news, and then, based on the sequence data of the user's historical browsing news, connect the historical browsing news before and after the click to construct a global news and news sequential relationship network.

[0022] The personalized news recommendation method based on user global interest migration perception, step (2) includes the following specific steps:

[0023] (2-1) For a given candidate news n i , the news title text sequence is represented as Where T i represents the title text of the i-th news, w i,j That is, the jth word in the title text of the i-th news, |T i | is the total number of words in the title text; the entity sequence of the news is represented as Among them E i Represents the title entity of the i-th news, e i,j That is, the jth entity in the title of the i-th news, |E i | is the total number of entities in the title; Use self-attention to learn the title text sequence representation matrix of news The calculation process is as follows:

[0024]

[0025] in, is the word vector matrix of the word sequence, the superscript T represents the matrix transpose operation, d w is the feature dimension of the word, Represents word w i,j The vector representation of Indicates news i The self-attention influence weight of the j-th word in , exp(·) represents the exponential function with the natural constant e as the base, It's news i The self-attention representation weight of the k-th word in , where k refers to the k-th word, It is the self-attention representation of the word. Each word is updated according to the normalized weight vector to obtain the text sequence representation matrix Afterwards, the self-attention mechanism is used to calculate the self-attention representation of the i-th news title text

[0026] Self-attention learning is also performed on the entities in the title to learn the entity sequence representation, and the entity sequence representation matrix of the news title is obtained. in Represent the self-attention representation of the jth entity in the title of the i-th news, and obtain the entity self-attention representation in the news title

[0027] (2-2) Through the representation matrix of the news title text sequence and entity sequence representation of news Perform cross attention to obtain the degree of association between words and entities, add the rows and columns as the weight values of words and entities respectively, and then aggregate the features of text sets and entity sets by weight to obtain the text-level cross-learning representation and entity-level cross-learning representations The specific calculation process is as follows:

[0028]

[0029] in Represents word w i,j The self-attention vector representation of Represents entity ei,k The self-attention vector representation of Indicates news i The cross-attention representation weight of the j-th word in , Indicates news i The cross attention influence weight of the j-th word in , Indicates news i The cross attention influence weight of the k-th word in , Indicates news i The cross attention representation weight of the k-th word in , d w and d e They are word and entity feature dimensions, and entity-level cross-learning representation The calculation method is the same as the text-level cross-learning representation

[0030] (2-3) The title text obtained in step (2-1) is represented by self-attention Title Entity Self-Attention Representation And the text-level cross-learning representation obtained in step (2-2) and entity-level cross-learning representations After adding and splicing, we can get the news content representation vector and the multi-view representation vector of the i-th news. The calculation is as follows:

[0031]

[0032] Where + represents the addition of two vectors, ∥ represents the concatenation operation, and the i-th news content representation vector The calculation is as follows:

[0033]

[0034] in are the linear layer weights, is the linear layer bias.

[0035] The personalized news recommendation method based on user global interest migration perception, wherein step (3) includes the following specific steps:

[0036] (3-1) Perform self-attention extraction of user sequence content, and obtain the news sequence matrix by calculating the news content representation of the user's historical browsing news sequence in As user u i The content representation vector of the first news item n1 browsed, Represents the length of the user's historical browsing sequence. The self-attention mechanism is applied to the matrix constructed by the news sequence vector set to update the information between news sequences to obtain the news self-attention matrix Then normalize it through the Softmax function to get the weight matrix Then multiply this weight matrix with the corresponding news vector set to realize the connection between each news and other news, and obtain the self-attention vector matrix of the user's historical browsing news in is user u i News j The self-attention vector of

[0037] (3-2) Using the attention mechanism at the user level to weight the self-attention vector set of aggregated news to represent the user's content interest preferences The calculation is as follows:

[0038]

[0039] in Represents user u i For news in the historical browsing sequence n j degree of preference, It is the parameter of the first linear layer. Its dimension is consistent with the dimension of the news content vector. It is nonlinearly transformed through the Tanh activation function and then sent to the second linear layer. is the parameter of the second linear layer, and the user implements the dimension d n Mapped to 1, each news vector corresponds to a weight. After normalizing all the weights of historical browsing news, the user's preference distribution for news in the historical browsing news sequence can be obtained. Finally, the weighted sum of the news self-attention vector matrix Get user content interest expression

[0040] The personalized news recommendation method based on user global interest migration perception, wherein step (4) includes the following specific steps:

[0041] (4-1) Modeling user migration interests based on sequence global perception, including the construction of a global news migration graph and the representation of migration interests based on migration perception;

[0042] First, we construct a global news migration graph: we connect the news in the user's historical browsing sequence into chains according to the chronological order of the user's browsing. Each chain can be connected to form a graph through the same news nodes. The order in which the news was originally browsed is the news propagation order, indicating that it spreads from the previous news to the next news. The order in reverse of the browsing order is the news influence order, indicating that the next news is affected by the previous news. At the same time, we also add the relationship between users and news into the graph, integrating the click relationship between users and news to obtain a global news migration graph. Where V u Represents the user node set, V n Represents the news node set, E C The edge set representing the click relationship, E P The edge set representing the propagation relationship, E I The set of edges representing the influence relationship;

[0043] (4-2) After constructing the global news migration graph, based on the various relationships in the global news migration graph, the user's migration interest in browsing news is modeled from a macro perspective. The formal expression is: for user u, in the global news migration graph According to the click relationship, we can get the neighbor news set Nei(u,Click)={n|n∈N,(u,n)∈E C For news n, the neighbor user set Nei(n,Click)={u|u∈U,(u,n)∈E is obtained according to the click relationship C According to the propagation relationship, we get the neighbor news set Nei(n,Propagate)={n|n∈N,(n,n)∈E P According to the influence relationship, we get the neighbor news set Nei(n,Influence)={n|n∈N,(n,n)∈E I The transfer interest learning model aggregates and learns information through a transfer-aware graph attention network, or Transition-GAT. The input is a user node representation and different neighboring news node representations. Nodes are aggregated based on their edge relationships to obtain user transfer interest representations, news representations, and transfer representations. In the first Transition-GAT, the initial user and news vectors are collected. The transfer network mainly learns the connection between news items. In the second Transition-GAT, the previously learned user and news vectors are collected to learn the transfer relationship between users and news items.

[0044] The input of Transition-GAT is the user representation matrix and News Representation Matrix in Represents user u i The initial characterization vector of Representative News j The initial characterization vector for user u i Browse NewsNei(u i ,Click), based on the graph attention mechanism, weighted aggregation of its neighbor news is used to obtain a new user migration interest representation The calculation formula is as follows:

[0045]

[0046] in Represents neighbor news node n j For user u i The attention score, Represents neighbor news node n j For user u i The degree of impact, Represents neighbor news node n k For user u i The attention score, is the transformation matrix parameter of the user feature vector in the first layer of Transition-GAT, is the transformation matrix parameter of the news feature vector in the first layer of Transition-GAT, d g is the dimension of the initial feature vector of users and news, ∥ represents the concatenation operation, It is the representation of user migration interest obtained by weighted aggregation of all news in the Nei(u1,Click) set through attention weight;

[0047] For the neighboring user nodes of the i-th news node, Nei(n i ,Click), its operation is similar to the user's news collection operation, and the news representation is obtained by calculating the attention weight and weighting For the information collection of communication relationship and influence relationship, the information collection of communication relationship and influence relationship is carried out at the same time, and then the corresponding news migration representation is obtained by calculating the weight. i , similar to the above graph aggregation methods can be performed on the communication relationship and influence relationship to obtain the news communication representation and news impact representation The migration representation of news is obtained by aggregating its impact representation and dissemination representation The calculation is as follows:

[0048]

[0049] Among them, Nei(ni ,Propagate) represents the connection with news node n i A collection of neighbor news with a propagation relationship, Represents neighbor news node n k For news node n i The attention score, Nei(n i ,Influence) represents the relationship with news node n i Neighborhood news collection with influence relationships, news dissemination vector and news impact vector The characteristics of the nodes of the propagation relationship and influence relationship are weighted and aggregated respectively, which clearly represents the relationship between news.

[0050] The personalized news recommendation method based on user global interest migration perception, wherein step (5) includes the following specific steps:

[0051] (5-1) Calculate the similarity between the user's content interest representation and the candidate news' content representation to obtain the user's content interest score; calculate the similarity between the user's migration interest representation and the candidate news' migration representation to obtain the user's migration interest score; user content interest score and user migration interest score The calculation of is as follows:

[0052]

[0053] where f u Represents the user's content interest representation, represents the content representation of candidate news i, gf u represents the user's transfer interest representation, represents the candidate news transfer representation, ⊙ represents the vector inner product;

[0054] (5-2) The content interest score and the transfer interest score are weighted and summed to obtain the final user's interaction probability for the candidate news, and the user u's interaction probability for the candidate news c is obtained. i The interaction probability The calculation formula is as follows:

[0055]

[0056] in Represents the normalized calculation of user u's response to candidate news c i Content interest score, Represents the normalized calculation of user u's response to candidate news c iwhere k is the length of the candidate news set of user u, and θ∈[0,1] is a hyperparameter representing the weight of user content interest. The interaction probability is obtained by weighting the content interest probability and the migration interest probability through θ. Finally, the top-k news list recommended in the candidate news set is returned according to the sorting of the interaction probabilities.

[0057] The personalized news recommendation method based on user global interest migration perception, wherein the system function display in step (6) includes a visual display of data analysis, experimental analysis and recommendation analysis, wherein the data analysis includes a graphical display of the time distribution of users browsing news, the distribution of news topics and subtopics, and the cumulative distribution of text length and users browsing news; the experimental analysis includes a bar chart display of the comparative experiments and ablation experiment results of the algorithm on a public data set, and a visual display of users' attention to news words; the recommendation analysis includes a display of an online news browsing interface, a display of a recommended news list after the user clicks to browse the news, a display of similar users and their browsing sequences, and a display of news migration relationships.

[0058] A personalized news recommendation system based on user global interest migration perception, which is operated by the personalized news recommendation method based on user global interest migration perception described in steps 1-7, and includes a data processing module, a business processing module, and a visualization analysis module;

[0059] The data processing module preprocesses the dataset offline, which is divided into news title preprocessing, user browsing news preprocessing, user-news graph construction, and news relationship graph construction. The preprocessed data is then classified and stored.

[0060] The business processing module mainly connects with system requirements and calls pre-trained models to generate data. This module contains three sub-modules: user content interest mining module, user migration interest mining module and joint recommendation module. Among them, the user content interest mining module models user content interests based on historical news data and news headline data browsed by users; the user migration interest mining module models user migration interests based on user-news graphs and news relationship graphs; the joint recommendation module combines users' content interests and migration interests to provide recommendation services for the visualization layer;

[0061] The visualization analysis module is an interactive interface module that the system provides services to users. It mainly provides users with visualization results of data analysis, experimental analysis and recommendation analysis. In addition, it provides users with interactive functions so that users can select news to browse, and make recommendations and recommendation analysis based on user browsing.

[0062] Beneficial effects: Compared with the prior art, the present invention adopts the above technical solution and has the following advantages:

[0063] (1) In personalized news recommendation, the present invention pays attention to the migration of users' interests in browsing news. A personalized news recommendation system based on the perception of users' global interest migration is designed: starting from a global perspective, a global migration graph of news is constructed based on the historical browsing sequences of global users. The click relationship and news migration relationship are convolved through the migration-aware graph attention mechanism; and the content interest and migration interest of the user are combined to make joint recommendations, thus achieving high-precision personalized recommendations.

[0064] (2) When modeling the content information in the news title, the present invention uses a multi-perspective associated news content modeling method, which can fully mine the news content information by combining the respective perspectives of the news title and news entity and the associated perspectives between them to represent the news content. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is the overall framework diagram of the algorithm of the present invention;

[0066] Figure 2 This is a schematic diagram of the algorithm for modeling the news content part;

[0067] Figure 3 It is a schematic diagram of the construction of the global news migration map;

[0068] Figure 4 This is a schematic diagram of the user migration interest model based on sequence global perception;

[0069] Figure 5 It is a system architecture diagram of the present invention; DETAILED DESCRIPTION

[0070] The technical solution of the present invention is described in detail below.

[0071] The following is only one embodiment of the present invention. The present invention has many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications based on the present invention. These corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

[0072] like Figure 1-5 As shown, the present invention provides a personalized news recommendation method based on user global interest migration perception, which includes the following steps:

[0073] 1. Preprocess the data of users’ historical browsing news and candidate news, and build a global news-to-news sequential relationship network and a click relationship network between users and news.

[0074] The news title text field and title entity field are extracted from the user's historically browsed news and candidate news. The corresponding news title word vector and entity vector are obtained as the initial vector representation through the entity vector trained on the Glove word vector. A sequence of historically browsed news for each user is constructed according to the chronological order of the user's browsing of news. At the same time, a click relationship network between the user and the historically browsed news is constructed. Then, based on the sequence data of the user's historically browsed news, the edges between the historically browsed news before and after the click are connected to construct a global sequential relationship network between news.

[0075] 2. Calculation of candidate news content representation

[0076] The attention mechanism is used to calculate the self-attention representation of the title text and entities of the candidate news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is calculated. The representation of each perspective and the cross-attention representation between the two perspectives are combined to perform multi-perspective fusion calculation to obtain the candidate news content representation:

[0077] (2-1) For a given candidate news n i , the news title text sequence is represented as Where T i represents the title text of the i-th news, w i,j That is, the jth word in the title text of the i-th news, |T i | is the total number of words in the title text; the entity sequence of the news is represented as Among them E i Represents the title entity of the i-th news, e i,j That is, the jth entity in the title of the i-th news, |E i | is the total number of entities in the title; Use self-attention to learn the title text sequence representation matrix of news The calculation process is as follows:

[0078]

[0079] in, is the word vector matrix of the word sequence, the superscript T represents the matrix transpose operation, d w is the feature dimension of the word, Represents word w i,j The vector representation of Indicates news i The self-attention influence weight of the j-th word in , exp(·) represents the exponential function with the natural constant e as the base, It's news i The self-attention representation weight of the j-th word in , k refers to the k-th word, It is the self-attention representation of the word. Each word is updated according to the normalized weight vector to obtain the text sequence representation matrix Afterwards, the self-attention mechanism is used to calculate the self-attention representation of the i-th news title text

[0080] Self-attention learning is also performed on the entities in the title to learn the entity sequence representation, and the entity sequence representation matrix of the news title is obtained. in Represent the self-attention representation of the jth entity in the title of the i-th news, and obtain the entity self-attention representation in the news title

[0081] (2-2) Through the representation matrix of the news title text sequence and entity sequence representation of news Perform cross attention to obtain the degree of association between words and entities, add the rows and columns as the weight values of words and entities respectively, and then aggregate the features of text sets and entity sets by weight to obtain the text-level cross-learning representation and entity-level cross-learning representations The specific calculation process is as follows:

[0082]

[0083] in Represents word w i,j The self-attention vector representation of Represents entity e i,k The self-attention vector representation of Indicates news i The cross-attention representation weight of the j-th word in , Indicates news i The cross attention influence weight of the j-th word in , Indicates news i The cross attention influence weight of the k-th word in , Indicates news i The cross attention representation weight of the k-th word in , d w and d e They are word and entity feature dimensions, and entity-level cross-learning representation The calculation method is the same as the text-level cross-learning representation

[0084] (2-3) The title text obtained in step (2-1) is represented by self-attention Title Entity Self-Attention Representation And the text-level cross-learning representation obtained in step (2-2) and entity-level cross-learning representations After adding and splicing, we can get the news content representation vector and the multi-view representation vector of the i-th news. The calculation is as follows:

[0085]

[0086] Where + represents the addition of two vectors, ∥ represents the concatenation operation, and the i-th news content representation vector The calculation is as follows:

[0087]

[0088] in are the linear layer weights, is the linear layer bias.

[0089] The overall implementation process of news content representation calculation based on multi-perspective association is shown in Algorithm 1:

[0090]

[0091]

[0092] 3. Calculation of User Content Interest Representation

[0093] The attention mechanism is used to calculate the self-attention representation of the title text and entities of the user's historical browsing news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is also calculated. The representations of the two perspectives and the cross-attention representation between the two perspectives are combined to perform multi-perspective fusion calculation to obtain the content representation of the user's historical browsing news. The content representation of the user's historical browsing news is then aggregated based on the attention mechanism to obtain the representation of the user's content interest. The specific steps include the following:

[0094] (3-1) Perform self-attention extraction of user sequence content, and obtain the news sequence matrix by calculating the news content representation of the user's historical browsing news sequence in As user u i The content representation vector of the first news item n1 browsed, Represents the length of the user's historical browsing sequence. The self-attention mechanism is applied to the matrix constructed by the news sequence vector set to update the information between news sequences to obtain the news self-attention matrix Then normalize it through the Softmax function to get the weight matrix Then multiply this weight matrix with the corresponding news vector set to realize the connection between each news and other news, and obtain the self-attention vector matrix of the user's historical browsing news in is user u i News j The self-attention vector of

[0095] (3-2) Using the attention mechanism at the user level to weight the self-attention vector set of aggregated news to represent the user's content interest preferences The calculation is as follows:

[0096]

[0097] in Represents user u i For news in the historical browsing sequence n j degree of preference, It is the parameter of the first linear layer. Its dimension is consistent with the dimension of the news content vector. It is nonlinearly transformed through the Tanh activation function and then sent to the second linear layer. is the parameter of the second linear layer, and the user implements the dimension d n Mapped to 1, each news vector corresponds to a weight. After normalizing all the weights of historical browsing news, the user's preference distribution for news in the historical browsing news sequence can be obtained. Finally, the weighted sum of the news self-attention vector matrix Get user content interest expression

[0098] 4. User Migration Interest Representation Calculation

[0099] The global news-to-news sequential relationship network and the click relationship network between users and news constructed in step (1) are integrated to construct a global news migration graph. The user node representation and different neighbor news node representations in the global news migration graph are used as input. A two-layer migration-aware graph attention network is used to perform information aggregation and learning. Finally, the user migration interest representation, news representation, and migration representation are obtained. The migration representation includes a propagation representation and an influence representation. The specific steps are as follows:

[0100] (4-1) Modeling user migration interests based on sequence global perception, including the construction of a global news migration graph and the representation of migration interests based on migration perception;

[0101] First, we construct a global news migration graph: we connect the news in the user's historical browsing sequence into chains according to the chronological order of the user's browsing. Each chain can be connected to form a graph through the same news nodes. The order in which the news was originally browsed is the news propagation order, indicating that it spreads from the previous news to the next news. The order in reverse of the browsing order is the news influence order, indicating that the next news is affected by the previous news. At the same time, we also add the relationship between users and news into the graph, integrating the click relationship between users and news to obtain a global news migration graph. Where V u Represents the user node set, V n Represents the news node set, E C The edge set representing the click relationship, E P The edge set representing the propagation relationship, E I The set of edges representing the influence relationship;

[0102] (4-2) After constructing the global news migration graph, based on the various relationships in the global news migration graph, the user's migration interest in browsing news is modeled from a macro perspective. The formal expression is: for user u, in the global news migration graph According to the click relationship, we can get the neighbor news set Nei(u,Click)={n|n∈N,(u,n)∈E C For news n, the neighbor user set Nei(n,Click)={u|u∈U,(u,n)∈E is obtained according to the click relationship C According to the propagation relationship, we get the neighbor news set Nei(n,Propagate)={n|n∈N,(n,n)∈E P According to the influence relationship, we get the neighbor news set Nei(n,Influence)={n|n∈N,(n,n)∈E I The transfer interest learning model aggregates and learns information through a transfer-aware graph attention network, or Transition-GAT. The input is a user node representation and different neighboring news node representations. Nodes are aggregated based on their edge relationships to obtain user transfer interest representations, news representations, and transfer representations. In the first Transition-GAT, the initial user and news vectors are collected. The transfer network mainly learns the connection between news items. In the second Transition-GAT, the previously learned user and news vectors are collected to learn the transfer relationship between users and news items.

[0103] The input of Transition-GAT is the user representation matrix and News Representation Matrix in Represents user u i The initial characterization vector of Representative News j The initial characterization vector for user u i Browse NewsNei(u i ,Click), based on the graph attention mechanism, weighted aggregation of its neighbor news is used to obtain a new user migration interest representation The calculation formula is as follows:

[0104]

[0105] in Represents neighbor news node n j For user u i The attention score, Represents neighbor news node n j For user u i The degree of impact, Represents neighbor news node n k For user u i The attention score, is the transformation matrix parameter of the user feature vector in the first layer of Transition-GAT, is the transformation matrix parameter of the news feature vector in the first layer of Transition-GAT, d g is the dimension of the initial feature vector of users and news, ∥ represents the concatenation operation, It is the representation of user migration interest obtained by weighted aggregation of all news in the Nei(u1,Click) set through attention weight;

[0106] For the neighboring user nodes of the i-th news node, Nei(n i ,Click), its operation is similar to the user's news collection operation, and the news representation is obtained by calculating the attention weight and weighting For the information collection of communication relationship and influence relationship, the information collection of communication relationship and influence relationship is carried out at the same time, and then the corresponding news migration representation is obtained by calculating the weight. i , similar to the above graph aggregation methods can be performed on the communication relationship and influence relationship to obtain the news communication representation and news impact representation The migration representation of news is obtained by aggregating its impact representation and dissemination representation The calculation is as follows:

[0107]

[0108] Among them, Nei(ni ,Propagate) represents the connection with news node n i A collection of neighbor news with a propagation relationship, Represents neighbor news node n k For news node n i The attention score, Nei(n i ,Influence) represents the relationship with news node n i Neighborhood news collection with influence relationships, news dissemination vector and news impact vector The characteristics of the nodes of the propagation relationship and influence relationship are weighted and aggregated respectively, which clearly represents the relationship between news.

[0109] The specific implementation algorithm for calculating user migration interest representation based on user global interest migration perception is as follows:

[0110]

[0111] 5. Joint Recommendation Model

[0112] Combine the candidate news content representation obtained in step (2), the user content interest representation obtained in step (3), the user migration interest representation, the news representation, and the migration representation obtained in step (4), build a joint recommendation module and perform news recommendation, calculate the similarity between the user content interest representation obtained in step (3) and the candidate news content representation obtained in step (2), and obtain the user content interest score, calculate the similarity between the user migration interest representation and the candidate news representation and the migration representation, and obtain the user migration interest score; perform weighted summation of the user content interest score and the user migration interest score to obtain the final user's interaction probability with the candidate news, and finally return the recommended top-k news list in the candidate news set according to the sorting of the interaction probability; specifically, the following steps are included:

[0113] (5-1) Calculate the similarity between the user's content interest representation and the candidate news' content representation to obtain the user's content interest score; calculate the similarity between the user's migration interest representation and the candidate news' migration representation to obtain the user's migration interest score; user content interest score and user migration interest score The calculation of is as follows:

[0114]

[0115] where f u Represents the user's content interest representation, represents the content representation of candidate news i, gf u represents the user's transfer interest representation, represents the candidate news transfer representation, ⊙ represents the vector inner product;

[0116] (5-2) The content interest score and the transfer interest score are weighted and summed to obtain the final user's interaction probability for the candidate news, and the user u's interaction probability for the candidate news c is obtained. i The interaction probability The calculation formula is as follows:

[0117]

[0118] in Represents the normalized calculation of user u's response to candidate news c i Content interest score, Represents the normalized calculation of user u's response to candidate news c i where k is the length of the candidate news set of user u, and θ∈[0,1] is a hyperparameter representing the weight of user content interest. The interaction probability is obtained by weighting the content interest probability and the migration interest probability through θ. Finally, the top-k news list recommended in the candidate news set is returned according to the sorting of the interaction probabilities.

[0119] 6. System function display

[0120] The system function display of the personalized news recommendation method based on the perception of user global interest migration includes the visualization of data analysis, experimental analysis and recommendation analysis. Among them, the data analysis includes the effect diagram of the time distribution of users browsing news, the distribution of news topics and sub-topics, and the effect diagram of text length and the cumulative distribution of users browsing news; the experimental analysis includes the bar chart display of the comparative experiments and ablation experiment results conducted on the algorithm on public datasets, as well as the visualization of users' attention to news words; the recommendation analysis includes the display of the online news browsing interface, the display of the recommended news list after the user clicks to browse the news, the display of similar users and their browsing sequences, and the display of news migration relationships.

[0121] The personalized news recommendation system based on user global interest migration perception, which is operated by a personalized news recommendation method based on user global interest migration perception of the present invention, includes a data processing module, a business processing module and a visualization analysis module;

[0122] The data processing module preprocesses the dataset offline, which is divided into news title preprocessing, user browsing news preprocessing, user-news graph construction, and news relationship graph construction. The preprocessed data is then classified and stored.

[0123] The business processing module mainly connects with system requirements and calls pre-trained models to generate data. This module contains three sub-modules: user content interest mining module, user migration interest mining module and joint recommendation module. Among them, the user content interest mining module models user content interests based on historical news data and news headline data browsed by users; the user migration interest mining module models user migration interests based on user-news graphs and news relationship graphs; the joint recommendation module combines users' content interests and migration interests to provide recommendation services for the visualization layer;

[0124] The visualization analysis module is an interactive interface module that the system provides services to users. It mainly provides users with visualization results of data analysis, experimental analysis and recommendation analysis. In addition, it provides users with interactive functions so that users can select news to browse, and make recommendations and recommendation analysis based on user browsing.

Claims

1. A personalized news recommendation method based on user global interest migration perception, characterized by: The method comprises the following steps: (1) Preprocess the data of users’ historical browsing news and candidate news, and construct a global news-to-news sequential relationship network and a click relationship network between users and news; (2) Calculation of candidate news content representation The attention mechanism is used to calculate the self-attention representation of the title text and entities of the candidate news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is also calculated. The representation of the candidate news content is obtained by multi-perspective fusion calculation by combining the representations of the two perspectives and the cross-attention representation between the two perspectives. (3) Calculation of User Content Interest Representation The attention mechanism is used to calculate the self-attention representation of the title text and entities of the user's historical browsing news from two different perspectives, namely the text perspective and the entity perspective. The cross-attention representation of the two perspective representations is also calculated. The representations of the two perspectives are combined with the cross-attention representation between the two perspectives to perform multi-perspective fusion calculation to obtain the content representation of the user's historical browsing news. The content representation of the user's historical browsing news is then aggregated based on the attention mechanism to obtain the representation of the user's content interest. (4) Calculation of User Migration Interest Representation The global news-to-news sequential relationship network and the click relationship network between users and news constructed in step (1) are integrated to construct a global news migration graph. The user node representation and different neighbor news node representations in the global news migration graph are used as inputs. A two-layer migration-aware graph attention network is used to perform information aggregation and learning. Finally, the user migration interest representation, news representation, and migration representation are obtained. The migration representation includes a propagation representation and an influence representation. (5) Joint recommendation model Combine the candidate news content representation obtained in step (2), the user content interest representation obtained in step (3), the user migration interest representation, the news representation, and the migration representation obtained in step (4), build a joint recommendation module and perform news recommendation, calculate the similarity between the user content interest representation obtained in step (3) and the candidate news content representation obtained in step (2), and obtain the user content interest score, calculate the similarity between the user migration interest representation and the candidate news representation and the migration representation, and obtain the user migration interest score; perform a weighted sum of the user content interest score and the user migration interest score to obtain the final user's interaction probability with the candidate news, and finally return a list of top-k recommended news in the candidate news set according to the sorting of the interaction probability; (6) System function display.

2. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: The specific method of step (1) is to extract the news title text field and title entity field from the user's historical browsing news and candidate news, and obtain the corresponding news title word vector and entity vector as the initial vector representation through the Glove word vector and the entity vector trained on Wikipedia respectively; construct each user's historical browsing news sequence according to the time sequence of the user browsing news, and at the same time construct a click relationship network between the user and the historical browsing news, and then according to the sequence data of the user's historical browsing news, connect the historical browsing news before and after the click to construct a global news and news sequential relationship network.

3. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: Step (2) includes the following specific steps: (2-1) For a given candidate news n i , the news title text sequence is represented as Where T i represents the title text of the i-th news, w i,j That is, the jth word in the title text of the i-th news, |T i | is the total number of words in the title text; the entity sequence of the news is represented as Among them E i Represents the title entity of the i-th news, e i,j That is, the jth entity in the title of the i-th news, |E i | is the total number of entities in the title; Use self-attention to learn the title text sequence representation matrix of news The calculation process is as follows: in, is the word vector matrix of the word sequence, the superscript T represents the matrix transpose operation, d w is the feature dimension of the word, Represents word w i,j The vector representation of Indicates news i The self-attention influence weight of the j-th word in , exp(·) represents the exponential function with the natural constant e as the base, It's news i The self-attention representation weight of the j-th word in , k refers to the k-th word, It is the self-attention representation of the word. Each word is updated according to the normalized weight vector to obtain the text sequence representation matrix Afterwards, the self-attention mechanism is used to calculate the self-attention representation of the i-th news title text Self-attention learning is also performed on the entities in the title to learn the entity sequence representation, and the entity sequence representation matrix of the news title is obtained. in Represent the self-attention representation of the jth entity in the title of the i-th news, and obtain the entity self-attention representation in the news title (2-2) Through the representation matrix of the news title text sequence and entity sequence representation of news Perform cross attention to obtain the degree of association between words and entities, add the rows and columns as the weight values of words and entities respectively, and then aggregate the features of text sets and entity sets by weight to obtain the text-level cross-learning representation and entity-level cross-learning representations The specific calculation process is as follows: in Represents word w i,j The self-attention vector representation of Represents entity e i,k The self-attention vector representation of Indicates news i The cross-attention representation weight of the j-th word in , Indicates news i The cross attention influence weight of the j-th word in , Indicates news i The cross attention influence weight of the k-th word in , Indicates news i The cross attention representation weight of the k-th word in , d w and d e They are word and entity feature dimensions, and entity-level cross-learning representation The calculation method is the same as the text-level cross-learning representation (2-3) The title text obtained in step (2-1) is represented by self-attention Title Entity Self-Attention Representation And the text-level cross-learning representation obtained in step (2-2) and entity-level cross-learning representations After adding and splicing, we can get the news content representation vector and the multi-view representation vector of the i-th news. The calculation is as follows: Where + represents the addition of two vectors, || represents the concatenation operation, and the i-th news content representation vector The calculation is as follows: in are the linear layer weights, is the linear layer bias.

4. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: Step (3) includes the following specific steps: (3-1) Perform self-attention extraction of user sequence content, and obtain the news sequence matrix by calculating the news content representation of the user's historical browsing news sequence in As user u i The content representation vector of the first news item n1 browsed, Represents the length of the user's historical browsing sequence. The self-attention mechanism is applied to the matrix constructed by the news sequence vector set to update the information between news sequences to obtain the news self-attention matrix Then normalize it through the Softmax function to get the weight matrix Then multiply this weight matrix with the corresponding news vector set to realize the connection between each news and other news, and obtain the self-attention vector matrix of the user's historical browsing news in is user u i News j The self-attention vector of (3-2) Using the attention mechanism at the user level to weight the self-attention vector set of aggregated news to represent the user's content interest preferences The calculation is as follows: in Represents user u i For news in the historical browsing sequence n j degree of preference, It is the parameter of the first linear layer. Its dimension is consistent with the dimension of the news content vector. It is nonlinearly transformed through the Tanh activation function and then sent to the second linear layer. is the parameter of the second linear layer, and the user implements the dimension d n Mapped to 1, each news vector corresponds to a weight. After normalizing all the weights of historical browsing news, the user's preference distribution for news in the historical browsing news sequence can be obtained. Finally, the weighted sum of the news self-attention vector matrix Get user content interest expression 5. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: Step (4) includes the following specific steps: (4-1) Modeling user migration interests based on sequence global perception, including the construction of a global news migration graph and the representation of migration interests based on migration perception; First, we construct a global news migration graph: we connect the news in the user's historical browsing sequence into chains according to the chronological order of the user's browsing. Each chain can be connected to form a graph through the same news nodes. The order in which the news was originally browsed is the news propagation order, indicating that it spreads from the previous news to the next news. The order in reverse of the browsing order is the news influence order, indicating that the next news is affected by the previous news. At the same time, we also add the relationship between users and news into the graph, integrating the click relationship between users and news to obtain a global news migration graph. Where V u Represents the user node set, V n Represents the news node set, E C The edge set representing the click relationship, E P The edge set representing the propagation relationship, E I The set of edges representing the influence relationship; (4-2) After constructing the global news migration graph, based on the various relationships in the global news migration graph, the user's migration interest in browsing news is modeled from a macro perspective. The formal expression is: for user u, in the global news migration graph According to the click relationship, we can get the neighbor news set Nei(n,Click)={n|n∈N,(u,n)∈E C For news n, the neighbor user set Nei(n,Click)={u|u∈U,(u,n)∈E is obtained according to the click relationship C According to the propagation relationship, we get the neighbor news set Nei(n,Propagate)={n|n∈N,(n,n)∈E P According to the influence relationship, we get the neighbor news set Nei(n,Influence)={n|n∈N,(n,n)∈E I The transfer interest learning model aggregates and learns information through a transfer-aware graph attention network, or Transition-GAT. The input is a user node representation and different neighboring news node representations. Nodes are aggregated based on their edge relationships to obtain user transfer interest representations, news representations, and transfer representations. In the first Transition-GAT, the initial user and news vectors are collected, and the connection between news is learned on the transfer network. In the second Transition-GAT, the previously learned user and news vectors are collected to learn the transfer relationship between users and news. The input of Transition-GAT is the user representation matrix and News Representation Matrix in Represents user u i The initial characterization vector of Representative News j The initial characterization vector for user u i Browse NewsNei(u i ,Click), based on the graph attention mechanism, weighted aggregation of its neighbor news is used to obtain a new user migration interest representation The calculation formula is as follows: in Represents neighbor news node n j For user u i The attention score, Represents neighbor news node n j For user u i The degree of impact, Represents neighbor news node n k For user u i The attention score, is the transformation matrix parameter of the user feature vector in the first layer of Transition-GAT, is the transformation matrix parameter of the news feature vector in the first layer of Transition-GAT, d g is the dimension of the initial feature vector of users and news, || represents the concatenation operation, It is the representation of user migration interest obtained by weighted aggregation of all news in the Nei(u1,Click) set through attention weight; For the neighboring user nodes of the i-th news node, Nei(n i ,Click), its operation is similar to the user's news collection operation, and the news representation is obtained by calculating the attention weight and weighting For the information collection of communication relationship and influence relationship, the information collection of communication relationship and influence relationship is carried out at the same time, and then the corresponding news migration representation is obtained by calculating the weight. i , the above graph aggregation method is performed on the communication relationship and influence relationship to obtain the news communication representation and news impact representation The migration representation of news is obtained by aggregating its impact representation and dissemination representation The calculation is as follows: Among them, Nei(n i ,Propagate) represents the connection with news node n i A collection of neighbor news with a propagation relationship, Represents neighbor news node n k For news node n i The attention score, Nei(n i ,Influence) represents the relationship with news node n i Neighborhood news collection with influence relationships, news dissemination vector and news impact vector The characteristics of the nodes of the propagation relationship and influence relationship are weighted and aggregated respectively, which clearly represents the relationship between news.

6. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: Step (5) includes the following specific steps: (5-1) Calculating the similarity between the user's content interest representation and the candidate news' content representation to obtain the user's content interest score, and calculating the similarity between the user's transfer interest representation and the candidate news' transfer interest representation to obtain the user's transfer interest score; User content interest rating and user migration interest score The calculation of is as follows: where f u Represents the user's content interest representation, represents the content representation of candidate news i, gf u represents the user's transfer interest representation, represents the candidate news transfer representation, ⊙ represents the vector inner product; (5-2) The content interest score and the transfer interest score are weighted and summed to obtain the final user's interaction probability for the candidate news, and the user u's interaction probability for the candidate news c is obtained. i The interaction probability The calculation formula is as follows: in Represents the normalized calculation of user u's response to candidate news c i Content interest score, Represents the normalized calculation of user u's response to candidate news c i where k is the length of the candidate news set of user u, and θ∈[0,1] is a hyperparameter representing the weight of user content interest. The interaction probability is obtained by weighting the content interest probability and the migration interest probability through θ. Finally, the top-k news list recommended in the candidate news set is returned according to the sorting of the interaction probabilities.

7. The personalized news recommendation method based on user global interest migration perception according to claim 1 is characterized in that: The system function display in step (6) includes a visual display of data analysis, experimental analysis and recommendation analysis, wherein the data analysis includes a graphical display of the time distribution of users browsing news, the distribution of news topics and subtopics, and the cumulative distribution of text length and users browsing news; the experimental analysis includes a bar chart display of the results of comparative experiments and ablation experiments conducted on the algorithm on a public data set, and a visual display of users' attention to news words; the recommendation analysis includes a display of the online news browsing interface, a display of a list of recommended news after the user clicks to browse the news, a display of similar users and their browsing sequences, and a display of news migration relationships.

8. A personalized news recommendation system based on user global interest migration perception, which is operated by the personalized news recommendation method based on user global interest migration perception according to any one of claims 1 to 7, characterized in that: The system includes a data processing module, a business processing module and a visual analysis module; The data processing module preprocesses the dataset offline, which is divided into news title preprocessing, user browsing news preprocessing, user-news graph construction, and news relationship graph construction. The preprocessed data is then classified and stored. The business processing module connects to system requirements and calls pre-trained models to generate data. This module contains three sub-modules: user content interest mining module, user migration interest mining module, and joint recommendation module. The user content interest mining module models user content interests based on historical news data and news headline data browsed by users; the user migration interest mining module models user migration interests based on user-news graphs and news relationship graphs; and the joint recommendation module combines users' content interests and migration interests to provide recommendation services for the visualization layer. The visualization analysis module is an interactive interface module that provides services to users. It provides users with visualization results of data analysis, experimental analysis and recommendation analysis. In addition, it provides users with interactive functions, allowing users to select news for browsing, and makes recommendations and recommendation analysis based on user browsing.

Images