Micro-blog retweet prediction model based on propagation tree mapping and interest denoising

Abstract

The application relates to a microblog forwarding prediction model based on propagation tree mapping and interest denoising, which comprises a propagation tree learning layer, an interest denoising learning layer and a prediction and training layer connected in sequence. The propagation tree learning layer determines a source microblog according to historical forwarding microblogs of a user; propagation content and propagation relationship derived from the source microblog are mapped into a propagation tree and initialized, then double graph pooling is carried out to generate a propagation tree representation; the interest denoising layer generates a historical forwarding microblog set representation according to the propagation tree representation and the historical forwarding microblogs of the user; interest denoising learning is carried out on the historical forwarding microblogs; and the prediction and training layer generates a forwarding prediction result. According to the double graph pooling mechanism, the propagation tree learning is carried out, the sufficiency and accuracy of the microblog representation are enhanced, the accuracy of the user interest representation is enhanced according to the interest denoising, and the accuracy of the microblog forwarding prediction is improved.

Description

Technical Field

[0001] This invention relates to a data analysis and prediction model, specifically a Weibo forwarding prediction model based on propagation tree mapping and interest denoising. Background Technology

[0002] In the digital age, social media platforms like Weibo have become core hubs for people to obtain and share information. Taking Sina Weibo as an example, as of the end of March 2024, its monthly active users reached 588 million, and this massive user base has accumulated a vast amount of data for the platform. How to conduct online public opinion analysis from this massive data has become a hot topic of common concern in multiple fields such as information resource management and computer science. Weibo forwarding prediction, as a key link in online public opinion analysis, has significant reference value for improving information dissemination efficiency, identifying trending topics, and strengthening public opinion monitoring.

[0003] Regarding Weibo retweet prediction, existing research primarily achieves the prediction task by calculating the similarity between the Weibo post to be retweeted and the user's interests. Since similarity calculation relies on accurate Weibo representations and user interest representations, learning these representations is the core of Weibo retweet prediction. For Weibo representation learning, existing research mainly learns Weibo content features based on information such as topic tags, text content, and emoticons to comprehensively represent Weibo using multi-source content features. For user interest representation learning, existing research mainly mines various interest representations, such as user interest categories, evolving interests, and diverse interests, from users' historical retweet sets to accurately characterize users' interest preferences.

[0004] For microblog representation learning, existing research mainly focuses on modeling microblog representations by mining content features to fully capture the semantics of microblogs and achieve microblog forwarding prediction. For example, Yin et al. proposed a multimodal perceptive microblog representation learning method that integrates text content and time information to predict users' microblog forwarding behavior by modeling the implicit relationships between different features. Wang et al. combined text content and numerical tags to learn microblog representations and used multi-task deep learning to model the potential associations between features to further improve microblog forwarding prediction performance. Firdaus et al. used topics and sentiment to learn microblog representations and then mined users' sentiment features for different topics based on their historical microblogs to achieve forwarding prediction. He Ruifang et al., based on modeling microblogs with text content and social context, used heterogeneous social relationships to enrich microblog representations, compensating for the shortcomings of microblogs being short and informal in expression. Wang et al. found that short text microblogs usually have obvious contextual semantic relationships and proposed a microblog embedding learning method based on an attention mechanism to enhance the word order learning ability of microblog representations and improve the accuracy of microblog forwarding prediction.

[0005] For learning user interest representations, existing research mainly mines user interests from historical Weibo reposts to accurately infer user preferences by understanding the content of those reposts. In the context of Weibo repost prediction, traditional research primarily quantifies user interests through manually defined statistical rules. For example, Xu Jianmin et al. quantified user interests by defining user reposting habits, reposting target habits, and reposting content habits, thereby predicting user reposting behavior. With the rapid development of deep learning technology, deep learning-based user interest learning methods such as recurrent neural networks and attention mechanisms have received widespread attention. For example, Gan et al. proposed a two-layer recurrent neural network that integrates attention mechanisms and provided a temporal modeling perspective for user interest representation by mining the evolution of user interests and movie appeal. Sun et al. found that users maintain friendships by reposting Weibo posts they are not interested in and proposed a causal attention mechanism to learn the causal influence of different interest factors on user reposting behavior, thereby effectively improving the performance of Weibo repost prediction. In the context of Weibo repost prediction, Zheng Cong et al. used a self-attention mechanism to extract the degree of user preference for reposted content and integrated multiple user preferences through the attention mechanism to comprehensively model user interest representations.

[0006] As a typical short text document, microblogs have limited content features. Relying solely on these features is insufficient to guarantee the sufficiency of the microblog's representation, thus affecting the accuracy of predicting microblog reposts. Summary of the Invention

[0007] The purpose of this invention is to provide a Weibo forwarding prediction model based on propagation tree mapping and interest denoising, so as to solve the problem of low accuracy of existing prediction models in predicting Weibo forwarding.

[0008] The objective of this invention is achieved as follows:

[0009] A Weibo forwarding prediction model based on propagation tree mapping and interest-based denoising includes:

[0010] The propagation tree learning layer, connected to the interest denoising layer, is used to map Weibo posts to propagation trees and generate Weibo representations that combine content features and propagation features through propagation tree learning.

[0011] An interest-based noise reduction layer, connected to both the propagation tree learning layer and the prediction and training layer, is used to learn noise reduction techniques from a set of users' historical Weibo reposts; and

[0012] The prediction and training layer, connected to the interest-based noise reduction layer, is used to predict the probability of a user forwarding a Weibo post.

[0013] Furthermore, the propagation tree learning layer determines the source Weibo based on the user's historical forwarded Weibo posts; the propagation content derived from the source Weibo is mapped into nodes, and the propagation relationships of the source Weibo are mapped into edges to generate a propagation tree; the nodes in the propagation tree are initialized; the initialized propagation tree is then subjected to double graph pooling to generate a propagation tree representation; the propagation content includes comment content and forwarded content, and the propagation relationships include forwarding relationships and comment relationships.

[0014] Furthermore, the propagation tree learning layer uses the skip-gram algorithm to first divide the text content into keywords, encode the keywords into word vectors, and then average-pool the word vectors to obtain the initial representation of the nodes.

[0015] Furthermore, the propagation tree learning layer performs propagation path pooling on the initialized propagation tree to generate a propagation path representation, and performs propagation tree pooling on the propagation tree after propagation path pooling to generate a propagation tree representation.

[0016] Furthermore, the propagation tree learning layer uses the number of keywords in a node as the node weight and generates a propagation path representation through weighted fusion.

[0017] Furthermore, the propagation tree learning layer performs weighted fusion of the root node representation and the propagation path representation using the hyperparameter θ to generate the propagation tree representation.

[0018] Furthermore, the interest denoising layer generates a historical Weibo post set representation based on the propagation tree representation and the user's historical Weibo posts; it then performs interest denoising learning on the historical Weibo post representation to generate a user interest representation.

[0019] Furthermore, the interest-based noise reduction layer encodes the ID of each Weibo post forwarded by the user in the past to generate an ID representation; it concatenates the ID representation and the propagation tree representation to generate a historical forwarded Weibo post representation; and it stacks the Weibo post representations according to the user's historical forwarding order to obtain a historical forwarded Weibo post set representation.

[0020] Furthermore, the interest denoising layer generates a Weibo time representation based on the Weibo posting time representation and the reposting time representation, concatenates the Weibo time representation and the historical reposting Weibo representation to generate a time-aware historical reposting Weibo set representation; maps the time-aware historical reposting Weibo set representation to the frequency domain using a fast Fourier transform; performs data denoising on the mapped frequency domain using a filter; and inversely maps the denoised data using an inverse fast Fourier transform to generate a user interest representation; the posting time representation and the reposting time representation are time representations that have been converted into time vectors and normalized.

[0021] Furthermore, after data denoising using filters, the denoised data is randomly deactivated and layer normalized. The calculation formula is as follows:

[0022]

[0023] in, The output of the residual connection, layer normalization, and random deactivation after the input data passes through the l-th filter is given. This is the output of the (l-1)th filter.

[0024] The calculation is performed based on the normalized data from the feedforward neural network, using the following formula:

[0025]

[0026] in, The output of the feedforward neural network is the data that has passed through the l-th filter. W1 and W2 are weight matrices, and b1 and b2 are bias vectors.

[0027] Furthermore, the prediction and training layer obtains the user's interest score based on the microblog representation to be predicted and the generated user interest representation, and selects the Top-K microblogs as the forwarding prediction results according to the score size; the microblog representation to be predicted is obtained by processing the microblog to be predicted through the propagation tree learning layer and the interest denoising layer.

[0028] This invention proposes a Weibo forwarding prediction model based on propagation tree mapping and interest denoising. By optimizing Weibo representation and user interest representation, the performance of Weibo forwarding prediction is further improved. This invention maps each Weibo post to a propagation tree, comprehensively learning the Weibo representation by integrating content features and propagation features. A dual graph pooling mechanism is designed for propagation tree learning to obtain a Weibo representation that integrates content features and propagation features, thereby enhancing the sufficiency and accuracy of the Weibo representation. Based on the user's historical forwarded Weibo set, a time-aware filter is designed to denoise the user interest representation, avoiding interference from noise information and enhancing the accuracy of the user interest representation. By combining propagation tree mapping and interest denoising techniques, this invention provides a new research perspective for multiple fields such as Weibo forwarding prediction and user behavior analysis. In practice, this research, by developing a Weibo forwarding prediction method, provides effective practical evidence for online public opinion analysis and governance, and thus provides effective technical support for creating a healthy online public opinion ecosystem. Attached Figure Description

[0029] Figure 1 This is a data processing flowchart of the present invention.

[0030] Figure 2 This is a schematic diagram of the propagation tree mapping.

[0031] Figure 3 This is a schematic diagram of dual graph pooling.

[0032] Figure 4(a) shows the comparison of the prediction performance of the present invention's prediction model with other propagation tree ablation models when the value of K in HR@K is 1; (b) shows the comparison of the prediction performance of the present invention's prediction model with other propagation tree ablation models when the value of K in HR@K is 5; (c) shows the comparison of the prediction performance of the present invention's prediction model with other propagation tree ablation models when the value of K in NDCG@K is 5; and (d) shows the comparison of the prediction performance of the present invention's prediction model with other propagation tree ablation models in the case of MRR.

[0033] Figure 5 (a) shows the comparison of the prediction performance of the present invention's prediction model with other time-aware filter ablation models when the value of K in HR@K is 1; (b) shows the comparison of the prediction performance of the present invention's prediction model with other time-aware filter ablation models when the value of K in HR@K is 5; (c) shows the comparison of the prediction performance of the present invention's prediction model with other time-aware filter ablation models when the value of K in NDCG@K is 5; and (d) shows the comparison of the prediction performance of the present invention's prediction model with other time-aware filter ablation models in the case of MRR.

[0034] Figure 6 (a) shows the prediction performance of the present invention's prediction model with different numbers of filters when the value of K in HR@K is 1; (b) shows the prediction performance of the present invention's prediction model with different numbers of filters when the value of K in HR@K is 5; (c) shows the prediction performance of the present invention's prediction model with different numbers of filters when the value of K in NDCG@K is 5; and (d) shows the prediction performance of the present invention's prediction model with different numbers of filters in the MRR case.

[0035] Figure 7 (a) shows the prediction performance of the present invention's prediction model under different fusion weights when the value of K in HR@K is 1; (b) shows the prediction performance of the present invention's prediction model under different fusion weights when the value of K in HR@K is 5; (c) shows the prediction performance of the present invention's prediction model under different fusion weights when the value of K in NDCG@K is 5; and (d) shows the prediction performance of the present invention's prediction model under different fusion weights in the MRR case.

[0036] Figure 8 In the figure, (a) shows the model performance of the prediction model of the present invention under different sparsity, and (b) shows the model performance of the FMLP-Rec model under different sparsity.

[0037] Figure 9In the diagram, (a) shows the distribution of keywords in the vector space of user 1's historical forwarded Weibo posts, (b) shows the distribution of keywords in the vector space of user 2's historical forwarded Weibo posts, (c) shows the distribution of keywords in the vector space of user 3's historical forwarded Weibo posts, and (d) shows the distribution of keywords in the vector space of user 4's historical forwarded Weibo posts.

[0038] Figure 10 (a) shows the score ranking obtained by the prediction model of this invention and the visualization results of users' actual forwarded Weibo posts; (b) shows the score ranking obtained by the FMLP-Rec model and the visualization results of users' actual forwarded Weibo posts; (c) shows the score ranking obtained by the Filter_nf model and the visualization results of users' actual forwarded Weibo posts; and (d) shows the score ranking obtained by the Filter_nt model and the visualization results of users' actual forwarded Weibo posts. Detailed Implementation

[0039] The present invention will now be described in further detail.

[0040] like Figure 1 As shown, this invention provides a Weibo forwarding prediction model based on propagation tree mapping and interest denoising. The model includes a propagation tree learning layer, an interest denoising layer, and a prediction and training layer. The propagation tree learning layer is connected to the interest denoising learning layer, the interest denoising learning layer is connected to both the propagation tree learning layer and the prediction and training layer, and the prediction and training layer is connected to the interest denoising layer.

[0041] The propagation tree learning layer determines the source Weibo post based on the user's historical reposts; the propagation content derived from the source Weibo post is mapped to nodes, and the propagation relationships of the source Weibo post are mapped to edges to generate a propagation tree; the nodes in the propagation tree are initialized; the initialized propagation tree is then subjected to double graph pooling to generate a propagation tree representation. The propagation content includes comment content and repost content, and the propagation relationships include repost relationships and comment relationships.

[0042] Weibo forwarding prediction aims to predict a user's next likely forward based on their historical forwarded Weibo posts. Assume the entire set of Weibo users is U and the entire set of Weibo posts is R, where any Weibo post r∈R and the historical forwarded Weibo post set of any user u∈U is R. u ={r u(1) ,r u(2) ,…,r u(t) …,r u(n)},and Where, r u(t) Let represent the t-th Weibo post forwarded by user u, and n represent the total number of Weibo posts forwarded by user u throughout history. Then, the Weibo forwarding prediction task can be represented as:

[0043] P u(n+1) =g(R)u ,r) (1)

[0044] Among them, P u(n+1) Let g(·) be the predicted probability of user u forwarding Weibo post r, and g(·) be the prediction method. In this invention, g(·) specifically refers to the method of prediction using a Weibo forwarding prediction model based on propagation tree mapping and interest denoising. Based on the predicted probability P... u(n+1) It can effectively achieve the task of predicting Weibo reposts.

[0045] like Figure 2 As shown, this invention integrates content information and dissemination information to transform any Weibo post into a single post. i Mapped as a propagation tree Among them, V i For a set of nodes, For edge set, P i This is the set of propagation paths. The nodes in the propagation tree include the root node v. i0 and propagation nodes The root node represents the source Weibo post, and the propagation nodes represent the propagated content (i.e., comments and reposts) derived from the source Weibo post. Therefore, the set of nodes in the propagation tree is denoted as .

[0046] This invention defines forwarding and commenting relationships as edges in the propagation tree. For example, <v ia ,v ib > indicates node v ia With node r ib The propagation relationship between them. Therefore, the edge set in the propagation tree is denoted as:

[0047] The path in a propagation tree refers to all the nodes traversed from the root node to any leaf node. For example, Figure 2 A propagation path p = {v i0 ,v i1 ,v i4} refers to the root node v i0 to leaf node v i4 All nodes traversed. Therefore, the propagation path set is denoted as: in, To propagate the tree i The Pth i One propagation path, P i To propagate the tree i The number of propagation paths, p ij Represents a propagation tree i The j-th propagation path, v ijk Represents a propagation tree i The k-th propagation node of the j-th propagation path, in Figure 2In the middle, v ij1 =v i0 p ij To propagate the tree i The number of nodes in the j-th propagation path.

[0048] Node initialization in the propagation tree refers to generating the vectorized representation of nodes in the propagation tree, that is, generating the initial representation of nodes based on the text content. The propagation tree learning layer uses the skip-gram algorithm to first divide the text content into keywords, encode the keywords into word vectors, and then average pool the word vectors to obtain the initial representation of the nodes.

[0049] like Figure 3 As shown, to generate a microblog representation that integrates content features and propagation features, this invention proposes a dual graph pooling mechanism for learning the propagation tree. Specifically, the dual graph pooling mechanism includes propagation path pooling and propagation tree pooling.

[0050] The propagation tree learning layer performs propagation path pooling on the initialized propagation tree, and then performs propagation tree pooling on the path-pooled propagation tree to generate a propagation tree representation. Specifically, propagation path pooling represents a path in the propagation tree as a vector. More specifically, propagation path pooling uses the number of keywords in a node as the node weight, and generates the propagation path representation through weighted fusion.

[0051] The formula for calculating propagation path pooling is:

[0052]

[0053] in, To represent the propagation path, For the propagation path p ij The number of keywords across all nodes. For node v on the propagation path ijk The number of keywords included. For node v ijk The initialization representation is given. The keywords are those obtained using the skip-gram algorithm during node initialization. Since the root node representation differs from the propagation node representation, the root node representation is not considered in the propagation path pooling.

[0054] Propagation tree pooling specifically refers to representing the propagation tree as a vector, and treating it as a propagation tree representation of Weibo. Specifically, propagation tree pooling uses a hyperparameter θ to weight and fuse the root node representation and the propagation path representation to generate the propagation tree representation.

[0055] The formula for calculating the propagation tree pooling process is:

[0056]

[0057] in, For Weibo r i The propagation tree representation, where θ is the fusion weight, This represents the root node.

[0058] Because the propagation tree learning process effectively considers the two propagation relationships of comments and reposts based on the text content, the Weibo propagation tree representation effectively integrates propagation features and content features.

[0059] The interest denoising layer in the prediction model of this invention is mainly used to eliminate the interference of noise information in the historical forwarded Weibo set on user interest learning and generate a more accurate representation of user interest. Specifically, it includes: generating a time-aware representation of the historical forwarded Weibo set and interest denoising.

[0060] The historical reposted Weibo post representation is generated by: encoding the ID of each Weibo post reposted by the user in the past into an ID representation; and concatenating the ID representation and the propagation tree representation to generate the historical reposted Weibo post representation.

[0061] Besides the propagation tree, embedding Weibo IDs also helps in mining user interests. First, the Weibo representation is obtained by combining the propagation tree and the ID. One-hot encoding is used to encode the ID of each Weibo post, and an embedding function maps the high-dimensional encoding matrix to a low-dimensional vector space to obtain the Weibo ID representation. The calculation formula is as follows:

[0062]

[0063] in, For Weibo r i The ID indicates that For Weibo r i One-hot encoding, W * and W * Let b be the weight matrix. * and b * This is the bias vector.

[0064] Secondly, the Weibo representation is obtained by concatenating the ID representation and the propagation tree representation. The calculation formula is as follows:

[0065]

[0066] Among them, e u(t) Let t be the Weibo post forwarded by user u. `concat(·)` is the operation to concatenate vectors. Let represent the ID of the t-th reposted Weibo post by user u. This ID is obtained by querying the ID vector of Weibo posts based on the user's historical reposted Weibo posts. Let t represent the propagation tree representation of the t-th retweet by user u. This is obtained by querying the propagation tree representation of a retweet post based on the user's historical retweet set.

[0067] Furthermore, by stacking the Weibo posts according to the user's historical forwarding order, we obtain the representation of the historical forwarded Weibo post set:

[0068] E u ={e u(1) ,e u(2) ,...,e u(n)} (6)

[0069] Among them, E u This represents the collection of Weibo posts forwarded by user u throughout their history.

[0070] Because user historical behavior data contains noisy information, directly mining user preferences from historical Weibo reposts can lead to inaccurate representation of user interests. This invention uses a time-aware filter to learn and reduce noise in user interests. The time-aware filter mainly includes time bias and filter noise reduction.

[0071] Time bias: Generate a time representation of a Weibo post based on its posting and reposting times, where the posting and reposting times are converted into time vectors and normalized.

[0072] Because noise in historical user data is often sudden, it can be effectively filtered out by analyzing the temporal sequence of user behavior. To improve the noise reduction capability for user interests, a time bias is introduced into the representation of a user's historical reposted Weibo posts. Specifically, a Weibo time representation is generated based on the posting and reposting times, and this is used as the time bias for the historical reposted Weibo post set, thereby better learning user interests. For example, the time "January 2, 2020, 3:04 AM" is converted into a time vector (2020, 1, 2, 3, 4); the time vector is then linearly normalized so that all dimensions of the vector take values ​​within the range [0, 1], thus eliminating the order-of-magnitude differences between the data in each dimension and improving the training effect of the model. By performing the above processing on the posting and reposting times of Weibo posts, the posting time representation and the reposting time representation are obtained. The quantization process of the Weibo time representation is shown below:

[0073]

[0074] in, Let the time of the user's t-th Weibo post be represented. This indicates the posting time on Weibo. This indicates the time it takes for a Weibo post to be forwarded.

[0075] By concatenating the time representation of Weibo posts and the historical reposted Weibo posts, a time-aware historical reposted Weibo post set representation is generated.

[0076] By incorporating a time bias (i.e., Weibo time representation) into the historical reposted Weibo representation generated in the first step, a time-aware historical reposted Weibo set representation is obtained. The calculation formula is as follows:

[0077]

[0078]

[0079] in, To incorporate the time-biased nature of Weibo, This is a collection of Weibo posts representing a historical perspective based on time.

[0080] After this, layer normalization and dropout are used to improve the stability of model training. The calculation process is as follows:

[0081]

[0082] in, This represents a time-aware set of forwarded Weibo posts that have undergone layer normalization and dropout.

[0083] User interest denoising: The historical forwarded Weibo set, after layer normalization and dropout, is mapped to the frequency domain using Fast Fourier Transform (FFT); data denoising is performed using filters; and the denoised data is inversely mapped using Inverse Fast Fourier Transform (IFFT) to generate user interest representations.

[0084] This invention employs L superimposed filters to learn and denoise time-aware user interests. An FFT is performed before each filter to provide a denoising environment, and an IFFT is performed after each filter to inversely map the denoised data back to its original format, thus achieving one-time user interest denoising. Therefore, the calculation formula mapped to the frequency domain is:

[0085]

[0086] in, This is the output of the l-th filter after FFT processing. The output of the (l-1)th filter after IFFT processing. The FFT operation is represented by the input data before the first filter, which is frequency-domain mapped and is represented by the time-aware historical Weibo repost set generated by formula (10).

[0087]

[0088] in, Let W be the noise reduction output of the l-th filter in the frequency domain, where W is the filter and ⊙ is the element-wise multiplication operation.

[0089]

[0090] in, The output of the IFFT processing of the l-th filter, This is an IFFT operation.

[0091] Furthermore, to alleviate the problems of gradient vanishing and training instability, and to further capture the nonlinear relationships of features, residual connections, layer normalization, and random deactivation are added after each IFFT operation, combined with a feedforward neural network to process the data. The calculation formula is as follows:

[0092]

[0093]

[0094] in, The output of the residual connection, layer normalization, and random deactivation after the input data passes through the l-th filter is given. This is the output of the (l-1)th filter. The data that has passed through the l-th filter is the output of the feedforward neural network.

[0095] After adding residual connections, layer normalization, random deactivation, and feedforward neural networks after each IFFT process The output of the data from the (l-1)th filter is the output after passing through the feedforward neural network.

[0096] After noise reduction training through L filters, the noise-reduced output can be obtained. in, This represents the user interest representation after noise reduction, which incorporates the features of the top n users' reposted Weibo posts.

[0097] The prediction and training layers obtain the user's interest score based on the microblog representation to be predicted and the user's interest representation, and select the Top-K microblogs as the forwarding prediction results according to the score.

[0098] The prediction and training layers perform a dot product operation between the microblog representation to be predicted and the user's interest representation to obtain the user's interest score. Based on the score, the Top-K microblogs are selected as the prediction results for forwarding. The microblog to be predicted is r. c The Weibo post that is yet to be predicted indicates e c The formula is:

[0099]

[0100] in, The Weibo post to be predicted is r c The ID indicates that The Weibo post to be predicted is r c The propagation tree indicates that, The Weibo post to be predicted is r ξ The time indicated on Weibo.

[0101] The interest score calculation process is as follows:

[0102]

[0103] Where, r u(1:s) ={r u(1) ,r u(2) ,…,r u(s)}, r u(1:s) r is a subset of the first s retweet posts by user u. c For the Weibo post to be predicted, P(r) u(s+1) =r c |r u(1:s) Let be the interest score of user u in the context of the first t (1≤s≤n) retweets. e represents the user's interest after noise reduction. c The Weibo post indicated that the prediction was pending. Vector transpose operation.

[0104] This invention uses a pairwise ranking loss function to calculate the difference between predicted and true values, and trains the model parameters using gradient descent. The loss function is shown below:

[0105]

[0106] Where Loss is the loss value, and σ(·) represents the activation function. If the value is true, it represents a Weibo post that the user actually forwarded. For negative values, the data is randomly sampled from Weibo posts that the user did not retweet.

[0107] The prediction model of this invention is evaluated.

[0108] Model Comparison: The prediction model of this invention is compared with other prediction models.

[0109] Currently, there is no publicly available benchmark dataset for predicting Weibo reposts. Therefore, this invention constructs a dataset by crawling real data from Sina Weibo using a Python web crawler. The crawled data mainly contains eight fields: user ID, Weibo ID, Weibo repost relationship, Weibo comment relationship, Weibo text content, Weibo comment content, Weibo posting time, and user reposting time. Based on the original data, data usability is enhanced through data preprocessing. The specific preprocessing operations include: (1) removing users with fewer than 3 repost records, using the last 2 Weibo posts reposted by the user for verification and testing, and using the remaining reposted Weibo posts for model training; (2) since user interests change dynamically over time, earlier Weibo posts are difficult to represent the user's recent interests. Therefore, only the 50 Weibo posts reposted by the user recently are retained; (3) according to the three-degree influence principle, information has an influence decay phenomenon during the propagation process. When the distance exceeds 3 steps, the influence almost disappears. Therefore, the propagation tree learning only considers nodes within 3 steps of the root node.

[0110] Table 1 shows the statistical information of the preprocessed dataset. Among them, the number of users represents the total number of users in the dataset, the number of microblogs represents the total number of microblogs in the dataset, the number of reposts represents the total number of times all microblogs were reposted, the time span represents the time span of all microblog reposts, the average number of reposts represents the average number of microblogs reposted per user, and the average microblog length represents the average number of keywords in the microblog.

[0111] Table 1: Dataset Statistics

[0112] numerical values 7513 22883 70631 9.40 19.98 2021.03～2021.09

[0113] Since Weibo repost prediction is essentially a sequence learning problem, this invention primarily uses mainstream sequence learning models as baseline models to verify the scientific validity and effectiveness of the prediction model. The specific details of the baseline model are as follows:

[0114] (1)SASRec model: A model based on attention mechanism, which mainly uses multi-head self-attention mechanism to capture users’ temporal interests.

[0115] (2) BERT4Rec model: an attention-based model that models user behavior sequences through a bidirectional self-attention mechanism to achieve time-series recommendation based on user dynamic interests.

[0116] (3) STOSA model: A model based on self-attention mechanism. This model uses Gaussian distribution to embed the user's interactive items and uses Wasserstein self-attention module to learn the positional relationship between interactive items.

[0117] (4) FMLP-Rec model: A filter-based model that uses filters to filter out noise factors in user interests.

[0118] (5) FMLP-Rec(con) model: When representing microblogs in the FMLP-Rec model, microblog content features are added to reflect that the propagation tree has certain advantages in prediction compared with content features.

[0119] To ensure experimental fairness and comparability, the main hyperparameters of the SASRec, BERT4Rec, STOSA, and FMLP-Rec models were determined in accordance with the original literature.

[0120] To quantitatively evaluate the performance of forwarding prediction, this invention employs evaluation metrics widely used in recommender systems and forwarding prediction to conduct experimental analysis. These metrics include: Top-K Hit Ratio (HR@K), Top-K Normalized Discounted Cumulative Gain (NDCG@K), and Mean Reciprocal Rank (MRR).

[0121] (1) HR@K: Whether the first K prediction results match the actual Weibo posts forwarded by the user. The higher the HR@K value, the better the prediction effect. The calculation formula is as follows:

[0122]

[0123] Among them, hit i Does the Top-K of the predicted forwarding results for the i-th user include the Weibo posts they actually forwarded? i ∈{0,1}.

[0124] (2) NDCG@K: The accuracy of the prediction results is evaluated based on the user interest score ranking. The higher the NDCG@K value, the better the prediction effect. The calculation formula is as follows:

[0125]

[0126] Among them, rank i Let rank be the ranking of the interest score of the i-th user's actual retweet of a Weibo post in the prediction results. i ∈[1,100].

[0127] (3) MRR: The ranking of the Weibo posts actually forwarded by users in the prediction results. The larger the MRR, the better the prediction effect. The calculation formula is as follows:

[0128]

[0129] Based on the formula derivation, HR@1 in this invention is equal to NDCG@1. Therefore, HR@{1,5,10}, NDCG@{5,10}, and MRR are used to evaluate the experimental results.

[0130] This invention was tested on an NVIDIA A100 PCIe GPU using Python 3.9 and PyTorch 2.0, with PyTorch being a mainstream deep learning framework. For all models, the model parameters were initialized using a Gaussian distribution by default, and the Adam optimizer was used for 200 training epochs with a batch size of 256. An early stopping strategy was employed, stopping model optimization when the training epochs exceeded 100 and the MRR (Mean Reduction Rate) stopped increasing after 10 epochs. For other hyperparameters, grid search was used to configure the optimal parameters for the model, where the hidden layer size ∈ {16, 32, 64, 128, 256}, the learning rate ∈ {0.0001, 0.0005, 0.001}, and the dropout rate ∈ {0.3, 0.5, 0.7}.

[0131] This experiment is mainly used to compare the forwarding prediction performance of the prediction model of this invention and the baseline model, so as to prove the effectiveness of the proposed model.

[0132] Table 2: Overall performance comparison results of the prediction model of this invention and the baseline model on the dataset.

[0133] SASRec 0.6754 0.7810 0.7985 0.7360 0.7416 0.7269 BERT4Rec 0.5508 0.7647 0.8104 0.6681 0.6831 0.6469 STOSA 0.7049 0.7999 0.8134 0.7593 0.7637 0.7504 FMLP-Rec 0.6812 0.7864 0.8046 0.7411 0.7470 0.7323 FMLP-Rec(con) 0.6905 0.7974 0.8235 0.7516 0.7600 0.7436 Prediction Model of this Invention 0.8109 0.8718 0.8901 0.8452 0.8512 0.8413 IMP 10.60％ 7.19％ 7.67％ 8.59％ 8.75％ 9.09％

[0134] As shown in Table 2, IMP represents the percentage improvement of the prediction model of this invention compared to the best baseline model. Compared to the better baseline model, the prediction model of this invention shows a 7%-10% improvement in performance across various evaluation metrics, indicating that the prediction model of this invention demonstrates superior Weibo forwarding prediction performance. Specifically, compared to the SASRec model, BERT4Rec model, and STOSA model, the prediction model of this invention utilizes a time-aware filter to learn user interests, which can effectively handle noise information in user interests; compared to the FMLP-Rec model, the prediction model of this invention incorporates a propagation tree when generating Weibo representations, compensating for the sparsity of Weibo content features. In summary, the superior recommendation performance of the prediction model of this invention is mainly attributed to two aspects: ① optimization of Weibo representation using a propagation tree; ② optimization of user interest representation using a time-aware filter.

[0135] Validation: The Weibo forwarding prediction model proposed in this invention includes two key designs: a propagation tree and a time-aware filter. To verify the effectiveness of these key designs, this section uses ablation experiments to validate their effectiveness.

[0136] Verification of the validity of the propagation tree.

[0137] To verify the effectiveness of the propagation tree, this invention derives three ablation models based on the prediction model of this invention, namely: (1) Tree_nt model: This model does not consider the propagation tree and generates the microblog representation based only on ID information and time information; (2) Tree_np model: This model does not consider the propagation features in the microblog propagation tree and generates the microblog representation based on ID information, time information and content features; (3) Tree_nr model: This model does not consider the content features in the microblog propagation tree and generates the microblog representation based on ID information, time information and propagation features.

[0138] like Figure 4 As shown, the prediction model of this invention (i.e., the proposed model) outperforms the Tree_nt, Tree_np, and Tree_nr models, proving that the introduction of the propagation tree can improve the representation capability of Weibo and help improve the performance of forwarding prediction. This indicates that both the content features and propagation features introduced based on the propagation tree are beneficial to improving the representation capability of Weibo. Furthermore, we can observe that the Tree_nr and Tree_np models outperform the Tree_nt model in all metrics, with the Tree_np model slightly outperforming the Tree_nt model. This suggests that considering only one of the content features or propagation features of the propagation tree can improve the performance of forwarding prediction to a certain extent, and the content features are slightly better than the propagation features.

[0139] Validation of the time-aware filter.

[0140] To verify the effectiveness of the time-aware filter, this invention derives three ablation models based on the prediction model of this invention: (1) Filter_nft model: This model does not consider the time-aware filter and uses a self-attention mechanism to learn user interests; (2) Filter_nf model: This model does not consider the filter and uses a self-attention mechanism that incorporates time features to learn user interests; (3) Filter_nt model: This model does not consider the time factor of the time-aware filter and only uses the filter to learn user interests.

[0141] like Figure 5 As shown, the prediction performance of the present invention's prediction model is superior to the three ablation models, proving that the time-aware filter can effectively filter noise in interests, which is beneficial to improving the performance of forwarding prediction. Furthermore, the present invention's prediction model outperforms the Filter_nf model, indicating that the method of noise reduction through filters can better uncover user interests compared to the self-attention mechanism; the present invention's prediction model and the Filter_nf model outperform the Filter_nft and Filter_nt models, indicating that Weibo has a strong timeliness, and time features are extremely important for predicting Weibo users' forwarding behavior.

[0142] Hyperparameter Validation: The Weibo forwarding prediction model proposed in this invention includes two important hyperparameters: fusion weight θ and the number of filters L. To investigate the impact of hyperparameters on the experiment, hyperparameter experiments were conducted on a dataset.

[0143] The impact of fusion weights on forwarding prediction.

[0144] The hyperparameter θ is used to fuse the root node representation and propagation path representation of the propagation tree pooling. It is an important hyperparameter for propagation tree learning and affects the representation of Weibo features. Therefore, it is necessary to verify the impact of this parameter on model performance.

[0145] like Figure 6 As shown, the forwarding prediction effect exhibits a trend of first increasing and then decreasing with the increase of the fusion weight θ. The prediction effect is better when the fusion weight θ is in the range [0.4, 0.6], and worse when the fusion weight θ is close to 0 or 1. This phenomenon indicates that considering only the source Weibo or the spreading Weibo when representing Weibo features will have a negative impact on forwarding prediction; while effectively fusing these two types of Weibo can improve the accuracy of Weibo expression and enhance the forwarding prediction effect.

[0146] Impact of the number of filters on forwarding prediction

[0147] Multiple filters can help the model reduce noise from different angles, enhancing the user's interest-based learning ability. The number of filters, L, determines the efficiency of noise processing and is an important hyperparameter of the interest-based noise reduction layer.

[0148] like Figure 7 As shown, the forwarding prediction performance exhibits a trend of first increasing and then decreasing with the increase of the number of filters L. The prediction performance is very good when the number of fusion filters L is around 8. This phenomenon indicates that learning user interests requires multiple filters for noise reduction. However, when the number of filters is too large, the model may identify useful information as noise during training, leading to a decrease in performance. Therefore, it is necessary to set the number of filters reasonably to obtain the optimal model performance.

[0149] Sparsity verification.

[0150] Data sparsity is a significant challenge in forwarding prediction. The number of Weibo forwards and the number of keywords in the source Weibo post are two important data points exhibiting sparsity in the experiment. This experiment compares the performance of the prediction model of this invention and the FMLP-Rec model under different sparsity conditions to examine their resilience to sparse data.

[0151] like Figure 8 As shown, in Figure 8In (a), the prediction accuracy for Weibo posts with very few reposts is extremely low. As the number of reposts increases, the model performance improves significantly. This indicates that predicting frequently interacting posts is relatively easy in the repost prediction task, while predicting unpopular posts remains challenging. The model performance improves with the increase in the number of keywords in the source Weibo post. The best repost prediction performance is achieved when the number of keywords in the source Weibo post reaches (20, 30). This indicates that Weibo posts with more keywords in the source Weibo post are easier to predict in the repost prediction task, but too many keywords can interfere with the repost prediction. Figure 8 (a) and Figure 8 (b) It can be observed that the prediction model of the present invention shows a more significant improvement in prediction performance when predicting Weibo posts with fewer reposts, and also shows some improvement when predicting Weibo posts with a high number of reposts and keywords from the source Weibo post. This indicates that the prediction model of the present invention has a better adaptability to sparse Weibo data compared to the comparison model.

[0152] To enhance the interpretability of the model, this invention uses a machine learning dimensionality reduction algorithm, T-SNE (t-Distributed Stochastic Neighbor Embedding), to reduce some important high-dimensional vectors in the experiment into two-dimensional vectors and visualize them.

[0153] The propagation tree enhances the representation of microblogs by fusing content features and propagation features, thereby improving the accuracy of representing users' historical retweet sets. To verify this, this invention selects four users with a relatively small number of retweets from the dataset and visualizes the distribution of keywords in their historical retweets in the vector space, using different colors to color the keywords in the source and propagation microblogs.

[0154] like Figure 9 As shown, based on the properties of collaborative filtering (items that the same user has interacted with are similar, and users who have interacted with the same item are similar), the keywords in a user's historical Weibo reposts should exhibit a high degree of aggregation. Figure 9 The relatively low number of reposts by the four users resulted in a scattered distribution of keywords, hindering the discovery of user interests. However, after adding the keywords from the disseminated Weibo posts, it was observed that the keyword distributions of users 2 and 4 exhibited a higher degree of aggregation, demonstrating that the propagation tree can enhance the representation of Weibo posts and improve the ability to discover user interests.

[0155] To verify that the filter can effectively improve the accuracy of users' historical interest representation, this invention visualizes the user interest representation obtained by the prediction model, FMLP-Rec, Filter_nf, and Filter_nt, and colors the user interest representation according to the score ranking of the user's actual forwarded Weibo posts.

[0156] like Figure 10As shown, compared to the other three models, the prediction model of this invention ranks closer to 1 in terms of the score of users actually forwarding Weibo posts, indicating that the time-aware filter has a significant advantage. When the model does not use the time filter, the ability to express user interests decreases sharply, seriously affecting the prediction of user forwarding behavior. In addition, among the four models, the forwarding prediction results are more accurate when the user interest representation is far from the center (0,0), indicating that the user interest has obvious characteristics that are more conducive to forwarding prediction. Moreover, the points in the figure are clustered into several groups of different sizes, and the colors of the groups are relatively uniform, indicating that the four models have similar accuracy in predicting forwarding for users with similar interests.

[0157] Yin et al.: Yin H, Yang S, Song X, et al. Deep fusion of multimodal features for social media retweet time prediction [J]. World Wide Web, 2021, 24(4): 1027-1044.

[0158] Wang et al. (Learning Weibo Retweet Prediction Based on Deep Multitask Learning) (Combining Text Content and Numerical Tags): Wang J, Yang Y. Tweet Retweet Prediction Based on Deep Multitask Learning[J].Neural Process Letters,2022,54:523-536.

[0159] Firdaus et al.: Firdaus SN, Ding C, Sadeghian A. Topic specific emotion detection for retweet prediction[J]. International Journal of Machine Learning and Cybernetics, 2019,10:2071-2083.

[0160] He Ruifang et al.: He Ruifang, Wang Haocheng, Liu Hongyu, et al. Multi-view microblog topic detection based on heterogeneous social context [J]. Journal of Software, 2023, 34(11): 5162-5178.

[0161] Wang et al. (A microblog embedding learning method based on attention mechanism): Wang L, Zhang Y, Yuan J, et al. FEBDNN: fusion embedding-based deep neural network for user retweeting behavior prediction on social networks[J]. Neural Computing and Applications, 2022, 34: 13219-13235.

[0162] Xu Jianmin et al.: Xu Jianmin, Han Kangkang, He Dandan, et al. Weibo forwarding prediction based on multiple forwarding habits [J]. Intelligence Magazine, 2020, 39(3):123-129,155.

[0163] Gan et al.: Gan M, Zhang

[0164] Sun et al.: Sun WJ, Liu XF. Deep attention framework for retweet prediction enriched with causal inferences[J].Applied Intelligence, 2023,53:24293–24313.

[0165] Zheng Cong et al.: Zheng Cong, Jiang Hao, Zhao Xiaoyue. Weibo forwarding prediction based on user preference features [J]. Journal of Wuhan University, 2023, 56(06):749-756.

Claims

1. A microblog forwarding prediction system based on propagation tree mapping and interest-based denoising, characterized in that, include: The propagation tree learning layer, connected to the interest denoising layer, is used to map Weibo posts to propagation trees and generate Weibo representations that combine content features and propagation features through propagation tree learning. The interest-based noise reduction layer is connected to the propagation tree learning layer and the prediction and training layer, respectively, and is used to learn noise reduction from the user's historical Weibo reposts. as well as The prediction and training layer, connected to the interest denoising layer, is used to predict the probability of a user forwarding a Weibo post. The propagation tree learning layer determines the source Weibo based on the user's historical forwarded Weibo posts; the propagation content derived from the source Weibo is mapped into nodes, and the propagation relationship of the source Weibo is mapped into edges to generate a propagation tree; The nodes in the propagation tree are initialized; the initialized propagation tree is then subjected to double graph pooling to generate a propagation tree representation; the propagation content includes comment content and forwarding content, and the propagation relationship includes forwarding relationship and comment relationship; The interest denoising layer generates a historical retweet set representation based on the propagation tree representation and the user's historical retweet Weibo posts; it then performs interest denoising learning on the historical retweet Weibo post representation to generate a user interest representation. The interest-based noise reduction layer encodes the ID of each Weibo post forwarded by the user in the past to generate an ID representation; it concatenates the ID representation and the propagation tree representation to generate a historical forwarded Weibo post representation; and it stacks the Weibo post representations according to the user's historical forwarding order to obtain a historical forwarded Weibo post set representation. The interest-based denoising layer generates a Weibo time representation based on the posting and forwarding times of the Weibo posts. It then concatenates the Weibo time representation with historical forwarded Weibo posts to generate a time-aware historical forwarded Weibo post set representation. The time-aware historical forwarded Weibo post set representation is mapped to the frequency domain using a Fast Fourier Transform (FFT). The mapped data is then denoised using a filter. Finally, the denoised data is inversely mapped using an Inverse Fast Fourier Transform (IFFT) to generate a user interest representation. The posting and forwarding times are converted into time vectors and normalized. After denoising the data using filters, the denoised data is randomly deactivated and layer normalized. The calculation formula is as follows: ， in, The output of the residual connection, layer normalization, and random deactivation after the input data passes through the l-th filter is given. This is the output of the (l-1)th filter; The calculation is performed based on the normalized data from the feedforward neural network, using the following formula: ， in, The data that has passed through the l-th filter is the output of the feedforward neural network. and This is the weight matrix. and This is the bias vector.

2. The microblog forwarding prediction system based on propagation tree mapping and interest denoising according to claim 1, characterized in that, The propagation tree learning layer uses the skip-gram algorithm to first divide the text content into keywords, encode the keywords into word vectors, and then average pool the word vectors to obtain the initial representation of the nodes.

3. The microblog forwarding prediction system based on propagation tree mapping and interest denoising according to claim 2, characterized in that, The propagation tree learning layer performs propagation path pooling on the initialized propagation tree to generate a propagation path representation, and then performs propagation tree pooling on the propagation tree after propagation path pooling to generate a propagation tree representation.

4. The microblog forwarding prediction system based on propagation tree mapping and interest denoising according to claim 3, characterized in that, The propagation tree learning layer uses the number of keywords in a node as the node weight, and generates a propagation path representation through weighted fusion; the propagation tree learning layer uses hyperparameters... The root node representation and the propagation path representation are weighted and fused to generate the propagation tree representation.

5. The microblog forwarding prediction system based on propagation tree mapping and interest denoising according to claim 1, characterized in that, The prediction and training layer obtains the user's interest score based on the microblog representation to be predicted and the generated user interest representation, and selects the Top-K microblogs as the forwarding prediction results according to the score size; the microblog representation to be predicted is obtained by processing the microblog to be predicted through the propagation tree learning layer and the interest denoising layer.

Images