Microblog forwarding prediction model based on propagation tree mapping and interest noise reduction

Abstract

The invention relates to a microblog forwarding prediction model based on propagation tree mapping and interest noise reduction. The microblog forwarding prediction model comprises a propagation tree learning layer, an interest noise reduction learning layer and a prediction and training layer which are connected in sequence. The propagation tree learning layer determines source microblogs according to the microblogs historically forwarded by the user; the method comprises the following steps: mapping propagation contents derived from a source microblog and a propagation relation into a propagation tree, initializing the propagation tree, and then carrying out double graph pooling to generate propagation tree representation; the interest noise reduction layer generates historical forwarding microblog set representation according to the propagation tree representation and the microblogs historically forwarded by the user; interest noise reduction learning is carried out on the historical forwarding microblogs; and the prediction and training layer generates a forwarding prediction result. According to the method, propagation tree learning is carried out according to a double graph pooling mechanism, the sufficiency and accuracy of microblog representation are enhanced, the accuracy of user interest representation is enhanced according to interest noise reduction, and the accuracy of microblog forwarding prediction is improved.

Description

Technical Field

[0001] The invention relates to a data analysis prediction model, in particular to a microblog forwarding prediction model based on propagation tree mapping and interest noise reduction. Background Art

[0002] In the digital age, social media platforms represented by Weibo have become the core hub for people to obtain and share information. Taking Sina Weibo as an example, as of the end of March 2024, its monthly active users have reached 588 million, and the huge user base has accumulated massive data for the Weibo platform. How to analyze online public opinion from massive data has become a hot topic of common concern in many fields such as information resource management and computer science. As a key link in online public opinion analysis, the accuracy of Weibo forwarding prediction is of great reference significance for improving information dissemination efficiency, mining hot topics, and strengthening public opinion monitoring.

[0003] Regarding microblog forwarding prediction, existing research mainly achieves the prediction task by calculating the similarity between the microblog to be forwarded and the user's interests. Since similarity calculation depends on accurate microblog representation and user interest representation, learning microblog representation and user interest representation is the core of microblog forwarding prediction. For microblog representation learning, existing research mainly learns microblog content features based on information such as topic tags, text content, and emoticons to represent microblogs with comprehensive multi-source content features. For user interest representation learning, existing research mainly mines multiple interest representations such as user interest categories, evolving interests, and diversified interests from the user's historical forwarded microblogs to accurately characterize the user's interest preferences.

[0004] For microblog representation learning, existing research mainly models microblog representation by mining the content features of microblogs, so as to achieve the microblog forwarding prediction task by fully capturing the semantics of microblogs. For example, Yin et al. proposed a multimodal-aware microblog representation learning method based on the text content and time information of microblogs, so as to predict the user's microblog forwarding behavior by modeling the implicit relationship between different features. Wang et al. combined the text content and digital labels of microblogs to learn microblog representations, and modeled the potential associations between features through multi-task deep learning to further improve the performance of microblog forwarding prediction. Firdaus et al. used topics and emotions to learn microblog representations, and then mined the emotional characteristics of users on microblogs with different topics based on the user's historical microblogs to achieve forwarding prediction. He Ruifang et al. enriched the representation of microblogs based on the text content and social context modeling of microblogs, making up for the shortcomings of microblogs being short in form and informal in expression. Wang et al. found that short text microblogs usually have obvious contextual semantic relationships, and proposed a microblog embedding learning method based on attention mechanism to enhance the word order learning ability of microblog representations and improve the accuracy of microblog forwarding prediction.

[0005] For learning user interest representation, existing research mainly mines user interests from the user's historical forwarded microblogs to accurately infer user preferences by understanding the user's historical forwarded content. In the context of microblog forwarding prediction, traditional research mainly quantifies user interests by manually defining statistical rules. For example, Xu Jianmin et al. quantified user interests by defining the user's forwarding behavior habits, forwarding object habits, and forwarding content habits, thereby predicting the user's forwarding behavior. With the rapid development of deep learning technology, deep learning-based user interest learning methods such as recurrent neural networks and attention mechanisms have received widespread attention. For example, Gan et al. proposed a two-layer recurrent neural network that integrates attention mechanisms, and provided a temporal modeling perspective for user interest representation by mining the evolution of user interests and movie appeal. Sun et al. found that users would maintain friendships by forwarding microblogs that they were not interested in, and proposed a causal attention mechanism to learn the causal effects of different interest factors on user forwarding behavior, thereby effectively improving the performance of microblog forwarding prediction. In the context of microblog forwarding prediction, Zheng Cong et al. extracted the user's preference for forwarded content based on a self-attention mechanism, and integrated multiple user preferences through an attention mechanism to comprehensively model user interest representation.

[0006] As a typical short text document, microblogs have limited content features. It is difficult to ensure the adequacy of microblog representation based solely on microblog content features, which in turn affects the accuracy of microblog forwarding predictions. Summary of the invention

[0007] The purpose of the present invention is to provide a microblog forwarding prediction model based on propagation tree mapping and interest noise reduction to solve the problem of low accuracy of microblog forwarding prediction by existing prediction models.

[0008] The object of the present invention is achieved in that:

[0009] A microblog forwarding prediction model based on propagation tree mapping and interest noise reduction, including:

[0010] The propagation tree learning layer is connected to the interest denoising layer and is used to map microblogs into propagation trees and generate microblog representations that integrate content features and propagation features through propagation tree learning;

[0011] The interest denoising layer is connected to the propagation tree learning layer and the prediction and training layer respectively, and is used to perform denoising learning on the user's historical forwarded microblog set; and

[0012] The prediction and training layer is connected to the interest noise reduction layer and is used to predict the probability of users forwarding microblogs.

[0013] Furthermore, the propagation tree learning layer determines the source Weibo based on the Weibo forwarded by the user in history; maps the propagation content derived from the source Weibo into nodes, maps the propagation relationship of the source Weibo into edges, and generates a propagation tree; initializes the nodes in the propagation tree; performs double graph pooling on the initialized propagation tree to generate a propagation tree representation; the propagation content includes comment content and forwarded content, and the propagation relationship includes forwarding relationship and comment relationship.

[0014] Furthermore, the propagation tree learning layer selects the skip-gram algorithm to first divide the keywords in the text content, encode the keywords into word vectors, and average pool the word vectors to obtain the initialization representation of the nodes.

[0015] Furthermore, the propagation tree learning layer performs propagation path pooling on the initialized propagation tree to generate a propagation path representation, and performs propagation tree pooling on the propagation tree after the propagation path pooling to generate a propagation tree representation.

[0016] Furthermore, the propagation tree learning layer uses the number of keywords of the node as the node weight and generates the propagation path representation through weighted fusion.

[0017] Furthermore, the propagation tree learning layer performs weighted fusion of the root node representation and the propagation path representation through a hyperparameter θ to generate a propagation tree representation.

[0018] Furthermore, the interest denoising layer generates a historical forwarded microblog set representation based on the propagation tree representation and the microblogs historically forwarded by the user; and performs interest denoising learning on the historical forwarded microblog representation to generate a user interest representation.

[0019] Furthermore, the interest denoising layer encodes the ID of each microblog forwarded by the user in the past to generate an ID representation; concatenates the ID representation and the propagation tree representation to generate a historical forwarded microblog representation; and stacks the microblog representations according to the user's historical forwarding order to obtain a historical forwarded microblog set representation.

[0020] Furthermore, the interest denoising layer generates a Weibo time representation based on the Weibo publishing time representation and forwarding time representation, concatenates the Weibo time representation and the historical forwarding Weibo representation, and generates a time-aware historical forwarding Weibo set representation; maps the time-aware historical forwarding Weibo set representation to the frequency domain according to the fast Fourier transform; uses a filter to denoise the data mapped to the frequency domain; inversely maps the denoised data through the inverse fast Fourier transform to generate a user interest representation; and converts the publishing time representation and the forwarding time representation into a time representation after the time is converted into a time vector and normalized.

[0021] Furthermore, after using the filter to denoise the data, random inactivation and layer normalization are performed on the denoised data, and the calculation formula is:

[0022]

[0023] in, is the output of the residual connection, layer normalization and random inactivation after the input data passes through the lth filter, is the output of the l-1th filter.

[0024] According to the feedforward neural network, the layer normalized data is calculated, and the calculation formula is:

[0025]

[0026] in, is the output of the feedforward neural network after the data passes through the lth filter, W1 and W2 are weight matrices, and b1 and b2 are bias vectors.

[0027] Furthermore, the prediction and training layer obtains the user's interest score based on the microblog representation to be predicted and the generated user interest representation, and selects the Top-K microblogs as forwarding prediction results based on the score; the microblog representation to be predicted is obtained by processing the microblog to be predicted through the propagation tree learning layer and the interest noise reduction layer.

[0028] The present invention proposes a microblog forwarding prediction model based on propagation tree mapping and interest denoising, and further improves the performance of microblog forwarding prediction by optimizing microblog representation and user interest representation. The present invention maps each microblog into a propagation tree, comprehensively learns microblog representation by integrating content features and propagation features, designs a dual graph pooling mechanism for propagation tree learning, and obtains microblog representations that integrate content features and propagation features to enhance the adequacy and accuracy of microblog representations; based on the user's historical forwarded microblog set, designs a time-aware filter to perform denoising learning on user interest representations to avoid the interference of noise information on user interest learning and enhance the accuracy of user interest representations. By combining propagation tree mapping and interest denoising technology, a new research perspective is provided for multi-field research such as microblog forwarding prediction and user behavior analysis. In practice, the research provides an effective practical basis for network public opinion analysis and governance by developing a microblog forwarding prediction method, and thus provides effective technical support for creating a good network public opinion ecology. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a data processing flow chart of the present invention.

[0030] Figure 2 It is a schematic diagram of propagation tree mapping.

[0031] Figure 3 It is a schematic diagram of dual image pooling.

[0032] Figure 4(a) is the comparison result of the prediction effect of the prediction model of the present invention and other propagation tree ablation models when the value of K in HR@K is 1, (b) is the comparison result of the prediction effect of the prediction model of the present invention and other propagation tree ablation models when the value of K in HR@K is 5, (c) is the comparison result of the prediction effect of the prediction model of the present invention and other propagation tree ablation models when the value of K in NDCG@K is 5, (d) is the comparison result of the prediction effect of the prediction model of the present invention and other propagation tree ablation models in the case of MRR.

[0033] Figure 5 (a) is the comparison result of the prediction effect of the prediction model of the present invention and other time-aware filter ablation models when the value of K in HR@K is 1, (b) is the comparison result of the prediction effect of the prediction model of the present invention and other time-aware filter ablation models when the value of K in HR@K is 5, (c) is the comparison result of the prediction effect of the prediction model of the present invention and other time-aware filter ablation models when the value of K in NDCG@K is 5, (d) is the comparison result of the prediction effect of the prediction model of the present invention and other time-aware filter ablation models in the case of MRR.

[0034] Figure 6 (a) is the prediction effect of the prediction model of the present invention under different numbers of filters when the value of K in HR@K is 1, (b) is the prediction effect of the prediction model of the present invention under different numbers of filters when the value of K in HR@K is 5, (c) is the prediction effect of the prediction model of the present invention under different numbers of filters when the value of K in NDCG@K is 5, (d) is the prediction effect of the prediction model of the present invention under different numbers of filters in the case of MRR.

[0035] Figure 7 (a) is the prediction effect of the prediction model of the present invention under different fusion weights when the value of K in HR@K is 1, (b) is the prediction effect of the prediction model of the present invention under different fusion weights when the value of K in HR@K is 5, (c) is the prediction effect of the prediction model of the present invention under different fusion weights when the value of K in NDCG@K is 5, (d) is the prediction effect of the prediction model of the present invention under different fusion weights in the case of MRR.

[0036] Figure 8 (a) shows the model performance of the prediction model of the present invention at different sparsities, and (b) shows the model performance of the FMLP-Rec model at different sparsities.

[0037] Fig. 9(a) is the distribution of keywords in user 1’s historical forwarded microblogs in the vector space, (b) is the distribution of keywords in user 2’s historical forwarded microblogs in the vector space, (c) is the distribution of keywords in user 3’s historical forwarded microblogs in the vector space, and (d) is the distribution of keywords in user 4’s historical forwarded microblogs in the vector space.

[0038] Fig.10 (a) is the score ranking obtained by the prediction model of the present invention and the visualization result of the actual forwarded microblogs of the users, (b) is the score ranking obtained by the FMLP-Rec model and the visualization result of the actual forwarded microblogs of the users, (c) is the score ranking obtained by the Filter_nf model and the visualization result of the actual forwarded microblogs of the users, and (d) is the score ranking obtained by the Filter_nt model and the visualization result of the actual forwarded microblogs of the users. DETAILED DESCRIPTION

[0039] The present invention is further described below.

[0040] like Figure 1 As shown, the present invention provides a microblog forwarding prediction model based on propagation tree mapping and interest denoising, the model includes a propagation tree learning layer, an interest denoising layer and a prediction and training layer, wherein the propagation tree learning layer is connected to the interest denoising learning layer, the interest denoising learning layer is respectively connected to the propagation tree learning layer and the prediction and training layer, and the prediction and training layer is connected to the interest denoising layer.

[0041] The propagation tree learning layer determines the source microblog based on the microblogs forwarded by the user in history; maps the propagation content derived from the source microblog into nodes, maps the propagation relationship of the source microblog into edges, and generates a propagation tree; initializes the nodes in the propagation tree; performs double graph pooling on the initialized propagation tree to generate a propagation tree representation. Among them, the propagation content includes comment content and forwarding content, and the propagation relationship includes forwarding relationship and comment relationship.

[0042] Weibo forwarding prediction aims to predict the next Weibo that a user may forward based on the user's historical forwarded Weibo. Assume that the complete set of Weibo users is U and the complete set of Weibo is R. For any Weibo r∈R, the historical forwarded Weibo set of any user u∈U is R. u = {r u(1) ,r u(2) ,…,r u(t) …,r u(n)},and Among them, r u(t) represents the tth forwarded microblog of user u, and n represents the number of historical forwarded microblogs of user u. Then the microblog forwarding prediction task can be expressed as:

[0043] P u(n+1) =g(Ru ,r) (1)

[0044] Among them, P u(n+1) is the predicted probability of user u forwarding microblog r, g(·) is the prediction method, which in the present invention specifically refers to the method of using the microblog forwarding prediction model based on propagation tree mapping and interest noise reduction to make predictions; according to the predicted probability P u(n+1) , which can effectively realize the Weibo forwarding prediction task.

[0045] like Figure 2 As shown, the present invention integrates content information and dissemination information to convert any microblog i Mapped into a propagation tree Among them, V i is a node set, is an edge set, P i is the propagation path set. The nodes in the propagation tree include the root node v i0 and propagation nodes The root node represents the source microblog, and the propagation node represents the propagation content (i.e., the comment content and forwarded content) derived from the source microblog. Therefore, the node set in the propagation tree is recorded as

[0046] The present invention determines the forwarding relationship and the comment relationship as the edges of the propagation tree. For example, <v ia ,v ib > represents node v ia With node r ib Therefore, the edge set in the propagation tree is recorded as:

[0047] The path of the propagation tree refers to all the nodes from the root node to any leaf node. For example, Figure 2 A propagation path p = {v i0 ,v i1 ,v i4} refers to the root node v i0 To leaf node v i4 All the nodes passed by. Therefore, the propagation path set is recorded as: in, For the propagation tree i P i propagation paths, P i For the propagation tree i The number of propagation paths, p ij Represents the propagation tree tree i The jth propagation path, v ijk Represents the propagation tree tree i The kth propagation node of the jth propagation path is Figure 2In, v ij1 =v i0 , p ij For the propagation tree i The number of nodes on the jth propagation path.

[0048] Node initialization in the propagation tree refers to generating vectorized representations of the nodes in the propagation tree, that is, generating the initialization representation of the nodes based on the text content. The propagation tree learning layer uses the skip-gram algorithm to first divide the keywords in the text content, encode the keywords into word vectors, and average pool the word vectors to obtain the initialization representation of the nodes.

[0049] like Figure 3 As shown, in order to generate a microblog representation that integrates content features and propagation features, the present invention proposes a dual graph pooling mechanism to learn the propagation tree. The dual graph pooling mechanism implementation steps specifically include propagation path pooling and propagation tree pooling.

[0050] The propagation tree learning layer performs propagation path pooling on the initialized propagation tree, and performs propagation tree pooling on the propagation tree after propagation path pooling to generate a propagation tree representation. Propagation path pooling specifically refers to representing a path in the propagation tree as a vector. Specifically, propagation path pooling uses the number of keywords of the node as the node weight and generates a propagation path representation through weighted fusion.

[0051] The calculation formula for propagation path pooling is:

[0052]

[0053] in, is the propagation path representation, is the propagation path p ij The number of keywords of all nodes on is the node v on the propagation path ijk The number of keywords included, For node v ijk The initialization representation of . Among them, the keywords are the keywords obtained by using the skip-gram algorithm when initializing the node. Since the root node representation is different from the propagation node representation, the root node representation is not considered in the propagation path pooling.

[0054] Propagation tree pooling specifically refers to representing the propagation tree as a vector and treating it as a microblog propagation tree representation. Specifically, propagation tree pooling performs weighted fusion of the root node representation and the propagation path representation through the hyperparameter θ to generate the propagation tree representation.

[0055] The calculation formula of the propagation tree pooling process is:

[0056]

[0057] in, For Weibo i The propagation tree representation of θ is the fusion weight. Represents the root node.

[0058] Since the two communication relationships of comments and forwarding are effectively considered based on the text content during the communication tree learning process, the Weibo communication tree representation effectively integrates the communication characteristics and content characteristics.

[0059] The interest denoising layer in the prediction model of the present invention is mainly used to eliminate the interference of noise information in the historical forwarded microblog set on user interest learning and generate a more accurate user interest representation, specifically including: generating time-aware historical forwarded microblog set representation and interest denoising.

[0060] Generate a historical forwarded microblog set representation: The interest denoising layer encodes the ID of each microblog forwarded by the user in the past to generate an ID representation; concatenate the ID representation and the propagation tree representation to generate a historical forwarded microblog representation.

[0061] In addition to the propagation tree, embedding the microblog ID is also helpful for mining user interests. First, the microblog representation is obtained by combining the microblog propagation tree and ID. The ID of each microblog is encoded using the one-hot encoding method, and the high-dimensional encoding matrix is ​​mapped to the low-dimensional vector space through the embedding function to obtain the ID representation of the microblog. The calculation formula is:

[0062]

[0063] in, For Weibo i The ID indicates that For Weibo i One-hot encoding, W * and W * is the weight matrix, b * and b * is the bias vector.

[0064] Secondly, the Weibo representation is obtained by concatenating the ID representation and the propagation tree representation of the Weibo. The calculation formula is:

[0065]

[0066] Among them, e u(t) is the microblog representation of the t-th repost of user u, concat(·) is the concatenation operation between vectors, The ID representation of the t-th forwarded microblog of user u is obtained by querying the ID vector of the microblog according to the historical forwarded microblog set of the user. The propagation tree representation of the t-th forwarded microblog by user u is obtained by querying the propagation tree representation of the microblog according to the user's historical forwarded microblog set.

[0067] Furthermore, the microblog representations are stacked according to the user's historical forwarding order to obtain the historical forwarded microblog set representation:

[0068] E u ={e u(1) ,e u(2) ,...,e u(n)} (6)

[0069] Among them, E u It is the historical set of forwarded microblogs of user u.

[0070] Since the historical behavior data of users contains noise information, mining user preferences directly through the user's historical forwarded microblog set will lead to inaccurate expression of user interests. The present invention uses a time-aware filter to perform noise reduction learning on user interests, wherein the time-aware filter mainly includes time bias and filter noise reduction.

[0071] Time offset: Generate a microblog time representation based on the microblog publishing time representation and forwarding time, where the publishing time and forwarding time are the times converted into time vectors and normalized.

[0072] Since the noise information in the user's historical data is bursty, this type of noise information can be better filtered through the user's behavior time sequence. In order to improve the user interest noise reduction ability, a time bias is introduced in the user's historical forwarded Weibo set representation. Specifically, a Weibo time representation is generated according to the Weibo release time and forwarding time, and it is used as the time bias of the historical forwarded Weibo set to better learn user interests. For example, the time "January 2, 2020 3:04" is converted to a time vector (2020,1,2,3,4); the time vector is linearly normalized so that all dimensions of the vector have a value range of [0,1], and the time representation is obtained, thereby eliminating the order of magnitude difference of each dimensional data and improving the training effect of the model. By performing the above processing on the Weibo release time and forwarding time, the release time representation and forwarding time representation are obtained. The quantization process of Weibo time representation is as follows:

[0073]

[0074] in, is the time of the user's t-th microblog, The time of posting on Weibo indicates that It represents the forwarding time of the Weibo.

[0075] The microblog temporal representation and the historical forwarded microblog representation are concatenated to generate a time-aware historical forwarded microblog set representation.

[0076] Incorporating the time bias (i.e., microblog time representation) into the historical forwarded microblog representation generated in step 1, we obtain the time-aware historical forwarded microblog set representation. The calculation formula is:

[0077]

[0078]

[0079] in, To incorporate the time bias of Weibo, Representation of a set of historical retweets as time-aware.

[0080] After that, layer normalization and dropout are used to improve the stability of model training. The calculation process is as follows:

[0081]

[0082] in, It is a time-aware representation of the historical retweet microblog set after layer normalization and dropout.

[0083] User interest denoising: Map the layer-normalized and dropout-time-aware historical forwarded microblog set representation to the frequency domain based on Fast Fourier Transform (FFT); use filters to denoise the data; and inversely map the denoised data through Inverse Fast Fourier Transform (IFFT) to generate user interest representation.

[0084] The present invention uses L superimposed filters to perform noise reduction learning on time-aware user interests. FFT needs to be performed before each filter to provide a noise reduction environment for the filter. IFFT needs to be performed after each filter to inversely map the noise-reduced data and convert it into the original format to achieve one-time user interest noise reduction. Therefore, the calculation formula mapped to the frequency domain is:

[0085]

[0086] in, is the output of the lth filter during FFT processing, is the output of the l-1th filter in IFFT processing, represents the FFT operation, and the input data for frequency domain mapping before the first filter is the time-aware historical forwarding microblog set generated by formula (10), that is,

[0087]

[0088] in, is the denoising output of the lth filter in the frequency domain, W is the filter, and ⊙ is the element-level multiplication operation.

[0089]

[0090] in, is the output of IFFT processing of the lth filter, This is the IFFT operation.

[0091] In addition, in order to alleviate the problems of gradient disappearance and training instability and further capture the nonlinear relationship of features, residual connections, layer normalization and random inactivation are added after each IFFT operation, and the data is processed in combination with a feedforward neural network. The calculation formula is:

[0092]

[0093]

[0094] in, is the output of the residual connection, layer normalization and random inactivation after the input data passes through the lth filter, is the output of the l-1th filter, is the output of the feedforward neural network after the data passes through the lth filter,

[0095] After adding residual connections, layer normalization, random dropout, and feedforward neural networks after each IFFT process, It is the output of the data output by the l-1th filter after passing through the feedforward neural network.

[0096] After learning the denoising of L filters, the denoised output can be obtained in, It represents the user interest representation after noise reduction, which integrates the features of the first n users’ forwarded microblogs.

[0097] The prediction and training layer obtains the user's interest score based on the microblog representation to be predicted and the user's interest representation, and selects the Top-K microblogs as the forwarding prediction results based on the score.

[0098] The prediction and training layer performs a dot product operation on the microblog representation to be predicted and the user interest representation to obtain the user's interest score. The top-K microblogs are selected as the forwarding prediction results based on the score. c The predicted microblogs represent c The formula is:

[0099]

[0100] in, Represents the microblog to be predicted c The ID indicates that Represents the microblog to be predicted c The propagation tree shows that Represents the microblog to be predicted ξ The Weibo time is expressed.

[0101] The interest score calculation process is as follows:

[0102]

[0103] Among them, r u(1:s) = {r u(1) ,r u(2) ,…,r u(s)}, r u(1:s) is the subset of the first s forwarded microblogs of user u, r c is the microblog to be predicted, P(r u(s+1) =r c |r u(1:s) ) is the interest score of user u in the context of the first t (1≤s≤n) forwarded microblogs, is the user interest representation after denoising, e c To be predicted, Weibo said, Vector transpose operation.

[0104] The present invention uses a ranking loss function based on the pairing method to calculate the gap between the predicted value and the true value, and trains the parameters of the model by the gradient descent method. The loss function is as follows:

[0105]

[0106] Among them, Loss is the loss value, σ(·) represents the activation function, is a truth value item, indicating the microblog that the user actually forwarded; It is a negative value item, which is randomly sampled from the microblogs that the user has not forwarded.

[0107] The prediction model of the present invention was evaluated.

[0108] Model comparison: The prediction model of the present invention is compared with other prediction models.

[0109] Currently, there is no public benchmark dataset for microblog forwarding prediction. Therefore, the present invention crawls real data from Sina Weibo through a Python crawler program to construct a dataset. The crawled data mainly includes 8 fields, namely: user ID, microblog ID, microblog forwarding relationship, microblog comment relationship, microblog text content, microblog comment content, microblog publishing time, and user forwarding microblog time. Based on the original data, data availability is enhanced through data preprocessing. The specific preprocessing operations include: (1) removing users with less than 3 forwarding records, using the last 2 microblogs forwarded by the user for verification and testing, and the remaining forwarded microblogs for model training; (2) since user interests change dynamically over time, premature microblogs are difficult to represent the user's recent interests. Therefore, only the 50 microblogs forwarded by the user in the recent period are retained; (3) according to the principle of three-degree influence, there is a phenomenon of influence attenuation in the process of information dissemination. When the distance exceeds 3 steps, the influence almost disappears. Therefore, the learning of the propagation tree only considers nodes within three steps from the root node.

[0110] As shown in Table 1, the statistical information of the preprocessed data set is obtained. Among them, the number of users represents the total number of users in the data set, the number of microblogs represents the total number of microblogs in the data set, the forwarding volume represents the total number of all microblogs forwarded, the time span represents the time span of all microblog forwarding, the average forwarding volume represents the average number of microblogs forwarded by users, and the average microblog length represents the average number of keywords in microblogs.

[0111] Table 1: Dataset statistics

[0112] Fields Number of users Number of Weibo Retweets Average forwarding volume Average length of microblogs Time span Numeric 7513 22883 70631 9.40 19.98 2021.03～2021.09

[0113] Since microblog forwarding prediction is essentially a sequence learning problem, the present invention mainly uses the mainstream sequence learning model as the baseline model to test the scientificity and effectiveness of the prediction model of the present invention. The specific introduction of the baseline model is as follows:

[0114] (1)SASRec model: a model based on attention mechanism, which mainly uses multi-head self-attention mechanism to capture user temporal interests.

[0115] (2) BERT4Rec model: an attention-based model that models user behavior sequences through a bidirectional self-attention mechanism to achieve temporal recommendations based on users’ dynamic interests.

[0116] (3) STOSA model: A model based on the self-attention mechanism, which uses Gaussian distribution to embed the items of user interaction and uses the Wasserstein self-attention module to learn the positional relationship between the interaction items.

[0117] (4) FMLP-Rec model: A filter-based model that uses filters to filter out noise factors in user interests.

[0118] (5) FMLP-Rec (con) model: When the FMLP-Rec model represents microblogs, the microblog content features are added to reflect that the propagation tree has certain advantages in prediction compared with the content features.

[0119] To ensure the fairness and comparability of the experiments, the main hyperparameters of the SASRec model, BERT4Rec model, STOSA model, and FMLP-Rec model are determined to be consistent with the original literature.

[0120] In order to quantitatively evaluate the forwarding prediction performance, the present invention uses evaluation indicators widely used in the recommendation system and forwarding prediction field for experimental analysis. The evaluation indicators include: Top-K Hit Ratio (HR@K), Top-K Normalized Discounted Cumulative Gain (NDCG@K), and Mean Reciprocal Rank (MRR).

[0121] (1) HR@K: Whether the top K prediction results hit the Weibo that the user actually forwarded. The higher the HR@K value, the better the prediction effect. The calculation formula is as follows:

[0122]

[0123] Among them, hit i Whether the Top-K of the forwarding prediction results for the i-th user contains the microblogs that he actually forwarded, hit i ∈{0,1}.

[0124] (2) NDCG@K: The accuracy of the prediction results is evaluated based on the ranking of user interest scores. The higher the NDCG@K value, the better the prediction effect. The calculation formula is as follows:

[0125]

[0126] Among them, rank i is the interest score ranking of the i-th user’s real forwarded microblog in the predicted results, rank i ∈[1,100].

[0127] (3) MRR: The ranking of the microblogs actually forwarded by the user in the prediction results. The larger the MRR, the better the prediction effect. The calculation formula is as follows:

[0128]

[0129] According to the formula, HR@1 in the present invention is equal to NDCG@1, so HR@{1,5,10}, NGCG@{5,10} and MRR are used to evaluate the experimental effect.

[0130] The present invention uses Python 3.9 and Pytorch 2.0 to conduct experiments on NVIDIA A100 PCIE GPU, where Pytorch is the mainstream framework for deep learning. For all models, the model parameters are initialized by Gaussian distribution by default, and the Adam optimizer is used to train 200 rounds with a batch size of 256; at the same time, an early stopping strategy is adopted to stop model optimization when the training rounds exceed 100 and the MRR does not increase after 10 rounds. For other hyperparameters, the optimal parameters for the model are configured by grid search, where the hidden layer size ∈ {16, 32, 64, 128, 256}, the learning rate ∈ {0.0001, 0.0005, 0.001}, and the drop rate ∈ {0.3, 0.5, 0.7}.

[0131] This experiment is mainly used to compare the forwarding prediction performance of the prediction model of the present invention and the baseline model to prove the effectiveness of the proposed model.

[0132] Table 2: Overall performance comparison results of the prediction model of the present invention and the baseline model on the data set

[0133] Model HR@1 HR@5 HR@10 NGCG@5 NGCG@10 MRR SASRec 0.6754 0.7810 0.7985 0.7360 0.7416 0.7269 BERT4Rec 0.5508 0.7647 0.8104 0.6681 0.6831 0.6469 STOSA 0.7049 0.7999 0.8134 0.7593 0.7637 0.7504 FMLP-Rec 0.6812 0.7864 0.8046 0.7411 0.7470 0.7323 FMLP-Rec(con) 0.6905 0.7974 0.8235 0.7516 0.7600 0.7436 Prediction model of the present invention 0.8109 0.8718 0.8901 0.8452 0.8512 0.8413 IMP 10.60％ 7.19％ 7.67％ 8.59％ 8.75％ 9.09％

[0134] As shown in Table 2, IMP represents the percentage improvement of the prediction model of the present invention compared to the optimal baseline model. Compared with the better baseline model, the prediction model of the present invention has a 7%-10% improvement in various evaluation indicators, which shows that the prediction model of the present invention exhibits better microblog forwarding prediction performance. Specifically, compared with the SASRec model, the BERT4Rec model, and the STOSA model, the prediction model of the present invention uses a time-aware filter to learn user interests, which can handle the noise information in user interests well; compared with the FMLP-Rec model, the prediction model of the present invention incorporates the propagation tree when generating microblog representations, which makes up for the lack of sparse microblog content features. In short, the reason why the prediction model of the present invention exhibits better recommendation performance is mainly attributed to two aspects: ① Optimization of microblog representation using a propagation tree; ② Optimization of user interest representation using a time-aware filter.

[0135] Validation: The microblog forwarding prediction model proposed in this paper contains two key designs: propagation tree and time-aware filter. To verify the effectiveness of the key designs, this section verifies the effectiveness of the two designs through ablation experiments.

[0136] Propagation tree validity verification.

[0137] To verify the effectiveness of the propagation tree, the present invention derives three ablation models based on the prediction model of the present invention, namely: (1) Tree_nt model: This model does not consider the propagation tree and only generates microblog representations based on ID information and time information; (2) Tree_np model: This model does not consider the propagation features in the microblog propagation tree and generates microblog representations based on ID information, time information and content features; (3) Tree_nr model: This model does not consider the content features in the microblog propagation tree and generates microblog representations based on ID information, time information and propagation features.

[0138] like Figure 4 As shown, the prediction effect of the prediction model of the present invention (i.e., the proposed model) is better than that of the Tree_nt model, the Tree_np model, and the Tree_nr model, proving that the introduction of the propagation tree can improve the microblog representation ability and help improve the forwarding prediction performance. This shows that the content features and propagation features introduced based on the propagation tree are both beneficial to improving the representation ability of microblogs. In addition, we can also observe that in terms of various indicators, the Tree_nr model and the Tree_np model are both better than the Tree_nt model, and the Tree_np model is slightly better than the Tree_nt model. This shows that considering only the content features and propagation features of the propagation tree can also improve the forwarding prediction effect to a certain extent, and the content features are slightly better than the propagation features.

[0139] Verification of the effectiveness of time-aware filters.

[0140] To verify the effectiveness of the time-aware filter, the present invention derives three ablation models based on the prediction model of the present invention, namely: (1) Filter_nft model: this model does not consider the time-aware filter and uses the self-attention mechanism to learn user interests; (2) Filter_nf model: this model does not consider the filter and uses the self-attention mechanism that incorporates time features to learn user interests; (3) Filter_nt model: this model does not consider the time factor of the time-aware filter and only uses the filter to learn user interests.

[0141] like Figure 5 As shown in the figure, the prediction effect of the prediction model of the present invention is better than the three ablation models, proving that the time-aware filter can effectively filter the noise in the interest, which is beneficial to improving the forwarding prediction performance. In addition, the prediction model of the present invention is better than the Filter_nf model, indicating that the method of noise reduction through the filter can better mine user interests compared with the self-attention mechanism; the prediction model of the present invention and the Filter_nf model are better than the Filter_nft model and the Filter_nt model, indicating that microblogs have a strong timeliness, and the time feature is extremely important for predicting the forwarding behavior of microblog users.

[0142] Hyperparameter verification: The microblog forwarding prediction model proposed in this invention contains two important hyperparameters: fusion weight θ and number of filters L. In order to explore the impact of hyperparameters on the experiment, a hyperparameter experiment was conducted on the dataset.

[0143] The impact of fusion weights on forwarding prediction.

[0144] The hyperparameter θ is used to fuse the root node representation of the propagation tree pooling and the propagation path representation. It is an important hyperparameter for propagation tree learning and affects the representation of microblog features. Therefore, it is necessary to verify the impact of this parameter on the model performance.

[0145] like Figure 6 As shown in the figure, with the increase of fusion weight θ, the forwarding prediction effect shows a trend of first rising and then falling. When the fusion weight θ is between the interval [0.4, 0.6], the prediction effect is better, and when the fusion weight θ is close to 0 or 1, the prediction effect is worse. This phenomenon shows that only considering the source microblog or the propagation microblog when representing microblog features will have a negative impact on forwarding prediction; and effectively fusing these two types of microblogs can improve the accuracy of microblog expression and enhance the forwarding prediction effect.

[0146] The impact of the number of filters on forwarding prediction

[0147] Multiple filters can help the model reduce noise from different angles and enhance the user's interest learning ability. The number of filters L determines the efficiency of noise processing and is an important hyperparameter for the interest denoising layer.

[0148] like Figure 7 As shown in the figure, as the number of filters L increases, the forwarding prediction effect also shows a trend of first rising and then falling. When the number of fusion filters L is around 8, the prediction effect is very good. This phenomenon shows that learning user interests requires multiple filter denoising, but when the number of filters is too large, the model will identify useful information as noise during the training process, resulting in a decrease in performance. Therefore, it is necessary to reasonably set the number of filters to obtain the best model performance.

[0149] Sparsity verification.

[0150] Data sparsity is an important challenge for forwarding prediction. The number of Weibo forwarding and the number of source Weibo keywords are two important data with sparsity in the experiment. This experiment compares the model performance of the prediction model of the present invention and the FMLP-Rec model under different sparsities to test the ability to resist sparse data.

[0151] like Figure 8 As shown, in Figure 8In (a), the prediction accuracy of microblogs with very few forwardings is extremely low. As the number of microblog forwardings increases, the model effect improves significantly, which shows that it is relatively easy to predict frequently interacted items in the forwarding prediction task, while predicting unpopular microblogs is still challenging; as the number of source microblog keywords increases, the model effect improves. When the number of source microblog keywords reaches (20,30], the forwarding prediction effect is the best. This shows that in the forwarding prediction task, the more microblog keywords the source microblog has, the easier it is to predict, but too many keywords will cause certain interference to the forwarding prediction. Comparison Figure 8 (a) and Figure 8 (b) It can be found that the prediction effect of the prediction model of the present invention is more obvious when predicting microblogs with less forwarding volume, and it is also improved to a certain extent when predicting microblogs with higher forwarding volume and number of source microblog keywords. This shows that the prediction model of the present invention has better adaptability to sparse microblog data than the comparison model.

[0152] In order to enhance the interpretability of the model, the present invention uses a machine learning dimensionality reduction algorithm T-SNE (t-Distributed Stochastic Neighbor Embedding) to reduce the dimensions of some important high-dimensional vectors in the experiment into two-dimensional vectors and visualize them.

[0153] The propagation tree enhances the microblog representation by fusing content features and propagation features, thereby improving the accuracy of the expression of the user's historical forwarded microblog set. In order to verify it, the present invention selects 4 users with a small number of forwarded microblogs from the data set, visualizes the distribution of keywords of their historical forwarded microblogs in the vector space, and uses different colors to color the keywords of the source microblogs and the propagated microblogs.

[0154] like Fig. 9 As shown in FIG, according to the nature of collaborative filtering (items interacted by the same user have similarities, and users who have interacted with the same item have similarities), the microblog keywords forwarded by the user in the past should show a higher degree of aggregation. Fig. 9 In the figure, due to the small number of reposts by the four users, the keyword distribution is relatively scattered, which is not conducive to the mining of user interests. However, after adding the keywords of the propagated microblogs, it can be found that the keyword distribution of users 2 and 4 shows a high degree of aggregation, proving that the propagation tree has the ability to enhance the representation of microblogs and improve the ability to mine user interests.

[0155] In order to verify that the filter can effectively improve the accuracy of the user's historical interest expression, the present invention visualizes the user interest expression obtained by the prediction model, FMLP-Rec, Filter_nf and Filter_nt of the present invention, and colors the user interest expression according to the score ranking rank of the user's actual forwarded microblogs.

[0156] like Fig.10As shown in the figure, compared with the other three models, the prediction model of the present invention has a score ranking closer to 1 for users' real forwarding of microblogs, indicating that the time-aware filter has obvious advantages. When the model does not use the time filter, the ability to express user interests will drop sharply, seriously affecting the prediction of user forwarding behavior. In addition, in the four models, when the user interest is far away from the center (0,0), the forwarding prediction result is more accurate, which shows that the user interest has obvious characteristics that are more conducive to forwarding prediction; and the points in the figure are clustered into several communities of different sizes, and the colors of the communities are relatively uniform, indicating that the four models have similar forwarding prediction accuracy for users with similar interests.

[0157] Yin et al.: Yin H, Yang S, Song X, et al. Deep fusion of multimodal features for social media retweet time prediction [J]. World Wide Web, 2021, 24(4): 1027-1044.

[0158] Wang et al. (Learning Weibo representation by combining text content and digital labels of Weibo): Wang J, Yang Y. Tweet Retweet Prediction Based on Deep Multitask Learning[J]. Neural Process Letters, 2022, 54: 523-536.

[0159] Firdaus et al.: Firdaus SN, Ding C, Sadeghian A. Topic specific emotion detection for retweet prediction[J]. International Journal of Machine Learning and Cybernetics, 2019,10:2071-2083.

[0160] He Ruifang, et al.: He Ruifang, Wang Haocheng, Liu Hongyu, et al. Multi-view microblog topic detection based on heterogeneous social context[J]. Journal of Software, 2023, 34(11): 5162-5178.

[0161] Wang et al. (A microblog embedding learning method based on attention mechanism): Wang L, Zhang Y, Yuan J, et al. FEBDNN: fusion embedding-based deep neural network for user retweeting behavior prediction on social networks[J]. Neural Computing and Applications, 2022, 34: 13219-13235.

[0162] Xu Jianmin et al.: Xu Jianmin, Han Kangkang, He Dandan, et al. Microblog forwarding prediction based on multiple forwarding habits[J]. Intelligence Journal, 2020, 39(3): 123-129,155.

[0163] Gan et al.: Gan M, Zhang

[0164] Sun et al.: Sun WJ, Liu XF. Deep attention framework for retweet prediction enriched with causal inferences[J].Applied Intelligence, 2023,53:24293–24313.

[0165] Zheng Cong et al.: Zheng Cong, Jiang Hao, Zhao Xiaoyue. Microblog forwarding prediction based on user preference characteristics[J]. Journal of Wuhan University, 2023, 56(06): 749-756.

Claims

1. A microblog forwarding prediction model based on propagation tree mapping and interest denoising, characterized in that: include: The propagation tree learning layer is connected to the interest denoising layer and is used to map microblogs into propagation trees and generate microblog representations that integrate content features and propagation features through propagation tree learning; The interest denoising layer is connected to the propagation tree learning layer and the prediction and training layer respectively, and is used to perform denoising learning on the user's historical forwarded microblog set; as well as The prediction and training layer is connected to the interest noise reduction layer and is used to predict the probability of users forwarding microblogs.

2. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 1 is characterized in that: The propagation tree learning layer determines the source microblog according to the microblogs forwarded by the user in history; maps the propagation content derived from the source microblog into nodes, maps the propagation relationship of the source microblog into edges, and generates a propagation tree; Initialize the nodes in the propagation tree; perform double graph pooling on the initialized propagation tree to generate a propagation tree representation; the propagation content includes comment content and forwarding content, and the propagation relationship includes forwarding relationship and comment relationship.

3. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 2 is characterized in that: The propagation tree learning layer uses the skip-gram algorithm to first divide the keywords in the text content, encode the keywords into word vectors, and average pool the word vectors to obtain the initialization representation of the nodes.

4. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 3 is characterized in that: The propagation tree learning layer performs propagation path pooling on the initialized propagation tree to generate a propagation path representation, and performs propagation tree pooling on the propagation tree after the propagation path pooling to generate a propagation tree representation.

5. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 4 is characterized in that: The propagation tree learning layer uses the number of keywords of the node as the node weight and generates the propagation path representation through weighted fusion; the propagation tree learning layer performs weighted fusion on the root node representation and the propagation path representation through the hyperparameter θ to generate the propagation tree representation.

6. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 2 is characterized in that: The interest denoising layer generates a historical forwarded microblog set representation based on the propagation tree representation and the microblogs historically forwarded by the user; performs interest denoising learning on the historical forwarded microblog representation to generate a user interest representation.

7. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 6 is characterized in that: The interest denoising layer encodes the ID of each microblog forwarded by the user in the past to generate an ID representation; concatenates the ID representation and the propagation tree representation to generate a historical forwarded microblog representation; and stacks the microblog representations according to the user's historical forwarding order to obtain a historical forwarded microblog set representation.

8. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 6 is characterized in that: The interest denoising layer generates a Weibo time representation based on the Weibo publishing time representation and forwarding time representation, concatenates the Weibo time representation and the historical forwarding Weibo representation, and generates a time-aware historical forwarding Weibo set representation; maps the time-aware historical forwarding Weibo set representation to the frequency domain according to the fast Fourier transform; uses a filter to denoise the data mapped to the frequency domain; inversely maps the denoised data through the inverse fast Fourier transform to generate a user interest representation; and converts the publishing time representation and the forwarding time representation into a time representation after the time is converted into a time vector and normalized.

9. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 6, characterized in that: After using the filter to denoise the data, the denoised data is randomly inactivated and layer normalized. The calculation formula is: in, is the output of the residual connection, layer normalization and random inactivation after the input data passes through the lth filter, is the output of the l-1th filter; According to the feedforward neural network, the layer normalized data is calculated, and the calculation formula is: in, is the output of the feedforward neural network after the data passes through the lth filter, W1 and W2 are weight matrices, and b1 and b2 are bias vectors.

10. The microblog forwarding prediction model based on propagation tree mapping and interest noise reduction according to claim 6, characterized in that: The prediction and training layer obtains the user's interest score based on the microblog representation to be predicted and the generated user interest representation, and selects the Top-K microblogs as the forwarding prediction result according to the score; the microblog representation to be predicted is obtained by processing the microblog to be predicted through the propagation tree learning layer and the interest noise reduction layer.

Images