A cascade popularity prediction method based on graph machine learning

Abstract

The application discloses a kind of cascade popularity prediction methods based on graph machine learning, comprising the following steps: S1, import original data set, divide into training set and test set, and define cascade data;S2, generate new cascade data by data reinforcement algorithm based on self-enhancement;S3, the user in cascade propagation network is embedded in network by graph representation learning algorithm, and the network embedding of each user is obtained, i.e. a low-dimensional embedding vector is generated for each user node;S4, the loss function for calculating cascade popularity is designed, to build cascade popularity prediction model, the training set is used to train the cascade popularity prediction model based on contrast learning, S5, the test set is input into the trained cascade popularity prediction model, and the cascade popularity prediction result is obtained by the trained cascade popularity prediction model.The method is an end-to-end new framework, which can model the internal and external attributes of information cascade simultaneously.

Description

Technical Field

[0001] This invention relates to the field of neural network technology in machine learning, specifically to a cascaded popularity prediction method based on graph neural networks. Background Technology

[0002] Cascade popularity prediction is a macro-level cascade prediction task that aims to predict the size of a cascade at a future point in time based on the propagation of the cascade over a period of time.

[0003] In recent years, online social networks have made tremendous progress, creating new platforms such as microblogs and Twitter, generating massive amounts of data every moment. The forwarding and sharing behavior among users spreads information and builds information cascade structures. For example, a photo or a link can be forwarded multiple times, potentially gaining access to a large number of users. Understanding information cascades has become crucial and can bring significant economic and social impacts, attracting considerable attention from academia and industry. Over the past few decades, it has been applied to various downstream tasks.

[0004] Most existing cascade popularity prediction models either simulate the generation of cascades through stochastic processes or simply make predictions using deep neural networks, failing to effectively track the internal and external properties of the cascade growth process, which may lead to poor model prediction performance. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing models that cannot integrate intra-cascade propagation patterns and inter-cascade interactions, and to provide a cascade popularity prediction method based on graph neural networks. This is a new end-to-end framework that can simultaneously model the internal and external attributes of information cascades.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A cascade popularity prediction method based on graph machine learning includes the following steps:

[0008] S1. Import the original dataset, divide it into training and test sets, and define the cascaded data;

[0009] S2. Generate new cascaded data using a self-enhancing data augmentation algorithm;

[0010] S3. The network embedding of users in the cascaded propagation network is performed by the graph representation learning algorithm to obtain the network embedding of each user, that is, to generate a low-dimensional embedding vector for each user node.

[0011] S4. Design a loss function to calculate cascade popularity, thereby constructing a cascade popularity prediction model, and train the cascade popularity prediction model based on contrastive learning using the training set.

[0012] The cascaded popularity prediction model uses contrastive loss as the loss function in the pre-training stage of the contrastive learning-based cascaded popularity prediction model; it uses mean squared logarithmic error as the loss function in the fine-tuning stage of the cascaded popularity prediction model, sets hyperparameters, and uses the adaptive momentum estimation Adam optimizer; it sets the initial learning rate, training batch size, and total training iterations t. The cascaded popularity prediction model is implemented based on a recurrent neural network (GRU) and uses a multilayer perceptron as the classifier. The specific steps are as follows:

[0013]

[0014]

[0015]

[0016]

[0017] in, Here are the trainable parameters, and tanh represents the hyperbolic tangent activation function. Let x represent the hidden layer vector of the GRU. vk σ represents the input at the current time step, and σ denotes the nonlinear activation function.

[0018] S5. Input the test set into the trained cascaded popularity prediction model, and obtain the cascaded popularity prediction results through the trained cascaded popularity prediction model.

[0019] Preferably, the cascaded data is defined as:

[0020] C k (t o )={(v j , t j ), j∈[1,N]}

[0021] Among them, t o For the observation time, (v j , t j ) for user v j At time t j In the cascading process, N represents the number of users participating in the cascading.

[0022] Preferably, in step S1, a cascade graph is defined using cascaded data:

[0023] For cascaded C k (t)={(v j , t j ), j∈[1,N]}, and its corresponding cascade graph is in For a set of nodes, Let be the set of edges, representing the forwarding relationships between nodes in a cascade.

[0024] Preferably, in step S1, a cascade graph is defined using cascaded data:

[0025]

[0026] in ε is a set of nodes containing all nodes in the dataset. g This indicates the forwarding relationship between nodes.

[0027] Preferably, in step S2, the method for generating new cascaded data through data augmentation is as follows:

[0028] The cascade diagram is as follows: Its corresponding cascade is C k (t)={(v j , t j ), j∈[1,N]}, based on observation time t o Calculate the snapshot observation time t s :

[0029] t s =t o *p, (0 < p ≤ 1)

[0030] Where p is a pre-set hyperparameter, based on the snapshot observation time t. s and the original cascaded C k (t)={(v j , t j Given j∈[1, N]}, we can obtain the enhanced cascade C corresponding to the original cascade. k (t s )={(v i , t i ), i∈[1,N]}, (t i <t s ) and enhanced cascade diagram

[0031] Preferably, in step S3, the method for embedding users in the cascaded propagation network using graph representation learning algorithms includes global structure learning on cascaded graphs based on RWPE and cascaded graph representation learning based on GRAPHWAVE.

[0032] Preferably, the global structure learning on the cascaded graph based on RWPE involves generating a random walk matrix RW = AD⁻¹ on the cascaded graph, where A is the cascaded graph. Given the adjacency matrix of A and D as the degree matrix of A, calculate node v based on the random walk matrix RW. i The embedding vector is:

[0033]

[0034] Where k is a pre-set hyperparameter.

[0035] Preferably, the cascaded graph representation learning based on GRAPHWAVE involves learning the structural embedding of each node based on the diffusion of spectral wavelets centered on the node.

[0036] This invention has the following characteristics and beneficial effects:

[0037] This invention proposes a unified framework that integrates intra-cascade and inter-cascade perspectives to model cascade popularity prediction.

[0038] This invention proposes a novel method for cascaded data augmentation and positive / negative sample pair generation. This method can simply and effectively help models understand the diffusion characteristics within a cascade. Simultaneously, this invention applies a novel graph embedding method to learn the influence of inter-cascade relationships on individual cascades by encoding user behavior. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a cascaded popularity prediction framework based on graph machine learning.

[0041] Figure 2 To visualize the cascaded representation of the encoder output, the coordinates of a point represent the position of the cascade in the vector space, and the color of the point represents the popularity of the cascade. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0043] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0044] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0045] This invention provides a cascaded popularity prediction method based on graph machine learning, such as... Figure 1 As shown, it includes the following steps:

[0046] S1: Import the original dataset and divide it into training and test sets;

[0047] S1.1: Define the cascaded data as:

[0048] C k (t o )={(v j , t j ), j∈[1,N]}

[0049] Among them, t o For the observation time, (v j , t j ) for user v j At time t j In the cascading process, N represents the number of users participating in the cascading.

[0050] S1.2: The cascade popularity prediction problem is defined as follows:

[0051] Given cascade C k (to Its final popularity at prediction time t p The number of users participating at any given time. Generally speaking, t p Size is t o Several times that.

[0052] S1.3: The cascade diagram is defined as follows:

[0053] For cascaded C k (t)={(v j , t j ), j∈[1,N]}, and its corresponding cascade graph is in For a set of nodes, Let be the set of edges, representing the forwarding relationships between nodes in a cascade.

[0054] S1.4: The cascade diagram is defined as follows:

[0055]

[0056] in ε is a set of nodes containing all nodes in the dataset. g This indicates the forwarding relationship between nodes.

[0057] S2: Generate new cascaded data using a self-enhancing data augmentation algorithm.

[0058] The cascade diagram is as follows: Its corresponding cascade is C k (t)={(v j , t j ), j∈[1,N]}, based on observation time t o Calculate the snapshot observation time t s :

[0059] t s =t o *p, (0 < p ≤ 1)

[0060] Where p is a pre-set hyperparameter, based on the snapshot observation time t. s and the original cascaded C k (t)={(v j , t j Given j∈[1, N]}, we can obtain the enhanced cascade C corresponding to the original cascade. k (t s )={(v i , t i ), i∈[1,N]}, t i <t s ) and enhanced cascade diagram

[0061] S3: The network embedding of users in the cascaded propagation network is performed by the graph representation learning algorithm to obtain the network embedding of each user, that is, to generate a low-dimensional embedding vector for each user node.

[0062] S3.1 The overall structure learning process on the cascaded graph based on RWPE (random walk positional embedding), the learning... The process includes: generating a random walk matrix RW = AD⁻¹ on the cascaded graph, where A is the cascaded graph. Given the adjacency matrix of A and D as the degree matrix of A, calculate node v based on the random walk matrix RW. i The embedding vector is:

[0063]

[0064] Where k is a pre-set hyperparameter.

[0065] S3.2 describes the cascaded graph representation learning based on GRAPHWAVE: learning the structural embedding of each node based on the diffusion of spectral wavelets centered on the node.

[0066] S4: Design a loss function to calculate cascade popularity, and train the cascade popularity prediction model based on contrastive learning using the training set;

[0067] Contrastive loss is used as the loss function in the pre-training stage of the cascaded popularity prediction model based on contrastive learning; mean square logarithmic error is used as the loss function in the fine-tuning stage of the cascaded popularity prediction model. Hyperparameters are set, and the adaptive momentum estimation Adam optimizer is used. The initial learning rate, training batch size, and total training iterations t are set. The cascaded popularity prediction model is implemented based on a recurrent neural network GRU and uses a multilayer perceptron as the classifier. The specific steps are as follows:

[0068]

[0069]

[0070]

[0071]

[0072] in, U r U z U h b r b z b hHere are the trainable parameters, and tanh represents the hyperbolic tangent activation function. h k Let x represent the hidden layer vector of the GRU. vk σ represents the input at the current time step, and σ denotes the nonlinear activation function.

[0073] S5: Input the test set into the trained cascaded popularity prediction model, and obtain the cascaded popularity prediction results through the trained cascaded popularity prediction model.

[0074] The trained cascade popularity prediction model can extract the internal growth characteristics of a single cascade and the characteristics of the overall cascade propagation structure during the cascade propagation process, judge the extracted characteristics, and then judge the final popularity of the cascade.

[0075] In this embodiment, the input cascaded graph is first augmented to obtain multiple sets of positive samples. One pair of positive samples is then input into the cascaded graph encoder to obtain a set of cascaded representations h1 and h2. After a linear transformation, transformed representations z1 and z2 are obtained. The model is then pre-trained using a contrastive loss function. After pre-training, the cascaded representations obtained by the cascaded graph encoder are output to an MLP-based predictor to obtain predicted labels. The model is then trained using mean squared logarithmic error to ultimately predict the popularity of the cascaded graphs.

[0076] Specifically, such as Figure 2 As shown, the cascaded popularity prediction framework in this embodiment produces results on three different real datasets (Weibo dataset, Twitter dataset, and APS dataset). For each dataset, we use 50% of the data as the training set, 25% as the test set, and 25% as the validation set. First, we perform self-supervised pre-training on the model using the training set, then we perform supervised fine-tuning training on the model using the training set, while simultaneously validating the prediction results on the test and validation sets. For each dataset, the t in step one... o The values ​​were tested under two different conditions. MSLE and MAPE in the table represent the prediction errors; the smaller the values, the better the model's cascade popularity prediction performance.

[0077] Table 1: Cascade popularity prediction performance on three datasets

[0078]

[0079] The remaining methods in the table are described below:

[0080] Feature-S&H extracts early popularity features from cascaded content to predict its long-term popularity. This model is primarily designed for traffic prediction problems in news media and online video websites. Reference: [Gabor Szabo and Bernardo A. Huberman. 2010. Predicting the Popularity of Online Content. Commun. ACM 53, 8 (aug 2010), 80–88.]

[0081] Feature-Linear, Feature-Deep: Researchers categorized the features in the cascade growth process into five classes. By inputting these features into linear regression and MLP models, we can predict cascade popularity. Reference: [Justin Cheng, Lada Adamic, P. Alex Dow, Jon Michael Kleinberg, and Jure Leskovec. 2014. Can cascades be predicted? . In Proceedings of the 23rd international conference on World Wide Web (WWW'14). ACM Press.]

[0082] TimeSeries: A model that treats cascades as time series and predicts cascade prevalence based on historical information provided by earlier prevalence measurements. References: [Henrique Pinto, Jussara M. Almeida, and Marcos A.] 2013.Using Early View Patterns to Predict the Popularity ofYoutube Videos.In Proceedings of the Sixth ACM International Conference onWeb Search and Data Mining(Rome,Italy)(WSDM'13).Association for ComputingMachinery,New York,NY,USA,365–374.】

[0083] DeepHawkes models cascades by combining the Hawkes generation process with an end-to-end deep learning approach. It considers three key factors in the Hawkes process—user influence, self-excitation mechanism, and time decay mechanism—inheriting the high interpretability of the Hawkes process and the high predictive power of deep learning. Reference: [Qi Cao, Huawei Shen, Keting Cen, Wentao Ouyang, and Xueqi Cheng. 2017. DeepHawkes: Bridging the Gap between Prediction and Understanding of Information Cascades. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (Singapore, Singapore) (CIKM'17). Association for Computing Machinery, New York, NY, USA, 1149–1158.]

[0084] CasCN combines Graph Convolutional Networks (GCNs) and Recurrent Neural Networks (RNNs) to extract topological and temporal features of cascaded graphs. Reference: [Xueqin Chen, Fan Zhou, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Fengli Zhang. 2019. Information diffusion prediction via recurrent cascades convolution. In 2019 IEEE 35th international conference on data engineering (ICDE). IEEE, 770–781.]

[0085] DMT-LIC: A multi-task model that performs simultaneous micro- and macro-level cascaded predictions through attention mechanisms and a shared representation layer. Reference: [Xueqin Chen, Kunpeng Zhang, Fan Zhou, Goce Trajcevski, Ting Zhong, and Fengli Zhang. 2019. Information cascades modeling via deep multi-task learning. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval. 885–888]

[0086] CasFlow: Predicting Cascade Popularity Based on Bayesian Learning. This method utilizes uncertainties at the node and cascade levels, employing variational inference and normalized flow to learn the posteriority of cascade distributions using a hierarchical variational information diffusion model. Reference: [Xovee Xu, Fan Zhou, Kunpeng Zhang, Siyuan Liu, and Goce Trajcevski. 2021. CasFlow: EXploring hierarchical structures and propagation uncertainty for cascade prediction. IEEE Transactions on Knowledge and Data Engineering (2021)]

[0087] CCGL proposes an application of contrastive learning in cascaded popularity prediction, using comparative learning to bridge the gap between supervised and unsupervised information cascaded modeling and prediction. Reference: [Xovee Xu, Fan Zhou, Kunpeng Zhang, and Siyuan Liu. 2022. CCGL: Contrastive Cascade Graph Learning. IEEE Transactions on Knowledge and Data Engineering (2022).]

[0088] As can be seen from the experimental results in Table 1, our proposed cascaded popularity prediction method has a higher overall prediction accuracy than some existing methods.

[0089] Therefore, this invention utilizes a novel end-to-end cascading popularity prediction framework and designs a data augmentation strategy using historical snapshots based on the cascading itself. To understand the cascading characteristics from a global perspective, the concept of a cascading graph is introduced, allowing for detailed observation of user behavior and achieving cascading popularity prediction superior to existing methods.

[0090] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A cascaded popularity prediction method based on graph machine learning, characterized in that, Includes the following steps: S1. Import the original dataset, divide it into training and test sets, and define the cascade; The cascading is defined as follows: ； in, For observation time, For users In time Participate in the cascade, The number of users participating in the cascading; Define a cascade graph using cascading: For cascades The corresponding cascade diagram is ,in For a set of nodes, Let be the set of edges, representing the forwarding relationships between nodes in the cascade; Define the cascade graph using cascading: ； in A node set, containing all nodes in the dataset. Indicates the forwarding relationship between nodes S2. Generate new cascades using a self-enhancing data hardening algorithm; The specific method is as follows: The cascade diagram is as follows: Its corresponding cascade is Based on observation time Calculate snapshot observation time : ； Where p is a pre-set hyperparameter based on the snapshot observation time. and original cascade This allows us to obtain an enhanced cascade corresponding to the original cascade. and enhanced cascade graph ; S3. The network embedding of users in the cascaded propagation network is performed by the graph representation learning algorithm to obtain the network embedding of each user, that is, to generate a low-dimensional embedding vector for each user node. The graph representation learning algorithm for embedding users in a cascaded propagation network includes RWPE-based global structure learning on cascaded graphs and GRAPHWAVE-based cascaded graph representation learning. The global structure learning on cascaded graphs based on RWPE involves generating a random walk matrix RW=AD⁻¹ on the cascaded graph, where A is the cascaded graph. Given the adjacency matrix of A and D as the degree matrix of A, calculate the nodes based on the random walk matrix RW. The embedding vector is: ； Where k is a pre-set hyperparameter; The cascaded graph representation learning based on GRAPHWAVE: learns the structural embedding of each node based on the diffusion of spectral wavelets centered on the node; S4. Design a loss function to calculate cascade popularity, thereby constructing a cascade popularity prediction model, and train the cascade popularity prediction model based on contrastive learning using the training set. The cascaded popularity prediction model uses contrastive loss as the loss function in the pre-training stage of the contrastive learning-based cascaded popularity prediction model; it uses mean squared logarithmic error as the loss function in the fine-tuning stage of the cascaded popularity prediction model, sets hyperparameters, and uses the adaptive momentum estimation Adam optimizer; it sets the initial learning rate, training batch size, and total training iterations t. The cascaded popularity prediction model is implemented based on a recurrent neural network (GRU) and uses a multilayer perceptron as the classifier. The specific steps are as follows: ； in, Here are the trainable parameters, and tanh represents the hyperbolic tangent activation function. This represents the hidden layer vector of the GRU. σ represents the input at the current time step, and σ denotes the nonlinear activation function. S5. Input the test set into the trained cascaded popularity prediction model, and obtain the cascaded popularity prediction results through the trained cascaded popularity prediction model.

Images