A popularity prediction method for online content based on contrastive learning

Abstract

The application discloses a popularity prediction method of online content based on contrast learning, which can be applied to scenarios such as message pushing in a social network. In order to solve the problem that it is very challenging to accurately predict the popularity of online content due to the large number of users and complex relationships in the social network, it is difficult to comprehensively capture the characteristics of message cascade propagation, the application first constructs a logical relationship graph between users and messages in the social network, and projects it into a relationship graph containing only message nodes. Then, the cascade propagation process of each message is modeled by obtaining the cascade graph time sequence of the cascade propagation of the message between users. At the same time, according to the popularity of the message, the positive and negative samples of the message anchor node are determined, a calculation method of contrast loss is designed, and contrast learning between different messages is realized by minimizing the contrast loss.

Description

Technical Field

[0001] This invention relates to network data prediction technology, specifically to technology for predicting the popularity of online content. Background Technology

[0002] The rapid development of internet technology has made communication between people more convenient, allowing them to publish or share content of interest anytime through social media platforms such as Weibo. Due to the ease of real-time sharing, tens of millions of messages are published or disseminated on social networks every day. Handling such a massive and complex volume of information presents a significant challenge for both social media platform administrators and users. On the one hand, pushing all online content to all users would be extremely costly in terms of platform management and maintenance; on the other hand, users are easily overwhelmed by the sheer volume of information they receive daily.

[0003] Predicting the popularity of online content on social networks involves forecasting the level of attention information will receive after a certain period of time. The aim is to predict and identify future trending information from a large amount of online content. This can save social platform administrators a lot of costs, and the quality of online content on the platform will be greatly improved. At the same time, users can selectively follow trending information, thereby avoiding the trouble caused by information overload.

[0004] Existing popularity prediction methods can be mainly divided into three categories: feature extraction-based methods, point process modeling-based methods, and deep learning-based methods.

[0005] (1) Feature extraction-based methods

[0006] Popularity prediction methods based on feature extraction mainly predict the popularity of messages by extracting user features, content features, temporal features, and structural features. However, this method relies too much on the quality of manually extracted features and cannot automatically learn feature representations.

[0007] (2) Point process modeling method

[0008] Popularity prediction methods based on point process modeling treat the propagation of messages in social networks as a process of arrival driven by user forwarding behavior, and model the rate function of this arrival process to predict message popularity. This method often requires modeling the rate function under certain constraints, but whether these constraints hold true in real-world scenarios is unknown.

[0009] (3) Deep learning-based methods

[0010] Deep learning-based popularity prediction methods learn the representations of message content features, user features, and temporal features in social networks through neural network models, thereby more accurately predicting the future popularity of messages. Liao D, Xu J, Li G, et al. Popularity prediction on online articles with deep fusion of temporal process and content features[C] / / Proceedings of the AAAIConference on Artificial Intelligence.2019,33(01):200-207. The temporal process and content features are modeled separately. The temporal process modeling uses LSTM to characterize the long-term growth trend and Attention CNN to automatically capture short-term fluctuations. The content feature modeling uses Hierarchical Attention Network (HAN) to learn text features and embedding technology to extract metadata features. However, in social platforms, the network structure formed by user interactions is often intricate, which has a significant impact on message popularity. Cao Q, Shen H, Gao J, et al. Popularity prediction on social platforms with coupled graph neural networks[C] / / Proceedings of the 13th International Conference on Web Search and Data Mining.2020:70-78. Considering the network structure, this paper uses two coupled graph neural networks to capture the interaction between the activation state of nodes and the propagation of influence. One graph neural network simulates the propagation of interpersonal influence through the user's activation state, while the other graph neural network models the update of each user's activation state through the interpersonal influence of neighbors. However, this method does not consider temporal characteristics, and cannot accurately predict when the popularity of a message surges in a short period of time.

[0011] However, existing popularity prediction methods still have the following problems:

[0012] (1) Popularity is a relative concept. Previous methods only considered the characteristics and propagation properties of the message, without taking this relativity into account, resulting in limited discriminative power in predicting the popularity of the message. Using a contrastive learning-based method can characterize this relativity, better distinguish between popular and unpopular messages, and obtain a popularity score that is closer to the true value.

[0013] (2) Messages spread among users to form a cascade network. The structure of the cascade network has a great influence on the popularity of messages. Most existing methods use graph convolutional networks to capture cascade features. However, for large-scale and complex networks, directly using graph convolutional networks will consume a lot of computing resources and costs.

[0014] (3) Social networks in reality are often complex and changeable. Therefore, the popularity of messages on the network may change drastically in a short period of time due to certain external factors, which will cause a large deviation between the predicted value and the actual value of popularity. Most online content popularity prediction methods do not take this into account. Summary of the Invention

[0015] The applicant's analysis revealed that while existing popularity prediction methods consider the cascading effects during message propagation, they do not comprehensively capture these cascading propagation characteristics. Some scholars have proposed using Graph Neural Networks (GCNs) to characterize these characteristics; however, GCNs repeatedly aggregate and update the features of neighboring nodes, requiring enormous computational power in large-scale and complex social networks. Furthermore, popularity is a relative concept; previous methods only considered message features and propagation characteristics without addressing this relativity, resulting in limited discriminative power in predicting message popularity. Using contrastive learning-based methods can characterize this relativity, better distinguishing between popular and unpopular messages, and yielding popularity values ​​closer to the true values. Simultaneously, in complex and dynamic social networks, message popularity can change drastically in a short period due to external factors such as weather and unforeseen events, leading to significant discrepancies between predicted and actual popularity values. Most current online content popularity prediction methods do not consider this. The technical problem this invention aims to solve is to propose an online content popularity prediction method that effectively captures the cascading characteristics during message propagation while considering the comparative relationships between different messages.

[0016] The technical solution adopted by this invention to solve the above-mentioned technical problems is a method for predicting the popularity of online content based on contrastive learning, comprising the following steps:

[0017] S1, via the message forwarding sequence C within a set time period. n,T The interaction matrix between users and messages is obtained by cascading the propagation sequence between users. Each element in the interaction matrix represents the number of times the corresponding user forwards the corresponding message, thus obtaining a bipartite graph of the interaction relationship between messages and users.

[0018] Projecting a bipartite graph onto a global graph: The global graph consists of a set of vertices formed by messages, and the number of edges between vertices of different messages is determined by the number of times they are forwarded by the same user;

[0019] S2. Utilize the cascading propagation sequence of messages among users to obtain the propagation order of messages among users within a set time period, thereby obtaining the cascading propagation graph sequence; use a graph representation algorithm to transform each propagation cascade graph in the cascading propagation graph sequence into a feature vector of the corresponding time series, thereby obtaining the feature vector of the time series within the set time period.

[0020] The feature vector of the time series is input into the gated loop unit to obtain the cascaded propagation feature vector corresponding to the message.

[0021] S3. The cascading propagation feature vectors of all messages and the global graph input graph convolutional network (GCN) obtain the high-order representation of the message nodes in the global graph. The high-order representation is then passed through a multilayer perceptron (MLP) to obtain the final predicted value of the message popularity.

[0022] In the training of Graph Convolutional Network (GCN) and Multilayer Perceptron (MLP), the loss function L is determined by the mean squared error (MSE). MSE And contrast loss L Con It consists of two parts:

[0023] L = L MSE +β·L Con

[0024] L MSE For mean square error loss, L CON To compare the loss functions, β is the weight parameter.

[0025] Contrast loss function L CON The calculation method is as follows:

[0026] Where pos represents the matching degree between the positive sample node and the anchor node, pos =<h(a),h(pp)> Neg represents the degree of matching between the negative sample node and the anchor node.<h(a),h(nn)> ,<A,B> Let A represent the vector inner product of A and B, h(a), h(pp), and h(nn) represent the feature embeddings corresponding to the anchor node set a, the positive sample node set pp, and the negative sample node set nn, respectively, and ones is a matrix of all 1s.

[0027] Specifically, the graph representation algorithm in step S2 is Node2Vec.

[0028] This invention proposes a method for predicting the popularity of online content based on contrastive learning. It constructs a logical relationship graph between users and messages using users' message forwarding behavior in social networks, and then projects this graph onto a relationship graph containing only message nodes. For each message in this relationship graph, it obtains the time series of its cascading propagation among users, transforms the cascading graph into a feature vector corresponding to the message using a graph representation algorithm, and obtains the cascading propagation feature vector for each message using a gated recurrent unit (GRU). Then, it learns the structural features of the message relationship graph using a graph convolutional network (GCN) to obtain a high-order representation of each message node. Furthermore, it designs a sampling strategy that samples based on the message's popularity to determine the positive and negative samples of the message anchor node. Further, it designs a method for calculating the contrastive loss function, achieving contrastive learning between different messages by minimizing the contrastive loss. Finally, it obtains the predicted popularity value of each message using an MLP. To the applicant's knowledge, this method is currently the first to apply contrastive learning to message popularity prediction in social networks.

[0029] The beneficial effects of this invention are:

[0030] 1. The user-message relationship graph is projected onto a bipartite graph to obtain a relationship graph between messages. The structural features of this relationship graph are learned using GCN. At the same time, the cascading propagation process of each message in the relationship graph is modeled. Compared with directly learning the user-message relationship graph, a richer feature representation can be obtained.

[0031] 2. This invention constructs a cascaded graph sequence of messages propagating among users and uses graph representation algorithms to map the nodes in the cascaded graph to a low-dimensional space, transforming them into a time series of feature vectors corresponding to the messages. Compared with directly processing graph data, this can reduce computational costs.

[0032] 3. By using a gated recurrent unit (GRU) to process the time series of message feature vectors, the problem of inaccurate popularity prediction due to a sudden surge in message popularity in a short period of time can be effectively avoided. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a scenario provided for an embodiment of the present invention;

[0034] Figure 2 This provides a framework for a contrastive learning-based popularity prediction method in the embodiments of the present invention. Detailed Implementation

[0035] The method for predicting the popularity of online content specifically includes the following steps:

[0036] S1, based on the forwarding sequence C of message n within time T. n,T and its cascaded propagation sequence V among users un,T Obtain the user and message interaction matrix A, and the element a in interaction matrix A. ij This represents the number of times user i forwards message j, thus obtaining the bipartite graph G representing the interaction relationship between message n and user u. u-n Matrix A is the user and message interaction relationship graph G. u-n The adjacency matrix;

[0037] The bipartite graph G u-n Projected onto a graph G containing only the message n vertices n On graph G, the projection method is as follows: The set of vertices V consists of messages from the social network, and E is the set of edges between vertices. Whether an edge exists between vertices depends on whether the users forwarding the messages are the same. Specifically, the number of edges between vertices of different messages is determined by the number of times they are forwarded by the same user. n G is a global graph representing the relationships between different messages. n ;

[0038] S2, for graph G n Each message n propagates among users. Users are considered vertices, and edges between user vertices represent message forwarding between those users. The cascading propagation graph sequence G is obtained based on the propagation order of message n among users in time T. n,T ={g n,1 ,g n,2 ,…,g n,T}, g n,i Let g be the propagation cascade graph of message n in time i, i = 1, ..., T; and let Node2Vec be used to transform the propagation cascade graph g. n,i Transform into the feature vector f corresponding to message n n,i Thus, the time series f of the feature vector of message n within time T is obtained. n ={f n,1 ,f n,2 ,…,f n,T};

[0039] f n The input is fed into a gated recurrent unit (GRU) to obtain the cascaded propagation feature vector h corresponding to message n. n ;

[0040] S3, global graph G n The adjacency matrix and G n The cascading propagation characteristics h of each message n The input graph convolutional network (GCN) obtains the high-order representation H of the message nodes in the global graph. The high-order representation H is then passed through a multilayer perceptron (MLP) to obtain the final predicted value of popularity.

[0041] During training, the implementation example designed a sampling strategy: given a set of anchor nodes a, for each message in a, messages with popularity belonging to the same category as the message are randomly selected as positive samples and added to the set pp; at the same time, messages with popularity not belonging to the same category as the message are randomly selected as negative samples and added to the set nn.

[0042] The loss function L is determined by the mean square error (MSE). MSE And contrast loss L Con It consists of two parts:

[0043] L = L MSE +β·L Con

[0044] Mean square error loss L MSE From the formula The calculation yielded, where y is the predicted value of popularity. i β represents the actual popularity value, m represents the total number of messages, and β is a parameter.

[0045] The embodiment designs a method for calculating the contrastive loss function, L. CON From the formula The calculation yields the following result: pos = (where pos represents the matching degree between the positive sample node and the anchor node).<h(a),h(pp)> The degree of matching between negative sample nodes and anchor nodes, neg =<h(a),h(nn)> ,<A,B> Let A represent the vector inner product of A and B, h(a), h(pp), and h(nn) represent the feature embeddings corresponding to the anchor node set a, the positive sample set pp, and the negative sample set nn, respectively; ones is a matrix of all 1s.

[0046] Application scenarios of the present invention are as follows Figure 1 As shown, on social media platforms such as Weibo and Zhihu, users can access and share content they are interested in anytime, anywhere. However, due to the massive scale and complex structure of social networks, users are prone to information overload when faced with a vast amount of information. To help users overcome this predicament and improve the quality of online content on social media platforms, this embodiment designs a method for predicting the popularity of online content based on contrastive learning, which can accurately predict the popularity of messages on social media platforms.

[0047] like Figure 2 The diagram illustrates the framework of the method in this embodiment, which includes three processes: obtaining the relationships between messages, modeling the message cascading propagation process, and comparative learning of the global graph. These three steps are described in detail below:

[0048] (1) Obtaining the relationships between messages.

[0049] Given a specific message k and its forwarding sequence within time T and Where, N k,T This represents the number of times message k is forwarded within time T, where t represents the forwarding time. k,T Let C represent the cascading propagation sequence of message k among users, and u represent the user who forwarded the message. This is achieved through the forwarding sequence C. k,T and V k,T The interaction matrix A between users and messages can be obtained. ij ) m×n Where m and n represent the total number of messages and users, respectively, and a ij Let A represent the number of interactions between user j and message i, that is, the number of times user j forwards message i; let matrix A be denoted as graph G. u-n =(V u-n E u-n If G is the adjacency matrix of ), then G u-n Defined as a user and message interaction relationship graph, V u-n Let E be the set of message vertices and user vertices in the graph. u-n For graph G u-n The set of edges represents the relationship between users and messages.

[0050] From Figure G u-n As can be seen from the construction process, the message vertex and the user vertex are divided into two vertex sets, and there are no edges within the two vertex sets. Therefore, G u-n This is a two-part graph. Since the popularity prediction problem is applied to different messages, and messages are associated with each other through users, in graph G... u-n In the graph G, message vertices are indirectly connected through a common user. To more directly explore the relationships between different messages, we will use a bipartite graph G. u-n Projected onto a graph G containing only message vertices n =(V n E n In this projection method, messages are treated as vertices, and the number of edges between different messages is determined by the number of times they are forwarded by the same user. Thus, we obtain a message global graph G representing the relationships between different messages. n .

[0051] (2) Modeling of message cascading propagation process.

[0052] In social networks, as time progresses, information publishers forward information to other users, who then selectively forward content they are interested in to even more users. Information spreads among users in a cascading diffusion manner, eventually forming a cascading propagation graph. We model the dynamic changes of this cascading graph. For each specific message n, we consider its cascading diffusion among users over time. Each time a message is forwarded by a user (denoted as a time step), the cascading graph is updated, thus obtaining the cascading propagation graph sequence G of message n within time T. n,T ={g n,1 ,g n,2 ,…,g n,T}, where g n,i =(V n,i E n,i V represents the cascade graph of message n at time i. n,i E is the set of user vertices. n,i This indicates their forwarding relationship.

[0053] Node2Vec is an algorithmic framework for learning continuous feature representations of nodes in a network, capable of mapping nodes in a graph to a low-dimensional feature space. Node2Vec employs a flexible neighborhood sampling strategy, considering both depth-first and breadth-first sampling strategies. Given a source node u, it simulates a random walk of fixed length l, c i The i-th node in the walk is generated by the following distribution:

[0054]

[0055] Where π vx Z is the unnormalized transition probability between nodes v and x, Z is the normalization constant, and E represents the set of two nodes with a forwarding relationship.

[0056] Node2Vec defines a second-order random walk, using parameters p and q to control the direction of the walk. Assuming a random walk starts from vertex t, passes through edge (t, v) to vertex v, and then the transition probability from vertex v to vertex x is defined as follows: π vx =α pq (t,x)·ω vx , where ω vx It is the weight of the edge between vertices v and x, α pq (t,x) is the multiplier factor, controlled by the hyperparameter p, which controls the probability of revisiting a previously visited vertex, and by the hyperparameter q, which controls whether to perform depth-first or breadth-first sampling.

[0057]

[0058] dtx This represents the length of the shortest path between vertex t and vertex x. Node2Vec, through its flexible and controllable neighborhood search strategy, is able to learn the structure of complex networks very well.

[0059] For large-scale, complex networks, directly processing cascaded graph sequences using neural networks would be computationally expensive; compared to graph data, processing vector data requires far fewer resources and computational power. This method uses Node2Vec to map the nodes in the cascaded graph to a low-dimensional space, thus transforming the cascaded graph g... n,i Transform into the feature vector f corresponding to message n n,i Thus, the time series f of the feature vectors of message n within time T is obtained. n ={f n,1 ,f n,2 ,…,f n,T}

[0060] Due to the structural complexity of social networks and the unpredictability of sudden events, the popularity of messages can change drastically in a short period of time. The Gated Recurrent Unit (GRU), a variant of the traditional Convolutional Neural Network (RNN), like the Long Short-Term Memory (LSTM) network, effectively alleviates the vanishing and exploding gradient problems in time series due to its special gate structure. It performs well on long sequences and has a simpler structure and fewer parameters compared to LSTM. Therefore, here we will use the time series f of the message feature vector. n The input is given to a gated recurrent unit (GRU), and the output is the cascaded propagation feature vector h corresponding to message n. n :

[0061] h n =GRU(f n )

[0062] In this way, we learn the propagation characteristics of each message, which is the feature representation of each message vertex in the global graph.

[0063] (3) Comparative learning of global graphs.

[0064] Graph Convolutional Neural Networks (GCNs) utilize the structure of graphs to obtain features of nodes in the neighborhood through convolution, exhibiting strong learning capabilities for graph representations. In (1), we have already obtained the global graph G that describes the relationships between different messages. n Global graph G n Each message is treated as a node, G n The cascading propagation characteristics h of each message n Also given in (2). h is the global graph G nAn abstract representation of the combination of feature vectors of all nodes in a given set can be understood as multiple h... n A high-dimensional vector composed of the global graph G. n This represents the relationship between different messages, and is also its adjacency matrix representation graph; Graph G n And Figure G n The features h of all nodes in the graph are input together into the GCN to obtain the high-order representation H of the nodes in the global graph, that is:

[0065] H = GCN(G n ,h)

[0066] H is the output of GCN, containing the embedded representations of all message nodes, in the cascading propagation feature h. n It incorporates the relationship features between message nodes on the basis of the previous one.

[0067] The popularity of a message is a relative concept. More specifically, only by comparing the popularity of different messages can we determine which message is more popular. Furthermore, messages with higher popularity are more likely to become even more popular in the future because they reach more users; conversely, messages with lower popularity will eventually become less popular due to their limited reach. We use comparative learning to capture this intuitive awareness.

[0068] We first propose a sampling strategy. We first coarsely classify the popularity of messages into three categories, h, m, and l, from highest to lowest popularity. Given a set of anchor nodes 'a', for each message in 'a', we randomly select messages belonging to the same category as the message with the same popularity as the message and add them to the set 'pp'. Simultaneously, we randomly select messages belonging to a different category as the message with the same popularity as the message and add them to the set 'nn'. In this implementation, we use labels to coarsely classify the popularity of messages; messages with higher popularity are more likely to become more popular in the future. The specific implementation process is detailed in Algorithm 1.

[0069]

[0070] Then, we designed a contrastive loss function L CON The calculation method is as follows:

[0071]

[0072] The degree of matching between positive sample nodes and anchor nodes pos =<h(a),h(pp)> The degree of matching between negative sample nodes and anchor nodes, neg =<h(a),h(nn)> ,<A,B> Let represent the vector dot product of A and B, and h(a), h(pp), and h(nn) represent the feature embeddings corresponding to the anchor node set a, the positive sample set pp, and the negative sample set nn, respectively; ones is a matrix of all 1s. Here, the pos and neg functions describe the matching degree between positive sample nodes, negative sample nodes, and anchor nodes, respectively. By minimizing the contrastive loss, contrastive learning between different messages can be achieved.

[0073] The loss function L is composed of the mean squared error (MSE) and the loss L. MSE And contrast loss L CON It consists of two parts, namely

[0074] L = L MSE +β·L Con

[0075]

[0076] in, y is the predicted value of popularity. i is the actual value of popularity, m is the total number of messages, and β is the preset weight parameter.

[0077] The higher-order representation H is processed by a multilayer perceptron (MLP) to obtain the final popularity value. Right now

[0078]

[0079] Finally, this model was trained on real datasets from multiple social platforms and used to predict the popularity of messages in new social networks.

[0080] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for predicting the popularity of online content based on contrastive learning, characterized in that, Includes the following steps: S1. Obtain the interaction matrix between users and messages by the forwarding sequence of messages within a set time and their cascading propagation sequence among users. Each element in the interaction matrix represents the number of times the corresponding user forwards the corresponding message, thereby obtaining a bipartite graph of the interaction relationship between messages and users. Projecting a bipartite graph onto a global graph: The global graph consists of a set of vertices formed by messages, and the number of edges between vertices of different messages is determined by the number of times they are forwarded by the same user; S2. Utilize the cascading propagation sequence of messages among users to obtain the propagation order of messages among users within a set time period, thereby obtaining the cascading propagation graph sequence; use a graph representation algorithm to transform each propagation cascade graph in the cascading propagation graph sequence into a feature vector of the corresponding time series, thereby obtaining the feature vector of the time series within the set time period. The feature vector of the time series is input into the gated loop unit to obtain the cascaded propagation feature vector corresponding to the message. S3. The cascaded propagation feature vectors of all messages are input into the global graph convolutional network (GCN) to obtain the high-order representation of the message nodes in the global graph. The high-order representation is then passed through a multilayer perceptron (MLP) to obtain the final predicted value of the message popularity. In the training of Graph Convolutional Network (GCN) and Multilayer Perceptron (MLP), the loss function L is determined by the mean squared error (MSE). MSE And contrast loss L Con It consists of two parts: L=L MSE +β·L Con L MSE For mean square error loss, L CON To compare the loss functions, β is the weight parameter; Contrast loss function L CON The calculation method is as follows: Where pos represents the matching degree between the positive sample node and the anchor node, pos =<h(a),h(pp)> Neg represents the degree of matching between the negative sample node and the anchor node.<h(a),h(nn)> , <,> denote the vector inner product, h(a), h(pp), h(nn) represent the feature embeddings corresponding to the anchor node set a, the positive sample node set pp, and the negative sample node set nn, respectively, and ones is a matrix of all 1s.

2. The method as described in claim 1, characterized in that, Mean square error loss L MSE : in Let y be the predicted popularity value of the i-th message. i Let m be the actual popularity value of the i-th message, and m be the total number of messages.

3. The method as described in claim 1, characterized in that, The samples used for training the Graph Convolutional Network (GCN) and the Multilayer Perceptron (MLP) were obtained in the following way: Given a set of anchor nodes a, for each message in the set of anchor nodes a, randomly select messages whose popularity belongs to the same category as the anchor node a as positive samples and add them to the set pp; at the same time, randomly select messages whose popularity does not belong to the same category as the anchor node a as negative samples and add them to the set nn.

4. The method as described in claim 1, characterized in that, The graph representation algorithm in step S2 is Node2Vec.

Images