Information diffusion prediction method based on community Transformer model

Abstract

The present invention discloses an information diffusion prediction method based on a community Transformer model, comprising: S1: inputting an adjacency matrix of a user social network graph and an information diffusion graph at time intervals; S2: identifying the community structure in the user social network; S3: calculating the multi-head attention of user node pairs; S4: using inter-layer gated residual connections; S5: obtaining user representations from the social network and the information diffusion graph; S6: performing context encoding and constructing a current diffusion sequence to predict future information diffusion cascades; the method of the present invention can more comprehensively and accurately capture the interaction preferences and diffusion behaviors between users, thereby improving the prediction performance.

Description

Technical Field

[0001] The present invention relates to the technical field of information prediction, and in particular to an information diffusion prediction method based on a community Transformer model. Background Art

[0002] Information cascade prediction is a research method that uses historical information cascades to predict future information dissemination behavior. With the development of social media and the internet, information dissemination has become an integral part of people's lives. Research on information cascade prediction is crucial for predicting and understanding the process of information dissemination, and can provide insights into social network marketing, news dissemination, information security, and other areas. A fundamental issue in information cascade prediction is how to measure the probability of information cascades. Predicting cascade probability requires considering multiple factors, such as the propagation path, propagation speed, and node influence. Researchers have employed various methods to measure the probability of information cascades, including random walk models, propagation tree models, and independent cascade models. The rapid development of deep learning in recent years has provided new insights and methods for information cascade prediction. A common approach is to use graph neural networks (GNNs) to model and predict information cascades. Compared to traditional machine learning methods, GNNs, by propagating information within a graph structure, can better capture the complex interactions between nodes. In existing technologies, although some methods can capture the interaction preferences and dissemination behaviors among users, they do not take into account that individuals are often influenced by the prevailing views and opinions within their communities. This limits the precise prediction of information dissemination paths, resulting in inaccurate predictions and limited prediction performance. Specifically, there are the following problems:

[0003] Lack of consideration for complex relationships and dynamic changes between nodes: Traditional probabilistic models and graph deep learning methods often struggle to account for the complex relationships and dynamic changes between nodes in social networks when predicting information cascades. This makes it difficult to accurately capture how individuals are influenced by the prevailing views and opinions in their communities, limiting the ability to accurately predict information dissemination paths.

[0004] Extensive focus on local information: Existing methods typically focus only on local information at the node level and fail to capture the interaction between community-level and global information. This limits the prediction performance of information cascade prediction and fails to fully grasp the propagation path and behavior of information throughout the social network.

[0005] Ignoring dynamic factors: Existing methods often ignore the dynamic changes in user preferences and time factors. This makes the prediction results inaccurate and unable to adapt to the actual changes and evolution in the process of information dissemination. Summary of the Invention

[0006] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide an information diffusion prediction method based on the community Transformer model.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] An information diffusion prediction method based on a community Transformer model includes the following steps:

[0009] S1: Input the user social network graph and the adjacency matrix of the information propagation graph at time intervals into the community-based Transformer model;

[0010] S2: The Louvain modularity optimization algorithm with resolution parameter restriction is used to identify community structures in user social networks as node attributes:

[0011]

[0012] in, and Represent the weighted degrees of the node’s external and internal connections, and They represent the total number of incoming and outgoing edges connected to the nodes in node C, and γ is the resolution parameter that controls the granularity of community division. It refers to the number of nodes in the network, also known as the size or scale of the network;

[0013] S3: Use node attributes to calculate multi-head attention for each user-node pair to capture relationships and long-range dependencies in graph-structured data;

[0014] S4: Use inter-layer gated residual connections to prevent over-smoothing of the model;

[0015] S5: Adopt heuristic strategies to obtain user representations from social networks and information diffusion graphs:

[0016]

[0017] The ⊙ operator represents the element-wise product, and then the resulting product is processed by the multi-layer perceptron, or MLP layer. Indicates time The user embed obtained, and Respectively refer to Embedding of user social networks and information diffusion graphs in the layer;

[0018] S6: Use a multi-head self-attention module for context encoding and use the learned user representation to construct the current propagation sequence to predict the future information propagation cascade:

[0019]

[0020] in H represents the number of attention heads, 、 、 ∈ is the parameter matrix, Q, K, V The matrices representing the correlation between nodes are obtained by multiplying the user node embedding with the query, key, and value weight matrices respectively; K T express K The transpose of the matrix is ​​used to calculate the similarity between the query and the key; d k The vector dimensions representing the query and key, used to scale the attention score; hi Represents the node representation obtained by the attention mechanism, which is obtained by converting the query ,key ,value Input into the attention function; Z represents the final node representation matrix.

[0021] Furthermore, the community-based Transformer model includes a community-aware heterogeneous graph transformer module, a time-enhanced attention module, and a multi-head attention-based predictor module. It can be applied to social media information dissemination prediction, disease spread prediction, and public opinion monitoring.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) The method of the present invention combines Graph Transformer with community information obtained through community detection algorithms to obtain representations of user social networks and information diffusion graphs, extract diverse interaction preferences and propagation behaviors within and between different user communities, and fuse them into heterogeneous graphs. This method can capture a variety of propagation behaviors and characteristics that may exist in different communities, and capture the interaction between community-level and global information, thereby improving prediction performance.

[0024] (2) The proposed method proposes a temporal enhanced attention module to enhance user representation and combines historical user representations to create a final user representation that captures the context-dependent nature of user dynamic preferences and ongoing diffusion paths. To enhance user representation, the module adopts multi-head attention, which improves the accuracy and robustness of user representation learning.

[0025] (3) The information diffusion prediction method based on the community Transformer model provided by the present invention can capture the interaction preferences and dissemination behaviors between users more comprehensively and accurately, and shows good performance in predicting the information dissemination path. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0027] Figure 1 It is a flow chart of the method of the present invention;

[0028] Figure 2 A detailed diagram of the workflow of the method of the present invention;

[0029] Figure 3 This is a structural diagram of the community-based Transformer model proposed in this invention. DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0031] Example 1

[0032] like Figures 1 and 2 Figure 1 shows an information diffusion prediction method based on the Community-Aware Transformer (CAT) model. Specifically, CAT extracts diverse interaction preferences and diffusion behaviors occurring within and across different user communities. CAT then employs Transformer-based message passing to capture global information and long-range dependencies in social and dynamic diffusion graphs. Furthermore, CAT incorporates dynamically changing user preferences into heterogeneous graphs via a temporally enhanced attention module. To enhance user representation, CAT employs multi-head attention to capture the contextual nature of ongoing diffusion paths.

[0033] The specific steps include:

[0034] S1: Input the user social network graph and the adjacency matrix of the information propagation graph at time intervals into the community-based Transformer model;

[0035] S2: The Louvain modularity optimization algorithm with resolution parameter restriction is used to identify community structures in user social networks as node attributes:

[0036]

[0037] in, and Represent the weighted degrees of the node’s external and internal connections, and They represent the total number of incoming and outgoing edges connected to the nodes in node C, respectively. γ is a resolution parameter that controls the granularity of community division, which refers to the number of nodes in the network, also known as the size or scale of the network.

[0038] S3: Use node attributes to calculate multi-head attention for each user-node pair to capture relationships and long-range dependencies in graph-structured data;

[0039] S4: Use inter-layer gated residual connections to prevent over-smoothing of the model;

[0040] S5: Adopt heuristic strategies to obtain user representations from social networks and information diffusion graphs:

[0041]

[0042] The ⊙ operator represents the element-wise product, and then the resulting product is processed by the multi-layer perceptron, or MLP layer. Indicates time The user embed obtained, and Respectively refer to Embedding of user social networks and information diffusion graphs in the layer;

[0043] S6: Use a multi-head self-attention module for context encoding and use the learned user representation to construct the current propagation sequence to predict the future information propagation cascade:

[0044]

[0045] in H represents the number of attention heads, 、 、 is the parameter matrix, Q, K, V The matrices representing the correlation between nodes are obtained by multiplying the user node embedding with the query, key, and value weight matrices respectively; K T express K The transpose of the matrix is ​​used to calculate the similarity between the query and the key; dk The vector dimensions representing the query and key, used to scale the attention score; hi Represents the node representation obtained by the attention mechanism, which is obtained by converting the query ,key ,value Input into the attention function; Z represents the final node representation matrix.

[0046] like Figure 3 As shown in FIG, the community-based Transformer model includes a community-aware heterogeneous GraphTransformer module, a time-enhanced attention module, and a multi-head attention-based predictor module.

[0047] Community-aware heterogeneous Graph Transformer module: The CAT model adopts a community-aware heterogeneous Graph Transformer module, which can effectively extract the diverse interaction preferences and propagation behaviors between and within different user communities.

[0048] Temporal-enhanced attention module: The CAT model introduces a temporal-enhanced attention module that takes into account dynamically changing user preferences and time factors. This enables the CAT model to better adapt to changes in user behavior during information dissemination, thereby improving prediction accuracy.

[0049] Multi-head attention-based predictor module: The CAT model uses a multi-head attention-based predictor module to capture the contextual nature of diffusion paths. Through this multi-head attention mechanism, the CAT model can better model relevant information in the diffusion path, thereby improving the accuracy of information diffusion prediction in social networks. Example

[0050] User Community Detection: To improve the generalization and robustness of the model, this paper proposes incorporating information about the community structure of the graph into the model. A community is a densely connected subset of nodes in a social graph, representing similarity or relatedness. After evaluating various community detection algorithms, this paper employs the Louvain modularity optimization algorithm with resolution constraints to identify the community structure of the network, and considers accurate detection of networks with overlapping social community structures. During message passing, CAT one-hot encodes the community to which each node's members belong and uses this as a node attribute.

[0051] Graph Multi-Head Attention: To capture relationships and long-range dependencies in graph-structured data, the graph multi-head attention mechanism is a useful tool. By allowing multiple attention heads to focus on different parts of the input graph simultaneously, it can effectively fuse information from different nodes and edges.

[0052] Temporal-enhanced attention module for enhanced user representation: First, the historical diffusion timeline is divided into n time periods to create n unique and diverse dynamic graphs. Each time period represents a snapshot of the diffusion graph. Then, the CAT method is used to extract user embeddings in each time period. CAT retrieves the user embedding corresponding to a specific user ID from all available embeddings to obtain a user embedding matrix. Finally, to obtain the final user embedding, the present invention proposes a temporal-enhanced attention module that combines historical user embeddings. Assuming that users forward information at timestamps, the time-sensitive attention mechanism is defined as follows:

[0053]

[0054] Among them, the function Used to generate time embeddings corresponding to time period IDs, which are initialized using a normal distribution. By embedding the user matrix With time embedding The final user embedding is Obtained by weighted summation.

[0055] The information diffusion prediction method based on the Community-Aware Transformer (CAT) model has the following applications:

[0056] Social Media Information Dissemination Prediction: CAT models are applied to social media information dissemination prediction. By modeling the interaction preferences and dissemination behaviors among social media users, information dissemination paths and effects can be predicted. For example, the popularity and spread rate of a topic on Twitter can be predicted, helping businesses and individuals better conduct social media marketing.

[0057] Disease spread prediction: CAT models are applied to disease spread prediction, predicting the path and speed of epidemic spread within social networks and providing targeted prevention and treatment plans. For example, CAT models can help predict the path and scale of virus spread within social networks, providing early warning and response plans.

[0058] Public opinion monitoring: Applying the CAT model to public opinion monitoring can predict the spread trends and influence of hot topics on social media, helping governments, businesses, and individuals better understand the public's attitudes and reactions to certain events or topics.

[0059] These implementation examples demonstrate the broad applicability and practicality of this invention, which can be used to predict and analyze information dissemination in a variety of fields and application scenarios. The CAT model provided by this invention can more comprehensively and accurately capture user interaction preferences and dissemination behaviors, and demonstrates excellent performance in predicting information dissemination paths, providing strong support for research and practice in related fields.

[0060] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for predicting information diffusion based on a community Transformer model, characterized in that: The following steps are involved: S1: Input the adjacency matrix of the user social network graph and the information diffusion graph at time intervals into the community-based Transformer model; S2: The Louvain modularity optimization algorithm with resolution parameter restriction is used to identify community structures in user social networks as node attributes: , in, and Represent the weighted degrees of the external and internal connections of a node, and They represent the total number of incoming and outgoing edges connected to the nodes in node C, γ is the resolution parameter that controls the granularity of community division, and m refers to the number of nodes in the network, also known as the size or scale of the network; S3: Use node attributes to calculate multi-head attention for each user-node pair to capture relationships and long-range dependencies in graph-structured data; S4: Use inter-layer gated residual connections to prevent over-smoothing of the model; S5: Adopt heuristic strategies to obtain user representation from social networks and information diffusion graphs: , The ⊙ operator represents the element-wise product, and then the resulting product is processed by a multi-layer perceptron, or MLP layer. Indicates at time The user embed obtained, and Respectively refer to Embedding of user social networks and information diffusion graphs in the layer; S6: Use multi-head self-attention modules for context encoding and use the learned user representations to construct the current propagation sequence to predict future information propagation cascades: , in H represents the number of attention heads, The parameter matrix, Q, K, V The matrices representing the correlation between nodes are obtained by multiplying the user node embedding with the query, key, and value weight matrices respectively; K T express K The transpose of the matrix is ​​used to calculate the similarity between the query and the key; d k The vector dimensions representing the query and key, used to scale the attention scores; hi represents the node representation obtained by the attention mechanism, which is obtained by converting the query ,key ,value Input to the attention function to calculate; represents the static embedding of the user; Z represents the final node representation matrix.

2. The information diffusion prediction method based on the community Transformer model according to claim 1 is characterized in that: The community-based Transformer model includes a community-aware heterogeneous Graph Transformer module, a time-enhanced attention module, and a multi-head attention-based predictor module.

3. The information diffusion prediction method based on the community Transformer model according to claim 1 is characterized in that: The information diffusion prediction method based on the community Transformer model can be applied to social media information propagation prediction, disease propagation prediction and public opinion monitoring.

Images