Information propagation prediction method based on co-attention fusion of sequential hypergraph neural network

Abstract

The present invention discloses an information propagation prediction method based on a sequence hypergraph neural network with common attention fusion, which includes three modules: a user feature learning module, which uses a graph convolutional neural network to learn user embedding; a cascade feature learning module, which constructs a sequence hypergraph neural network based on isovariable diffusion to capture complex cascade irregular connections, so as to learn the cascade embedding that encapsulates the internal and external features of the cascade; a feature fusion and prediction module, which learns the interdependence between cascade features and user features through a common attention mechanism, captures the complex relationship and mutual influence between them, and then calculates the possibility of potential user infection. The present invention realizes more effective information propagation prediction, comprehensively considers the external influence of other cascades, and combines a novel feature fusion technology based on stacking multiple common attention layers, thereby improving the prediction ability of information diffusion.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and information propagation prediction, and in particular to an information propagation prediction method based on a sequence hypergraph neural network with common attention fusion. Background Art

[0002] Information propagation prediction is an important research direction in the field of social network analysis. It uses factors such as cascade propagation relationships and user characteristics to study how information spreads between users and predict the next user to forward the message. With the widespread use of social media platforms, the mechanism of information propagation is becoming increasingly complex, which has a profound impact on the formulation of social marketing strategies, the identification of false information, and the guidance of public opinion.

[0003] At present, information diffusion prediction methods can be roughly divided into two categories: feature engineering-based methods and deep learning-based methods. Feature engineering-based methods design effective prediction models by analyzing factors such as the topological structure of social networks, user behavior patterns, and information content. Although such methods can provide interpretable prediction results to a certain extent, they often require a lot of manual participation and domain knowledge support, and have limitations when dealing with large-scale data and complex network structures. In recent years, with the rapid development of artificial intelligence technology, deep learning-based methods have begun to be widely used in the field of information diffusion prediction. These methods use the powerful representation learning ability of neural networks to automatically extract and learn complex features in data, avoiding tedious feature engineering. In particular, technologies such as graph neural networks, recursive neural networks, and attention mechanisms have been proven to have significant advantages in simulating the dynamics of information diffusion in social networks. In addition, emerging technologies such as hypergraph neural networks provide new possibilities for processing high-order complex relationships in social networks. Hypergraph neural networks extend neural networks to hypergraph structures, can process irregular data that cannot be represented as grids but can only be represented as graphs, can better simulate high-order relationships and group behaviors between users, and are of great significance for improving the accuracy of information diffusion prediction.

[0004] Existing deep learning-based methods mainly focus on mining the social influence and group homogeneity characteristics of early infected users from social network structures and historical cascade events, which often ignores the influence of external factors in the information dissemination process. Secondly, when integrating cascade representations with user preferences, current models tend to directly fuse features, ignoring the potential correlation between features, which may lead to information redundancy and reduce the prediction accuracy and generalization ability of the model. Therefore, the current field of information dissemination prediction needs more effective models to solve these challenges.

[0005] At present, the methods in the field of information diffusion prediction often ignore the influence of external information in the process of information diffusion, and the fusion of feature information is too simple and direct. The actual social network environment is full of many parallel information diffusion flows, in which not only the propagation mechanism within a single information flow is complex and changeable, but also there are many interactions and influences between the cross-propagation of different information flows. Therefore, it is particularly important to characterize these external factors and their influence on the information diffusion process, especially in the modeling of cascade relationships. Secondly, the direct fusion of the current model when integrating cascade representation and user preferences ignores the potential correlation between features, which may lead to information redundancy and reduce the prediction accuracy and generalization ability of the model. Based on these two points, the present invention constructs a sequence hypergraph neural network based on equivariance diffusion, regards each information propagation as a hyperedge, and learns the correlation between hyperedges. And combined with a common attention mechanism based on stacking multiple common attention layers, learn the interdependence between cascade features and user features, and capture the complex relationship and mutual influence between them. Summary of the invention

[0006] Purpose of the invention: The present invention provides an information propagation prediction method of a sequential hypergraph neural network based on common attention fusion to solve the deficiencies of the above-mentioned prior art.

[0007] Technical solution: An information propagation prediction method based on a sequence hypergraph neural network with common attention fusion, comprising the following steps:

[0008] S1. User feature learning

[0009] A social network graph is constructed based on the known information of user batch attention. Considering that the structure of the user social network is relatively stable, a two-layer graph convolutional network is used to learn the user's embedded representation, and finally a static representation of all users is obtained.

[0010] S2. Cascade feature learning

[0011] A hypergraph is used to construct a propagation cascade graph. Each piece of information propagation is regarded as a hyperedge. Considering that the influence between information is usually short-term related, the previous cascades are decomposed into several subsets based on timestamps, and a hypergraph is constructed for each subset. A hypergraph neural network based on equivariance diffusion is used to obtain the cascade embedding of each subset, and the cascade embeddings of different timestamps are connected through the residual gating mechanism.

[0012] S3. Feature fusion and prediction

[0013] First, user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where he,t Represents the cascade embeddings obtained at different timestamps in step S2, and then stacks two co-attention layers to learn the interdependence between cascade features and user features through the co-attention mechanism, capture the complex relationship and mutual influence between them, and then calculate the possibility of potential user infection.

[0014] Furthermore, in user feature learning, the layer-by-layer propagation rule of the graph convolutional network is:

[0015]

[0016] Where W F is a trainable weight matrix, and They are the self-circulating social network graph G F The adjacency matrix and degree matrix of .

[0017] Furthermore, step S2 includes constructing a hypergraph neural network based on equivariance diffusion, specifically including:

[0018] (1) Node to hyperedge

[0019] For each node on the hypergraph, firstly transform its node features through a multi-layer perceptron, then aggregate the node features to the associated hyperedges to obtain the hyperedge representation The specific form is as follows:

[0020]

[0021]

[0022] in, is a multilayer perceptron. The multilayer perceptrons in this model share parameters. is the node feature, the initial feature

[0023] (2) Hyperedge to node

[0024] The hyperedge is represented as Broadcast to relevant nodes and calculate the information from the hyperedge to the node through the multi-layer perceptron The specific form is as follows: is a multilayer perceptron:

[0025]

[0026] (3) Update node features

[0027] For each node in the hypergraph, update the node’s features using a multilayer perceptron The specific form is as follows:

[0028]

[0029] in is a multilayer perceptron, d v Represents the degree information of the node;

[0030] (4) Update hyperedge features

[0031] For each hyperedge, the updated node features are aggregated into the hyperedge, and then a multi-layer perceptron is used Update Hyperedge Features The specific form is as follows:

[0032]

[0033] In addition, step S2 includes introducing a residual gating mechanism to propagate residual information between different timestamps. The mechanism generates the initial embedding of each node by combining the dynamic embedding and user embedding of the node in each time period, including the following calculations:

[0034] The initial embedding of a node at timestamp t+1 can be calculated as:

[0035]

[0036] Where W R and z R denote the transformation matrix and vector of the gating mechanism respectively, while σ(·) denotes the tanh function, using the user representation learned from the user feature learning module As the user's initial embedding, Represents the hypergraph node features obtained by the hypergraph neural network based on equivariance diffusion. The value g calculated by the gating function is used to control the percentage of residual information retained. The hypergraph is sequentially connected through this residual gating mechanism.

[0037] Furthermore, the feature fusion in step S3 is gradual, and the output of each layer is used as the input of the next layer; in the first joint attention layer, the user embedding is merged and cascade embedding C D To generate fused embedding Then, in the second joint attention layer, C D and Merge to generate the final fused embedding R CA To further enhance the fusion; the output vector of each common attention layer is d-dimensional;

[0038] Here is the calculation process of the first joint attention layer:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] represents the result of user embedding after being processed by the joint attention mechanism, MA represents the attention mechanism, which takes into account the cascade embedding C D Embedded in user The impact of. Yes The output of is the result of applying a feed-forward neural network, FFN is a feed-forward neural network used to increase nonlinearity and the ability to capture complex patterns. Represents the result of the cascade embedding after the common attention mechanism, which takes into account the user embedding Embed C in Cascade D The impact of. Yes The output of the feed-forward neural network. is the final output of the first co-attention layer, which will be passed to the next co-attention layer. is the projection matrix of the first joint attention layer, represents the concatenation of vectors, It is converted to a d+1-dimensional representation before being input into the next CA layer.

[0045] Furthermore, step S3 for feature fusion and prediction includes calculating the user's diffusion probability by the following method:

[0046]

[0047] Where W p Is R CA Transformation matrix mapped to user-specific space, using Mask m To block users who have been activated before the prediction.

[0048] Combined with the implementation of the above method, the method constructs a user feature learning module, a cascade feature learning module and a feature fusion and prediction module, specifically:

[0049] User feature learning module, which uses graph convolutional neural network to obtain user embedding;

[0050] Cascade feature learning module, used to obtain cascade embeddings that encapsulate cascade internal and external features;

[0051] The feature fusion and prediction module uses the cascade embedding query process and the two-layer joint attention mechanism to obtain the fused features of the cascade embedding and the user embedding, and then calculates the possibility of potential user propagation.

[0052] The method includes a feature fusion layer and propagation prediction in a feature fusion and prediction module. The feature fusion layer includes a cascade embedding query process and a two-layer joint attention mechanism, which avoids information leakage by carefully selecting pre-attend embedding; the joint attention layer has attention flows from user to cascade and from cascade to user, which realizes subtle integration of user preferences and cascade dynamics, and utilizes the interdependence of the two to enhance the prediction ability of the model.

[0053] Furthermore, in the cascade embedding query process, for a given target cascade, the representation of the cascade is read in the most recent time interval before the user participates in the cascade, thereby reducing the risk of information leakage. This process requires the user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where h e,t is the concatenated embedding obtained at different timestamps in step S2;

[0054] The joint attention block operates by utilizing the query from one modality and the keys and values ​​from the other modality, with the query matrix serving as the residual information after the multi-head attention sub-layer.

[0055] Beneficial effects: The present invention achieves more effective prediction of information diffusion, comprehensively considers the external influence of other cascades, and combines a novel feature fusion technology based on stacking multiple common attention layers, thereby improving the prediction ability of information diffusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a diagram of the framework of a sequence hypergraph neural network based on common attention fusion;

[0057] Figure 2 is the joint attention block diagram;

[0058] Figure 3 These are the experimental results of the comparative experiment conducted by the present invention on four data sets with different training ratios, where: (a) corresponds to the Twitter data set, (b) corresponds to the Douban data set, (c) corresponds to the Android data set, and (d) corresponds to the Christianity data set. DETAILED DESCRIPTION

[0059] To illustrate the technical solution provided by the present invention in detail, a further introduction is given below in conjunction with the accompanying drawings.

[0060] The present invention discloses an information propagation prediction method based on a sequence hypergraph neural network with common attention fusion, which is used to predict information propagation. The method comprises:

[0061] User feature learning module, which uses graph convolutional neural network to learn user embedding;

[0062] Cascade feature learning module, constructs a sequence hypergraph neural network based on isovariable diffusion to capture complex cascade irregular connections, so as to learn the cascade embedding that encapsulates the internal and external features of the cascade;

[0063] The feature fusion and prediction module learns the interdependencies between cascade features and user features through a joint attention mechanism, captures the complex relationships and mutual influences between them, and then calculates the probability of potential user infection.

[0064] The present invention achieves more effective information propagation prediction, comprehensively considers the external influence of other cascades, and combines a novel feature fusion technology based on stacking multiple common attention layers, thereby improving the prediction ability of information diffusion.

[0065] Specifically, the method described in the present invention can be implemented according to the following steps.

[0066] An information propagation prediction method based on a sequence hypergraph neural network with common attention fusion, characterized by comprising the following steps:

[0067] S1. User feature learning (user feature learning module)

[0068] A social network graph is constructed based on the known information of user batch attention. Considering that the structure of the user social network is relatively stable, a two-layer graph convolutional network is used to learn the user's embedded representation to obtain the static representation of all users.

[0069] Specifically, a two-layer graph convolutional network is used to learn the embedded representation of users, and finally the static representation of all users is obtained.

[0070] The layer-by-layer propagation rule of the graph convolutional network is:

[0071]

[0072] Where W F is a trainable weight matrix, and They are the self-circulating social network graph G F The adjacency matrix and degree matrix of .

[0073] S2. Cascade feature learning (cascade feature learning module)

[0074] A hypergraph is used to construct a propagation cascade graph. Each piece of information propagation is regarded as a hyperedge. Considering that the influence between information is usually short-term related, the previous cascades are decomposed into several subsets based on timestamps, and a hypergraph is constructed for each subset. A hypergraph neural network based on equivariance diffusion is used to obtain the cascade embedding of each subset, and the cascade embeddings of different timestamps are connected through the residual gating mechanism.

[0075] The implementation steps of the hypergraph neural network based on equivariance diffusion and the residual gating mechanism are introduced in detail below.

[0076] Hypergraph Neural Network Based on Equivariance Diffusion

[0077] (1) Node to hyperedge

[0078] In this step, for each node on the hypergraph, its node features are first transformed through a multi-layer perceptron. Then, the node features are aggregated to the associated hyperedges to obtain the hyperedge representation The specific form is as follows:

[0079]

[0080]

[0081] in, is a multilayer perceptron, and the multilayer perceptrons in this model share parameters. is the node feature, the initial feature

[0082] (2) Hyperedge to node

[0083] This step achieves equivariance. Broadcast to relevant nodes and calculate the information from the hyperedge to the node through the multi-layer perceptron The specific form is as follows: is a multilayer perceptron:

[0084]

[0085] (3) Update node features

[0086] For each node on the hypergraph, a multilayer perceptron is used to update the node’s features based on the node’s features, the information from the hyperedge to the node, the node’s input features, and the node’s degree information. The specific form is as follows:

[0087]

[0088] in is a multilayer perceptron, d vRepresents the degree information of a node.

[0089] (4) Update hyperedge features

[0090] For each hyperedge, the updated node features are aggregated into the hyperedge, and then a multi-layer perceptron is used Update Hyperedge Features The specific form is as follows:

[0091]

[0092] in, and All represent multi-layer perceptrons, which are used here for the convenience of different expressions and calculations of each layer.

[0093] (5) Residual gating mechanism

[0094] In order to propagate residual information between different timestamps, the present invention introduces a residual gating mechanism. This mechanism generates the initial embedding of each node by combining the dynamic embedding and user embedding of the node in each time period. The initial embedding of the node at timestamp t+1 can be calculated as:

[0095]

[0096] Where W R and z R denote the transformation matrix and vector of the gating mechanism, respectively, and σ(·) denotes the tanh function. Using the user representation learned from the user feature learning module As the user's initial embedding, Represents the hypergraph node features obtained by the hypergraph neural network based on equivariance diffusion. The value g calculated by the gating function is used to control the percentage of residual information retained. The hypergraph is sequentially connected through this residual gating mechanism.

[0097] S3, feature fusion and prediction (feature fusion and prediction module)

[0098] First, user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where h e,t It is the cascade embedding obtained at different timestamps in step S2, and then two co-attention layers are stacked to learn the interdependence between cascade features and user features through the co-attention mechanism, capture the complex relationship and mutual influence between them, and then calculate the possibility of potential user infection.

[0099] This module includes feature fusion layer and propagation prediction. The specific implementation of each part will be introduced below.

[0100] (1) Feature Fusion Layer

[0101] The feature fusion layer includes a cascade embedding query process and a two-layer co-attention mechanism. This design avoids information leakage by carefully selecting pre-attended embeddings. The co-attention layer has user-to-cascade and cascade-to-user attention flows, which enables subtle integration of user preferences and cascade dynamics, effectively exploiting their interdependencies to enhance the model's predictive power.

[0102] (2) Cascading embedded query

[0103] In order to emphasize the interactions captured in the cascade, the present invention implements a cascade embedding query mechanism. Unlike methods that only utilize the embedding of the last activated cascade, the present invention reads the embeddings of all activated cascades to represent the target cascade. Specifically, for a given target cascade, the representation of the cascade is read in the most recent time interval before the user participated in the cascade, thereby reducing the risk of information leakage. This process requires the user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where h e,t is the concatenated embedding obtained at different timestamps in step S2.

[0104] 1) Joint Attention Mechanism

[0105] The joint attention block operates by leveraging the query from one modality and the key and value from the other modality. The query matrix serves as the residual information after the multi-head attention sub-layer. When the query (Q) originates from the user, and the key (K) as well as the value (V) come from the cascade, the attention value calculated using the query and key can measure the similarity between the user and the cascade. Therefore, the cascade is weighted accordingly, enabling more precise focus on the user region related to the cascade and understanding the interdependencies between various cascade embeddings and user embeddings.

[0106] The co-attention layer connects two co-attention blocks in parallel, each of which is tasked with processing a different set of features. Each co-attention block computes the query, key, and value separately. Subsequently, the key and value from one co-attention block are provided as input to the other, enabling information exchange between the two blocks. The outputs of the two co-attention blocks are concatenated and processed through a fully connected layer to produce a fused representation. This layer effectively models the dense interaction between different modalities by facilitating information exchange.

[0107] In order to achieve deep integration of user features and cascade features, the present invention stacks two co-attention layers. This fusion process is gradual, and the output of each layer is used as the input of the next layer. For example, in the first co-attention layer, the user embedding and cascade embedding C D To generate fused embedding Then, in the second joint attention layer, C D and Merge to generate the final fused embedding R CA To further enhance the fusion. The output vector of each common attention layer is d-dimensional.

[0108] Here is the calculation process of the first joint attention layer:

[0109]

[0110]

[0111]

[0112]

[0113]

[0114] represents the result of user embedding after being processed by the joint attention mechanism, MA represents the attention mechanism, which takes into account the cascade embedding C D Embedded in user The impact of. Yes The output of is the result of applying a feed-forward neural network, FFN is a feed-forward neural network used to increase nonlinearity and the ability to capture complex patterns. Represents the result of the cascade embedding after the common attention mechanism, which takes into account the user embedding Embed C in Cascade D The impact of. Yes The output of the feed-forward neural network. is the final output of the first co-attention layer, which will be passed to the next co-attention layer. is the projection matrix of the first joint attention layer, represents the concatenation of vectors, It is converted to a d+1-dimensional representation before being input into the next CA layer.

[0115] 2) Propagation prediction

[0116] The present invention calculates the diffusion probability of a user by the following method:

[0117]

[0118] Where W p Is R CA Transformation matrix mapped to user-specific space, using Mask m To block users who have been activated before prediction, that is, if user u i Participate in cascade c in step j m , and Training with cross entropy loss:

[0119]

[0120] Where θ represents all the parameters that need to be learned in the model. i Participate in cascade $c at step j m ,y ji =1, otherwise y ji =0.

[0121] The technical effect of the present invention is further verified through experiments.

[0122] The present invention was implemented in PyTorch 2.1.2 on an NVIDIA GeForce RTX 4060 GPU. For all datasets, the training set, validation set, and test set were divided into 8:1:1. The model was optimized using the Adam optimizer and the learning rate was set to 0.001. The Dropout rate was set to 0.3. The batch size was set to 64, and the embedding size was also set to 64. The number of layers of the three MLPs in the model was set to 1, and the hidden dimension was 128; the number of layers of the classifier was set to 2, and the hidden dimension was 96. The depth of the common attention layer was 2. After experimental verification, the Twitter and Douban datasets were finally divided into 8 time blocks; the Android and Christianity datasets were divided into 3 time blocks. For the baseline model, the settings provided in the original model were retained.

[0123] To verify the universality and effectiveness of the model, the present invention has conducted experiments in three different application scenarios, including Twitter and Douban datasets and question-and-answer data from Android forums. The specific situation of the dataset is summarized in Table 1.

[0124] Table 1: Dataset details

[0125] Dataset user connect cascade Average length Twitter 12627 309631 3442 32.60 Douban 12232 396580 3475 21.76 Android 9958 48573 679 33.3

[0126] Twitter: We extracted the propagation paths and timestamps of tweets posted on Twitter during October 2010, and created a social relationship network based on follow relationships.

[0127] Douban: It is mainly extracted from the books shared by users on Douban. If users participate in the same discussion more than 20 times, they are considered friends and a social relationship network is built from this.

[0128] Android: Mainly comes from the question-and-answer interactions on the StackExchange community, including user questions, answers, and other interactions. These interactions form a social network among users.

[0129] Information propagation prediction is regarded as a retrieval task, and the candidate set is obtained by predicting the propagation probability of all candidate users, and then sorted according to the predicted probability. Therefore, the present invention selects Hits@K and Map@K as evaluation indicators.

[0130] 1) Hits@K: The infection probability ratio associated with the top K actual infected users.

[0131] 2) Map@K: considers the presence of actual users and their specific ranking positions in the ranking prediction results to evaluate specific ranking.

[0132] The model is compared with several classic information diffusion prediction methods, including:

[0133] DeepDiffuse: Using recurrent neural networks and attention mechanisms to predict potentially infected users and infection times in social networks based on timestamp data.

[0134] Topo-LSTM: aims to enhance the propagation model by considering the social relationships between users and extends the standard LSTM model.

[0135] NDM: A micro-cascade model is established based on the relaxation assumption, combining the attention mechanism and convolutional neural network to alleviate the long-term dependency problem.

[0136] SNIDSA: A structural attention module is defined to introduce structural features, and a recurrent neural network is subsequently used to model and predict the propagation.

[0137] FOREST: Predicting multi-scale propagation from both micro and macro perspectives, adopts a novel context extraction algorithm based on recurrent neural networks to better utilize user information embedded in social network graphs.

[0138] Inf-VAE: We selectively exploit social variables using graph neural networks based on user social relationships and design a novel fusion network to learn social and temporal variables.

[0139] DyHGCN: We construct a heterogeneous graph based on user follow and forwarding relationships, and then use a graph convolutional network to learn user representations.

[0140] MSHGAT: Dynamically capture user preferences using a sequential hypergraph attention network enhanced with memory and integrate them with user social graph representation.

[0141] Tables 2, 3, 4, and 5 show the comprehensive performance of our model and the baseline model. It is obvious from the results that our model significantly outperforms the existing baseline models in terms of prediction performance. Compared with other models, our model uses hypergraphs to capture the internal and external influences of cascades, and introduces a joint attention mechanism to effectively fuse user embeddings and cascade embeddings, thereby reducing redundancy. Therefore, on datasets with rich cascade information such as Twitter and Douban, our model comprehensively captures the cascade information and performs better in feature fusion. Compared with the state-of-the-art models, Hits@100 is improved by 16% and MAP@100 is improved by 18%. In addition, on datasets with less cascade information such as Android and Christianity, our model also shows improvements, with Hits@100 increasing by 2% and MAP@100 increasing by 1%. This shows that our model performs well in handling complex information propagation scenarios and can also maintain good performance in scenarios with relatively simple cascade propagation.

[0142] Table 2: Experimental results on Twitter dataset

[0143]

[0144]

[0145] Table 3: Experimental results on the Douban dataset

[0146]

[0147] Table 4: Experimental results on the Android dataset

[0148]

[0149] Table 5: Experimental results on the Christianity dataset

[0150]

[0151]

[0152] In order to evaluate the contribution of each module of the present invention, the following ablation studies are performed on different parts of the present model:

[0153] w / o EDHNN: Replace the equivariant diffusion based hypergraph neural network with a hypergraph neural network.

[0154] w / o CA: Replace the co-attention mechanism with a gated fusion strategy.

[0155] w / o UE: Ignore user embedding.

[0156] w / o CE: Ignore cascade embeddings.

[0157] Observing Tables 6 and 7, the proposed model shows good rationality. First, it is observed that when the user embedding captured by the social relationship graph is removed, the prediction performance drops significantly. This shows the impact of user features in social networks on the effectiveness of the prediction model. Similarly, removing the cascade embedding extracted from the propagation cascade hypergraph also leads to a significant drop in prediction performance, further demonstrating the effectiveness of integrating the two feature representations. In addition, when the hypergraph neural network based on equivariance diffusion is replaced by the hypergraph neural network, a significant drop in prediction performance can be noticed on the Twitter and Douban datasets, and a slight drop is observed on the Android and Christian datasets. Considering the relatively sparse cascade information in the Android and Christian datasets, this reflects that the hypergraph neural network based on equivariance diffusion is able to comprehensively learn the internal and external effects of cascade information, enabling it to better improve the prediction performance when dealing with complex cascade information datasets. Finally, it is observed that after removing the co-attention fusion module, the prediction performance drops significantly, especially on the Twitter and Douban datasets. This shows that when the quality of the cascade embedding is high, the introduction of the co-attention mechanism can better fuse user and cascade features.

[0158] Table 6: Ablation experiments on Twitter and Douban datasets

[0159]

[0160]

[0161] Table 7: Ablation experiments on Android and Christian datasets

[0162]

[0163] In the field of information propagation prediction, an outstanding propagation prediction model must show stable and excellent performance on data sets of different qualities. Therefore, the present invention conducts a series of comparative experiments on four data sets with different training ratios to strictly evaluate the effectiveness and progress of the present model. Figure 3 As shown, the experimental results show that compared with the performance level achieved by other models using 90% training data, our model is able to achieve comparable results using only 60% training data. It is worth noting that even compared with the most advanced model MS-HGAT, our model is able to achieve similar results to MS-HGAT using 90% training data when the training data is reduced to 70%. This finding not only highlights the overall advantages of the hypergraph neural network based on equivariance diffusion in cascade feature extraction, but also emphasizes the efficiency of the joint attention mechanism in the fusion of user embedding and cascade embedding.

Claims

1. An information propagation prediction method based on a sequence hypergraph neural network with common attention fusion, characterized in that: The steps include: S1. User feature learning A social network graph is constructed based on the known information of user batch attention. Considering that the structure of the user social network is relatively stable, a two-layer graph convolutional network is used to learn the user's embedded representation to obtain the static representation of all users. S2. Cascade feature learning A hypergraph is used to construct a propagation cascade graph. Each piece of information propagation is regarded as a hyperedge. Considering that the influence between information is usually short-term related, the previous cascades are decomposed into several subsets based on timestamps, and a hypergraph is constructed for each subset. A hypergraph neural network based on equivariance diffusion is used to obtain the cascade embedding of each subset, and the cascade embeddings of different timestamps are connected through the residual gating mechanism. S3. Feature fusion and prediction First, user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where h e,t It is the cascade embedding obtained at different timestamps in step S2, and then two co-attention layers are stacked to learn the interdependence between cascade features and user features through the co-attention mechanism, capture the complex relationship and mutual influence between them, and then calculate the probability of potential user infection; The feature fusion is gradual, and the output of each layer is used as the input of the next layer; In the first joint attention layer, user embeddings are merged and cascade embedding C D To generate fused embedding Then, in the second joint attention layer, C D and Merge to generate the final fused R CA To further enhance the fusion; the output vector of each common attention layer is d-dimensional; Here is the calculation process of the first joint attention layer: represents the result of user embedding after being processed by the joint attention mechanism, MA represents the attention mechanism, which takes into account the cascade embedding C D Embedded in user The impact of Yes The result after applying a feed-forward neural network to the output of FFN, which is a feed-forward neural network used to increase nonlinearity and the ability to capture complex patterns; Represents the result of the cascade embedding after the common attention mechanism, which takes into account the user embedding Embed C in Cascade D The impact of Yes The result after applying the feedforward neural network to the output of is the final output of the first joint attention layer, which will be passed to the next joint attention layer; is the projection matrix of the first joint attention layer, represents the concatenation of vectors, It is converted to a d+1-dimensional representation before being input into the next CA layer.

2. The information propagation prediction method based on the common attention fusion sequence hypergraph neural network according to claim 1 is characterized in that: In user feature learning, the layer-by-layer propagation rule of the graph convolutional network is: Where W F is a trainable weight matrix, A F and They are the self-circulating social network graph G F The adjacency matrix and degree matrix of .

3. The information propagation prediction method based on the common attention fusion sequence hypergraph neural network according to claim 1 is characterized in that: Step S2 includes constructing a hypergraph neural network based on equivariance diffusion, specifically including: (1) Node to hyperedge For each node on the hypergraph, firstly transform its node features through a multi-layer perceptron, then aggregate the node features to the associated hyperedges to obtain the hyperedge representation The specific form is as follows: in, is a multilayer perceptron. The multilayer perceptrons in this model share parameters. is the information from node u to hyperedge e, is the node feature, the initial feature (2) Hyperedge to node The hyperedge is represented as Broadcast to relevant nodes and calculate the information from the hyperedge to the node through the multi-layer perceptron The specific form is as follows: is a multilayer perceptron: (3) Update node features For each node in the hypergraph, update the node’s features using a multilayer perceptron The representation is as follows: in is a multilayer perceptron, d v Represents the degree information of the node; (4) Update hyperedge features For each hyperedge, the updated node features are aggregated into the hyperedge, and then a multi-layer perceptron is used Update Hyperedge Features The specific form is as follows:

4. The information propagation prediction method based on common attention fusion sequential hypergraph neural network according to claim 1 is characterized in that: Step S2 includes introducing a residual gating mechanism to propagate residual information between different timestamps. The mechanism generates the initial embedding of each node by combining the dynamic embedding and user embedding of the node in each time period, including the following calculations: The initial embedding of a node at timestamp t+1 can be calculated as: Where W R and z R denote the transformation matrix and vector of the gating mechanism respectively, while σ(·) denotes the tanh function, using the user representation learned from the user feature learning module As the user's initial embedding, Represents the hypergraph node features obtained by the hypergraph neural network based on equivariance diffusion. The value g calculated by the gating function is used to control the percentage of residual information retained. The hypergraph is sequentially connected through this residual gating mechanism.

5. The information propagation prediction method based on common attention fusion sequential hypergraph neural network according to claim 1 or 4, characterized in that: Step S3 for feature fusion and prediction includes calculating the user's diffusion probability by the following method: Where W p Is R CA Transformation matrix mapped to user-specific space, using Mask m To block users who have been activated before prediction, if user u i Participate in cascade c in step j m , and Training with cross entropy loss: Where θ represents all the parameters that need to be learned in the model. i Participate in cascade c at step j m ,y ji =1, otherwise y ji =0.

6. The information propagation prediction method based on common attention fusion sequential hypergraph neural network according to claim 1 is characterized in that: include: User feature learning module, which uses graph convolutional neural network to obtain user embedding; Cascade feature learning module, used to obtain cascade embeddings that encapsulate cascade internal and external features; The feature fusion and prediction module uses the cascade embedding query process and the two-layer joint attention mechanism to obtain the fused features of the cascade embedding and the user embedding, and then calculates the possibility of potential user propagation.

7. The information propagation prediction method based on common attention fusion sequential hypergraph neural network according to claim 6 is characterized in that: The method includes a feature fusion layer and propagation prediction in a feature fusion and prediction module, wherein the feature fusion layer includes a cascaded embedding query process and a two-layer joint attention mechanism, and information leakage is avoided by carefully selecting pre-attendance embedding; The joint attention layer has both user-to-cascade and cascade-to-user attention flows, achieving a subtle integration of user preferences and cascade dynamics, leveraging their interdependence to enhance the model’s predictive power.

8. The information propagation prediction method based on common attention fusion sequential hypergraph neural network according to claim 7 is characterized in that: In the cascade embedding query process, for a given target cascade, the representation of the cascade is read in the most recent time interval before the user participates in the cascade, thereby reducing the risk of information leakage. This process requires the user u i Time when information is disseminated Aligned with the pre-stored timestamps, the relevant concatenated embeddings are then retrieved to obtain where h e,t represents the concatenated embeddings obtained at different timestamps in step S2; The joint attention block operates by utilizing the query from one modality and the keys and values ​​from the other modality, with the query matrix serving as the residual information after the multi-head attention sub-layer.

Images