Information propagation prediction system based on intercascade relationships

Abstract

The application relates to an information propagation prediction system based on inter-cascade relationships, comprising: a heterogeneous graph convolution module for acquiring a network node feature vector of a heterogeneous graph; a user attention module for processing the network node feature vector obtained by the heterogeneous graph convolution module by using an attention mechanism, distributing weights to the internal relationships between users, and obtaining a new feature vector; an inter-cascade relationship graph convolution module for acquiring inter-cascade relationships from a cascade graph; a time decay network module for introducing time decay information into the feature representation of the cascade; a multi-head self-attention module for adjusting the cascade feature weight with the time decay information to obtain a final cascade representation; and an information propagation prediction output module for calculating the probability distribution of each user node in the next time node by using the final cascade representation. The system is beneficial to improving the information cascade prediction effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer dialogue, and particularly relates to an information propagation prediction system based on inter-cascade relationships. BACKGROUND

[0002] Information cascade prediction aims to analyze the user relationship and user cascade diffusion sequence generated in social media. According to the target of the information cascade prediction method, it can be divided into macro information cascade prediction and micro information cascade prediction. The former mainly uses the user relationship graph and user diffusion sequence in social media to predict the final cascade increment size after a period of time; and the latter is more focused on user level modeling, and its main purpose is to use the user relationship graph and user diffusion sequence in social media to predict the probability of the network node being activated at the subsequent time node. The present application mainly focuses on the micro level information cascade prediction.

[0003] The information cascade prediction research method can be divided into two categories: traditional information cascade prediction method and deep learning-based information cascade prediction method.

[0004] The traditional information cascade prediction method mainly includes the following three types: 1) feature-based modeling method; the feature-based method mainly extracts features from the original data, including user attributes, user content, time features, etc., and uses machine learning methods for prediction. Since the acquisition of these features often requires very high human and material costs, it is difficult to promote. 2) generation-based modeling method. The generation-based method only models the propagation process of each message, and ignores the popularity information implied by the information cascade in the information propagation process, resulting in poor interpretability of the prediction model. 3) diffusion model-based method. The diffusion-based method is superior to the first two methods in representing the diffusion process of information, but it is largely dependent on the underlying diffusion model, so it performs poorly in a relatively large network.

[0005] At present, the information cascade prediction method based on deep learning can be divided into two categories: information cascade prediction method based on diffusion sequence and information cascade prediction method based on topology structure. 1) The information cascade prediction method based on diffusion sequence aims to predict the information cascade according to the given diffusion sequence. Previous work assumes that there is a prior diffusion model in the information diffusion process, and then the information diffusion is predicted. For example, linear threshold model or independent cascade model. Although it successfully captures the invisible relationship between users, the effectiveness of its method depends largely on the assumed prior diffusion model, which is difficult to verify in the real world. With the development of neural network model, researchers use recurrent neural network to construct information cascade prediction model, so as to adaptively learn the path representation in the cascade diffusion sequence, capture the sequence characteristics in the information cascade, and predict the information cascade, so as to get rid of the assumed basic diffusion model. Most of the information cascade prediction methods based on diffusion sequence regard the cascade prediction task as a sequence prediction task, predict the diffusion user in sequence, and explore the influence of historical diffusion sequence on future diffusion. However, the social network relationship structure between users is not considered, which leads to poor performance in identifying and predicting information flow. 2) The inspiration of information cascade prediction method based on topology structure comes from the application of image and text in deep learning. The information cascade prediction method based on topology structure believes that in real life, people will have some common interests and hobbies, and if a person forwards or follows a certain type of news or tweet, the probability of his friends forwarding this type of news or tweet will increase. Based on this assumption, researchers use graph convolution network to consider the node structure information in the user social relationship graph together with the information carried by itself, so that the fitting and generalization ability of the model is greatly improved. For similar nodes in the topology graph, this method will generate close feature vector representation to represent the similar interests and hobbies between users. However, due to the low sensitivity of graph neural network to time sequence characteristics, a recurrent neural network is introduced to capture the time sequence structure characteristics of information cascade. However, this method ignores the influence of other information on the current diffusion path in the information propagation process, and cannot capture the global forwarding relationship, so it is difficult to model the complexity in the diffusion process. SUMMARY

[0006] The purpose of the present application is to provide an information propagation prediction system based on inter cascade relationship, which is helpful to improve the effect of information cascade prediction.

[0007] To achieve the above purpose, the technical scheme adopted by the present application is: an information propagation prediction system based on inter cascade relationship, comprising:

[0008] The heterogeneous graph convolution module is used for acquiring the network node feature vector of the heterogeneous graph.

[0009] a user attention module configured to process the network node feature vector obtained by the heterogeneous graph convolution module using an attention mechanism, and to assign weights to the internal relationship between users to obtain a new feature vector;

[0010] a cascade interrelation graph convolution module configured to obtain the intercascade relationship from the cascade graph;

[0011] a time decay network module configured to introduce time decay information into the feature representation of the cascade;

[0012] a multi-head self-attention module configured to adjust the cascade feature weight with the time decay information to obtain a final cascade representation;

[0013] an information propagation prediction output module configured to calculate the probability distribution of each user node in the next time node using the final cascade representation.

[0014] Further, a heterogeneous graph is constructed using a user social relationship graph and a user social behavior graph, the heterogeneous graph convolution module aggregates the network node feature vector of the heterogeneous graph, and iterates in the subsequent training process to gradually update the network node feature.

[0015] Further, the user attention module processes the network node feature vector obtained by the heterogeneous graph convolution module using an attention mechanism, reassigns the importance weights of the user social relationship and the user behavior relationship, captures the internal relationship between users, and obtains a new feature vector; each user u i and its neighbor users u j ∈{u1,u2,u3,…,u i-1} between the attention score calculation formula is as follows:

[0016]

[0017] where x i and x j represent the feature vectors obtained after learning by the heterogeneous graph convolution module, β ji is the relationship score between the user u i and the user u j , <,> represents the inner product operation between vectors, W t represents the transformation matrix of the current target user, W c represents the transformation matrix of the neighbor user of the current target user, W t and W c distinguish the cooperative feature relationship between users in the prediction of different cascades;

[0018] Therefore, the target user u iThe feature representation vector representing the relationship between the user and the neighbor nodes is obtained after the user attention module The calculation formula is as follows:

[0019]

[0020] Then, the obtained feature vector is spliced with the feature vector obtained after learning by the heterogeneous graph convolution module, and the weight size is adjusted through a full connection layer to obtain the vector representation of the user u i :

[0021]

[0022] h i =σ(h i W h +b h )

[0023] Wherein, concat represents the splicing operation, W h and b h are the parameter matrices of training, and σ is the activation function.

[0024] Further, the inter-cascade relationship graph convolution module extracts the inter-cascade relationship from the cascade graph, obtains the influence of other diffusion cascades on the current cascade in the propagation process, and thereby constructs a cascade relationship graph; then the network nodes in the cascade relationship graph are aggregated by the method of graph convolution to obtain an additional node representation, which is fused with the feature vector obtained by the user attention module to obtain a feature vector with inter-cascade relationship.

[0025] Further, the time decay network module obtains the influence of the time factor on the user infection according to the time node information in the cascade; first, the time interval of each node infected in the cascade is converted into a series of one-hot vectors, then the time interval is weighted and distributed through a full connection layer to obtain the influence weight factor of the time node on the user infection, and finally the influence of the historical user's forwarding time information on the user forwarding in the current cascade is obtained through BiLSTM; wherein, the calculation formula of the weight assignment for each time interval is as follows:

[0026] u k =σ(W t t k +b t )

[0027] Wherein, t k represents the time point when the user k is infected, W t and b t represent the trainable parameter matrix;

[0028] The update process of the BiLSTM at time t is as follows:

[0029] i t = σ(W i [h t-1 , x t ]+b i )

[0030] f i = σ(W f [h t-1 , x t ]+b f )

[0031] o t = σ(W o [h t-1 , x t ]+b o )

[0032] C t = f t *C t-1 +i t *σ(W C [h t-1 , x t ]+b C )

[0033] h t = o t *tanh(C t )

[0034] where i t , f t and o t represent the input gate, the forget gate and the output gate, respectively, C t represents the state of the vector at time t, h t represents the hidden layer state, W and b are the parameter matrix and vector to be learned, x t is the initial vector of the node, and σ is a nonlinear activation function.

[0035] Further, the multi-head self-attention module applies a multi-head self-attention with a mask to the feature vector with a heterogeneous graph structure relationship, a cascading relationship in the information propagation process and time decay information, so as to adjust the weight dependence between different relationships and obtain the final cascading representation; the query Q, the key K and the value V in the multi-head attention are all cascading feature vectors represented by S in the system; the specific calculation formula is as follows:

[0036]

[0037]

[0038]

[0039] A = [a1; a2; a3;... ; a H ]W O

[0040] wherein, W O is a weight matrix to be learned, Z is a concatenated representation vector, H is the number of heads of multi-head self-attention, d is the dimension of the vector, d k = d / H, d k is a scaling factor, and M is a mask matrix.

[0041] Further, the information propagation prediction output module uses the obtained final concatenated representation to perform information propagation prediction, calculates the probability distribution of each user node in the next time node, and thus obtains the prediction result of the information cascade.

[0042] Further, the system adjusts the model parameters according to the forward propagation of information and the backward propagation of errors in the training phase of the model, and gradually optimizes the target loss function.

[0043] Compared with the prior art, the present application has the following beneficial effects: an information propagation prediction system based on inter-cascade relationship is provided, which can learn the influence of other diffusion cascades on the current diffusion cascade in the propagation process by fusing the internal relationship between cascades, take the time decay effect into consideration, capture the influence brought by the time factor in the cascade diffusion process, and use user attention to strengthen the relationship between users, thereby better realizing the prediction of information cascade. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a system composition block diagram of an embodiment of the present application.

[0045] Figure 2 is a system model architecture diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0046] The present application will be further described below in conjunction with the drawings and embodiments.

[0047] It should be pointed out that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0048] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0049] As shown in Figure 1 , 2 The embodiment provides a system for predicting information propagation based on inter-cascade relationship, which comprises a heterogeneous graph convolution module, a user attention module, an inter-cascade relationship graph convolution module, a time decay network module, a multi-head self-attention module and an information propagation prediction output module.

[0050] The heterogeneous graph convolution module is used for obtaining a network node feature vector of a heterogeneous graph.

[0051] The user attention module is used for processing the network node feature vector obtained by the heterogeneous graph convolution module by using an attention mechanism, distributing weights of internal relationships between users, and obtaining a new feature vector.

[0052] The inter-cascade relationship graph convolution module is used for obtaining inter-cascade relationships from a cascade graph.

[0053] The time decay network module is used for introducing time decay information into a feature representation of a cascade.

[0054] The multi-head self-attention module is used for adjusting a cascade feature weight with time decay information to obtain a final cascade representation.

[0055] The information propagation prediction output module is used for calculating a probability distribution of each user node in a next time node by using the final cascade representation.

[0056] The configuration and functions of each module are described in detail below.

[0057] 1) Heterogeneous graph convolution module

[0058] The following describes how the heterogeneous graph convolution module obtains an initial network node vector.

[0059] In the embodiment, first, a user social relationship graph and a user social behavior graph are used to construct a heterogeneous graph, then a heterogeneous graph convolution module is used to aggregate a network node feature vector of the heterogeneous graph, and an iterative operation is performed in a subsequent training process to gradually update the network node feature.

[0060] 2) User attention module

[0061] The following describes how the user attention module processes the network node feature vector obtained by the previous module.

[0062] The user attention module processes the network node feature vector obtained by the heterogeneous graph convolution module using an attention mechanism to more finely integrate user social relationships and user social behavior, reassign the importance weights of user social relationships and user behavior relationships, capture the inherent relationship between users, and thus obtain a new feature vector; each user u i and its neighbor users u j ∈{u1,u2,u3,…,u i-1} The calculation formula of the attention score between the target user and its neighbor users is as follows:

[0063]

[0064] where x i and x j represent the feature vectors obtained after learning by the heterogeneous graph convolution module, β ji is the relationship score between user u i and user u j , <,> represents the inner product operation between vectors, W t represents the transformation matrix of the current target user, W c represents the transformation matrix of the neighbor users of the current target user, W t and W c distinguish the cooperative feature relationship between users in the prediction of different cascades.

[0065] Therefore, the target user u i obtains the feature representation vector representing the relationship between the user and its neighbor nodes after the user attention module The calculation formula is as follows:

[0066]

[0067] Then, the obtained feature vector is spliced with the feature vector obtained after learning by the heterogeneous graph convolution module, and the weight size is adjusted through a fully connected layer to obtain the vector representation of user u i :

[0068]

[0069] h i =σ(h′ i W h +b h )

[0070] where concat represents the splicing operation, W hand b h is the activation function, and W k and b t are trainable parameter matrices.

[0071] 3) Inter-cascade relation graph convolution module

[0072] The following describes how the inter-cascade relation graph convolution module introduces inter-cascade relations into the network feature node representation described above.

[0073] In the process of information diffusion, different information cascades may have mutual promotion relations. Neither the user social relation graph nor the user social behavior graph takes the relation between cascades into consideration. Therefore, in this embodiment, the inter-cascade relation graph convolution module extracts inter-cascade relations from the cascade graph, obtains the influence of other diffusion cascades on the current cascade in the propagation process, and thus constructs a cascade relation graph. Then, the network nodes in the cascade relation graph are aggregated by the method of graph convolution to obtain an additional node representation, which is fused with the feature vector obtained by the user attention module to obtain a feature vector with inter-cascade relations.

[0074] 4) Time decay network module

[0075] The following describes how the time decay network module introduces time decay information into the feature representation of a cascade.

[0076] The time at which a previous user forwards a message may affect the series of operations of the user associated with it, and the influence of the forwarded message weakens slowly over time. In this embodiment, the time decay network module obtains the influence of the time factor on user infection according to the time node information in the cascade. First, the time interval at which each node in the cascade is infected is converted into a series of one-hot vectors, and then the time interval is weighted by a fully connected layer to obtain the influence weight factor of the time node on user infection. Finally, the influence of the forwarding time information of the historical user on the forwarding of the user in the current cascade is obtained by BiLSTM. The calculation formula for assigning weights to each time interval is as follows:

[0077] u k =σ(W t t k +b t )

[0078] where t k represents the time point at which user k is infected, W t and b t represent trainable parameter matrices.

[0079] The update process of the BiLSTM at time t is as follows:

[0080] i t =σ(Wi [h t-1 ,x t ]+b i )

[0081] f t =σ(W f [h t-1 ,x t ]+b f )

[0082] o t =σ(W o [h t-1 ,x t ]+b o )

[0083] C t =f t *C t-1 +i t *σ(W C [h t-1 ,x t ]+b C )

[0084] h t =o t *tanh(C t )

[0085] Among them, i t f t and o t These represent the input gate, forget gate, and output gate, respectively. (C) t h represents the state of the vector at time t. t Let W and b represent the hidden layer states, respectively, and let x be the parameter matrix and vector to be learned. t Let σ be the node initialization vector, and σ be the nonlinear activation function.

[0086] 5) Multi-head self-attention module

[0087] The multi-head self-attention module applies a masked multi-head self-attention to feature vectors containing heterogeneous graph structural relationships, cascade relationships in the information propagation process, and time decay information, thereby adjusting the weight dependencies between different relationships to obtain the final cascaded representation. In this system, the query Q, key K, and value V in the multi-head attention are all cascaded feature vectors, denoted by S; the specific calculation formula is as follows:

[0088]

[0089]

[0090]

[0091] A = [a1; a2; a3;... ; a H ]W O

[0092] wherein, W O is a weight matrix to be learned, Z is a concatenated representation vector, H is the number of heads of multi-head self-attention, d is the dimension of the vector, d k = d / H, d k is a scaling factor, and M is a mask matrix.

[0093] 6) information propagation prediction output module

[0094] The information propagation prediction output module uses the obtained final concatenated representation to perform information propagation prediction, calculates the probability distribution of each user node in the next time node, and thus obtains the prediction result of the information cascade.

[0095] In this embodiment, the heterogeneous graph convolution module first obtains the initial feature vector of the network node, balances the weight thereof through the user attention module, so that the vector can better represent the internal relationship between users, and through the introduction of the inter-cascade relationship of the inter-cascade relationship graph convolution module and the time decay effect of the time decay network module, the vector representing the user feature carries the inter-cascade relationship and the time influence factor. The multi-head self-attention of the multi-head self-attention module is used to weight the information carried in the vector, the attention score is calculated, the final concatenated representation is obtained, and finally the probability distribution of each user node being activated in the next time point is obtained through the information propagation prediction output module, and thus the prediction result of the information cascade is obtained.

[0096] In this embodiment, the system adjusts the model parameters during the training phase of the model according to the forward propagation of information and the backward propagation of errors, and gradually optimizes the target loss function.

[0097] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0098] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0099] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0100] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0101] The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application to other forms. Any person skilled in the art can make modifications and variations to the disclosed technical content without departing from the technical scope of the present application. Any simple modification, equivalent variation and modification of the above embodiments according to the technical essence of the present application, which does not deviate from the technical solution of the present application, still belongs to the protection scope of the present application.

Claims

1. An information propagation prediction system based on cascading relationships, characterized in that, include: The heterogeneous graph convolution module is used to obtain the feature vectors of network nodes in a heterogeneous graph. The user attention module is used to process the network node feature vectors obtained by the heterogeneous graph convolution module using the attention mechanism, and to assign weights to the intrinsic relationships between users to obtain new feature vectors. The convolution module for cascade relationship graphs is used to extract cascade relationships from cascade graphs. The time decay network module is used to incorporate time decay information into the cascaded feature representation; The multi-head self-attention module is used to adjust the weights of the cascaded features with time decay information to obtain the final cascaded representation. The information propagation prediction output module is used to calculate the probability distribution of each user node in the next time node using the final concatenated representation; The user attention module uses an attention mechanism to process the network node feature vectors obtained by the heterogeneous graph convolution module, redistributing the importance weights of user social relationships and user behavioral relationships to capture the inherent relationships between users, thereby obtaining new feature vectors; each user u in the user set U i With its neighboring user u j ∈{u1,u2,u3,…,u i-1 The formula for calculating the attention score between} is as follows: Where, x i and x j β represents the feature vector obtained after learning by the heterogeneous graph convolution module. ji For user u i With user u j Relationship scores, <,> represent inner product operations between vectors, W t W represents the transformation matrix of the current target user. c W represents the transformation matrix of the current target user's neighboring users. t and W c The cooperative relationship between users is distinguished during the prediction of different cascading processes; Therefore, the target user u i After passing through the user attention module, we obtain a feature representation vector representing the relationship between the user and their neighboring nodes. The calculation formula is as follows: Then, the obtained feature vector is concatenated with the feature vector learned by the heterogeneous graph convolution module, and the weights are adjusted through a fully connected layer to obtain the user u. i Vector representation of: h i =σ(h i ′ W h +b h ) Where concat represents the concatenation operation, W h and b h Let be the parameter matrix for training, and σ be the activation function; The time decay network module obtains the impact of time factors on user infection based on the time node information in the cascade. First, the time interval of infection of each node in the cascade is converted into a series of one-hot vectors. Then, the time interval is weighted by a fully connected layer to obtain the influence weight factor of the time node on user infection. Finally, the influence of the forwarding time information of historical users on the forwarding of users in the current cascade is obtained by BiLSTM.

2. The information propagation prediction system based on cascading relationships according to claim 1, characterized in that, A heterogeneous graph is constructed using user social relationship graphs and user social behavior graphs. The heterogeneous graph convolution module aggregates the feature vectors of network nodes in the heterogeneous graph and performs iterative operations during subsequent training to gradually update the network node features.

3. The information propagation prediction system based on cascading relationships according to claim 1, characterized in that, The convolutional module for the cascade relationship graph extracts the cascade relationships from the cascade graph, obtains the influence of other diffusion cascades on the current cascade during the propagation process, and thus constructs the cascade relationship graph. Then, it aggregates the network nodes in the cascade relationship graph using graph convolution to obtain an additional node representation, which is then fused with the feature vector obtained by the user attention module to obtain a feature vector with cascade relationships.

4. The information propagation prediction system based on cascading relationships according to claim 1, characterized in that, The formula for assigning weights to each time interval is as follows: u k =σ(W t t k +b t ) Among them, t k W represents the time point at which user k was infected. t and b t Represents the trainable parameter matrix; The update process of BiLSTM at time t is as follows: i t =σ(W i [h t-1 ,x t ]+b i ) f t =σ(W f [h t-1 ,x t ]+b f ) the t =σ(W o [h t-1 ,x t ]+b o ) C t =f t *C t-1 +i t *σ(W C [h t-1 ,x t ]+b C ) h t = no t *fish(C) t ) Among them, i t f t and o t These represent the input gate, forget gate, and output gate, respectively. (C) t h represents the state of the vector at time t. t Let W and b represent the hidden layer states, respectively, and let x be the parameter matrix and vector to be learned. t Let σ be the node initialization vector, and σ be the nonlinear activation function.

5. The information propagation prediction system based on cascading relationships according to claim 1, characterized in that, The multi-head self-attention module applies a masked multi-head self-attention to feature vectors containing heterogeneous graph structural relationships, cascade relationships in the information propagation process, and time decay information, thereby adjusting the weight dependencies between different relationships to obtain the final cascaded representation. In this system, the query Q, key K, and value V in the multi-head attention are all cascaded feature vectors, denoted by S. The specific calculation formula is as follows: A=[a1;a2;a3;…;a H W O in, W O Let Z be the weight matrix to be learned, H be the number of heads in the multi-head self-attention system, and d be the dimension of the vector. k =d / H, d k is the scaling factor, and M is the mask matrix.

6. The information propagation prediction system based on inter-cascade relationships according to claim 1, characterized in that, The information propagation prediction output module uses the obtained final cascaded representation to predict information propagation, calculates the probability distribution of each user node in the next time node, and thus obtains the prediction result of information cascading.

7. The information propagation prediction system based on inter-cascade relationships according to claim 1, characterized in that, During the model training phase, the system continuously adjusts the model parameters based on the forward propagation of information and the backward propagation of errors, thereby gradually optimizing the target loss function.

Images