Information cascade prediction method and device based on spatio-temporal characteristics and content preferences

Abstract

The invention provides an information cascade prediction method and device based on spatio-temporal characteristics and content preferences, and the method comprises the steps: obtaining the social relation data of a user, calling a GCN model to generate the social relation data, and obtaining the social relation embedded characteristics of the user; calling a GRU model to construct and obtain user space-time features, and performing feature construction based on information cascading on the user space-time features to obtain user space-time cascading features; the method comprises the following steps: acquiring information propagation content in which a user participates, constructing a multi-modal joint feature based on the information propagation content, and determining an interest influence factor; and finally, determining the dot product similarity between the space-time cascade features of the user and the interest influence factors, and predicting the participation probability of the user for the information propagation content according to the dot product similarity. Through the method and the device, the technical problem of poor prediction effect of the information cascade prediction task in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of information dissemination technology, and in particular to an information cascade prediction method and device based on spatiotemporal characteristics and content preference. Background Art

[0002] As social networks have become increasingly dominant on the Internet, the corresponding way of information dissemination has changed. Understanding the spread of information on social networking platforms can help applications such as information recommendation and public opinion control. The process of information diffusion is also called cascade. The cascade mainly records the relevant information of when the information is transmitted from the communicator to the receiver. The information cascade prediction task is to predict the key attribute characteristics of the future by observing the early cascade, and then judge the possibility of participating in information dissemination.

[0003] In the recent research on information cascade prediction task, a prediction method based on neural network is proposed in the prior art. The information cascade prediction task is modeled by considering the social relationship between users, and the structural characteristics of social relationship graph and dynamic diffusion graph are learned at the same time to improve the accuracy of information cascade prediction. However, the prediction method based on neural network faces the following difficulties: First, the problem of modeling the relationship between users. Since the information in social network is spread and diffused based on the network constructed by user interaction behavior, users are the decisive factor in the process of information dissemination. Users establish attention relationships, share content, and forward content to make information flow with the help of interactive behaviors between users, because it is difficult to build complex relationships between users. Second, the problem of modeling the spatiotemporal characteristics of cascades. The cascade structure will also change over time. It is also a difficult problem to consider the dynamic characteristics of cascades in the modeling process. Third, the modeling of information content features. The information dissemination process will also be affected by the content of the information itself. It is necessary to re-combine the information content in the information cascade prediction task. However, the information dissemination content in the current social network will contain multimodal data including text and pictures. These three difficulties affect the prediction accuracy of the information cascade prediction task, resulting in poor prediction results. Summary of the invention

[0004] The present invention provides an information cascade prediction method and device based on spatiotemporal characteristics and content preferences. On the one hand, modeling is performed in combination with user social relationships to construct user spatiotemporal cascade features. On the other hand, user interest preference features are constructed according to information dissemination content. Information cascade prediction is performed by combining the two features, thereby solving the technical problem of low prediction accuracy of information cascade prediction tasks in the prior art, which leads to poor prediction results.

[0005] The present invention provides an information cascade prediction method based on spatiotemporal characteristics and content preference, which specifically includes the following steps: Obtain the user's social relationship data, and call the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; Based on the social relationship embedding features, the GRU model is called to construct a user spatiotemporal feature, and the user spatiotemporal feature is subjected to feature construction based on information cascade to obtain a user spatiotemporal cascade feature; Acquire information dissemination content that the user will participate in, construct a multimodal joint feature based on the information dissemination content, and determine an interest influence factor according to the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; The dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor is determined, and the user's participation probability in the information dissemination content is predicted based on the dot product similarity.

[0006] In some embodiments, the step of calling the GRU model to construct and obtain user spatiotemporal features based on the social relationship embedding features includes: Acquire user historical cascade data, and divide the user historical cascade data into multiple cascade subgraphs according to timestamps; Aggregating the social relationship embedding feature with the cascade subgraph to obtain a cascade subgraph embedding feature corresponding to each timestamp; Determine the time embedding feature corresponding to each timestamp; Inputting the cascade subgraph embedding features and the time embedding features into the GCN model to extract spatial features to obtain a spatial feature sequence; The GRU model is called to extract features from the spatial feature sequence to obtain user spatiotemporal features.

[0007] In some embodiments, the step of constructing the user spatiotemporal features based on information cascade to obtain the user spatiotemporal cascade features includes: Performing gated fusion on the social relationship embedding feature and the user spatiotemporal feature to obtain a fused spatiotemporal feature sequence; Determine the position embedding code corresponding to each sequence element in the fused spatiotemporal feature sequence, and concatenate the position embedding code with the sequence element to obtain a spatiotemporal cascade feature sequence; The spatiotemporal cascade feature sequence is input into the Transformer model for masked attention mechanism processing to obtain user spatiotemporal cascade features.

[0008] In some embodiments, the information dissemination content includes a visual image and a text segment corresponding to the visual image, and the constructing of a multimodal joint feature based on the information dissemination content includes: Calling the image encoder of the CLIP model to perform image encoding on each of the visual pictures to obtain initial image features; Inputting the initial image features into the Transformer model for embedding coding to obtain image coding features of the visual picture; Calling the text encoder of the CLIP model to encode the text fragment corresponding to each of the visual images to obtain text encoder features, and merging the text encoding features to obtain the overall text encoding features; The image coding features are gated and fused with the overall text coding features to obtain multimodal joint features.

[0009] In some embodiments, inputting the initial image features into a Transformer model for embedding coding to obtain the embedded coding features of the visual image includes: Each of the initial image features is concatenated as a sequence element to obtain an embedded sequence, and a CLS tag is inserted into the head of the embedded sequence as the first sequence element to obtain an image embedded sequence; Determine the position code of each sequence element in the image embedding sequence, and concatenate the position code with the sequence element to obtain a concatenation feature; Each concatenated feature in the image embedding sequence is input into the Transformer model for encoding processing to obtain the embedded encoding feature of the visual image.

[0010] In some embodiments, determining the interest influencing factor according to the multimodal joint feature includes: Extract the user's participation information embedding features based on the user's historical interaction data, and call the Transformer model to encode the participation information embedding features to obtain the user's interest embedding features; Performing average pooling processing on the user interest embedding features to obtain average pooling features; Determine the feature similarity between the average pooled feature and the multimodal joint feature, and use the feature similarity as an interest influencing factor.

[0011] In some embodiments, the participation probability is predicted by calling an information cascade prediction model, and the training method of the information cascade prediction model includes: Use user social relationship data samples, information dissemination content samples, user historical cascade data samples, and user historical interaction data samples as model training samples; Inputting the model training sample into the information cascade prediction model for forward propagation to obtain the predicted participation probability of the user for the information dissemination content sample; Determining the actual participation probability of users for the information dissemination content sample; A cross entropy loss function is constructed according to the predicted participation probability and the actual participation probability, and the cross entropy loss function is used to perform back propagation in the information cascade prediction model to update the parameters of the information cascade prediction model.

[0012] The present invention also provides an information cascade prediction device based on spatiotemporal characteristics and content preferences, which specifically includes the following modules: An acquisition module is used to acquire the user's social relationship data and call the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; A construction module, used to call the GRU model to construct a user spatiotemporal feature based on the social relationship embedding feature, and perform feature construction based on information cascade on the user spatiotemporal feature to obtain a user spatiotemporal cascade feature; A determination module, used to obtain information dissemination content that the user will participate in, construct a multimodal joint feature based on the information dissemination content, and determine an interest influence factor according to the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; The prediction module is used to determine the dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor, and predict the user's participation probability for the information dissemination content based on the dot product similarity.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the information cascade prediction method based on spatiotemporal features and content preferences as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described information cascade prediction methods based on spatiotemporal features and content preferences.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned information cascade prediction methods based on spatiotemporal features and content preferences.

[0016] The information cascade prediction method based on spatiotemporal features and content preferences provided by the present invention, on the one hand, combines user social relationship data for modeling, and then combines user spatiotemporal features to construct user spatiotemporal cascade features. On the other hand, a multimodal joint feature is constructed according to the information dissemination content, and then the user's interest preference features are determined, and then the information cascade prediction is performed by combining the spatiotemporal cascade features and the interest preference features, which solves the technical problem of low prediction accuracy of information cascade prediction tasks in the prior art, resulting in poor prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the information cascade prediction method based on spatiotemporal characteristics and content preferences provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the information cascade graph embedding layer based on spatiotemporal features provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the principle of extracting image coding features by the visual information processing module provided by the present invention.

[0021] Figure 4 It is a principle diagram of the training process of the information cascade prediction model provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the information cascade prediction device based on spatiotemporal characteristics and content preferences provided by the present invention.

[0023] Figure 6 It is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1 Describe the information cascade prediction method based on spatiotemporal characteristics and content preference of the present invention, Figure 1 is a flow chart of the information cascade prediction method based on spatiotemporal features and content preferences provided by the present invention, such as Figure 1 As shown, the method includes the following steps.

[0026] In step 101, the user's social relationship data is obtained, and the GCN model is called to generate the social relationship data to obtain the user's social relationship embedding features.

[0027] First, to extract the characteristics of the user, it is necessary to consider the user's social relationship. Because each user has different social relationships, and each user has different preferences for different information dissemination content. Based on this, the present invention obtains the user's social relationship data, and then uses all users as nodes, and uses the social relationships between users as undirected edges based on the social relationship data, so as to construct a user social network relationship graph based on the nodes and undirected edges, and the information dissemination content can be freely propagated between different user nodes in the graph according to the undirected edges. Since graph convolutional neural networks (GCN) are suitable for graph structured data, the present invention calls the GCN model here to generate social relationship data and obtain the user's social relationship embedding features. That is, the GCN model is used to extract the characteristics of each node (user) in the user social network relationship graph to obtain the user's social relationship embedding features, which are expressed as , the extraction process is expressed as the following formula (1) and formula (2): (1)

[0028] (2)

[0029] In the above formula (1) and formula (2), Indicates the node (user) in the GCN model The vector representation of the layer (i.e., social relationship embedding features), is the input of the first layer of the GCN model, which is a feature vector that can be randomly initialized, and , N represents the number of nodes (users), and d represents the dimension of the feature vector. represents the adjacency matrix constructed by the GCN model based on the user social network relationship graph, E represents the identity matrix, Represents the adjacency matrix The matrix obtained by fusing the identity matrix E is express The degree matrix of Degree matrix The matrix obtained by fusing the identity matrix E is Represents a trainable weight matrix. Represents the activation function of the network layer in the GCN model.

[0030] Through step 101, the user's social relationship data is used to construct the user's social relationship embedding features using the GRU model, which can accurately represent the user's characteristics under complex social network relationships, and solve the technical problem that it is difficult to construct complex relationships between users in the existing neural network-based prediction method.

[0031] In step 102, based on the social relationship embedding features, the GRU model is called to construct the user's spatiotemporal features, and the user's spatiotemporal features are subjected to feature construction based on information cascade to obtain user spatiotemporal cascade features.

[0032] The GCN model is used to generate the user's social relationship embedding features, that is, to determine the user's feature vector representation. Next, based on the social relationship embedding features, the GRU model is called to build the user's spatiotemporal features. Since the user's spatiotemporal features need to be integrated with the user's cascade information, the user's cascade information needs to be extracted first.

[0033] The embodiment of the present invention first obtains the user's historical cascade data, and divides the user's historical cascade data into multiple cascade subgraphs according to timestamps. Since there is spatial and temporal correlation between each cascade subgraph, the modeling process of the dynamically changing information cascade graph is converted into an aggregation problem of spatial and temporal features.

[0034] See also Figure 2 , Figure 2 is a schematic diagram of an information cascade graph embedding layer based on spatiotemporal features provided by the present invention, such as Figure 2 As shown, a multi-layer GCN network is used to aggregate user information and cascade information. The social relationship embedding features and cascade subgraphs are aggregated at the input layer to obtain the cascade subgraph embedding features corresponding to each timestamp, which is recorded as , where t represents the timestamp. Since each cascade subgraph is segmented according to the timestamp, each timestamp also corresponds to a cascade subgraph embedding feature. Figure 2 In the example, the n cascaded subgraph embedding features are represented as “ , , …, The corresponding n timestamps are expressed as “t, t-1, …, tn”.

[0035] Next, determine the time embedding feature corresponding to each timestamp, that is, construct the corresponding time embedding feature according to the timestamp. Figure 2As shown in , for each of the n timestamps, a corresponding time embedding feature is constructed. In the spatial feature extraction stage, the cascaded subgraph embedding features and the time embedding features are input into the GCN model for spatial feature extraction to obtain a spatial feature sequence. Considering that the gated recurrent unit (GRU) can better process long sequences and has a simpler structure, in the spatiotemporal feature extraction stage, the GRU model is called to extract features from the spatial feature sequence to obtain user spatiotemporal features, so that the GRU model can understand the dynamic changes of the data in the spatial feature sequence, so that the user spatiotemporal features can better simulate the feature representation of the user at the corresponding timestamp.

[0036] like Figure 2 As shown, at timestamp t-1 in the spatiotemporal feature extraction phase, the user spatiotemporal features are obtained in the t-1th gated recurrent unit (GRU) in the GRU model. After that, it will be input into the GCN model. In the GCN model, the time embedding feature of the next timestamp t is combined , extract the spatial feature sequence at timestamp t, and then input the spatial feature sequence at timestamp t into the tth gated recurrent unit to obtain the user spatiotemporal feature at timestamp t Therefore, at the subsequent timestamp t+1, the corresponding user spatiotemporal features The construction process can be expressed as the following formula (3) and formula (4): (3)

[0037] (4)

[0038] In the above formula (3), t represents the timestamp, represents the time embedding feature corresponding to timestamp t, , represents the dimension of temporal embedding features, Represents the user's spatiotemporal features at timestamp t , at the first timestamp (i.e. t=0), the user's spatiotemporal features Using social relationship embedding features (L represents the number of layers of the GCN network) for initialization. represents the adjacency matrix in the user social network relationship graph at timestamp t+1, , N represents the number of nodes (users) in the user social network relationship graph. express The processing function of the network layer in the model. represents the activation function of the gated unit in the GRU model, and GRU represents the processing function of the gated unit in the GRU model.

[0039] When constructing user spatiotemporal features, the embodiment of the present invention integrates corresponding time embedding features according to timestamps on the basis of user social relationship embedding features, so that the user spatiotemporal features can accurately express the features of the user at each timestamp, and also considers the possible dynamic changes of user spatiotemporal features between timestamps, and uses the timing characteristics of the GRU model to continuously output the user spatiotemporal features corresponding to each timestamp in the order of timestamps, thereby further improving the representation accuracy of user spatiotemporal features. The technical problem of the prior art prediction method based on neural networks that it is difficult to construct cascade dynamic features in the modeling process because the cascade structure will also change over time is solved.

[0040] After determining the user's spatiotemporal features, the user's spatiotemporal features are then subjected to feature construction based on information cascade to obtain the user's spatiotemporal cascade features. In order to obtain the user's content preference features at different timestamps, the embodiment of the present invention uses a gated fusion mechanism to fuse the user's social information with the user's spatiotemporal features, then splices the randomly initialized position embedding features, and finally uses a masked self-attention mechanism to learn sequence representation.

[0041] Specifically, first, for the user time features corresponding to each timestamp, the social relationship embedding features are gated and fused with the user spatiotemporal features to obtain a fused spatiotemporal feature sequence.

[0042] Among them, for the user time feature corresponding to each timestamp, the calculation process of gated fusion is shown in the following formula (5): (5)

[0043] In the above formula (5), represents the processing function of gated fusion, Represents the user's social relationship embedding features, represents the user's spatiotemporal characteristics, It represents the weight parameter for measuring the social relationship embedding features and spatiotemporal features. The calculation formula is as follows: (6)

[0044] In the above formula (6), represents the natural exponential function, Represents the weight parameter The corresponding weight matrix. Represents the user social relationship embedding feature The corresponding coding feature is calculated as shown in the following formula (7): Representing user spatiotemporal characteristics The corresponding coding feature is calculated as shown in the following formula (8): (6)

[0045] (7)

[0046] In the above formulas (6) and (7), Represents the user social relationship embedding feature The corresponding weight matrix is, Representing user spatiotemporal characteristics The corresponding weight matrix, tanh represents the hyperbolic tangent function.

[0047] Through the above formulas (5) to (8), the social relationship embedding features at each timestamp can be gated and fused with the user spatiotemporal features, and then the results of the gated fusion at each timestamp can be sequenced to obtain a fused spatiotemporal feature sequence.

[0048] Next, determine the position embedding code corresponding to each sequence element in the fused spatiotemporal feature sequence, denoted as , and then concatenate the position embedding code with the sequence element to obtain a cascade feature sequence. Here, the corresponding position feature is incorporated into each sequence element. Therefore, each sequence element in the cascade feature sequence is denoted as h, which is expressed as the following formula (9):

[0049] In the above formula (9), Represents the user's social relationship embedding features, represents the user's spatiotemporal characteristics, It represents the result of gated fusion under the timestamp. Indicates the corresponding position embedding code.

[0050] Finally, the cascade feature sequence is input into the Transformer encoder model for masked attention mechanism processing to obtain the user spatiotemporal cascade features.

[0051] Considering that the Transformer model is a model based on the attention mechanism, the sequence feature representation of the spatiotemporal cascade feature sequence can be learned through the attention mechanism. Therefore, the Transformer model is called here to extract the features of the cascade feature sequence to obtain the user's spatiotemporal cascade feature H. In addition, since some nodes (users) in the user social network relationship graph may be activated users, for example, a user has disseminated (forwarded or shared) information to some other users before the current timestamp, the possibility of the user continuing to disseminate information to other users after the current timestamp will be very small. Therefore, in the cascade feature sequence, the sequence elements corresponding to certain timestamps cannot represent the specific features of the user and are irrelevant sequence elements. Therefore, the present invention adopts a masked self-attention mechanism, firstly performs sequence masking on the cascade feature sequence, that is, masks some sequence elements in the cascade feature sequence, and then inputs the masked cascade feature sequence into the Transformer model, and uses the multi-head attention network in the Transformer model to process the cascade feature sequence with the self-attention mechanism to obtain the cascade feature after each head in the multi-head attention network is processed. , the processing process of the masked self-attention mechanism is expressed as the following formula (10): (10)

[0052] In the above formula (10), X represents the sequence element in the cascade feature sequence obtained by sequence masking. , , They represent the weight matrices corresponding to the query vector, key vector, and value vector of the sequence element X in the self-attention mechanism, respectively. represents the processing function of the self-attention mechanism in the Transformer model, which is expressed as the following formula (11): (11)

[0053] In the above formula (11), Q, K, and V represent the query vector, key vector, and value vector in the attention mechanism respectively, and d represents the vector dimension. Represents the sequence elements for sequence masking in the concatenated feature sequence. Represents the normalized exponential function commonly used in the attention mechanism processing process.

[0054] Through the above formulas (10) and (11), the cascade features of each head in the multi-head attention network can be calculated Next, we can extract the cascade sequence based on the user's historical cascade data, and then find the cascade feature corresponding to the user's last timestamp based on the cascade sequence, thereby determining the cascade feature at each timestamp. The cascade order is calculated and concatenated according to the cascade order to obtain the overall cascade feature combination. Then, the overall cascade feature combination is subjected to feature mapping by the last multilayer perceptron (MLP) in the Transformer model to obtain the final user spatiotemporal cascade feature H, which is expressed as the following formula (12): (12)

[0055] In the above formula (12), MLP represents the processing function of the multi-layer perceptron. Represents the number of heads of the multi-head attention network in the Transformer model, It represents the overall cascade feature combination. The final user spatiotemporal cascade feature H can accurately characterize the dynamic cascade features of users in complex social network relationships under time and space conditions.

[0056] Therefore, through the above steps 101 to 102, the user's social relationship embedding features are constructed using the GRU model through the user's social relationship data, which can accurately represent the user's characteristics under complex social network relationships. When constructing the user's spatiotemporal features, on the basis of the user's social relationship embedding features, the corresponding time embedding features are integrated according to the timestamp, so that the user's spatiotemporal features can accurately express the user's characteristics at each timestamp, and also consider the possible dynamic changes of the user's spatiotemporal features between timestamps. By using the timing characteristics of the GRU model, the user's spatiotemporal features corresponding to each timestamp are continuously output in the order of timestamps, thereby further improving the representation accuracy of the user's spatiotemporal features. Finally, the corresponding cascade information is integrated into the user's spatiotemporal features, and the final user's spatiotemporal cascade features are extracted through the Transformer model, which accurately represents the dynamic cascade features of the user under complex social network relationships under time and space conditions.

[0057] Continue to see Figure 1 , in step 103, the information dissemination content that the user will participate in is obtained, a multimodal joint feature is constructed based on the information dissemination content, and an interest influencing factor is determined based on the multimodal joint feature.

[0058] After constructing the spatiotemporal cascade features of the user (i.e., the dynamic cascade features of the user under the complex social network relationship) in step 102, the content preference features of the user are determined to characterize the content preference of the user for information dissemination content, so as to predict the probability of the user's participation in the information dissemination content.

[0059] First, it is necessary to obtain the information dissemination content. The information dissemination content in the embodiment of the present invention belongs to the collected information graphic data, specifically including visual pictures and text fragments corresponding to the visual pictures. For example, the information dissemination content can be a post, an advertisement or a news, which contains a text introduction and corresponding pictures. The text introduction can be divided into multiple text fragments according to the pictures, and each text fragment introduces a picture. The picture here is a visual picture, and the text fragment introducing the picture is the text fragment corresponding to the visual picture. Since the information graphic data involves two forms, text and image, the information dissemination content is also called multimodal joint information. Therefore, the present invention here constructs multimodal joint features based on the information dissemination content, which is explained in detail below.

[0060] For multimodal joint information, the present invention adopts the Contrastive Language-Image Pre-training (CLIP) model to extract multimodal joint features. For visual images, in the information dissemination content, there are problems such as different sizes and color mismatches in visual images. Therefore, the visual images are transformed here to make the visual images adapt to the CLIP model. First, the shape of the visual image is changed to a uniform width of 224 and height of 224. Cubic spline interpolation is used when scaling the image. Then the visual image data is normalized, and the data range is transformed into a standard normal distribution. Finally, the processed visual image is input into the image encoder of the CLIP model. The corresponding image coding features can be obtained by encoding through the image encoder. For information dissemination content (for example, a post, recorded as ) The jth visual image corresponding to , the initial image features can be obtained through the image encoder of the CLIP model, denoted as If there is no corresponding visual image in the information dissemination content, the initial image features are all initialized to 0 vectors.

[0061] Considering that the information dissemination content in social networks may contain multiple visual images, the importance of each visual image varies, and the position of the visual image affects the expression of the overall information dissemination content to a certain extent. In order to express the differences in importance and position between visual images, the present invention constructs a visual information processing module based on the Transformer model, and further extracts image features from the initial image features obtained by the CLIP model through the visual information processing module to obtain the image coding features of the visual images in the information dissemination content. Here, the initial image features are also extracted. The input is input into the Transformer model for embedded coding to obtain the image coding features of the visual picture.

[0062] See Figure 3 , Figure 3 The schematic diagram of the principle of extracting image coding features by the visual information processing module provided by the present invention is shown below. Figure 3 To explain the feature extraction process of the visual information processing module in detail. Figure 3 As shown in the figure, firstly, according to the distribution of visual images in the information dissemination content, the initial image features of each visual image are As sequence elements, the sequence elements are then concatenated to obtain an embedded sequence. For example, according to the distribution of illustrations in a post, the initial image features of each illustration are used as sequence elements, and the embedded sequence of the illustrations is concatenated.

[0063] In order to realize the aggregation of information features of multiple visual images in information dissemination content, that is,

[0064] The information of multiple pictures in a post is aggregated to obtain global features.

[0065] Insert the CLS tag as the first sequence element into the head of the embedding sequence to obtain the image embedding sequence, that is, add the "[CLS]" tag at the first position of the embedding sequence of the image. Figure 3 As shown in , the “*” represents the CLS tag inserted at position 0 of the sequence. In addition, in order to avoid bias towards a certain visual image in the embedding sequence, the embedding of the “[CLS]” tag is set to random initialization and is continuously updated during training to encode the statistical characteristics of the entire embedding sequence.

[0066] like Figure 3 In the example, after inserting the CLS tag to obtain the image embedding sequence, the position encoding of each sequence element in the image embedding sequence is determined. , and concatenate the position code with the sequence element to obtain the concatenated feature , expressed as the following formula (13): (13)

[0067] Finally, each concatenated feature in the image embedding sequence is input into the Transformer model for encoding processing to obtain the embedded encoding features of the visual image. This is similar to the processing process of the cascade feature sequence mentioned above. Some visual images in the information dissemination content may have been forwarded or shared by users and cannot represent the specific features of the information dissemination content. These visual images are irrelevant sequence elements, so the mask mechanism is still used here. The image embedding sequence is first sequence-masked, that is, some sequence elements in the image embedding sequence are randomly masked (the inserted CLS tags are not masked), and then the masked image embedding sequence is input into the Transformer model. Figure 3As shown in Figure 1, the multi-head attention network in the Transformer Encoder is used to process the image embedding sequence with a self-attention mechanism, and then the result of the self-attention mechanism is input into the final MLP layer, and the final output visual image embedding coding feature is recorded as , expressed as the following formula (14): (14)

[0068] In the above formula (14), Represents the processing function of the Transformer model encoding, A sequence element representing mask processing.

[0069] For the text fragments corresponding to the visual images, since each text fragment corresponds to a visual image, for the information dissemination content, the text fragments of each visual image can be intercepted in the form of a sliding window, and then the text encoder of the CLIP model is called to encode the text fragments corresponding to each visual image to obtain the text encoder features, which are recorded as Then the text encoding features of each text segment are combined to obtain the overall text encoding features, recorded as , the calculation process is expressed as the following formula (15): (15)

[0070] In the above formula (15), n represents the total number of text segments obtained by the sliding window, i represents the i-th information dissemination content, and j represents the j-th text segment in the information dissemination content.

[0071] Finally, the image encoding features The overall encoding features of the text Here, the present invention uses a gated fusion mechanism to learn the importance of different modal information and the interaction between the models, fuses the image features with the text features, and obtains the multimodal joint features, which are recorded as , the multimodal joint features can accurately express the information characteristics of the information dissemination content, which is convenient for subsequent matching with user preferences to complete information dissemination. Gated fusion can be achieved by using a neural network model or a feature fusion algorithm. The calculation process is expressed as the following formula (16): (16)

[0072] In the above formula (16), Represents the processing function of gated fusion.

[0073] Therefore, the embodiment of the present invention extracts the image features and text features of the information dissemination content respectively, takes into account the position of each visual image in the information dissemination content and the corresponding text fragment, and fuses these image features and text features to obtain multimodal joint features, so as to accurately express the information features of the information dissemination content. This solves the problem that in the prior art prediction method based on neural networks, the information dissemination content in social networks may have multimodal data including text and pictures, and the information dissemination process may be affected by the content of the information itself, resulting in difficulty in modeling the information content features.

[0074] After extracting the multimodal joint features of the information dissemination content, the interest influencing factors are determined based on the multimodal joint features to express the user's interest characteristics.

[0075] Considering that user interests change over time, and that the information dissemination content that users have interacted with in the early stage will have an impact on their later interests, the information dissemination content that users have participated in is arranged in chronological order. In order to determine the user's interest characteristics, the user's historical interaction data is first obtained, that is, the interaction record data of the information dissemination content that the user has participated in. Then, the embedded feature extraction algorithm is used to extract the interaction features of the user's historical interaction data as the user's participation information embedding features, denoted as W, and According to the above formula (16), the multimodal joint feature of each information dissemination content is recorded as , so the multimodal joint features of the information dissemination content to be predicted here can be recorded as w, and , indicating that each piece of information dissemination must be predicted.

[0076] Next, the Transformer model is called to encode the participation information embedding feature W to obtain the user interest embedding feature Because user interests change over time, and the information disseminated in the early interaction will affect the user's later interests, the Transformer encoding layer is called again here to learn the temporal characteristics of user interests and extract the user interest embedding features containing temporal characteristics. The encoding process is expressed as the following formula (17):

[0077] In the above formula (17), Represents the processing function of the Transformer encoding layer.

[0078] Considering the large amount of historical user interaction data, user interest embedding features The feature dimension is very large, so before calculating the user interest features, the user interest embedding features Perform average pooling to obtain average pooling features, then determine the feature similarity I between the average pooling features and the multimodal joint features w, and use the feature similarity I as the interest influencing factor. Here, the feature similarity can be calculated as the Euclidean distance or cosine similarity. The higher the feature similarity, the more compatible the corresponding information dissemination content is with the user's interests, and the more likely the user is to participate in the information dissemination content, share or forward it. The lower the feature similarity, the less compatible the corresponding information dissemination content is with the user's interests, and the more likely the user is to participate in the information dissemination content, and the lower the probability of sharing or forwarding the information dissemination content. The formula for calculating the interest influencing factor is expressed as follows (18):

[0079] In the above formula (18), represents the processing function of the average pooling operation, represents the user interest embedding feature, Represents the multimodal joint features to be predicted. The function for calculating feature similarity is expressed as the following formula (19): (19)

[0080] In the above formula (19), and They represent two feature vectors whose feature similarity needs to be calculated, and d represents two feature vectors , The feature dimension of , i represents the ordinal number of the feature dimension.

[0081] Through the above step 103, the present invention extracts the multimodal joint features from the information dissemination content, and then calculates the user interest features based on the user's historical interaction data, thereby calculating the user's interest influence factor based on the multimodal joint features and the user interest features, thereby integrating the information content and the user's preference, and making a preliminary estimate of the possibility of the user's participation in the information dissemination content, which can further improve the accuracy of the cascade prediction.

[0082] Continue to see Figure 1 In step 104, the dot product similarity between the user's spatiotemporal cascade features and the interest influencing factors is determined, and the user's participation probability in the information dissemination content is predicted based on the dot product similarity.

[0083] After determining the user's interest influencing factors, the next step is to combine the user's dynamic cascade features under complex social network relationships (i.e., the final user spatiotemporal cascade features H obtained in step 102) to make a final prediction on whether the user is likely to participate in the information dissemination content.

[0084] The embodiment of the present invention directly calculates the dot product similarity between the user's spatiotemporal cascade feature H and the interest influence factor I, and then predicts the user's participation probability for the information dissemination content based on the dot product similarity, which is recorded as The prediction process can be implemented by MLP, and the formula is expressed as follows (20): (20)

[0085] In the above formula (20), It represents the calculation of the dot product similarity between the user's spatiotemporal cascade feature H and the interest influencing factor I, and MLP represents the processing function of the multi-layer perceptron to predict the participation probability.

[0086] Through the above step 104, the participation probability of the user in the information dissemination content is finally predicted. According to the participation probability, targeted information diffusion can be carried out on the user to achieve accurate dissemination of the information content.

[0087] In some embodiments, the participation probability of the user participating in the information dissemination content predicted in step 104 can also be predicted by calling the information cascade prediction model. The information cascade prediction model specifically includes modules such as the GCN model, the GRU model, the Transformer model, the MLP model, and the CLIP model. The prediction process of the information cascade prediction model is introduced in conjunction with 4. Figure 4 It is a principle diagram of the training process of the information cascade prediction model provided by the present invention.

[0088] First, you need to obtain training samples for the information cascade prediction model. Here you can collect user social relationship data samples, information dissemination content samples that users will participate in, user historical cascade data samples, and user historical interaction data samples, and use these samples as model training samples.

[0089] Then, these model training samples are input into the information cascade prediction model for forward propagation to obtain the predicted participation probability of users for the information dissemination content samples. The forward propagation process is described below.

[0090] Specific as Figure 4As shown in the figure, in the process of forward propagation, at the beginning, based on the user's social relationship data sample, the GCN model is first called to generate the user's social relationship embedding features, and then based on the user's historical cascade data sample, the cascade subgraph and cascade sequence are generated respectively. By calling the GCN model and the GRU model, the user's time features are generated according to the cascade subgraph and the social relationship embedding features. By calling the Transformer model, the user's spatiotemporal cascade features are generated according to the cascade sequence and the user's time features. Next, based on the information dissemination content samples that the user will participate in, the CLIP model is called to extract the multimodal joint features of the information dissemination content samples, and then based on the user's historical interaction data samples, the Transformer model is called to generate the user's interest embedding features. The next step is to calculate the similarity between the user's interest embedding features and the multimodal joint features as the interest influencing factor. Finally, the dot product similarity between the interest influencing factor and the user's spatiotemporal cascade feature is calculated and input into the MLP to obtain the predicted participation probability, that is, the user's participation probability for the information dissemination content sample. . In this way, the forward propagation process of the information cascade prediction model ends.

[0091] Next, determine the user's actual participation probability for the information dissemination content sample, that is, determine the user's actual participation in the information dissemination content sample based on the actual information dissemination environment, denoted as y. Then, based on the predicted participation probability and the actual participation probability, construct the cross entropy loss function Loss. The construction process is expressed as the following formula (21): (twenty one)

[0092] In the above formula (21), n ​​represents the number of information dissemination content samples, and i represents the i-th information dissemination content sample. It represents the predicted participation probability obtained in the forward propagation process of the information cascade prediction model, and y represents the actual participation probability of the user for the information dissemination content sample.

[0093] After constructing the cross entropy loss function, the cross entropy loss function is used to back propagate in the information cascade prediction model. The process of back propagation is also the process of calculating the gradient. The gradient is optimized by the gradient descent algorithm to update the parameters of the information cascade prediction model, so that the cross entropy loss function gradually converges.

[0094] The training process is also an iterative process. In each iteration, forward propagation and backward propagation are performed. In each iteration, the forward propagation process needs to determine whether the cross entropy loss function Loss has decreased significantly. If it does not decrease significantly, it means that the cross entropy loss function has converged. Then the predicted participation probability is output, and this iteration ends, that is, the training process ends. If it still decreases significantly, it means that the cross entropy loss function has not converged, and backward propagation continues to update the parameters of the information cascade prediction model.

[0095] The trained information cascade prediction model can be used directly for prediction. According to the input user information and information dissemination samples, the user's participation probability in the information dissemination content can be directly predicted. According to the predicted participation probability, the dissemination of information on the social networking platform can be understood very intuitively, so as to make targeted information diffusion and complete the cascade process, providing assistance for applications such as information recommendation and public opinion control.

[0096] Through the embodiments of the present invention, on the one hand, modeling is performed in combination with user social relationship data, and then user spatiotemporal features are combined to construct user spatiotemporal cascade features. On the other hand, multimodal joint features are constructed according to the content of information dissemination, and then the user's interest preference features are determined, and then information cascade prediction is performed in combination with spatiotemporal cascade features and interest preference features, thereby overcoming the three difficulties of user relationship modeling, cascade spatiotemporal feature modeling, and information content feature modeling in the prediction method based on neural networks in the prior art one by one, and solving the technical problem that the prediction accuracy of the information cascade prediction task in the prior art is low, resulting in poor prediction effect.

[0097] The information cascade prediction device based on spatiotemporal characteristics and content preferences provided by the present invention is described below. The information cascade prediction device based on spatiotemporal characteristics and content preferences described below and the information cascade prediction method based on spatiotemporal characteristics and content preferences described above can be referenced to each other.

[0098] Figure 5 Schematic diagram of the structure of the information cascade prediction device based on spatiotemporal characteristics and content preferences provided by the present invention. Figure 5 As shown, the present invention provides an information cascade prediction device based on spatiotemporal features and content preferences, and the device specifically includes the following modules: an acquisition module 501, a construction module 502, a determination module 503, and a prediction module 504. Among them, the acquisition module 501 is used to acquire the user's social relationship data, and call the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; the construction module 502 is used to call the GRU model to construct the user's spatiotemporal features based on the social relationship embedding features, and perform feature construction based on information cascade on the user's spatiotemporal features to obtain the user's spatiotemporal cascade features; the determination module 503 is used to acquire the information dissemination content that the user will participate in, construct a multimodal joint feature based on the information dissemination content, and determine the interest influence factor according to the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; the prediction module 504 is used to determine the dot product similarity between the user's spatiotemporal cascade feature and the interest influence factor, and predict the user's participation probability for the information dissemination content according to the dot product similarity.

[0099] Figure 6 is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor (processor) 610 , a communication interface (Communications Interface) 620 , a memory (memory) 630 and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other through the communication bus 640 . The processor 610 can call the logic instructions in the memory 630 to execute the information cascade prediction method based on spatiotemporal features and content preferences, the method including: obtaining the user's social relationship data, and calling the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; based on the social relationship embedding features, calling the GRU model to construct the user's spatiotemporal features, and performing feature construction based on the information cascade on the user's spatiotemporal features to obtain the user's spatiotemporal cascade features; obtaining the information dissemination content that the user will participate in, constructing a multimodal joint feature based on the information dissemination content, and determining the interest influence factor based on the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; determining the dot product similarity between the user's spatiotemporal cascade feature and the interest influence factor, and predicting the user's participation probability in the information dissemination content based on the dot product similarity.

[0100] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the information cascade prediction method based on spatiotemporal features and content preferences provided by the above methods, the method including: obtaining the user's social relationship data, and calling the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; based on the social relationship embedding features, calling the GRU model to construct the user's spatiotemporal features, and performing feature construction based on information cascade on the user's spatiotemporal features to obtain user spatiotemporal cascade features; obtaining the information dissemination content that the user will participate in, constructing a multimodal joint feature based on the information dissemination content, and determining an interest influencing factor based on the multimodal joint feature, wherein the interest influencing factor is used to characterize the user's content preference for the information dissemination content; determining the dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor, and predicting the user's participation probability in the information dissemination content based on the dot product similarity.

[0102] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the information cascade prediction method based on spatiotemporal features and content preferences provided by the above-mentioned methods, the method comprising: obtaining the user's social relationship data, and calling the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; based on the social relationship embedding features, calling the GRU model to construct the user's spatiotemporal features, and performing feature construction based on information cascade on the user's spatiotemporal features to obtain the user's spatiotemporal cascade features; obtaining the information dissemination content in which the user will participate, constructing a multimodal joint feature based on the information dissemination content, and determining an interest influencing factor based on the multimodal joint feature, wherein the interest influencing factor is used to characterize the user's content preference for the information dissemination content; determining the dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor, and predicting the user's participation probability in the information dissemination content based on the dot product similarity.

[0103] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information cascade prediction method based on spatiotemporal characteristics and content preference, characterized in that: The method comprises: Obtain the user's social relationship data, and call the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; Based on the social relationship embedding features, the GRU model is called to construct a user spatiotemporal feature, and the user spatiotemporal feature is subjected to feature construction based on information cascade to obtain a user spatiotemporal cascade feature; Acquire information dissemination content that the user will participate in, construct a multimodal joint feature based on the information dissemination content, and determine an interest influence factor according to the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; The dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor is determined, and the user's participation probability in the information dissemination content is predicted based on the dot product similarity.

2. The information cascade prediction method based on spatiotemporal features and content preferences according to claim 1 is characterized in that: The embedding feature based on the social relationship, Call the GRU model to build the user's spatiotemporal features, including: Acquire user historical cascade data, and divide the user historical cascade data into multiple cascade subgraphs according to timestamps; Aggregating the social relationship embedding feature with the cascade subgraph to obtain a cascade subgraph embedding feature corresponding to each timestamp; Determine the time embedding feature corresponding to each timestamp; Inputting the cascade subgraph embedding features and the time embedding features into the GCN model to extract spatial features to obtain a spatial feature sequence; The GRU model is called to extract features from the spatial feature sequence to obtain user spatiotemporal features.

3. The information cascade prediction method based on spatiotemporal characteristics and content preference according to claim 1 is characterized in that: The step of constructing the user spatiotemporal features based on information cascade to obtain user spatiotemporal cascade features includes: Performing gated fusion on the social relationship embedding feature and the user spatiotemporal feature to obtain a fused spatiotemporal feature sequence; Determine the position embedding code corresponding to each sequence element in the fused spatiotemporal feature sequence, and concatenate the position embedding code with the sequence element to obtain a spatiotemporal cascade feature sequence; The spatiotemporal cascade feature sequence is input into the Transformer model for masked attention mechanism processing to obtain user spatiotemporal cascade features.

4. The information cascade prediction method based on spatiotemporal characteristics and content preference according to claim 1 is characterized in that: The information dissemination content includes a visual image and a text segment corresponding to the visual image, and the multimodal joint feature is constructed based on the information dissemination content, including: Calling the image encoder of the CLIP model to perform image encoding on each of the visual pictures to obtain initial image features; Inputting the initial image features into the Transformer model for embedding coding to obtain image coding features of the visual picture; Calling the text encoder of the CLIP model to encode the text fragments corresponding to each of the visual images to obtain text encoder features, and merging the text encoding features to obtain the overall text encoding features; The image coding features are gated and fused with the overall text coding features to obtain multimodal joint features.

5. The information cascade prediction method based on spatiotemporal characteristics and content preference according to claim 4 is characterized in that: The step of inputting the initial image features into the Transformer model for embedding coding to obtain the embedded coding features of the visual image includes: Each of the initial image features is concatenated as a sequence element to obtain an embedded sequence, and a CLS tag is inserted into the head of the embedded sequence as the first sequence element to obtain an image embedded sequence; Determine the position code of each sequence element in the image embedding sequence, and concatenate the position code with the sequence element to obtain a concatenation feature; Each concatenated feature in the image embedding sequence is input into the Transformer model for encoding processing to obtain the embedded encoding feature of the visual image.

6. The information cascade prediction method based on spatiotemporal characteristics and content preference according to claim 1, characterized in that: The determining of the interest influencing factor according to the multimodal joint feature includes: Extract the user's participation information embedding features based on the user's historical interaction data, and call the Transformer model to encode the participation information embedding features to obtain the user's interest embedding features; Performing average pooling processing on the user interest embedding features to obtain average pooling features; Determine the feature similarity between the average pooled feature and the multimodal joint feature, and use the feature similarity as an interest influencing factor.

7. The information cascade prediction method based on spatiotemporal characteristics and content preference according to claim 1 is characterized in that: The participation probability is predicted by calling an information cascade prediction model, and the training method of the information cascade prediction model includes: Use user social relationship data samples, information dissemination content samples, user historical cascade data samples, and user historical interaction data samples as model training samples; Inputting the model training sample into the information cascade prediction model for forward propagation to obtain the predicted participation probability of the user for the information dissemination content sample; Determining the actual participation probability of users for the information dissemination content sample; A cross entropy loss function is constructed according to the predicted participation probability and the actual participation probability, and the cross entropy loss function is used to perform back propagation in the information cascade prediction model to update the parameters of the information cascade prediction model.

8. An information cascade prediction device based on spatiotemporal characteristics and content preference, characterized in that: The device comprises: An acquisition module is used to acquire the user's social relationship data and call the GCN model to generate the social relationship data to obtain the user's social relationship embedding features; A construction module, used to call the GRU model to construct a user spatiotemporal feature based on the social relationship embedding feature, and perform feature construction based on information cascade on the user spatiotemporal feature to obtain a user spatiotemporal cascade feature; A determination module, used to obtain information dissemination content that the user will participate in, construct a multimodal joint feature based on the information dissemination content, and determine an interest influence factor according to the multimodal joint feature, wherein the interest influence factor is used to characterize the user's content preference for the information dissemination content; The prediction module is used to determine the dot product similarity between the user's spatiotemporal cascade feature and the interest influencing factor, and predict the user's participation probability for the information dissemination content based on the dot product similarity.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the information cascade prediction method based on spatiotemporal features and content preferences as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the information cascade prediction method based on spatiotemporal features and content preferences as described in any one of claims 1 to 6 is implemented.

Images