Information Propagation Prediction Model Training Method, Prediction Method and System Based on Training during Testing

Abstract

The present invention provides a training method, a prediction method and a system for an information propagation prediction model based on training at test time. A directed social graph and a sequential diffusion hypergraph are constructed according to a user set and a cascade set, and a training set and a test set are constructed. The information propagation prediction model includes a macroscopic prediction branch, a microscopic prediction branch and an auxiliary self-supervised learning branch introducing MAE, and the three branches share a feature extractor. During training, the model is jointly trained based on the training set, and a consistency loss is introduced to ensure that the optimization directions of the auxiliary task and the main task for the parameters of the feature extractor are consistent. During testing, training at test time is carried out with the goal of minimizing the loss of the auxiliary self-supervised learning task for a specific test sample, the feature extractor is adjusted to adapt to the sample distribution, and an information propagation prediction model is obtained by combining a task-specific prediction head. The present invention can achieve multi-scale information propagation prediction, solve the distribution shift problem in the information propagation prediction task, and improve the generalization performance of the model in different test environments.

Description

Technical Field

[0001] The present invention relates to the technical field of information dissemination prediction, and in particular, to a training method, a prediction method, and a system for an information dissemination prediction model based on training during testing. Background Art

[0002] The popularity of online social media has made the dissemination and forwarding of information extremely convenient. With a large number of information cascades being tracked and recorded in real time, researchers have begun to explore the laws and potential mechanisms of information dissemination in social networks through the Information Diffusion Prediction (IDP) task. Currently, the research on information dissemination prediction mainly focuses on two aspects. On the one hand, from a macroscopic perspective, it predicts the scale of information dissemination to evaluate the future popularity of the entire information cascade. On the other hand, from a microscopic perspective, it analyzes the individual behaviors of users to identify the next user most likely to disseminate the information. Through this task, researchers can monitor and intervene in the information dissemination process, providing key support for applications such as false information control, hot topic detection, and online marketing.

[0003] Previous studies usually regarded the information dissemination prediction at the macroscopic and microscopic levels as independent tasks, while recent studies have combined the two and proposed a multi-scale prediction paradigm for joint training. Whether it is an independent prediction model or a multi-scale prediction model, they all follow the traditional learning paradigm: the model is first trained on the training set, and then the model parameters are fixed and directly evaluated on the test samples. The effectiveness of this paradigm depends on the independent and identically distributed assumption, that is, it is assumed that the training samples and the test samples are from the same data distribution. However, due to the openness of social media platforms and the inherent uncertainty and variability of user behaviors, the independent and identically distributed assumption often fails to hold in actual social networks. For example, even if the initial disseminators are similar, minor differences in subsequent forwarding users may lead to significant changes in future dissemination dynamics. Similarly, although the dissemination scales are similar, the themes of the information and the characteristics of the users participating in the forwarding may also vary greatly. This distribution shift between the source domain and the target domain may result in the knowledge obtained from the training cascades being unable to be effectively generalized to the test samples, resulting in poor model generalization ability and a decline in prediction performance.

[0004] Currently, the common strategies to alleviate the distribution shift problem mainly include Unsupervised Domain Adaptation (UDA) and Domain Generalization (DG). These two methods are each targeted at different application scenarios and strive to reduce the distribution difference between the source domain and the target domain in different ways, thereby improving the generalization ability of the model in cross-domain tasks.

[0005] The UDA assumption allows for simultaneous access to labeled data in the source domain and unlabeled data in the target domain, aiming to utilize the unlabeled data in the target domain to bridge the distributional differences between the source and target domains, for example, by minimizing statistical differences between domains or using adversarial learning to learn domain-invariant features. However, in the information propagation prediction task, the target domain data is not available during the training phase, and test cascades often arrive in the form of real-time data streams and require online inference. In addition, most UDA algorithms focus on single-source single-target domain adaptation. When faced with multiple unknown target domains, repeatedly performing the unsupervised domain adaptation process is not only inefficient but also impractical.

[0006] Compared with UDA, DG is a more challenging task. The DG assumption is that the prior information of the target domain is unknown. By using data from multiple source domains to train a general model, it is expected that the model will have generalization ability on all unseen domains. Some of these studies focus on learning domain-invariant feature representations, while others adopt data augmentation strategies in the data or feature space. However, DG fails to fully consider the characteristics of different domains, usually resulting in suboptimal generalization performance and limiting its performance in complex cross-domain tasks.

[0007] Generally speaking, both UDA and DG are limited to the training phase, attempting to estimate the distributional differences between the source and target domains by designing complex models. However, the distributional shifts in the information propagation prediction task often occur instantaneously and unpredictably, making it impractical to pre-estimate all potential distributional shifts. When encountering large distributional shifts, these methods may still experience significant performance degradation and are difficult to apply to such tasks. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a method, system, and prediction method for training an information propagation prediction model based on training at test time to eliminate or improve one or more defects existing in the prior art.

[0009] On the one hand, the present invention provides a method for training an information propagation prediction model based on training at test time, the method comprising the following steps:

[0010] Construct a user set and a historical cascade set, the user set containing multiple users, and the historical cascade set containing multiple cascades, each cascade recording the propagation process of an information item through corresponding users over time; construct a directed social graph based on the user set and the social relationships between users, divide the historical cascade set according to the time stamps to construct a sequence diffusion hypergraph; divide the training sample set and the test sample set according to the initial release time of each cascade, and add true labels to the macro prediction task based on the cascade size.

[0011] Construct an information propagation prediction model, where the information propagation prediction model includes a macro prediction branch, a micro prediction branch, and an auxiliary self-supervised learning branch, and the three branches share a feature extractor; among them, input the directed social graph and the sequence diffusion hypergraph into the feature extractor, learn the user social relationship representation, user dynamic interaction representation, and shared cascade representation, and input them into the three branches; in the macro prediction branch, perform sequence enhancement on each cascade in the historical cascade set to generate an enhanced training environment, perform sequence modeling specific to the macro task based on the user dynamic interaction representation to obtain a refined cascade representation, and splice it with the shared cascade representation, and then predict by the macro predictor to obtain the cascade scale; in the micro prediction branch, perform sequence modeling specific to the micro task based on the enhanced training environment and the user social relationship representation to obtain a refined cascade representation, and splice it with the shared cascade representation, and then predict by the micro predictor to obtain the next spreading user; in the auxiliary self-supervised learning branch, randomly mask some users in each cascade, convert the user sequence into a user embedding sequence according to the shared cascade representation, set the visible user embedding and the masked user embedding, input the visible user embedding into the encoder to generate the visible representation of the visible user, and input the visible representation and the masked user embedding into the decoder to reconstruct the embedding of each user in the user embedding sequence;

[0012] Use the training sample set to train the information propagation prediction model, construct a joint loss of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss, and optimize the information propagation prediction model with the goal of minimizing the joint loss;

[0013] When using the test sample set to test and train the optimized information propagation prediction model, fix the parameters of the macro prediction branch and the micro prediction branch, and optimize the parameters of the auxiliary self-supervised learning branch and the feature extractor with the goal of minimizing the auxiliary self-supervised learning task loss to obtain the final information propagation prediction model.

[0014] In some embodiments of the present invention, inputting the directed social graph and the sequence diffusion hypergraph into the feature extractor to learn the user social relationship representation, user dynamic interaction representation, and shared cascade representation includes:

[0015] The feature extractor includes a social relationship encoder, a diffusion interaction encoder, and a shared representation module;

[0016] Input the directed social graph into the social relationship encoder, and extract the user social relationship representation through a multi-layer graph convolutional neural network;

[0017] Input the sequence diffusion hypergraph into the diffusion interaction encoder, introduce the gated fusion strategy, and capture the dynamic interactions between users within each time interval through the hypergraph neural network to extract the user dynamic interaction representation;

[0018] For each enhanced cascade, retrieve the social relationship representation and the dynamic interaction representation of each user and input them into the shared representation module. Model the context interaction between users through the long short-term memory network to obtain the shared cascade representation shared by the macro task and the micro task.

[0019] In some embodiments of the present invention, in the macro prediction branch or the micro prediction branch, perform sequence enhancement on each cascade in the historical cascade set to generate an enhanced training environment, including:

[0020] Perform enhancement on each cascade in the historical cascade set using the sequence enhancement strategy of user masking and user rearrangement.

[0021] In some embodiments of the present invention, after performing sequence modeling specific to the macro task based on the user dynamic interaction representation, obtain a refined cascade representation, splice it with the shared cascade representation, and perform prediction by the macro predictor to obtain the cascade scale, including:

[0022] For each enhanced cascade, input the user embedding obtained according to the user dynamic interaction representation into the macro long short-term memory network to generate a macro cascade representation;

[0023] Splice the macro cascade representation with the shared cascade representation to obtain a macro spliced representation;

[0024] Input the macro spliced representation into the macro multi-layer perceptron for macro prediction to obtain the cascade scale.

[0025] In some embodiments of the present invention, after performing sequence modeling specific to the micro task based on the enhanced training environment and the user social relationship representation, obtain a refined cascade representation, splice it with the shared cascade representation, and perform prediction by the micro predictor to obtain the next spreading user, including:

[0026] For each enhanced cascade, input the user embedding obtained according to the user social relationship representation into the micro long short-term memory network to generate a micro cascade representation;

[0027] Splice the micro cascade representation with the shared cascade representation to obtain a micro spliced representation;

[0028] Input the micro spliced representation into the micro multiple multi-layer perceptrons for micro prediction to obtain the next spreading user.

[0029] In some embodiments of the present invention, the auxiliary self-supervised learning branch adopts an asymmetric encoder-decoder structure, and the method includes:

[0030] Randomly mask some users in each cascade, and replace their user labels with a unified mask label;

[0031] Convert the user sequence into a user embedding sequence according to the shared cascade representation; set visible user embeddings for visible users, and share the same masked user embedding for masked users;

[0032] Input the visible user embeddings into the encoder, and generate visible representations of visible users by multiple Transformer layers;

[0033] Input the visible representations and the masked user embeddings into the decoder, and reconstruct the embeddings of each user in the user embedding sequence by multiple Transformer layers.

[0034] In some embodiments of the present invention, constructing a joint loss of a macro prediction task loss, a micro prediction task loss, an auxiliary self-supervised learning task loss, and a task consistency loss includes:

[0035] The macro prediction task loss adopts the mean absolute error loss, and the calculation formula is:

[0036] ;

[0037] where, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, represents the parameters of the macro prediction branch; is a hyperparameter; represents the number of enhanced training environments;

[0038] The micro prediction task loss adopts the cross-entropy loss;

[0039] The auxiliary self-supervised learning task loss adopts the reconstruction loss;

[0040] The task consistency loss is used to ensure that the optimization directions of the macro prediction task, the micro prediction task, and the auxiliary self-supervised learning task for the parameters of the feature extractor are consistent, and the calculation formula is:

[0041] ;

[0042] where, represents the parameters of the feature extractor; represents the parameters of the auxiliary self-supervised learning branch; Parameters representing the macro prediction branch; Parameters representing the micro prediction branch; Gradients of the macro prediction task loss with respect to the parameters of the feature extractor; Gradients of the micro prediction task loss with respect to the parameters of the feature extractor; Gradients of the auxiliary self-supervised learning task loss with respect to the parameters of the feature extractor.

[0043] In some embodiments of the present invention, optimizing the information propagation prediction model with the objective of minimizing the joint loss includes:

[0044] Jointly optimizing the information propagation prediction model by minimizing the weighted sum of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss. The calculation formula is:

[0045]

[0046] ;

[0047] Wherein, Represents the macro prediction task loss; Represents the micro prediction task loss; Represents the auxiliary self-supervised learning task loss; Represents the task consistency loss; Represents the parameters of the feature extractor; Represents the parameters of the auxiliary self-supervised learning branch; Represents the parameters of the macro prediction branch; Represents the parameters of the micro prediction branch; Represents hyperparameters.

[0048] On the other hand, the present invention also provides an information propagation prediction method based on training at test time. The method includes:

[0049] Obtaining the cascade to be predicted;

[0050] Inputting the cascade into an information propagation prediction model trained by the information propagation prediction model training method based on training at test time as described above to obtain the cascade scale and the next propagation user.

[0051] On the other hand, the present invention also provides an information propagation prediction system based on training at test time. When the system is executed, it implements the steps of any one of the methods mentioned above.

[0052] The present invention provides a training method, a prediction method and a system for an information propagation prediction model based on training at test time, including: constructing a directed social graph and a sequential diffusion hypergraph according to a user set and a cascade set, and thereby constructing a training set and a test set; constructing an information propagation prediction model, which includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch introducing MAE, and the three branches share a feature extractor; introducing invariant learning in the macro prediction branch and the micro prediction branch to improve the generalization ability of the model to multi-environment distributions; during training, jointly training the information propagation prediction model based on the training set, and introducing a consistency loss to ensure that the optimization directions of the auxiliary task and the main task for the feature extractor parameters are consistent; during testing, performing training at test time with the goal of minimizing the loss of the auxiliary self-supervised learning task for a specific test sample, adjusting the feature extractor to adapt to the distribution of this sample, and then combining with a task-specific prediction head to obtain the information propagation prediction model. The present invention can achieve multi-scale information propagation prediction, solve the distribution shift problem in the information propagation prediction task, and improve the generalization performance of the model in different test environments.

[0053] Additional advantages, objects, and features of the present invention will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following, or can be learned from the practice of the present invention. The objects and other advantages of the present invention can be realized and obtained by the structure specifically pointed out in the specification and the drawings.

[0054] Those skilled in the art will understand that the objects and advantages that can be achieved by the present invention are not limited to the above specifically described, and the above and other objects that the present invention can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and do not limit the present invention. In the drawings:

[0056] Figure 1 It is a schematic diagram of the steps of a training method for an information propagation prediction model based on training at test time in an embodiment of the present invention.

[0057] Figure 2 It is a flow diagram of a training method for an information propagation prediction model based on training at test time in an embodiment of the present invention.

[0058] Figure 3 It is an overall framework diagram of an information propagation prediction model in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

[0060] Herein, it should also be noted that in order to avoid obscuring the present invention due to unnecessary details, only the structures and / or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details less related to the present invention are omitted.

[0061] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0062] Herein, it should also be noted that if not otherwise specified, the term "connection" in this document can not only refer to direct connection, but also represent indirect connection with an intermediate.

[0063] In the following, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0064] In order to solve the problems of poor generalization ability and poor prediction performance of the prediction model caused by the distribution shift problem in the existing information propagation prediction technical solution, the present invention provides a method for training an information propagation prediction model based on training at test time, as Figure 1 shown, the method includes the following steps S101 to S104:

[0065] Step S101: Construct a user set and a historical cascade set, wherein the user set includes multiple users, and the historical cascade set includes multiple cascades, and each cascade records the propagation process of an information item realized by corresponding users over time. Construct a directed social graph according to the user set and the social relationships between users, divide the historical cascade set according to timestamps, and construct a sequence diffusion hypergraph; divide the training sample set and the test sample set according to the initial release time of each cascade, and add real labels to the macro prediction task based on the cascade size.

[0066] Step S102: Construct an information propagation prediction model, which includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch, and the three branches share a feature extractor.

[0067] Among them, the directed social graph and the sequential diffusion hypergraph are input into the feature extractor to learn the user social relationship representation, the user dynamic interaction representation, and the shared cascade representation, and are input into three branches. In the macro prediction branch, each cascade in the historical cascade set is sequence-enhanced to generate an enhanced training environment. After performing sequence modeling specific to the macro task based on the user dynamic interaction representation, a refined cascade representation is obtained, which is concatenated with the shared cascade representation and predicted by the macro predictor to obtain the cascade scale. In the micro prediction branch, based on the enhanced training environment and the user social relationship representation, after performing sequence modeling specific to the micro task, a refined cascade representation is obtained, which is concatenated with the shared cascade representation and predicted by the micro predictor to obtain the next spreading user. In the auxiliary self-supervised learning branch, some users in each cascade are randomly masked. According to the shared cascade representation, the user sequence is transformed into a user embedding sequence. The visible user embedding and the masked user embedding are set. The visible user embedding is input into the encoder to generate the visible representation of the visible user. The visible representation and the masked user embedding are input into the decoder to reconstruct the embedding of each user in the user embedding sequence.

[0068] Step S103: Train the information propagation prediction model using the training sample set, construct a joint loss of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss, and optimize the information propagation prediction model with the goal of minimizing the joint loss.

[0069] Step S104: When testing and training the optimized information propagation prediction model using the test sample set, fix the parameters of the macro prediction branch and the micro prediction branch, and optimize the parameters of the auxiliary self-supervised learning branch and the feature extractor with the goal of minimizing the auxiliary self-supervised learning task loss to obtain the final information propagation prediction model.

[0070] As Figure 2 shown, it is a flowchart of the training method of the information propagation prediction model based on training during testing, including two parts: joint training and training during testing.

[0071] In step S101, a training sample set and a test sample set for model training and testing are constructed.

[0072] Construct a user set and an observed historical cascade set, where each cascade records the propagation process of information items in ascending order of time. Exemplarily, the user set is denoted as and the historical cascade set is denoted as , then each cascade , where represents that user forwarded the information item at the timestamp .

[0073] Construct a directed social graph based on the user set and the historical cascade set. Divide the historical cascade set into multiple subsets based on timestamps to construct a sequential diffusion hypergraph. Exemplarily, the directed social graph is denoted as , where is the user set, and is the edge set, representing the social relationships between users. The sequential diffusion hypergraph is denoted as , where , and respectively represent the user set and the hyperedge set in the th time interval. In each diffusion hypergraph , a hyperedge only connects users who participate in the same cascade within this time interval.

[0074] In some embodiments, sort the release times of all cascades and divide the training sample set, validation sample set, and test sample set according to a ratio of 8:1:1.

[0075] In step S102, construct an information propagation prediction model, as shown in Figure 3 . It is the overall structure of the model and the process of joint training. The information propagation prediction model includes a macro prediction branch, a micro prediction branch, and an auxiliary self-supervised learning branch. The three branches share a feature extractor and are each equipped with an independent prediction head. Among them, the macro prediction main branch uses a task-specific prediction head to perform macro prediction tasks, the micro prediction main branch uses a task-specific prediction head to perform micro prediction tasks, and the auxiliary self-supervised learning branch fine-tunes the shared feature extractor through self-supervised learning to make it adapt to specific test cascades. Specifically:

[0076] Input the directed social graph and the sequential diffusion hypergraph into the feature extractor to learn the user social relationship representation, user dynamic interaction representation, and shared cascade representation, capture the social relationships and global interactions between users, and finally input them into the three branches.

[0077] In some embodiments, the feature extractor includes three modules: a social relationship encoder, a diffusion interaction encoder, and a shared representation module.

[0078] Input the directed social graph into the social relationship encoder. Through a multi-layer graph convolutional network (Graph Convolutional Network, GCN), extract the user social relationship representation, which can be denoted as , represents the size of the user set, and represents the dimension.

[0079] The sequence diffusion hypergraph is input into the diffusion interaction encoder, and the cross-cascade user interactions are modeled by the Hypergraph Neural Networks (HGNN) at each time interval. Among them, message aggregation includes two steps: node-to-hyperedge aggregation and hyperedge-to-node aggregation. Considering that a single hypergraph neural network only captures the interactions within a specific time interval, it is insufficient to comprehensively model the evolution of the cascade. To address this limitation, the present invention introduces a gated fusion strategy, which sequentially integrates the interactions learned by the hypergraph neural network at different time intervals to obtain the final user dynamic interaction representation, denoted as , represents the size of the user set, represents the dimension.

[0080] Finally, a long short-term memory network is introduced to model the context interactions between users in the cascade. Specifically: for each cascade, the corresponding user embeddings are retrieved from the user social relationship representation and the user dynamic interaction representation respectively, and arranged in the original order of the cascade, such as . At each time step of the long short-term memory network, the two sets of embeddings of the corresponding users are used as inputs, so as to obtain the -dimensional representation of each user in the cascade. This representation is the cascade representation shared by subsequent macro tasks and micro tasks, denoted as .

[0081] The user social relationship representation, user dynamic interaction representation, and shared cascade representation learned by the feature extractor are respectively input into three branches to implement specific tasks.

[0082] In the macro prediction branch, invariant learning is introduced. Deep neural networks often tend to utilize spurious associations that are easy to fit (i.e., shortcut strategies) when solving problems. Invariant learning is a technique aimed at solving this problem. Its goal is to reveal the invariant causal relationship between features and target labels across environments, while reducing the impact of variable spurious associations, so as to achieve satisfactory out-of-distribution generalization under distribution changes. Therefore, the present invention adopts Variance Risk Extrapolation to train an invariant model that can generalize to unseen offset test data.

[0083] First, as shown in (c) environment construction in Figure 3 , multiple training environments are constructed from the historical cascade set. For each cascade, typical sequence augmentation strategies, such as user masking and user rearrangement, are used for augmentation to generate augmented training environments, and the macro prediction tasks of each environment are processed independently.

[0084] For each augmented cascade , from the above user dynamic interaction representation User embedding composition , and input it into a macro long short-term memory network to generate a macro-cascade representation, denoted as .

[0085] The macro-cascade representation and the shared cascade representation are concatenated and then input into a macro multi-layer perceptron (MLP) for macro prediction to obtain the cascade scale. As Figure 3 shown, a task-specific long short-term memory network (LSTM) and a predictor form a macro prediction branch, and its parameters are .

[0086] In the micro prediction branch, it is similar to the macro prediction branch, except for its learning objective.

[0087] The micro prediction branch and the macro prediction branch adopt the same enhanced prediction environment, and the micro prediction tasks of each environment are processed independently.

[0088] For each enhanced cascade, the user embedding composition from the above user social relationship representation is used and input into a micro long short-term memory network to generate a micro-cascade representation.

[0089] The micro-cascade representation and the shared cascade representation are concatenated and then input into multiple micro multi-layer perceptrons (MLP) for micro prediction to obtain the next propagated user. As Figure 3 shown, a task-specific long short-term memory network (LSTM) and a predictor form a micro prediction branch, and its parameters are .

[0090] The key to Test-Time Training (TTT) is to select an appropriate auxiliary self-supervised learning task. The present invention proposes to introduce a self-supervised learning task based on Masked Autoencoder (MAE) in the auxiliary self-supervised learning branch and introduce a consistency loss during subsequent training to align the self-supervised learning task with the main tasks (macro prediction task loss and micro prediction task loss).

[0091] In the auxiliary self-supervised learning branch, as Figure 3 shown, the MAE-based auxiliary self-supervised learning adopts an asymmetric encoder-decoder structure.

[0092] Considering any cascade, randomly mask some of its users and replace their user labels with a unified mask label. Subsequently, use the shared cascade representation learned by the feature extractor to convert the user sequence into a user embedding sequence. For visible users , their user embedding is , for masked users, the same embedding is shared .

[0093] In the encoder module, the visible user embedding is input into the encoder, and the visible representation of the visible user is generated by multiple Transformer layers, denoted as . In the decoder module, the visible representation and the masked user embedding are input into the decoder, and the embeddings of all users in the user embedding sequence are reconstructed by multiple Transformer layers.

[0094] In step S103, a training sample set is used to train the information propagation prediction model, constructing a joint loss of a macro prediction task loss, a micro prediction task loss, an auxiliary self-supervised learning task loss, and a task consistency loss, and aiming to minimize the joint loss to optimize the information propagation prediction model.

[0095] In some embodiments, considering that the macro prediction task belongs to a regression task, the mean absolute error loss is selected as the loss function in a single environment. The objective of the present invention is to increase the similarity of training risks between different environments while reducing the average training risk. The specific optimization objective is as shown in formula (1) below:

[0096] ; (1)

[0097] where, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, represents the parameters of the macro prediction branch; is a hyperparameter used to control the balance between reducing the average risk and forcing the risks to be equal; represents the number of enhanced training environments.

[0098] In some embodiments, the micro prediction task loss uses the cross-entropy loss.

[0099] In some embodiments, the auxiliary self-supervised learning task loss uses the reconstruction loss.

[0100] In some embodiments, considering that the gradient correlation between the auxiliary self-supervised learning task and the loss functions of the main tasks (macro prediction task and micro prediction task) is crucial for the success of training during testing, the present invention proposes a task consistency loss to ensure that all tasks maintain the same optimization direction for the shared feature extractor parameters, thereby improving the overall performance.

[0101] The definition of the task consistency loss is as shown in formula (2):

[0102] ; (2)

[0103] Among them, represents the parameters of the feature extractor; represents the parameters of the auxiliary self-supervised learning branch; represents the parameters of the macro prediction branch; represents the parameters of the micro prediction branch; represents the gradient of the macro prediction task loss with respect to the parameters of the feature extractor; represents the gradient of the micro prediction task loss with respect to the parameters of the feature extractor; represents the gradient of the auxiliary self-supervised learning task loss with respect to the parameters of the feature extractor.

[0104] By minimizing the weighted sum of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss, the information propagation prediction model is jointly calculated as shown in Equation (3):

[0105]

[0106] ; (3)

[0107] Among them, represents the macro prediction task loss; represents the micro prediction task loss; represents the auxiliary self-supervised learning task loss; represents the task consistency loss; represents the parameters of the feature extractor; represents the parameters of the auxiliary self-supervised learning branch; represents the parameters of the macro prediction branch; represents the parameters of the micro prediction branch; represents the hyperparameter.

[0108] Finally, the parameters of the feature extractor and the parameters of the auxiliary self-supervised learning branch and the parameters of the macro prediction branch and the parameters of the micro prediction branch .

[0109] In step S104, as Figure 2 shown, during the test training phase, the optimized information propagation prediction model is tested and trained using the test sample set, fixing the macro prediction branch and the parameters of the micro prediction branch , to minimize the reconstruction loss Taking this as the goal, multi-step gradient descent is applied to fine-tune the parameters of the feature extractor, as shown in formula (4):

[0110] ; (4)

[0111] By optimizing the loss of the auxiliary self-supervised learning task, the learned parameters and are adjusted for a specific test cascade, enabling the model to extract better representations, effectively capture complex user interaction features, and even handle cascades that have never been seen before.

[0112] The present invention also provides an information propagation prediction method based on training during testing, and this method includes the following steps S201 to S202:

[0113] Step S201: Obtain the cascade to be predicted.

[0114] Step S202: Input the cascade into the information propagation prediction model trained by the information propagation prediction model training method based on training during testing described above, and obtain the cascade scale and the next propagating user.

[0115] In some embodiments, after the information propagation prediction model training method based on training during testing described above, after training during testing, the final feature extractor parameters are obtained. The feature extractor parameters are respectively combined with the parameters of the macro prediction branch and the parameters of the micro prediction branch to form two models: , . Among them, model is used for macro prediction, and model is used for micro prediction.

[0116] Correspondingly, the present invention also provides an information propagation prediction system based on training during testing, and when this system is executed, it implements the steps of any one of the methods described above.

[0117] Correspondingly, the present invention also provides an electronic device, which includes a computer device. The computer device includes a processor and a memory. Computer instructions are stored in the memory, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0118] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing edge computing server deployment method are implemented. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium well known in the art.

[0119] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0120] It should be clear that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0121] In the present invention, the features described and / or illustrated for one embodiment can be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0122] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and modifications can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A training method for an information propagation prediction model based on training during testing, characterized in that, The method includes the following steps: Construct a user set and a historical cascade set. The user set contains multiple users, and the historical cascade set contains multiple cascades. Each cascade records the propagation process of an information item realized by the corresponding user over time; construct a directed social graph according to the user set and the social relationships between users, divide the historical cascade set according to the time stamps, and construct a sequence diffusion hypergraph; divide the training sample set and the test sample set according to the initial release time of each cascade, and obtain the true labels of the macro prediction task based on the cascade size; Construct an information propagation prediction model. The information propagation prediction model includes a macro prediction branch, a micro prediction branch, and an auxiliary self-supervised learning branch. The three branches share a feature extractor; among them, input the directed social graph and the sequence diffusion hypergraph into the feature extractor, learn to obtain the user social relationship representation, the user dynamic interaction representation, and the shared cascade representation, and input them into the three branches; among them, the feature extractor includes a social relationship encoder, a diffusion interaction encoder, and a shared representation module; input the directed social graph into the social relationship encoder, and extract the user social relationship representation through a multi-layer graph convolutional neural network; input the sequence diffusion hypergraph into the diffusion interaction encoder, introduce a gated fusion strategy, and capture the dynamic interaction between users within each time interval through a hypergraph neural network to extract the user dynamic interaction representation; retrieve the social relationship representation and the dynamic interaction representation of each user and input them into the shared representation module, model the context interaction between users through a long short-term memory network, and obtain the shared cascade representation shared by the macro task and the micro task; in the macro prediction branch, enhance each cascade in the historical cascade set by using a sequence enhancement strategy of user masking and user rearrangement to generate an enhanced training environment, based on the user dynamic interaction representation, obtain a refined cascade representation based on a macro long short-term network, and splice it with the shared cascade representation, and predict by a macro predictor to obtain the cascade size; in the micro prediction branch, based on the enhanced training environment and the user social relationship representation, obtain a refined cascade representation based on a micro long short-term memory network, and splice it with the shared cascade representation, and predict by a micro predictor to obtain the next propagating user; in the auxiliary self-supervised learning branch, randomly mask some users in each cascade, convert the user sequence into a user embedding sequence according to the shared cascade representation, set the visible user embedding and the masked user embedding, input the visible user embedding into the encoder to generate the visible representation of the visible user, and input the visible representation and the masked user embedding into the decoder to reconstruct the embeddings of each user in the user embedding sequence; Use the training sample set to train the information propagation prediction model, construct a joint loss of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss, and optimize the information propagation prediction model with the goal of minimizing the joint loss; When testing and training the optimized information propagation prediction model using the described test sample set, fix the parameters of the macro prediction branch and the micro prediction branch, and aim to minimize the loss of the auxiliary self-supervised learning task to optimize the parameters of the auxiliary self-supervised learning branch and the feature extractor, obtaining the final information propagation prediction model.

2. The method for training an information dissemination prediction model based on training during testing according to claim 1, wherein Based on the user dynamic interaction representation, obtain a refined cascade representation using a macro long short-term network and concatenate it with the shared cascade representation, and then perform prediction by the macro predictor to obtain the cascade scale, including: For each enhanced cascade, input the user embedding obtained according to the user dynamic interaction representation into the macro long short-term memory network to generate a macro cascade representation; Concatenate the macro cascade representation with the shared cascade representation to obtain a macro concatenated representation; Input the macro concatenated representation into a macro multi-layer perceptron for macro prediction to obtain the cascade scale.

3. The method for training an information dissemination prediction model based on training during testing according to claim 1, wherein Based on the enhanced training environment and the user social relationship representation, obtain a refined cascade representation using a micro long short-term memory network and concatenate it with the shared cascade representation, and then perform prediction by the micro predictor to obtain the next propagating user, including: For each enhanced cascade, input the user embedding obtained according to the user social relationship representation into the micro long short-term memory network to generate a micro cascade representation; Concatenate the micro cascade representation with the shared cascade representation to obtain a micro concatenated representation; Input the micro concatenated representation into a micro multi-layer perceptron for micro prediction to obtain the next propagating user.

4. The method for training an information dissemination prediction model based on training during testing according to claim 1, characterized in that The auxiliary self-supervised learning branch adopts an asymmetric encoder-decoder structure, and the method includes: Randomly mask some users in each cascade and replace their user labels with a unified mask label; Convert the user sequence into a user embedding sequence according to the shared cascade representation; set visible user embeddings for visible users and share the same masked user embedding for masked users; Input the visible user embeddings into the encoder to generate visible representations of visible users by multiple Transformer layers; Input the visible representations and the masked user embeddings into the decoder to reconstruct the embeddings of each user in the user embedding sequence by multiple Transformer layers.

5. The method for training an information dissemination prediction model based on training during testing according to claim 1, wherein Construct a joint loss of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss, including: The macro prediction task loss uses the mean absolute error loss, and the calculation formula is: ； Among them, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, represents the parameters of the macro prediction branch; is a hyperparameter; represents the number of enhanced training environments; The micro prediction task loss uses the cross-entropy loss; The auxiliary self-supervised learning task loss uses the reconstruction loss; The task consistency loss is used to ensure that the optimization directions of the macro prediction task, the micro prediction task, and the auxiliary self-supervised learning task for the feature extractor parameters are consistent, and the calculation formula is: ； Among them, represents the parameters of the auxiliary self-supervised learning branch; represents the parameters of the micro prediction branch.

6. The training method of the information propagation prediction model based on training during testing according to claim 1, wherein Aim to minimize the joint loss to optimize the information propagation prediction model, including: Jointly optimize the information propagation prediction model by minimizing the weighted sum of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss. The calculation formula is as follows: ； Among them, represents the loss of the macro prediction task; represents the loss of the micro prediction task; represents the loss of the auxiliary self-supervised learning task; represents the task consistency loss.

7. An information propagation prediction method based on training during testing, characterized in that, The method includes: Obtain the cascade to be predicted; Input the cascade into the information propagation prediction model trained by the information propagation prediction model training method based on training at test time according to any one of claims 1 to 6, to obtain the cascade scale and the next propagating user.

8. An information dissemination prediction system based on training during testing, characterized in that, When the system is executed, the steps of the method according to any one of claims 1 to 7 are implemented.

Images