Information propagation prediction model training method, prediction method and system based on training in test

Abstract

The invention provides an information propagation prediction model training method, prediction method and system based on training during testing. The method comprises the following steps: constructing a directed social graph and a sequence diffusion hypergraph according to a user set and a cascade set, and constructing a training set and a testing set; the information propagation prediction model comprises a macroscopic prediction branch, a microscopic prediction branch and an auxiliary self-supervised learning branch introducing MAE, and the three branches share a feature extractor; during training, performing joint training on the model based on a training set, and introducing consistency loss to ensure that the optimization directions of the auxiliary task and the main task on the parameters of the feature extractor are consistent; during testing, training during testing is carried out by taking minimization of auxiliary self-supervised learning task loss as a target aiming at a specific test sample, a feature extractor is adjusted to adapt to sample distribution, and an information propagation prediction model is acquired in combination with a task-specific prediction head. According to the method, multi-scale information propagation prediction can be realized, the problem of distribution offset in an information propagation prediction task is solved, and the generalization performance of the model in different test environments is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of information propagation prediction, and in particular to an information propagation prediction model training method based on test-time training, a prediction method and a system. Background Art

[0002] The popularity of online social media has made the dissemination and forwarding of information extremely convenient. As a large number of information cascades are tracked and recorded in real time, researchers have begun to explore the laws and potential mechanisms of information dissemination in social networks through the task of Information Diffusion Prediction (IDP). At present, the research on information dissemination prediction mainly focuses on two aspects. On the one hand, it predicts the scale of information dissemination from a macro perspective to evaluate the future popularity of the entire information cascade; on the other hand, it analyzes individual user behavior from a micro perspective to identify the next user who is most likely to spread the information. Through this task, researchers can monitor and intervene in the process of information dissemination, providing key support for applications such as false information control, hot topic detection, and online marketing.

[0003] Previous studies usually regard information propagation prediction at the macro and micro levels as independent tasks, while recent studies combine the two and propose a multi-scale prediction paradigm for joint training. Both independent prediction models and multi-scale prediction models follow the traditional learning paradigm: the model is first trained on the training set, and then the model parameters are fixed and directly evaluated on the test sample. The effectiveness of this paradigm relies on the independent and identically distributed assumption, which assumes that the training samples and test samples come from the same data distribution. However, due to the openness of social media platforms and the inherent uncertainty and variability of user behavior, the independent and identically distributed assumption is often difficult to hold in actual social networks. For example, even if the initial spreaders are similar, small differences in subsequent forwarding users may lead to significant changes in future propagation dynamics. Similarly, despite similar propagation scales, the topic of the information and the characteristics of the users involved in the forwarding may be very different. This distribution shift between the source domain and the target domain may result in the knowledge acquired from the training cascade being unable to be effectively generalized to the test sample, resulting in poor model generalization ability and reduced prediction performance.

[0004] At present, common strategies to alleviate the distribution shift problem mainly include unsupervised domain adaptation (UDA) and domain generalization (DG). These two methods are aimed at different application scenarios and strive to reduce the distribution difference between the source domain and the target domain in different ways, thereby improving the generalization ability of the model in cross-domain tasks.

[0005] UDA assumes that both labeled data in the source domain and unlabeled data in the target domain can be accessed simultaneously, and aims to use the unlabeled data in the target domain to bridge the distribution difference between the source and target domains, for example by minimizing the statistical difference between the domains or using adversarial learning to learn domain-invariant features. However, in information propagation prediction tasks, the target domain data is not available during the training phase, and the test cascade often arrives in the form of real-time data streams, requiring online reasoning. In addition, most UDA algorithms focus on single-source single-target domain adaptation. When faced with multiple unknown target domains, repeatedly performing the unsupervised domain adaptation process is not only inefficient but also impractical.

[0006] Compared with UDA, DG is a more challenging task. DG assumes that the prior information of the target domain is unknown, and trains a general model by leveraging data from multiple source domains in the hope that the model will have generalization capabilities on all unseen domains. Some of these studies focus on learning domain-invariant feature representations, while others adopt data augmentation strategies in data or feature space. However, DG fails to fully consider the characteristics of different domains, which usually leads to suboptimal generalization effects and limits its performance in complex cross-domain tasks.

[0007] In general, UDA and DG are limited to the training phase, attempting to estimate the distribution difference between the source domain and the target domain by designing complex models. However, the distribution shift in information propagation prediction tasks often occurs instantly and is unpredictable, which makes it impractical to estimate all potential distribution shifts in advance. When encountering a large distribution shift, these methods may still experience significant performance degradation, making them difficult to apply to such tasks. Summary of the invention

[0008] In view of this, the embodiments of the present invention provide an information propagation prediction model training method, prediction method and system based on test-time training to eliminate or improve one or more defects existing in the prior art.

[0009] In one aspect, the present invention provides a method for training an information propagation prediction model based on test-time training, the method comprising the following steps: Constructing a user set and a historical cascade set, wherein the user set includes multiple users, and the historical cascade set includes multiple cascades, each cascade recording the propagation process of an information item through the corresponding user according to time; constructing a directed social graph according to the user set and the social relationship between users, dividing the historical cascade set according to the timestamp, and constructing a sequence diffusion hypergraph; dividing the training sample set and the test sample set according to the initial release time of each cascade, and adding a true label to the macro prediction task based on the cascade size; An information propagation prediction model is constructed, wherein the information propagation prediction model includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch, and the three branches share a feature extractor; wherein the directed social graph and the sequence diffusion hypergraph are input into the feature extractor, and the user social relationship representation, the user dynamic interaction representation and the shared cascade representation are learned and input into the three branches; in the macro prediction branch, each cascade in the historical cascade set is sequence enhanced to generate an enhanced training environment, and a macro-task-specific sequence modeling is performed according to the user dynamic interaction representation to obtain a refined cascade representation, which is spliced ​​with the shared cascade representation and predicted by the macro predictor to obtain a cascade representation. Cascade scale; in the micro prediction branch, according to the enhanced training environment and the user social relationship representation, a micro task-specific sequence modeling is performed to obtain a refined cascade representation, which is spliced ​​with the shared cascade representation and predicted by the micro predictor to obtain the next propagation user; in the auxiliary self-supervised learning branch, some users in each cascade are randomly masked, and the user sequence is converted into a user embedding sequence according to the shared cascade representation, and the visible user embedding and the masked user embedding are set, and the visible user embedding is input into the encoder to generate a visible representation of the visible user, and the visible representation and the masked user embedding are input into the decoder to reconstruct the embedding of each user in the user embedding sequence; The information propagation prediction model is trained using the training sample set, a joint loss of macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss is constructed, and the information propagation prediction model is optimized with the goal of minimizing the joint loss; The optimized information propagation prediction model is trained during testing using the test sample set, the parameters of the macro prediction branch and the micro prediction branch are fixed, and the parameters of the auxiliary self-supervised learning branch and the feature extractor are optimized with the goal of minimizing the auxiliary self-supervised learning task loss to obtain the final information propagation prediction model.

[0010] In some embodiments of the present invention, the directed social graph and the sequence diffusion hypergraph are input into the feature extractor to learn and obtain user social relationship representation, user dynamic interaction representation and shared cascade representation, including: The feature extractor includes a social relationship encoder, a diffusion interaction encoder, and a shared representation module; Inputting the directed social graph into the social relationship encoder, and extracting the user social relationship representation through a multi-layer graph convolutional neural network; Inputting the sequence diffusion hypergraph into the diffusion interaction encoder, introducing a gated fusion strategy, and capturing the dynamic interactions between users in each time interval through a hypergraph neural network to extract the user dynamic interaction representation; For each enhanced cascade, the social relationship representation and dynamic interaction representation of each user are retrieved and input into the shared representation module, and the contextual interaction between users is modeled through a long short-term memory network to obtain a shared cascade representation shared by macro tasks and micro tasks.

[0011] In some embodiments of the present invention, in the macro prediction branch or the micro prediction branch, performing sequence enhancement on each cascade in the historical cascade set to generate an enhanced training environment includes: Each cascade in the historical cascade set is enhanced by adopting a sequence enhancement strategy of user masking and user rearrangement.

[0012] In some embodiments of the present invention, a macro-task-specific sequence modeling is performed according to the user dynamic interaction representation to obtain a refined cascade representation, which is concatenated with the shared cascade representation and predicted by a macro predictor to obtain a cascade scale, including: For each enhanced cascade, the user embedding obtained according to the user dynamic interaction representation is input into a macro long short-term memory network to generate a macro cascade representation; Concatenating the macroscopic cascade representation with the shared cascade representation to obtain a macroscopic concatenated representation; The macro concatenated representation is input into a macro multilayer perceptron for macro prediction to obtain the cascade scale.

[0013] In some embodiments of the present invention, according to the enhanced training environment and the user social relationship representation, a micro-task-specific sequence modeling is performed to obtain a refined cascade representation, which is concatenated with the shared cascade representation and predicted by a micro-predictor to obtain the next propagation user, including: For each enhanced cascade, the user embedding obtained according to the user social relationship representation is input into a micro long short-term memory network to generate a micro cascade representation; Concatenating the microscopic cascade representation with the shared cascade representation to obtain a microscopic concatenated representation; The micro-concatenated representation is input into a micro-multi-layer perceptron for micro-prediction to obtain the next propagation user.

[0014] In some embodiments of the present invention, the auxiliary self-supervised learning branch adopts an asymmetric encoder-decoder structure, and the method includes: Randomly mask some users in each cascade and replace their user tags with a unified mask tag; Converting the user sequence into a user embedding sequence according to the shared cascade representation; setting visible user embeddings for visible users, and sharing the same masked user embeddings for masked users; Embedding the visible user into an input encoder, and generating a visible representation of the visible user by multiple Transformer layers; The visible representation and the masked user embedding are input into a decoder, and the embedding of each user in the user embedding sequence is reconstructed by multiple Transformer layers.

[0015] In some embodiments of the present invention, a joint loss of a macro prediction task loss, a micro prediction task loss, an auxiliary self-supervised learning task loss, and a task consistency loss is constructed, including: The macro prediction task loss adopts the mean absolute error loss, which is calculated as follows: ; in, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, A parameter representing the macro prediction branch; is a hyperparameter; Indicates the number of enhanced training environments; The micro prediction task loss adopts cross entropy loss; The auxiliary self-supervised learning task loss adopts reconstruction loss; The task consistency loss is used to ensure that the optimization direction of the feature extractor parameters of the macro prediction task, the micro prediction task and the auxiliary self-supervised learning task are consistent, and the calculation formula is: ; in, represents parameters of the feature extractor; Representing parameters of the auxiliary self-supervised learning branch; A parameter representing the macro prediction branch; A parameter representing the micro-prediction branch; represents the gradient of the macro prediction task loss with respect to the feature extractor parameters; represents the gradient of the micro prediction task loss with respect to the feature extractor parameters; Represents the gradient of the auxiliary self-supervised learning task loss with respect to the feature extractor parameters.

[0016] In some embodiments of the present invention, optimizing the information propagation prediction model with the goal of minimizing the joint loss includes: The information propagation prediction model is jointly optimized by minimizing the weighted sum of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss and the task consistency loss. The calculation formula is: ; in, represents the macro prediction task loss; represents the micro prediction task loss; represents the auxiliary self-supervised learning task loss; represents the task consistency loss; represents parameters of the feature extractor; Representing parameters of the auxiliary self-supervised learning branch; A parameter representing the macro prediction branch; A parameter representing the micro-prediction branch; represents a hyperparameter.

[0017] On the other hand, the present invention also provides an information propagation prediction method based on test-time training, the method comprising: Get the cascade to be predicted; The cascade is input into the information propagation prediction model trained based on the information propagation prediction model training method based on test-time training as described above to obtain the cascade scale and the next propagation user.

[0018] On the other hand, the present invention also provides an information propagation prediction system based on test-time training, which implements the steps of any of the methods mentioned above when executed.

[0019] The present invention provides a training method, prediction method and system of an information propagation prediction model based on test-time training, including: constructing a directed social graph and a sequence diffusion hypergraph according to a user set and a cascade set, thereby constructing a training set and a test set; constructing an information propagation prediction model, which includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch introducing MAE, and the three branches share a feature extractor; introducing invariant learning in the macro prediction branch and the micro prediction branch to improve the generalization ability of the model to multi-environment distribution; during training, the information propagation prediction model is jointly trained based on the training set, and consistency loss is introduced to ensure that the optimization direction of the auxiliary task and the main task for the feature extractor parameters is consistent; during testing, for a specific test sample, the test-time training is performed with the goal of minimizing the auxiliary self-supervised learning task loss, the feature extractor is adjusted to adapt to the distribution of the sample, and then the information propagation prediction model is obtained in combination with a task-specific prediction head. The present invention can realize multi-scale information propagation prediction, solve the distribution offset problem in the information propagation prediction task, and improve the generalization performance of the model under different test environments.

[0020] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0021] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings: Figure 1 Schematic diagram of the steps of a method for training an information propagation prediction model based on test-time training in one embodiment of the present invention.

[0023] Figure 2 Schematic diagram of the flow of a method for training an information propagation prediction model based on test-time training in one embodiment of the present invention.

[0024] Figure 3 It is an overall framework diagram of the information propagation prediction model in one embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0026] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0027] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0028] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0030] In order to solve the problem of poor generalization ability and prediction performance of the prediction model caused by distribution offset in the existing information propagation prediction technology solutions, the present invention provides an information propagation prediction model training method based on test-time training, such as Figure 1 As shown, the method includes the following steps S101-S104: Step S101: Construct a user set and a historical cascade set, wherein the user set includes multiple users, the historical cascade set includes multiple cascades, and each cascade records the propagation process of an information item through the corresponding user according to time. Construct a directed social graph based on the user set and the social relationship between users, divide the historical cascade set according to timestamps, and construct a sequence diffusion hypergraph; divide the training sample set and the test sample set according to the initial release time of each cascade, and add a true label to the macro prediction task based on the cascade size.

[0031] Step S102: construct an information propagation prediction model, which includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch, and the three branches share a feature extractor.

[0032] Among them, the directed social graph and the sequence diffusion hypergraph are input into the feature extractor, and the user social relationship representation, user dynamic interaction representation and shared cascade representation are learned and input into three branches; in the macro prediction branch, each cascade in the historical cascade set is sequence enhanced to generate an enhanced training environment, and the macro task-specific sequence modeling is performed according to the user dynamic interaction representation to obtain a refined cascade representation, which is concatenated with the shared cascade representation and predicted by the macro predictor to obtain the cascade scale; in the micro prediction branch, according to the enhanced training environment and user social relationship representation, the micro task-specific sequence modeling is performed to obtain a refined cascade representation, which is concatenated with the shared cascade representation and predicted by the micro predictor to obtain the next propagation user; in the auxiliary self-supervised learning branch, some users in each cascade are randomly masked, and the user sequence is converted into a user embedding sequence according to the shared cascade representation, the visible user embedding and the masked user embedding are set, the visible user embedding is input into the encoder, the visible representation of the visible user is generated, the visible representation and the masked user embedding are input into the decoder, and the embedding of each user in the user embedding sequence is reconstructed.

[0033] Step S103: Use the training sample set to train the information propagation prediction model, construct the joint loss of macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss, and optimize the information propagation prediction model with the goal of minimizing the joint loss.

[0034] Step S104: Use the test sample set to train the optimized information propagation prediction model during testing, fix the parameters of the macro prediction branch and the micro prediction branch, optimize the parameters of the auxiliary self-supervised learning branch and the feature extractor with the goal of minimizing the auxiliary self-supervised learning task loss, and obtain the final information propagation prediction model.

[0035] like Figure 2 As shown, it is a flow chart of the information propagation prediction model training method based on test-time training, which includes two parts: joint training and test-time training.

[0036] In step S101, a training sample set and a test sample set are constructed for model training and testing.

[0037] Construct a user set and a set of observed historical cascades, where each cascade records the propagation process of the information item in ascending time order. For example, the user set is denoted as , historical cascade collection , then each cascade ,in Indicates user In timestamp Forwarded information item .

[0038] A directed social graph is constructed based on the user set and the historical cascade set, and the historical cascade set is divided into multiple subsets based on timestamps to construct a sequence diffusion hypergraph. For example, the directed social graph is denoted as ,in, For the user set, is an edge set, representing the social relationship between users. The sequence diffusion hypergraph is denoted as ,in, , and Respectively represent The user set and hyperedge set in each diffusion hypergraph In , a hyperedge only connects users who participate in the same cascade within the time interval.

[0039] In some embodiments, the release times of all cascades are sorted, and the training sample set, the validation sample set, and the test sample set are divided according to a ratio of 8:1:1.

[0040] In step S102, an information propagation prediction model is constructed, such as Figure 3The figure shows the overall structure of the model and the joint training process. The information propagation prediction model includes a macro prediction branch, a micro prediction branch, and an auxiliary self-supervised learning branch. The three branches share a feature extractor and are each equipped with an independent prediction head. The main macro prediction branch uses a task-specific prediction head to perform macro prediction tasks, the main micro prediction branch uses a task-specific prediction head to perform micro prediction tasks, and the auxiliary self-supervised learning branch fine-tunes the shared feature extractor through self-supervised learning to adapt it to a specific test cascade. Specifically: The directed social graph and sequence diffusion hypergraph are input into the feature extractor to learn the user social relationship representation, user dynamic interaction representation and shared cascade representation, which capture the social relationships and global interactions between users and are finally input into three branches.

[0041] In some embodiments, the feature extractor includes three modules: a social relationship encoder, a diffuse interaction encoder, and a shared representation module.

[0042] The directed social graph is input into the social relationship encoder, and the user social relationship representation is extracted through a multi-layer graph convolutional neural network (GCN), which can be recorded as , represents the size of the user set, Represents dimension.

[0043] The sequential diffusion hypergraph is input into the diffusion interaction encoder, and the cross-cascade user interactions are modeled in each time interval by the Hypergraph NeuralNerworks (HGNN), where message aggregation includes two steps: node-to-hyperedge aggregation and hyperedge-to-node aggregation. Considering that a single hypergraph neural network only captures the interactions in a specific time interval, it is not enough to fully model the evolution of the cascade. To address this limitation, the present invention introduces a gated fusion strategy, which sequentially integrates the interactions learned by the hypergraph neural network in different time intervals to obtain the final user dynamic interaction representation, which can be recorded as , represents the size of the user set, Represents dimension.

[0044] Finally, a long short-term memory network is introduced to model the contextual interactions between users in the cascade. Specifically: for each cascade, the corresponding user embeddings are found from the user social relationship representation and the user dynamic interaction representation, and arranged in the original order of the cascade, such as At each time step of the LSTM network, the two sets of embeddings corresponding to the user are taken as input, thus obtaining the embeddings of each user in the cascade. dimensional representation, which is a cascade representation shared by subsequent macro tasks and micro tasks, denoted as .

[0045] The user social relationship representation, user dynamic interaction representation and shared cascade representation learned by the feature extractor are input into three branches respectively to achieve specific tasks.

[0046] In the macro prediction branch, invariant learning is introduced. Deep neural networks tend to exploit spurious correlations that are easy to fit (i.e., shortcut strategies) when solving problems. Invariant learning is a technique designed to solve this problem, which aims to reveal the invariant causal relationship between features and target labels across environments while reducing the impact of variable spurious correlations, thereby achieving satisfactory distribution out-of-distribution generalization under distribution changes. Therefore, the present invention adopts variance risk extrapolation to train an invariant model that can generalize to invisible offset test data.

[0047] First, if Figure 3 As shown in (c) environment construction, multiple training environments are constructed from historical cascade sets. For each cascade, typical sequence enhancement strategies such as user masking and user reordering are used to enhance and generate enhanced training environments. The macro prediction tasks of each environment are processed independently.

[0048] For each enhanced cascade , from the above user dynamic interaction representation User embedding composition , and input into the macroscopic long short-term memory network to generate a macroscopic cascade representation, denoted as .

[0049] Representing the macro cascade With shared cascade representation After concatenation, it is input into the macro multi-layer perceptron (MLP) for macro prediction to obtain the cascade scale. Figure 3 As shown in Figure 2, the task-specific long short-term memory network (LSTM) and the predictor form the macro prediction branch, whose parameters are .

[0050] In the micro prediction branch, it is similar to the macro prediction branch, the difference lies in its learning objective.

[0051] The micro prediction branch and the macro prediction branch use the same enhanced prediction environment, and the micro prediction tasks of each environment are processed independently.

[0052] For each enhanced cascade, user embeddings from the above user social relationship representation are constructed and input into a micro long short-term memory network to generate a micro cascade representation.

[0053] After the micro-cascade representation is concatenated with the shared cascade representation, it is input into multiple micro-multilayer perceptrons (MLPs) for micro-prediction to obtain the next propagation user. Figure 3 As shown in Figure 2, the task-specific long short-term memory network (LSTM) and the predictor constitute the micro prediction branch, whose parameters are .

[0054] The key to test-time training (TTT) is to select appropriate auxiliary self-supervised learning tasks. This paper proposes to introduce a self-supervised learning task based on Masked Autoencoder (MAE) in the auxiliary self-supervised learning branch, and introduce consistency loss in subsequent training to align the self-supervised learning task with the main task (macro prediction task loss and micro prediction task loss).

[0055] In the auxiliary self-supervised learning branch, such as Figure 3 As shown, the MAE-based auxiliary self-supervised learning adopts an asymmetric encoder-decoder structure.

[0056] Consider any cascade, randomly mask some of its users, and replace their user tags with a uniform mask tag. Then, use the shared cascade representation learned by the feature extractor Convert the user sequence to a user embedding sequence. For visible users , whose user embedding is , for masked users, they share the same embedding .

[0057] In the encoder module, embed the visible user Input encoder, multiple Transformer layers generate visible representation of visible user, denoted as In the decoder module, the visible representation and masking user embeds The decoder is input and multiple Transformer layers are used to reconstruct the embeddings of all users in the user embedding sequence.

[0058] In step S103, the information propagation prediction model is trained using a training sample set to construct a joint loss of macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss, with the goal of minimizing the joint loss to optimize the information propagation prediction model.

[0059] In some embodiments, considering that the macro prediction task belongs to a regression task, the mean absolute error loss is selected as the loss function in a single environment. The goal of the present invention is to reduce the average training risk while increasing the similarity of training risks between different environments. The specific optimization goal is as shown in formula (1): ; (1) in, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, Represents the parameters of the macro prediction branch; is a hyperparameter that controls the balance between reducing the average risk and enforcing equal risks; Indicates the number of enhanced training environments.

[0060] In some embodiments, the micro prediction task loss adopts cross entropy loss.

[0061] In some embodiments, the auxiliary self-supervised learning task loss adopts reconstruction loss.

[0062] In some embodiments, given that the gradient correlation between the loss functions of the auxiliary self-supervised learning task and the main task (macro prediction task and micro prediction task) is crucial to the success of training during testing, the present invention proposes a task consistency loss to ensure that all tasks maintain consistent optimization directions for shared feature extractor parameters, thereby improving overall performance.

[0063] The definition of task consistency loss is shown in formula (2): ; (2) in, Represents the parameters of the feature extractor; Represents the parameters of the auxiliary self-supervised learning branch; Represents the parameters of the macro prediction branch; represents the parameters of the micro-prediction branch; represents the gradient of the macro prediction task loss with respect to the feature extractor parameters; represents the gradient of the micro-prediction task loss with respect to the feature extractor parameters; represents the gradient of the auxiliary self-supervised learning task loss with respect to the feature extractor parameters.

[0064] By minimizing the weighted sum of the macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss, the information propagation prediction model is combined, and the calculation formula is shown in formula (3): ; (3) in, represents the macro prediction task loss; represents the micro-prediction task loss; represents the auxiliary self-supervised learning task loss; represents the task consistency loss; Represents the parameters of the feature extractor; Represents the parameters of the auxiliary self-supervised learning branch; Represents the parameters of the macro prediction branch; represents the parameters of the micro-prediction branch; Represents a hyperparameter.

[0065] Finally, the parameters of the feature extractor are obtained , parameters of the auxiliary self-supervised learning branch , parameters of the macro prediction branch , parameters of micro-prediction branches .

[0066] In step S104, Figure 2 As shown in the figure, during the test training phase, the optimized information propagation prediction model is tested and trained using the test sample set, and the macro prediction branch is fixed. and the parameters of the micro-predicted branches , to minimize the reconstruction loss of the auxiliary self-supervised learning task As the target, multi-step gradient descent is applied to fine-tune the parameters of the feature extractor, as shown in formula (4): ; (4) By optimizing the auxiliary self-supervised learning task loss, the learned parameters and Adjustments are made to specific test cascades, allowing the model to extract better representations, effectively capture complex user interaction characteristics, and even handle cascades that have never been seen before.

[0067] The present invention also provides an information propagation prediction method based on test-time training, the method comprising the following steps S201-S202: Step S201: Obtain the cascade to be predicted.

[0068] Step S202: input the cascade into the information propagation prediction model trained based on the information propagation prediction model training method based on test-time training described above, and obtain the cascade scale and the next propagation user.

[0069] In some embodiments, after the information propagation prediction model training method based on test-time training described above, the final feature extractor parameters are obtained after the test-time training. , the feature extractor parameters The parameters of the macro prediction branch are , parameters of micro-prediction branches The structure can be composed of two models: , Among them, the model For macro forecasting, models For micro forecasting.

[0070] Corresponding to the above method, the present invention also provides an information propagation prediction system based on test-time training, which implements the steps of any of the methods described above when executed.

[0071] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0072] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0073] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0074] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0075] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for training an information propagation prediction model based on test-time training, characterized in that: The method comprises the following steps: Construct a user set and a historical cascade set, wherein the user set includes multiple users, and the historical cascade set includes multiple cascades, each cascade recording the propagation process of an information item through the corresponding user according to time; construct a directed social graph according to the user set and the social relationship between users, divide the historical cascade set according to the timestamp, and construct a sequence diffusion hypergraph; divide the training sample set and the test sample set according to the initial release time of each cascade, and obtain the true label of the macro prediction task based on the cascade size; An information propagation prediction model is constructed, wherein the information propagation prediction model includes a macro prediction branch, a micro prediction branch and an auxiliary self-supervised learning branch, and the three branches share a feature extractor; wherein the directed social graph and the sequence diffusion hypergraph are input into the feature extractor, and the user social relationship representation, the user dynamic interaction representation and the shared cascade representation are learned and input into the three branches; in the macro prediction branch, each cascade in the historical cascade set is sequence enhanced to generate an enhanced training environment, and a macro-task-specific sequence modeling is performed according to the user dynamic interaction representation to obtain a refined cascade representation, which is spliced ​​with the shared cascade representation and predicted by the macro predictor to obtain a cascade representation. Cascade scale; in the micro prediction branch, according to the enhanced training environment and the user social relationship representation, a micro task-specific sequence modeling is performed to obtain a refined cascade representation, which is spliced ​​with the shared cascade representation and predicted by the micro predictor to obtain the next propagation user; in the auxiliary self-supervised learning branch, some users in each cascade are randomly masked, and the user sequence is converted into a user embedding sequence according to the shared cascade representation, and the visible user embedding and the masked user embedding are set, and the visible user embedding is input into the encoder to generate a visible representation of the visible user, and the visible representation and the masked user embedding are input into the decoder to reconstruct the embedding of each user in the user embedding sequence; The information propagation prediction model is trained using the training sample set, a joint loss of macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss is constructed, and the information propagation prediction model is optimized with the goal of minimizing the joint loss; The optimized information propagation prediction model is trained during testing using the test sample set, the parameters of the macro prediction branch and the micro prediction branch are fixed, and the parameters of the auxiliary self-supervised learning branch and the feature extractor are optimized with the goal of minimizing the auxiliary self-supervised learning task loss to obtain the final information propagation prediction model.

2. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: Inputting the directed social graph and the sequence diffusion hypergraph into the feature extractor, learning to obtain user social relationship representation, user dynamic interaction representation and shared cascade representation, including: The feature extractor includes a social relationship encoder, a diffusion interaction encoder, and a shared representation module; Inputting the directed social graph into the social relationship encoder, and extracting the user social relationship representation through a multi-layer graph convolutional neural network; Inputting the sequence diffusion hypergraph into the diffusion interaction encoder, introducing a gated fusion strategy, and capturing the dynamic interactions between users in each time interval through a hypergraph neural network to extract the user dynamic interaction representation; For each enhanced cascade, the social relationship representation and dynamic interaction representation of each user are retrieved and input into the shared representation module, and the contextual interaction between users is modeled through a long short-term memory network to obtain a shared cascade representation shared by macro tasks and micro tasks.

3. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: In the macro prediction branch or the micro prediction branch, performing sequence enhancement on each cascade in the historical cascade set to generate an enhanced training environment includes: Each cascade in the historical cascade set is enhanced by adopting a sequence enhancement strategy of user masking and user rearrangement.

4. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: After performing sequence modeling specific to a macro task according to the user dynamic interaction representation, a refined cascade representation is obtained, and the cascade representation is concatenated with the shared cascade representation, and the cascade scale is obtained by a macro predictor for prediction, including: For each enhanced cascade, the user embedding obtained according to the user dynamic interaction representation is input into a macro long short-term memory network to generate a macro cascade representation; Concatenating the macroscopic cascade representation with the shared cascade representation to obtain a macroscopic concatenated representation; The macro concatenated representation is input into a macro multilayer perceptron for macro prediction to obtain the cascade scale.

5. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: According to the enhanced training environment and the user social relationship representation, a micro-task-specific sequence modeling is performed to obtain a refined cascade representation, which is concatenated with the shared cascade representation and predicted by a micro-predictor to obtain the next propagation user, including: For each enhanced cascade, the user embedding obtained according to the user social relationship representation is input into a micro long short-term memory network to generate a micro cascade representation; Concatenating the microscopic cascade representation with the shared cascade representation to obtain a microscopic concatenated representation; The micro-concatenated representation is input into a micro-multi-layer perceptron for micro-prediction to obtain the next propagation user.

6. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: The auxiliary self-supervised learning branch adopts an asymmetric encoder-decoder structure, and the method includes: Randomly mask some users in each cascade and replace their user tags with a unified mask tag; Converting the user sequence into a user embedding sequence according to the shared cascade representation; setting visible user embeddings for visible users, and sharing the same masked user embeddings for masked users; Embedding the visible user into an input encoder, and generating a visible representation of the visible user by multiple Transformer layers; The visible representation and the masked user embedding are input into a decoder, and the embedding of each user in the user embedding sequence is reconstructed by multiple Transformer layers.

7. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: Construct a joint loss of macro prediction task loss, micro prediction task loss, auxiliary self-supervised learning task loss and task consistency loss, including: The macro prediction task loss adopts the mean absolute error loss, which is calculated as follows: ； in, represents the mean absolute error loss in a single training environment, represents the parameters of the feature extractor, A parameter representing the macro prediction branch; is a hyperparameter; Indicates the number of enhanced training environments; The micro prediction task loss adopts cross entropy loss; The auxiliary self-supervised learning task loss adopts reconstruction loss; The task consistency loss is used to ensure that the optimization direction of the feature extractor parameters of the macro prediction task, the micro prediction task and the auxiliary self-supervised learning task are consistent, and the calculation formula is: ； in, represents parameters of the feature extractor; Representing parameters of the auxiliary self-supervised learning branch; A parameter representing the macro prediction branch; A parameter representing the micro-prediction branch; represents the gradient of the macro prediction task loss with respect to the feature extractor parameters; represents the gradient of the micro prediction task loss with respect to the feature extractor parameters; Represents the gradient of the auxiliary self-supervised learning task loss with respect to the feature extractor parameters.

8. The information propagation prediction model training method based on test-time training according to claim 1, characterized in that: The information propagation prediction model is optimized with the goal of minimizing the joint loss, including: The information propagation prediction model is jointly optimized by minimizing the weighted sum of the macro prediction task loss, the micro prediction task loss, the auxiliary self-supervised learning task loss, and the task consistency loss. The calculation formula is: ； in, represents the macro prediction task loss; represents the micro prediction task loss; represents the auxiliary self-supervised learning task loss; represents the task consistency loss; represents parameters of the feature extractor; Representing parameters of the auxiliary self-supervised learning branch; A parameter representing the macro prediction branch; A parameter representing the micro-prediction branch; Represents a hyperparameter.

9. A method for predicting information propagation based on test-time training, characterized in that: The method comprises: Get the cascade to be predicted; The cascade is input into the information propagation prediction model trained by the information propagation prediction model training method based on test-time training described in claims 1 to 8 to obtain the cascade scale and the next propagation user.

10. An information propagation prediction system based on test-time training, characterized in that: When the system is executed, the steps of the method according to any one of claims 1 to 9 are implemented.

Images