Anti-data drift recommendation method based on hybrid expert and dynamic graph neural network

Abstract

The application discloses an anti-data deviation recommendation method based on a mixed expert and dynamic graph neural network, and belongs to the technical field of intelligent recommendation systems. The dynamic graph neural network comprises a user portrait encoder and an expert mixed decoder. The method comprises the following steps: step 1, collecting and fusing multi-source data of users from a social platform to construct comprehensive features; step 2, constructing a user portrait encoder based on variational inference to perform dynamic coding and generate original latent representations; step 3, constructing an intervention network to extrapolate the original latent representations into a mixture of multiple intervened latent representations; step 4, constructing an expert mixed decoder to decode and aggregate the original latent representations and the intervened latent representations to generate a final recommendation result; and step 5, designing a composite training target and a double-layer optimization strategy. The application can maintain high accuracy and stability even in the case of significant data distribution deviation, and significantly improves user experience and platform activity.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent recommendation systems, and particularly relates to a data drift resistant recommendation method based on a hybrid expert and dynamic graph neural network. BACKGROUND

[0002] With the rapid development of social platforms, the behavior data (such as posting, liking, forwarding, commenting, and following) and social relationship network structure of users on the platform show obvious dynamic evolution characteristics. User interests can quickly change due to hot topics, popular content, friend interactions, or external events, resulting in continuous migration of user behavior distribution and content popularity distribution over time. This dynamic nature makes the recommendation model trained based on historical behavior prone to performance degradation when actually deployed, i.e., the so-called "distribution drift" problem. How to effectively adapt to the evolution of user behavior preferences in social platforms and maintain stable recommendation performance in distribution outside environments has become a key challenge for the development of recommendation technology. Currently, the mainstream methods to improve the generalization ability of social platform recommendation systems in distribution drift environments mainly include two categories: (1) data augmentation-based methods. This type of method forces the model to learn in different "simulated environments" by perturbing user interaction behavior data or constructing virtual scenarios, thereby improving its robustness in real distribution changes. For example, some studies attempt to generate multiple training environments by randomly adding or deleting interaction records, perturbing user feature vectors, or dividing different time segments. The model learns in these environments simultaneously, thereby enhancing its cross-distribution stability. However, this type of method often relies on limited pre-defined rules, making it difficult to comprehensively cover the complex change scenarios that may occur in user behavior and content ecology in social platforms. (2) disentangled representation learning-based methods. This type of method attempts to split user interest representation into "long-term stable core interest" and "short-term dynamic variable interest", and uses the core interest part for recommendation to enhance the model's generalization ability in distribution changes. For example, some solutions identify stable interest themes in user history as domain-invariant features, while viewing rapidly changing behavior patterns over time as domain-specific features, which are only used in the training stage. However, in actual social platform environments, user long-term interests and short-term interests are often highly coupled, with hot events, social interactions, and personalized behaviors intertwined, making it difficult for this type of method to accurately separate the two, and the computational overhead for large-scale social graph data is huge.

[0003] Although the above methods alleviate the out-of-distribution generalization problem to some extent, there are still the following core shortcomings in the social platform recommendation scene: (1) Dependence on complex graph operations: whether it is data augmentation or feature disentanglement, most existing methods directly perform complex operations on the original social network graph or user behavior sequence. For example, data augmentation methods usually generate virtual data by pre-set rules (such as randomly modifying user relationship edges, perturbing behavior features, etc.), but the evolution of user interest in a real social platform is often the result of multiple factors such as hot topics, social interactions and external events. The complexity makes it difficult for these rules to be fully covered. Similarly, methods based on feature disentanglement attempt to separate a user's long-term interest from their short-term interest, but in actual platforms the two are highly intertwined, making direct separation not only computationally expensive, but also difficult to obtain reliable results. (2) The modeling of user interest evolution is too static or simple: some methods implicitly assume a static assumption when designed, such as assuming that there is always a part of the user's interest that is "stable and unchanging", and another part that is "dynamic and variable". However, in a social platform, the importance and content of a hot topic are constantly changing over time, for example, a user may focus on a hot topic for a short period of time and then quickly switch to other areas. This static assumption makes it difficult for existing methods to adapt flexibly to the real dynamic environment. SUMMARY

[0004] In order to solve the problem of performance degradation of recommendation model caused by the distribution shift of behavior data due to factors such as rapid replacement of hot topics and evolution of user interest circles in social platforms, an anti-data shift recommendation method based on mixed experts and dynamic graph neural networks is proposed. The method first integrates the multi-source behavior data, content features and social relationship network of users on the social platform, and maps these heterogeneous information to a low-dimensional continuous latent space through a user portrait encoder based on variational inference, to obtain a more expressive latent representation. Then, by intervening in the latent distribution, more diverse and rich interest evolution patterns are generated, thereby improving the generalization ability of the recommendation model. Finally, a mixed expert decoder is designed, where each expert models a specific latent interest distribution shift, and an alignment loss is combined to enhance the generalization ability of each expert, thereby improving the performance of the recommendation model in the out-of-distribution generalization scenario.

[0005] The technical solution of the present application is as follows:

[0006] An anti-data shift recommendation method based on mixed experts and dynamic graph neural networks, the dynamic graph neural network comprising a user portrait encoder and a mixed expert decoder; specifically comprising the following steps:

[0007] Step 1, collect and integrate multi-source data of users from a social platform, and construct comprehensive features;

[0008] Step 2: Construct a user profile encoder based on variational inference to perform dynamic encoding and generate the original latent representation;

[0009] Step 3: Construct an intervention network to extrapolate the original latent representations into a mixture of multiple intervention-related latent representations;

[0010] Step 4: Construct an expert hybrid decoder to decode and aggregate the original latent representation and the latent representation after intervention to generate the final recommendation result;

[0011] Step 5: Design composite training objectives and a two-layer optimization strategy.

[0012] Furthermore, the specific process of step 1 is as follows:

[0013] Step 1.1: The collected multi-source data includes static attribute data and dynamic interaction behavior data; among which, static attribute data includes user personal information; dynamic interaction behavior data includes behavior sequence data and time dynamic information; behavior sequence data includes posting, liking, forwarding, commenting, and collecting; time dynamic information includes the timestamps of the behavior.

[0014] Step 1.2: Extract the self-features of users or interaction targets respectively; interaction targets are divided into two categories: one is posts or comments, and the other is target users; user personal information, posts, comments, and target user homepage text all belong to text information; the specific process of extracting self-features is as follows: first, the text is preprocessed, including word segmentation, removal of stop words and noise symbols; then, the preprocessed words are matched with the dictionary in the text analysis tool LIWC, the frequency of each word in different psychological, linguistic, and semantic categories is counted, and normalization is performed to obtain the corresponding semantic feature vector; the semantic feature vector extracted from the user's personal information is defined as the user's self-feature vector. At the present moment its own characteristics The semantic feature vector extracted from the interaction target is defined as the interaction target. At the present moment its own characteristics ;

[0015] Step 1.3: Perform one-hot encoding based on the interaction type to obtain interaction features; interaction types include posting, liking, forwarding, commenting, collecting, following, and unfollowing; during the one-hot encoding process, each interaction type is assigned a fixed feature position, and when an interaction occurs, the corresponding position is set to 1, and the remaining positions are set to 0;

[0016] Step 1.4: Extract the temporal features of the user or interaction target; for users At the present moment Historical interaction sequence , which includes the most recent Each interaction record corresponds to a historical timestamp. Encode each timestamp in the historical interaction sequence to obtain the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows:

[0017] ;

[0018] in, , All of these are trainable parameters; Encoding dimension for time;

[0019] Similarly, for the interaction target At the present moment Historical interaction sequence Each timestamp is encoded to obtain the historical interaction sequence. Time feature matrix ;

[0020] Step 1.5: Multiply each of the user's features by the learnable parameter matrix to obtain the user's complete features:

[0021] ;

[0022] in, For the current moment user Complete features; , , All are learnable parameters; For the current moment Interactive features;

[0023] Multiplying each feature of the interactive target by the learnable parameter matrix yields the complete features of the interactive target:

[0024] ;

[0025] in, For the current moment Interaction Target Complete features; , , All are learnable parameters; For the current moment Interactive features;

[0026] Step 1.6: Concatenate the user's complete features with the complete features of the interaction target to obtain the comprehensive features:

[0027] ;

[0028] in, For the current moment Its comprehensive characteristics.

[0029] Furthermore, in step 2, the specific working process of the user profile encoder based on variational inference is as follows:

[0030] Step 2.1: For the current time... Comprehensive characteristics Perform multi-head attention encoding:

[0031] ;

[0032] ;

[0033] ;

[0034] in, For the current moment Attention representation; This is a multi-head attention mechanism; , The first The, the One point of attention; Learnable parameters for the multi-head attention output layer; For attention mechanisms; , The first Individual attention head query ,key ,value Learnable parameters; Representative layer normalization; It is the transpose symbol; The key vector dimension for each attention head; For activation functions;

[0035] Step 2.2: Concatenate the current attention representation with the previous hidden state and input them into two independent feedforward neural networks to generate the mean and standard deviation of a Gaussian distribution, respectively.

[0036] ;

[0037] ;

[0038] ;

[0039] in, It is a feedforward neural network; These are the input features of the feedforward neural network; , , , All are learnable parameters; For the previous moment The hidden state; For activation functions; For the current moment The mean of a Gaussian distribution; For the current moment The standard deviation of the Gaussian distribution; A feedforward neural network for generating the mean; To generate a feedforward neural network for standard deviation;

[0040] Step 2.3: Obtain the approximate posterior distribution based on variational inference:

[0041] ;

[0042] in, Indicates the current moment Given , and Then, the approximate posterior distribution of the latent representation is inferred through the user profile encoder; Indicates the current moment The original latent representation obtained by sampling from an approximate posterior distribution; Indicates the current time The interactive relationship structure; Indicates a Gaussian distribution; Represents the diagonal covariance matrix;

[0043] The sampled data is obtained from the approximate posterior distribution using a reparameterization method, and the formula is as follows:

[0044] ;

[0045] in, Standard Gaussian noise, i.e. ; The unit covariance matrix; This is element-wise multiplication;

[0046] Step 2.4. Iteratively update the hidden state with the gating recurrent unit to model the dynamic evolution of interest over time:

[0047] ;

[0048] where, and are the hidden state at the current time and the previous time ; denotes the update function of the gating recurrent neural network.

[0049] Further, in the step 3, the intervention network is composed of independent perturbers , where is the learnable parameter of the th perturber;

[0050] Each perturber receives the original Gaussian distribution parameters and outputs a new set of Gaussian distribution parameters , and uses the reparameterization method to sample the latent representation after user intervention from the new Gaussian distribution parameters:

[0051] ;

[0052] ;

[0053] where, , are the mean and standard deviation of the Gaussian distribution of the th perturber at the current time ; denotes the latent representation sampled from the approximate posterior distribution after intervention of the th perturber at the current time ;

[0054] Finally, perturbers correspond to latent representations after intervention ;

[0055] The perturber uses two independent linear transformations to map the input , respectively:

[0056] ;

[0057] ;

[0058] where, 、 、 、 are the learnable parameters of the th perturber; is the activation function;

[0059] The training objective of the intervention network is to maximize the variance between each group of new Gaussian distribution parameters, which is defined as follows:

[0060] ;

[0061] where, is the intervention loss; and are hyperparameters, respectively controlling the weights of the mean and standard deviation diversity; is the variance calculation.

[0062] Further, in step 4, the expert mixture decoder contains a gating network and parallel expert decoders; the first expert decoder serves as a reference model to process the original latent representation ; the remaining expert decoders process the intervened latent representations , one expert decoder corresponding to one perturber; the gating network calculates the weight of each expert decoder, as follows:

[0063] ;

[0064] where, is the weight of the th expert decoder; , are learnable parameter matrices;

[0065] The final aggregated representation is obtained by weighting and summing the outputs of all expert decoders:

[0066] ;

[0067] where, is the aggregated representation at the current time ; is the th expert decoder;

[0068] In the expert mixture decoding stage, the vectors of the user and the interaction target are taken out from the aggregated representation , respectively, and averaged as their final representations:

[0069] ;

[0070] ;

[0071] wherein, , are the final representations of the user and the interaction target at the current time ; is an average operation; and are the index lengths of the user and the interaction target at the current time ;

[0072] The inner product of the user final representation and the interaction target final representation is calculated, and normalized to obtain the interest probability of the user to the interaction target:

[0073] ;

[0074] wherein, represents the interest probability of the user to the interaction target at the current time ; is an activation function;

[0075] All interest probabilities are sorted in descending order, and the interaction targets corresponding to the top interest probabilities are output as the final recommendation results.

[0076] Further, in step 5, the composite training target contains two parts: evidence lower bound loss and alignment loss.

[0077] The evidence lower bound loss is a loss function of the user portrait encoder based on variational inference, used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: reconstruction loss of interaction relationship structure and KL divergence loss; the specific formula is:

[0078] ;

[0079] ;

[0080] ;

[0081] wherein, is the evidence lower bound loss; is the reconstruction loss of the interaction relationship structure; is the KL divergence loss; is the total number of time steps; is the current time previous interaction structure; is the current time previous comprehensive feature; is the current time , given , generates a conditional probability distribution; is the current time , based on a defined conditional prior distribution; is the current time , given , and , the approximate posterior distribution of the latent representation inferred by the user portrait encoder; is the current time , given , and , the approximate posterior distribution of the latent representation inferred by the first perturber; is the KL divergence;

[0082] The formula for the alignment loss is as follows:

[0083] ;

[0084] wherein , are the approximate posterior distributions of the first, second expert decoder, respectively; The double-layer optimization strategy includes inner-layer optimization and outer-layer optimization. When the inner-layer optimization is updated, the parameters of the intervention network

[0085] are updated, and when the outer-layer optimization is updated, the parameters of the encoder and the decoder are updated; ;

[0086] The update formula of the inner-layer optimization is as follows:

[0087] ;

[0088] wherein is the optimal parameter of the intervention network;

[0089] The update formula of the outer-layer optimization is as follows:

[0090] ;

[0091] wherein to optimize parameters of the encoder and the decoder; is a hyperparameter that controls the strength of alignment.

[0092] The beneficial technical effects brought by the present application are as follows.

[0093] (1) Feasibility and efficiency for social platform: most existing technologies directly operate on the original social relationship network or large-scale interaction graph data (such as modifying historical interaction data, user features, etc.), which has high computational cost and is difficult to apply in large-scale social platforms. The present application models user behavior, content interaction and social relationship data in a low-dimensional, continuous latent space, avoiding complex operations at the original network level, making dynamic recommendation for large-scale social platform data more feasible.

[0094] (2) Adapt to dynamic evolution of user interest: existing technologies based on disentanglement often imply the assumption of "fixed long-term user interest", but in social platforms, users will quickly shift their interest circles with new topics, popular content or social influences. This static assumption cannot meet the actual needs. The expert mixed decoder used in the present application dynamically allocates expert decoding weights for each user interaction (such as likes, forwards or comments) using a gating network, thereby being able to adapt to changes in user short-term behavior in real time, while actively learning the user's long-term stable interest by combining the alignment loss, achieving a balance between short-term hotspots and long-term preferences.

[0095] (3) Enhanced generalization ability and recommendation effect: existing methods based on data augmentation only rely on limited rules (such as random edge deletion, feature perturbation) to simulate distribution changes, which is difficult to cover the diversity of user behavior evolution in social platforms (such as from following hot topics to long-term interest in interest communities). The present application better covers complex situations such as user interest mutation and content ecological evolution by intervening in the distribution parameters in the latent space. In specific social platform recommendation tasks (such as content recommendation, etc.), the present application can still maintain high accuracy and stability under significant distribution changes, significantly improving user experience and platform activity. BRIEF DESCRIPTION OF DRAWINGS

[0096] Figure 1 is a flowchart of the present application based on a mixed expert and dynamic graph neural network anti-data drift recommendation method. DETAILED DESCRIPTION

[0097] The present application will be further described in detail below in conjunction with the drawings and specific embodiments:

[0098] The application provides a data drift resistant recommendation method based on a hybrid expert and dynamic graph neural network. The method is a dynamic graph neural network framework including a user portrait encoder and an expert hybrid decoder. The core of the method is to model the time sequence interaction structure of the user and the interaction target as a dynamic graph. The user portrait encoder is used to obtain a portrait representation capable of representing the current interest state of the user, and the preference distribution is modeled and intervened in the corresponding low-dimensional latent space. Finally, the expert hybrid decoder can accurately capture the data drift phenomenon caused by the dynamic evolution of the user interest. The specific purposes are as follows.

[0099] Purpose one: effectively depict the user portrait. The application aims to integrate multi-source heterogeneous data of the user to construct a comprehensive feature vector as the basis of the user behavior portrait. The portrait not only includes the static attributes of the user, but more importantly, can capture the time dynamics and preference intensity in the behavior sequence, providing a solid data basis for predicting the evolution trend of the user interest.

[0100] Purpose two: avoid direct operation on the original complex graph data, and turn to model the distribution drift in the latent space which is easier to handle. The application does not directly modify the interaction relationship structure or features in the social network, but first compresses the original complex graph data into a low-dimensional latent space using a user portrait encoder based on variational inference. In the space, the distribution drift is simulated and processed, which is more efficient and feasible than operating in the original data space.

[0101] Purpose three: generate more abundant and diverse user interest evolution trends by intervening in the latent distribution to improve the generalization ability of the social platform recommendation model. The application proposes a new "latent distribution extrapolation" strategy. Through a special intervention network, the probability distribution parameters (i.e. mean and standard deviation) of the user portrait in the latent space are systematically disturbed, thereby generating a series of new latent distributions as proxies for the possible evolution direction of the user's future interest. This way can simulate a wider range of distribution changes in a parameterized form, which has stronger coverage and flexibility than limited data enhancement rules.

[0102] Purpose four: use the expert hybrid decoder to specialize in processing different latent interest evolution patterns. For the generated multiple latent interest distributions, the application uses an expert hybrid decoder, which includes multiple parallel "expert" decoders. Each expert specializes in processing a specific latent interest distribution drift, thereby achieving fine modeling of the user's different interest evolution possibilities.

[0103] As shown in Figure 1 The core idea of the application is to enhance the generalization ability of the model by simulating diversified latent distribution drift under the dynamic graph neural network framework, which specifically includes the following steps:

[0104] Step 1, collect multi-source data of users from a social platform and fuse to build comprehensive features; the specific process is:

[0105] Step 1.1, the collected multi-source data specifically includes: static attribute data (such as user personal information) and dynamic interaction behavior data; wherein, the dynamic interaction behavior data further includes behavior sequence data (such as posting, liking, forwarding, commenting, collecting and other interaction behaviors), time dynamic information (such as the time stamp of the behavior occurrence, etc.). In the social platform, user behavior is dynamic and diverse. In order to build an accurate user portrait, the present application encodes each interaction of the user and the content. The core of this stage is to convert the behavior of the user at a certain time point (such as posting, liking, forwarding, commenting, collecting) into a vector representation that can be understood by a machine learning model, that is, to extract behavior features and time features from dynamic interaction behavior data, and integrate with static attribute (user personal information) data to generate a comprehensive feature vector. Specifically, the present application formalizes a single interaction into a triple , wherein represents the user, represents the interaction target (post or comment, target user, etc.), represents the current time when the interaction occurs. In order to depict the historical dependence of the interaction, the interaction target of the user before the current time is denoted as , and the user who interacts with the interaction target before the current time is denoted as . In order to represent such dynamic interaction, first, the first-order historical interaction sequence of the user before the current time is obtained by backtracking the interaction behavior of the user. Then, the first-order historical interaction sequence of the interaction target before the current time is obtained by backtracking the interaction behavior of the interaction target in the same way. Considering that the scale of interaction data is large, in order to avoid information redundancy, the present application sets a truncation strategy for the historical interaction sequence, only keeping the latest interaction records.

[0106] Step 1.2, extract the self features of the user or the interaction target from the static attribute data or the dynamic interaction behavior data, respectively;

[0107] For the historical interaction sequence of the user at the current time , it contains interaction records, and each interaction record corresponds to an interaction target The interaction target can be a post or a comment, or a target user. The interaction target at the current time The historical interaction sequence Also contains Interaction records, each of which corresponds to a user In the feature construction process, the personal information of the user (such as the personal homepage text) is regarded as static attribute data, and the post, the comment, and the homepage text of the target user belong to text information. To ensure the consistency of the feature extraction method, the present application adopts a text analysis tool LIWC based on a dictionary to uniformly process the text content of the user and the interaction target. Specifically, first, the analysis text (including the user's personal homepage text, the post or comment text, and the target user's homepage text) is preprocessed, including word segmentation, removal of stop words and noise symbols. Subsequently, the preprocessed vocabulary is matched with the LIWC dictionary, the occurrence frequency in different psychological, linguistic and semantic categories (such as emotion, cognition, sociality, time, etc.) is counted, and normalization processing is performed, so as to obtain a semantic feature vector. Finally, the semantic feature vector extracted from the user's personal information is defined as the user's at the current time Self-characteristics ; the semantic feature vector extracted from the interaction target (post or comment, target user, etc.) is defined as the interaction target at the current time Self-characteristics Each feature contains vectors, corresponding to the length of the historical interaction sequence and (That is, the interaction records correspond to vectors).

[0108] Step 1.3, One-Hot encoding according to the interaction type to obtain interaction features;

[0109] Interaction type refers to the different categories of interactive behaviors that occur between a user and the interaction target. The number of interaction categories may vary under different distribution offset scenarios. For example, in a typical scenario on a social platform, interaction types include posting, liking, forwarding, commenting, collecting, following, and unfollowing. Since interaction types are essentially discrete categorical data, directly inputting them into the model would lead to a lack of comparability between features. Therefore, this invention uses one-hot encoding for representation. In the one-hot encoding process, each interaction type is assigned a fixed feature position. When an interaction occurs, the corresponding position is set to 1, and all other positions are set to 0. For example, when the interaction type is "like," the encoding of the position corresponding to "like" is 1, while the positions of other interaction types are all 0, thus obtaining a vector representation with the same dimension as the number of interaction categories. In this way, the equidistance of different interaction types in the numerical space can be ensured, avoiding false ordering relationships between categories. Finally, at the current moment... The interaction features are defined as This feature vector can accurately characterize the types of interaction behaviors between the user and the interaction target, and provide input for subsequent feature fusion and modeling.

[0110] Step 1.4: Extract the temporal characteristics of users or interaction targets from dynamic interaction behavior data;

[0111] For users At the present moment Historical interaction sequence , which includes the most recent Each interaction record corresponds to a historical timestamp. To characterize the temporal features of the interaction, this invention encodes each timestamp in the historical interaction sequence, thereby obtaining the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows:

[0112] ;

[0113] in, , All of these are trainable parameters; Encoding dimension for time; For the current moment With historical moments The time interval. Therefore, the user At the current time The time feature matrix can be represented as It consists of a historical interaction sequence In The time of each interaction record is composed of a encoded vector obtained through time encoding.

[0114] Similarly, for the interaction target At the present moment Historical interaction sequence Each timestamp is encoded to obtain the historical interaction sequence. Time feature matrix Therefore, the interactive target At any moment Time feature matrix Also through corresponding historical interaction sequences In The time-encoded vector of each interaction record is obtained, and its length is consistent with that of the historical interaction sequence.

[0115] Step 1.5: To unify the dimensions of different features, this invention multiplies all user features by the learnable parameter matrix to obtain the user's complete features:

[0116] ;

[0117] in, For the current moment user Complete features; , , These are all learnable parameters used to align features of different dimensions; For the current moment Interactive features.

[0118] Similarly, by multiplying each feature of the interactive target by the learnable parameter matrix, we obtain the complete features of the interactive target:

[0119] ;

[0120] in, For the current moment Interaction Target Complete features; , , These are all learnable parameters used to align features of different dimensions; For the current moment Interactive features.

[0121] Step 1.6: Concatenate the user's complete features with the complete features of the interaction target to obtain the comprehensive features:

[0122] ;

[0123] in, For the current moment The comprehensive features fully characterize the interaction features between the user and the interaction target at that moment, providing input for subsequent user interest modeling and recommendation.

[0124] Step 2: Construct a user profile encoder based on variational inference to perform dynamic encoding and generate the original latent representation; then construct an accurate user profile.

[0125] In the dynamic graph neural network framework, the user profiling encoder, combined with a Transformer-based multi-head attention mechanism, can effectively capture the sequential dependencies in user behavior patterns. Unlike traditional encoders that output deterministic vectors, the user profiling encoder of this invention does not output a deterministic vector, but rather the parameters (including mean and standard deviation) of a latent Gaussian distribution, which probabilistically models the user's current comprehensive preferences. This not only expresses the uncertainty of user interests but also provides a flexible latent space representation for the simulation and intervention of subsequent interest evolution. For example, if a user first browses digital products and then follows review videos related to those products, this continuity of interest represents an evolutionary path of user interests. To better capture the dynamic evolution of social networks, this invention addresses the sequential dependencies of each node (user) at each moment. or interactive target Initialize the current time. state vector and The state of nodes is updated through a recurrent neural network; this mechanism enables the model to remember the user's long-term historical behavior and connect new interactions with existing interests.

[0126] The specific working process of a user profile encoder based on variational inference is as follows:

[0127] Step 2.1: First, for the current time... Comprehensive characteristics Multi-head attention encoding is performed because For users With the target of interaction By concatenating features, multi-dimensional dependencies between them can be captured:

[0128] ;

[0129] ;

[0130] ;

[0131] in, is the attention representation at the current time step ; is the multi-head attention mechanism; , are the first th, th attention head, respectively; are the learnable parameters of the multi-head attention output layer; is the attention mechanism; , are the learnable parameters of the first th attention head query , , , respectively; denotes layer normalization; is the transpose symbol; is the dimension of the key vector for each attention head; is the number of attention heads; is the activation function; Through the multi-head attention, the user portrait encoder can capture the diversified relationship patterns between users and interaction targets in different representation subspaces in parallel, so as to improve the expression ability of the dynamic graph neural network.

[0132] Step 2.2, then, the attention representation at the current time step is concatenated with the hidden state at the previous time step to input into two independent feed-forward neural networks (FFN) to generate the mean and standard deviation of the Gaussian distribution, respectively:

[0133] ;

[0134] ;

[0135] ;

[0136] wherein is the feed-forward neural network; is the input feature of the feed-forward neural network; , , , are all learnable parameters; is the hidden state at the previous time step ; is the activation function; is the mean of the Gaussian distribution at the current time step ; is the standard deviation of the Gaussian distribution at the current time step ; a feedforward neural network for generating the mean; a feedforward neural network for generating the standard deviation;

[0137] Step 2.3, the approximate posterior distribution of the latent representation obtained based on the variational inference is:

[0138] ;

[0139] wherein, represents the approximate posterior distribution of the latent representation; represents the latent representation sampled from the approximate posterior distribution at the current time point , given , and the approximate posterior distribution of the latent representation inferred by the user portrait encoder; represents the original latent representation sampled from the approximate posterior distribution at the current time point ; represents the interaction relationship structure at the current time point ; represents a Gaussian distribution; represents a diagonal covariance matrix.

[0140] In order to enable the model to be trained end-to-end by gradient descent, the present application adopts a reparameterization method to sample the original latent representation from the approximate posterior distribution as follows:

[0141] ;

[0142] wherein, is a standard Gaussian noise, i.e. ; is a unit covariance matrix; is an element-wise multiplication.

[0143] Step 2.4, in order to further model the process of dynamic evolution of interest over time, a gated recurrent unit is adopted to iteratively update the hidden state:

[0144] ;

[0145] wherein, and are the hidden states at the current time point and the previous time point ; represents an update function of the gated recurrent neural network. This design enables the model to maintain long-term memory while associating the newly generated interaction with the existing interest pattern, thereby effectively capturing the dynamic evolution characteristics of the interaction relationship structure in the social network.

[0146] The user portrait encoder adopts a probabilistic encoding mode, and the output is the current time The mean of the Gaussian distribution And the standard deviation Then, a reparameterization method is used to sample a time-evolving original latent distribution from the parameters of the Gaussian distribution So as to provide greater flexibility and expression ability for the subsequent latent distribution intervention stage.

[0147] Step 3, in order to simulate the diversified distribution offset of user preferences in the social platform, the present application constructs an intervention network for latent distribution extrapolation in the latent space generated by the user portrait encoder of the dynamic graph neural network based on variational inference; the intervention network is the core of simulating user interest evolution, which includes a group of learnable "perturbators" that systematically and diversely modify the original user latent preference distribution parameters (mean And the standard deviation Output by the user portrait encoder, thereby generating a plurality of new, intervened latent representations These newly generated distributions are used as "agents" of unknown interest shifts that the user may have in the real world (for example, interest broadening or interest migration). The intervention network maximizes the difference between the generated distributions through a special loss function to ensure that as wide an interest evolution scenario as possible can be simulated.

[0148] The present application extrapolates the original latent representation to a mixture of a plurality of intervened latent representations through the intervention network;

[0149] The intervention network is composed of Independent perturbators , The learnable parameters of the first Perturbator; wherein each perturbator receives the original Gaussian distribution parameters And outputs a new set of Gaussian distribution parameters And a reparameterization method is used to sample the user's intervened latent representation from the above parameters:

[0150] ;

[0151] ;

[0152] Wherein, , The mean and standard deviation of the Gaussian distribution of the first Perturbator at the current time ; Indicates the mean and standard deviation of the Gaussian distribution of the first Perturbator at the current time the latent representation sampled from the approximate posterior distribution after the intervention of the perturber; is a standard Gaussian noise, i.e. .

[0153] In implementation, the perturber adopts two independent linear transformations to respectively map the input , :

[0154] ;

[0155] ;

[0156] wherein, , , , are the learnable parameters of the first perturber; is an activation function.

[0157] In order to ensure that the generated distribution has sufficient diversity and can cover various test distributions in the social platform (for example, the short attention of users to a newly released movie, the discussion triggered by a sudden news event), the training target of the intervention network aims to maximize the variance between the new Gaussian distribution parameters of each group; the intervention loss calculation formula is defined as follows:

[0158] ;

[0159] wherein, is the intervention loss; and are hyperparameters, respectively used to control the weights of the mean and the standard deviation diversity; is the variance calculation.

[0160] Step 4, constructing an expert mixed decoder to decode and aggregate the original latent representation and the latent representation after intervention to generate the final recommendation result;

[0161] Step 4.1, for the generated multiple latent interest distributions, the present application adopts an expert mixed decoder for processing;

[0162] The goal of the expert mixed decoder is to decode the latent representation generated by the user portrait encoder based on variational inference into a specialized user preference representation under the dynamic graph neural network framework; wherein the first expert decoder handles the original latent representation without intervention as a reference expert represent the user's current core interest; the rest of the expert decoders handle the intervened latent representations respectively , each expert decoder is thus responsible for decoding a specific latent interest corresponding to one perturber , which is used to process the output of each perturber; the gating network computes the weights of each expert decoder to determine the contribution of each expert decoder in the aggregation of the final user representation

[0163] In the present invention, there are expert decoders, among which the first expert decoder serves as the reference expert, responsible for decoding the original approximate posterior distribution without intervention This expert decoder focuses on learning the user's long-term, stable core interest; the rest of the expert decoders then decode the perturbed approximate posterior distributions generated by the intervention network respectively These expert decoders are trained to adapt to different latent interest evolution patterns, such as the user's short-term interest in a specific hot topic; each expert decoder is a learnable nonlinear function ;

[0164] In order to achieve adaptive selection between different experts, the gating network receives the intervened latent representation as input and computes the weights of the expert decoders; the introduction of noise helps to achieve more balanced expert utilization in training:

[0165] ;

[0166] where is the weight of the expert decoder; , are learnable parameter matrices; and are activation functions;

[0167] The final aggregated representation is obtained by weighted summation of the outputs of all expert decoders:

[0168] ;

[0169] where is the aggregated representation at the current time ; is the expert decoder.

[0170] Because during the multi-source data fusion process in step 1 at the current time Aggregate representation of input For users and interaction target The features are concatenated and their relationship is modeled through an attention mechanism. Therefore, in the expert hybrid decoding stage, it is necessary to analyze the aggregated representation. Extract users from the middle and interaction target The vectors are averaged to obtain their final representation:

[0171] ;

[0172] ;

[0173] in, , Each represents the current time. user Interaction goals The final representation; For averaging operations; and Each represents the current time. user and interaction target The corresponding index length.

[0174] Step 4.2: Make predictions using the final representations of users and interaction goals to complete the recommendation task;

[0175] In recommendation tasks, by calculating user... With the target of the interaction (or the post) The similarity is used to predict new content that a user might be interested in. Specifically, in order to determine the user's... With the target of interaction At the present moment Whether interaction will occur can be determined by calculating the inner product of the user's final representation and the final representation of the interaction target, then normalizing the result to obtain the probability of the user's interest in the interaction target:

[0176] ;

[0177] in, Indicates user At the present moment For the interaction target The probability of interest; For activation functions; , Each represents the current time. user Interaction goals the final representation of the user's interest;

[0178] all interest probabilities are sorted in descending order, and the top interaction targets corresponding to the interest probabilities are output as the final recommendation result.

[0179] Step 5, in order to obtain a core preference representation with stability to interest evolution, the present application designs the training process of the dynamic graph neural network as a double-layer optimization problem. Specifically, the goal of the outer-layer optimization is to learn a representation that can robustly capture the user's core interest, and to minimize the composite training target (composed of the evidence lower bound loss and the alignment loss) by updating the encoder and decoder parameters; while the goal of the inner-layer optimization is to generate a diverse latent distribution, and to minimize its corresponding intervention loss by updating the parameters of the intervention network.

[0180] The composite training target contains two parts: the evidence lower bound loss (including the reconstruction loss of the interaction relationship structure and the KL divergence loss), and the alignment loss; the training optimization is performed by weighting the evidence lower bound loss and the alignment loss. This ensures that the model can accurately encode and reconstruct user behavior, and also remains robust in the face of simulated distribution shifts, thereby improving the recommendation performance in real dynamic environments.

[0181] Step 5.1, the evidence lower bound loss is the core loss function of the user portrait encoder based on variational inference, used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: one is the reconstruction loss of the dynamic graph neural network reconstructing the interaction relationship , used to measure the accuracy of the model in reconstructing the interaction relationship structure between the user and the interaction target; the other is the KL divergence loss , used to constrain the difference between the approximate posterior distribution corresponding to each perturber and its conditional prior :

[0182] ;

[0183] ;

[0184] ;

[0185] wherein, is the evidence lower bound loss, composed of the reconstruction loss of the interaction relationship structure and the KL divergence loss ; is the total number of time steps; is the number of perturbers and latent distributions; represents the latent representation obtained by sampling from the approximate posterior distribution at the current time ; represents the latent representation obtained by sampling from the approximate posterior distribution at the current time latent representation sampled from the approximate posterior distribution after the intervention of the jth perturber; interaction structure before the current time step; aggregate features before the current time step;

[0186] Step 5.2, Alignment loss Lalignis crucial to ensure the model learns an invariant representation. This loss function aims to minimize the expected distance between the output representation of each “perturbed” expert and that of the “reference” expert. This forces the model to keep its core user profile representation stable in the face of various interest shifts:

[0187]

[0188] ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​a latent representation sampled from the approximate posterior distribution after the intervention of the perturber; represents the current time a latent representation sampled from the approximate posterior distribution.

[0189] Step 5.3, the double-layer optimization strategy of the application includes inner-layer optimization and outer-layer optimization, which are used to update two categories of independent parameters under the dynamic graph neural network framework, wherein the parameters of the intervention network are updated in the inner-layer optimization and the parameters of the encoder and the decoder are updated in the outer-layer optimization .

[0190] Inner-layer optimization: for the fixed parameters of the main model (encoder, decoder, etc.) , the parameters of the intervention network are optimized , and the goal is to maximize the diversity of the generation distribution and constantly make new and unpredictable interest changes for the decoder. The update formula is:

[0191] ;

[0192] wherein, is the optimal parameter of the intervention network; is the intervention loss.

[0193] Outer-layer optimization: fixing the optimal intervention network parameters , then optimizing the parameters of the main model (encoder, decoder, etc.) , and the goal is to minimize the weighted sum of the evidence lower bound loss and the alignment loss. The update formula is:

[0194] ;

[0195] wherein, is the optimal parameter of the main model (encoder, decoder, etc.); is a hyperparameter for controlling the alignment strength; is the reconstruction loss of the interaction relationship structure; is the KL divergence loss; is the alignment loss.

[0196] The application carries out experiments in a plurality of different distribution shift application scenarios to verify the feasibility and superiority of the method of the application in the scene of feature distribution and historical interaction data distribution shift. In the experiment process, the method proposed in the application (hereinafter referred to as MoVE) is compared with a plurality of prior art schemes, including an empirical risk minimization method ERM, an invariant risk minimization method IRM, a dynamic graph attention network DIDA based on disentanglement intervention, a dynamic graph invariant learning method SILD, a dynamic graph attention method TGAT, a dynamic graph method TGN based on a recurrent neural network, a method CAWN based on causal anonymous walk, a method DygFormer based on Transformer, and a dynamic graph method FreeDyg based on frequency domain enhancement. Finally, in the application scenario of user interest evolution and distribution shift, the experimental results of each method are comprehensively compared and analyzed, so as to further verify the effectiveness and superiority of the method of the application in practical application.

[0197] The application first evaluates a plurality of interaction behaviors of users in a social platform scene, including post and like request on Reddit platform within one month, and editing record on Wikipedia platform. Specifically, the Reddit dataset contains 10984 nodes and 672447 interactions, and the Wikipedia dataset contains 9227 nodes and 157474 interactions. The experimental results are shown in Table 1, wherein the best result is indicated by bold, and the suboptimal result is indicated by underlining. In the two datasets, the feature distribution of the test set is significantly shifted. If the model cannot effectively utilize the invariant mode across the distribution in the data, its performance on the test set will obviously decrease.

[0198] The results in Table 1 show that in the case that the user interest quickly migrates from one topic circle to another, the application can maintain a high recommendation accuracy, while the comparative method obviously decreases in performance after the interest changes.

[0199] Table 1 Recommendation prediction results under user feature behavior distribution shift (average accuracy%)

[0200] .

[0201] In order to further verify the ability of the application in dealing with out-of-distribution data, in addition to verification in the social platform scene, the application also carries out experiments in the historical interaction data distribution shift scene.

[0202] Specifically, the UN Trade dataset covers trade records between countries over the past 30 years, containing 299 nodes and 507497 interactions; the UN Vote dataset covers voting records between countries at the United Nations General Assembly over the past 70 years, containing 201 nodes and 1035742 interactions. Due to the continuous evolution of the geopolitical landscape, the distribution of these two datasets also changes significantly in different time periods.

[0203] Two settings were used in the experiment: in direct learning, during the training phase, the model can access the test data in addition to the training data; in inductive learning, during the training phase, the model cannot observe any samples of the test data and can only make inferences based on the learned knowledge during testing, so this setting is more challenging. The experimental results are shown in Table 2, where the best results are marked in bold and the suboptimal results are marked in underlined.

[0204] Table 2 shows that the present application can effectively identify potential environmental changes and avoid over-reliance on false correlations, thereby maintaining consistent excellent recommendation performance in various dynamic environments.

[0205] Table 2 shows that the present application can effectively identify potential environmental changes and avoid over-reliance on false correlations, thereby maintaining consistent excellent recommendation performance in various dynamic environments.

[0206] .

[0207] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A data-biased recommendation method based on hybrid experts and dynamic graph neural networks, characterized in that, The dynamic graph neural network includes a user profile encoder and an expert hybrid decoder; specifically, it includes the following steps: Step 1: Collect and integrate multi-source user data from social platforms to construct comprehensive features; Step 2: Construct a user profile encoder based on variational inference to perform dynamic encoding and generate the original latent representation; Step 3: Construct an intervention network to extrapolate the original latent representations into a mixture of multiple intervention-related latent representations; Intervention network by An independent disturbance composition, For the first Learnable parameters for each disturbance; Each perturbator receives the original Gaussian distribution parameters. It outputs a new set of Gaussian distribution parameters. The latent representation after user intervention is obtained by sampling from the new Gaussian distribution parameters using a reparameterization method. ； ； in, , Each represents the current time. No. The mean and standard deviation of the Gaussian distribution of each disturbance; Indicates the current moment From the The latent representation obtained by sampling from the approximate posterior distribution after intervention by a perturbator; For the current moment The mean of a Gaussian distribution; For the current moment The standard deviation of the Gaussian distribution; Standard Gaussian noise, i.e. ; final, Each disturbance corresponds to... Potential representations after intervention ; Disruptor Two independent linear transformations are used to transform the input respectively. , Perform mapping: ； ； in, , , , All are the first Learnable parameters for each disturbance; For activation functions; The training objective of the intervention network is to maximize the variance among the parameters of the new Gaussian distribution in each group. The intervention loss is defined by the following formula: ； in, To intervene in losses; and These are all hyperparameters, used to control the mean. and standard deviation Weight of diversity; For variance calculation; Step 4: Construct an expert hybrid decoder to decode and aggregate the original latent representation and the latent representation after intervention to generate the final recommendation result; The expert hybrid decoder includes a gating network and A parallel expert decoder; the first expert decoder As a reference model, it processes the original latent representation. The remaining expert decoders Processing latent representations after intervention separately One expert decoder corresponds to one perturber; Indicates the current moment The original latent representation obtained by sampling from an approximate posterior distribution; Step 5: Design composite training objectives and a two-layer optimization strategy; The composite training objective consists of two parts: evidence lower bound loss and alignment loss; The evidence lower bound loss is the loss function of the variational inference-based user profile encoder, used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: the reconstruction loss of the interaction relationship structure and the KL divergence loss; the specific formula is as follows: ； ； ； in, Loss of the lower bound of evidence; The loss is the reconstruction loss of the interaction relationship structure; For KL divergence loss; This represents the total number of time steps. For the current moment The previous interaction structure; For the current moment Previous comprehensive characteristics; For the previous moment The hidden state; Indicates the current moment Given generate The conditional probability distribution; Indicates the current moment ,based on Defined conditional prior distribution; Indicates the current moment Given , and Then, the approximate posterior distribution of the latent representation is inferred through the user profile encoder; Indicates the current moment Given , and Afterwards, through the first The approximate posterior distribution of the latent representation inferred by each perturbation device; Let KL divergence be a metric. Alignment loss The formula is: ； in, , The first and the second, respectively. Approximate posterior distribution of an expert decoder; The two-layer optimization strategy includes inner-layer optimization and outer-layer optimization. The inner-layer optimization updates the parameters of the intervention network. During outer layer optimization, the parameters of the encoder and decoder are updated. ; The update formula for inner layer optimization is: ； in, The parameters are those of the optimal intervention network; The update formula for outer layer optimization is: ； in, The optimal parameters for the encoder and decoder; It is a hyperparameter that controls the alignment strength.

2. The data-biased recommendation method based on hybrid experts and dynamic graph neural networks according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: The collected multi-source data includes static attribute data and dynamic interaction behavior data; among which, static attribute data includes user personal information; dynamic interaction behavior data includes behavior sequence data and time dynamic information; behavior sequence data includes posting, liking, forwarding, commenting, and collecting; time dynamic information includes the timestamps of the behavior. Step 1.2: Extract the self-features of users or interaction targets respectively; interaction targets are divided into two categories: one is posts or comments, and the other is target users; user personal information, posts, comments, and target user homepage text all belong to text information; the specific process of extracting self-features is as follows: first, the text is preprocessed, including word segmentation, removal of stop words and noise symbols; then, the preprocessed words are matched with the dictionary in the text analysis tool LIWC, the frequency of each word in different psychological, linguistic, and semantic categories is counted, and normalization is performed to obtain the corresponding semantic feature vector; the semantic feature vector extracted from the user's personal information is defined as the user's self-feature vector. At the present moment its own characteristics The semantic feature vector extracted from the interaction target is defined as the interaction target. At the present moment its own characteristics ; Step 1.3: Perform one-hot encoding based on the interaction type to obtain interaction features; interaction types include posting, liking, forwarding, commenting, collecting, following, and unfollowing; during the one-hot encoding process, each interaction type is assigned a fixed feature position, and when an interaction occurs, the corresponding position is set to 1, and the remaining positions are set to 0; Step 1.4: Extract the temporal features of the user or interaction target; for users At the present moment Historical interaction sequence , which includes the most recent Each interaction record corresponds to a historical timestamp. Encode each timestamp in the historical interaction sequence to obtain the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows: ； in, , All of these are trainable parameters; Encoding dimension for time; Similarly, for the interaction target At the present moment Historical interaction sequence Each timestamp is encoded to obtain the historical interaction sequence. Time feature matrix ; Step 1.5: Multiply each of the user's features by the learnable parameter matrix to obtain the user's complete features: ； in, For the current moment user Complete features; , , All are learnable parameters; For the current moment Interactive features; Multiplying each feature of the interactive target by the learnable parameter matrix yields the complete features of the interactive target: ； in, For the current moment Interaction Target Complete features; , , All are learnable parameters; For the current moment Interactive features; Step 1.6: Concatenate the user's complete features with the complete features of the interaction target to obtain the comprehensive features: ； in, For the current moment Its comprehensive characteristics.

3. The data-biased recommendation method based on hybrid experts and dynamic graph neural networks according to claim 1, characterized in that, In step 2, the specific working process of the user profile encoder based on variational inference is as follows: Step 2.1: For the current time... Comprehensive characteristics Perform multi-head attention encoding: ； ； ； in, For the current moment Attention representation; This is a multi-head attention mechanism; , The first The, the One point of attention; Learnable parameters for the multi-head attention output layer; For attention mechanisms; , The first Individual attention head query ,key ,value Learnable parameters; Representative layer normalization; It is the transpose symbol; The key vector dimension for each attention head; For activation functions; Step 2.2: Concatenate the current attention representation with the previous hidden state and input them into two independent feedforward neural networks to generate the mean and standard deviation of a Gaussian distribution, respectively. ； ； ； in, It is a feedforward neural network; These are the input features of the feedforward neural network; , , , All are learnable parameters; For activation functions; A feedforward neural network for generating the mean; To generate a feedforward neural network for standard deviation; Step 2.3: Obtain the approximate posterior distribution based on variational inference: ； in, Indicates the current moment Given , and Then, the approximate posterior distribution of the latent representation is inferred through the user profile encoder; Indicates the current time The interactive relationship structure; Indicates a Gaussian distribution; Represents the diagonal covariance matrix; The sampled data is obtained from the approximate posterior distribution using a reparameterization method, and the formula is as follows: ； in, The unit covariance matrix; This is element-wise multiplication; Step 2.4: Iteratively update the hidden state using gated recurrent units, thereby modeling the dynamic evolution of interest over time. ； in, and Each represents the current time. and the previous moment The hidden state; This represents the update function of a gated recurrent neural network.

4. The data-biased recommendation method based on hybrid experts and dynamic graph neural networks according to claim 3, characterized in that, In step 4, the gating network calculates the weights of each expert decoder using the following formula: ； in, For the first The weights of each expert decoder; , All are learnable parameter matrices; The final aggregate representation is obtained by weighted summation of the outputs of all expert decoders: ； in, For the current moment Aggregate representation; For the first One expert decoder; In the expert hybrid decoding stage, from the aggregate representation Extract users from the middle and interaction target The vectors are averaged to obtain their final representation: ； ； in, , Each represents the current time. user Interaction goals The final representation; For averaging operations; and Each represents the current time. user and interaction target The corresponding index length; Calculate the inner product of the user's final representation and the interaction goal's final representation, and normalize it to obtain the probability of the user's interest in the interaction goal: ； in, Indicates user At the present moment For the interaction target The probability of interest; For activation functions; Sort all interest probabilities in descending order and output the first... The interaction target corresponding to each interest probability is used as the final recommendation result.

Images