Anti-data migration recommendation method based on hybrid expert and dynamic graph neural network

Abstract

The invention discloses an anti-data migration recommendation method based on a hybrid expert and a dynamic graph neural network, and belongs to the technical field of intelligent recommendation systems. The dynamic graph neural network comprises a user portrait encoder and an expert hybrid decoder; comprising the following steps: step 1, collecting multi-source data of a user from a social platform and fusing the multi-source data to construct a comprehensive feature; 2, constructing a user portrait encoder based on variational inference for dynamic encoding, and generating an original potential representation; step 3, constructing an intervention network for extrapolating the original potential representation into a mixture of a plurality of intervened potential representations; 4, constructing an expert hybrid decoder, and decoding and aggregating the original potential representation and the intervened potential representation to generate a final recommendation result; and 5, designing a composite training target and a double-layer optimization strategy. According to the method, high accuracy and stability can be still kept under the condition of significant data distribution offset, and the user experience and the platform activeness are significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent recommendation systems, and particularly relates to a data drift resistant recommendation method based on a hybrid expert and dynamic graph neural network. BACKGROUND

[0002] With the rapid development of social platforms, the behavior data (such as posting, liking, forwarding, commenting, and following) and social relationship network structure of users on the platform show obvious dynamic evolution characteristics. User interests can quickly change due to hot topics, popular content, friend interactions, or external events, resulting in continuous migration of user behavior distribution and content popularity distribution over time. This dynamic nature makes the recommendation model trained based on historical behavior prone to performance degradation when actually deployed, i.e., the so-called "distribution drift" problem. How to effectively adapt to the evolution of user behavior preferences in social platforms and maintain stable recommendation performance in distribution outside environments has become a key challenge for the development of recommendation technology. Currently, the mainstream methods to improve the generalization ability of social platform recommendation systems in distribution drift environments mainly include two categories: (1) data augmentation-based methods. This type of method forces the model to learn in different "simulated environments" by perturbing user interaction behavior data or constructing virtual scenarios, thereby improving its robustness in real distribution changes. For example, some studies attempt to generate multiple training environments by randomly adding or deleting interaction records, perturbing user feature vectors, or dividing different time segments. The model learns in these environments simultaneously, thereby enhancing its cross-distribution stability. However, this type of method often relies on limited pre-defined rules, making it difficult to comprehensively cover the complex change scenarios that may occur in user behavior and content ecology in social platforms. (2) disentangled representation learning-based methods. This type of method attempts to split user interest representation into "long-term stable core interest" and "short-term dynamic variable interest", and uses the core interest part for recommendation to enhance the model's generalization ability in distribution changes. For example, some solutions identify stable interest themes in user history as domain-invariant features, while viewing rapidly changing behavior patterns over time as domain-specific features, which are only used in the training stage. However, in actual social platform environments, user long-term interests and short-term interests are often highly coupled, with hot events, social interactions, and personalized behaviors intertwined, making it difficult for this type of method to accurately separate the two, and the computational overhead for large-scale social graph data is huge.

[0003] Although the above methods alleviate the out-of-distribution generalization problem to some extent, there are still the following core shortcomings in the social platform recommendation scene: (1) Dependence on complex graph operations: whether it is data augmentation or feature disentanglement, most existing methods directly perform complex operations on the original social network graph or user behavior sequence. For example, data augmentation methods usually generate virtual data by pre-set rules (such as randomly modifying user relationship edges, perturbing behavior features, etc.), but the evolution of user interest in a real social platform is often the result of multiple factors such as hot topics, social interactions and external events, and its complexity makes it difficult for these rules to be fully covered. Similarly, methods based on feature disentanglement attempt to separate a user's long-term interest from their short-term interest, but in an actual platform, the two are highly intertwined, making it difficult to obtain reliable results, not only is the computational overhead huge, but also. (2) The modeling of user interest evolution is too static or simple: some methods implicitly assume a static assumption when they are designed, such as assuming that there is always a part of the user's interest that is "stable and unchanging", and another part that is "dynamic and variable". However, in a social platform, the importance and content of a hot topic are constantly changing over time, for example, a user may focus on a hot topic for a short period of time, and then quickly switch to other areas. This static assumption makes it difficult for existing methods to adapt flexibly to the real dynamic environment. SUMMARY

[0004] In order to solve the problem of the decline in the performance of a recommendation model caused by the distribution shift of behavior data in a social platform due to factors such as the rapid replacement of hot topics and the evolution of a user's interest circle, the present application proposes an anti-data shift recommendation method based on a hybrid expert and dynamic graph neural network; the method first integrates the analysis of a user's multi-source behavior data, content features and social relationship network on a social platform, fuses these heterogeneous information, maps it to a low-dimensional continuous latent space through a user portrait encoder based on variational inference, to obtain a more expressive latent representation; then, by intervening in the latent distribution, more rich and diverse interest evolution patterns are generated, thereby improving the generalization ability of the recommendation model; finally, a hybrid expert decoder is designed, wherein each expert models a specific latent interest distribution shift, and combines an alignment loss to enhance the generalization ability of each expert, thereby improving the performance of the recommendation model in the out-of-distribution generalization scenario.

[0005] The technical solution of the present application is as follows: An anti-data shift recommendation method based on a hybrid expert and a dynamic graph neural network, the dynamic graph neural network comprising a user portrait encoder and a hybrid expert decoder; specifically comprising the following steps: Step 1, collect and fuse the multi-source data of users from a social platform, and construct comprehensive features; Step 2, construct a user portrait encoder based on variational inference for dynamic encoding to generate original latent representation; Step 3, construct an intervention network for extrapolating the original latent representation into a mixture of multiple intervened latent representations; Step 4, construct an expert mixture decoder to decode and aggregate the original latent representation and the intervened latent representation to generate the final recommendation result; Step 5, design a composite training target and a double-layer optimization strategy.

[0006] Further, the specific process of step 1 is as follows: Step 1.1, the collected multi-source data includes static attribute data and dynamic interaction behavior data; wherein the static attribute data includes user personal information; the dynamic interaction behavior data includes behavior sequence data and time dynamic information; the behavior sequence data includes posting, liking, forwarding, commenting, and collecting; the time dynamic information includes the timestamp of the behavior occurrence; Step 1.2, extract the self-features of the user or the interaction target respectively; the interaction target is divided into two categories: one is a post or a comment, and the other is a target user; the user personal information, the post, the comment, and the homepage text of the target user all belong to text information; the specific process of extracting self-features is as follows: first, pre-process the text, including word segmentation, removing stop words and noise symbols; then match the pre-processed words with the dictionary in the text analysis tool LIWC, count the occurrence frequency of each word in different psychological, linguistic and semantic categories, and normalize to obtain the corresponding semantic feature vector; define the semantic feature vector extracted from the user personal information as the user 's self-feature at the current time ; define the semantic feature vector extracted from the interaction target as the interaction target 's self-feature at the current time ; Step 1.3, one-hot encoding according to the interaction type to obtain the interaction feature; the interaction type includes posting, liking, forwarding, commenting, collecting, following and unfollowing; in the one-hot encoding process, each interaction type is assigned a fixed feature position, when a certain interaction occurs, the corresponding position is set to 1, and the rest are set to 0; Step 1.4, extract the time feature of the user or the interaction target; for the user 's historical interaction sequence at the current time , which contains the latest interaction records, each interaction record corresponds to a historical timestamp Encode each timestamp in the historical interaction sequence to obtain the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows: ; in, , All of these are trainable parameters; Encoding dimension for time; Similarly, for the interaction target At the present moment Historical interaction sequence Each timestamp is encoded to obtain the historical interaction sequence. Time feature matrix ; Step 1.5: Multiply each of the user's features by the learnable parameter matrix to obtain the user's complete features: ; in, For the current moment user Complete features; , , All are learnable parameters; For the current moment Interactive features; Multiplying each feature of the interactive target by the learnable parameter matrix yields the complete features of the interactive target: ; in, For the current moment Interaction Target Complete features; , , All are learnable parameters; For the current moment Interactive features; Step 1.6: Concatenate the user's complete features with the complete features of the interaction target to obtain the comprehensive features: ; in, For the current moment Its comprehensive characteristics.

[0007] Further, the specific working process of the user portrait encoder based on variational inference in step 2 is as follows: Step 2.1, multi-head attention encoding is performed on the comprehensive features of the current moment ; ; ; Wherein, is the attention representation of the current moment ; is the multi-head attention mechanism; , , are the first , the second attention head respectively; is the learnable parameter of the multi-head attention output layer; is the attention mechanism; , are the learnable parameters of the first attention head query , key , value respectively; represents layer normalization; is the transpose symbol; is the key vector dimension of each attention head; is the activation function; Step 2.2, the attention representation of the current moment is spliced with the hidden state of the last moment to input into two independent feedforward neural networks to generate the mean and standard deviation of Gaussian distribution respectively: ; ; ; Wherein, is the feedforward neural network; is the input feature of the feedforward neural network; , , , are all learnable parameters; is the hidden state of the last moment ; is the activation function; is the mean of Gaussian distribution of the current moment ; is the standard deviation of Gaussian distribution of the current moment ; is the feedforward neural network for generating the mean;​​ a feedforward neural network for generating standard deviation; Step 2.3, obtaining an approximate posterior distribution based on variational inference: ; wherein, represents the current time , given , and , an approximate posterior distribution of the latent representation inferred by the user portrait encoder; represents the original latent representation sampled from the approximate posterior distribution at the current time ; represents the interaction relationship structure at the current time ; represents a Gaussian distribution; represents a diagonal covariance matrix; is sampled from the approximate posterior distribution by using a reparameterization method, and the formula is: ; wherein, is a standard Gaussian noise, that is ; is a unit covariance matrix; is an element-wise multiplication; Step 2.4, iteratively updating the hidden state by using a gated recurrent unit, and further modeling the process of dynamic evolution of interest over time: ; wherein, and are the hidden states at the current time and the previous time ; represents an update function of the gated recurrent neural network.

[0008] Further, in the step 3, the intervention network is composed of independent perturbers , is a learnable parameter of the th perturber; Each perturber receives the original Gaussian distribution parameter , and outputs a new set of Gaussian distribution parameters , and samples the latent representation of the user after intervention from the new Gaussian distribution parameter by using a reparameterization method: ; ; wherein, , are the mean and standard deviation of the Gaussian distribution of the th perturber at the current time ; is the latent representation sampled from the approximate posterior distribution of the th perturber after intervention; Finally, the th latent representation of the th perturber after intervention is obtained ; , are mapped by two independent linear transformations: ; ; wherein, , , , are the learnable parameters of the th perturber; is an activation function; The training objective of the intervention network is to maximize the variance between the new Gaussian distribution parameters of each group, and the intervention loss calculation formula is defined as follows: ; wherein, is the intervention loss; and are hyperparameters, respectively used to control the weights of the mean and standard deviation diversity; is the variance calculation.

[0009] Further, in step 4, the expert mixed decoder contains a gating network and parallel expert decoders; the first expert decoder serves as a reference model to process the original latent representation ; the remaining expert decoders process the latent representations after intervention , one expert decoder corresponding to one perturber; the gating network calculates the weight of each expert decoder, and the formula is as follows: ; wherein, is the weight of the th expert decoder; , are learnable parameter matrices; The final aggregated representation is obtained by weighted sum of the outputs of all expert decoders: ; wherein, is the aggregated representation at the current time ; is the th expert decoder; In the expert mixture decoding stage, the vectors of the user and the interaction target are taken out from the aggregated representation respectively, and the average is taken as the final representation thereof: ; ; wherein, , are the final representations of the user and the interaction target at the current time respectively; is the average operation; and are the index lengths of the user and the interaction target at the current time respectively; The inner product of the final representation of the user and the final representation of the interaction target is calculated, and normalized to obtain the interest probability of the user to the interaction target: ; wherein, represents the interest probability of the user to the interaction target at the current time ; is an activation function; All the interest probabilities are sorted in descending order, and the interaction targets corresponding to the top interest probabilities are output as the final recommendation results.

[0010] Further, in step 5, the composite training target contains two parts: evidence lower bound loss and alignment loss; The evidence lower bound loss is a loss function of the user portrait encoder based on variational inference, used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: reconstruction loss of the interaction relationship structure and KL divergence loss; the specific formula is: ; ; ; wherein, is the evidence lower bound loss; is the reconstruction loss of the interaction structure; is the KL divergence loss; is the total time step; is the interaction structure before the current time ; is the comprehensive feature before the current time ; denotes the conditional probability distribution of , given ; denotes the approximate posterior distribution of the latent representation , given , based on the conditional prior distribution defined by ; denotes the approximate posterior distribution of the latent representation , given , and , approximated by the user profile encoder; denotes the approximate posterior distribution of the latent representation , given , and , approximated by the th perturber; is the KL divergence; the alignment loss is formulated as: ; wherein, , are the approximate posterior distributions of the first, the th expert decoder, respectively; The double-layer optimization strategy includes inner-layer optimization and outer-layer optimization. The parameters of the intervention network are updated in the inner-layer optimization, and the parameters of the encoder and the decoder are updated in the outer-layer optimization; The update formula of the inner-layer optimization is: ; wherein, is the optimal parameter of the intervention network; The update formula of the outer-layer optimization is: ; wherein, is the optimal parameter of the encoder and the decoder; is a hyperparameter that controls the strength of alignment.

[0011] The beneficial technical effects brought by the present application are as follows.

[0012] (1) Feasibility and efficiency for social platform: most existing technologies directly operate on the original social relationship network or large-scale interaction graph data (such as modifying historical interaction data, user features, etc.), which has high computational cost and is difficult to apply in large-scale social platforms. The present application avoids complex operations at the original network level by mapping user behavior, content interaction and social relationship data into a low-dimensional, continuous latent space, making dynamic recommendation for large-scale social platform data more feasible.

[0013] (2) Adapt to dynamic evolution of user interest: existing technologies based on disentanglement often imply the assumption of "fixed long-term user interest", but in social platforms, users will quickly shift their interest circles with new topics, popular content or social influences. This static assumption cannot meet the actual needs. The expert mixed decoder used in the present application dynamically allocates expert decoding weights for each user interaction (such as likes, forwards or comments) using a gating network, thereby being able to adapt to changes in user short-term behavior in real time, while actively learning the long-term stable interest of the user by combining the alignment loss, achieving a balance between short-term hotspots and long-term preferences.

[0014] (3) Enhanced generalization ability and recommendation effect: existing methods based on data augmentation only rely on limited rules (such as random edge deletion, feature perturbation) to simulate distribution changes, which is difficult to cover the diversity of user behavior evolution in social platforms (such as from following hot topics to long-term interest in interest communities). The present application better covers complex situations such as user interest mutation and content ecological evolution by intervening in the distribution parameters in the latent space. In specific social platform recommendation tasks (such as content recommendation, etc.), the present application can still maintain high accuracy and stability under significant distribution changes, significantly improving user experience and platform activity. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flowchart of the anti-data drift recommendation method of the present application based on mixed experts and dynamic graph neural networks. DETAILED DESCRIPTION

[0016] The present application will be further described in detail below in conjunction with the drawings and specific embodiments: The application provides a data drift resistant recommendation method based on a hybrid expert and dynamic graph neural network. The method is a dynamic graph neural network framework including a user portrait encoder and an expert hybrid decoder. The core of the method is to model the time sequence interaction structure of the user and the interaction target as a dynamic graph. The user portrait encoder is used to obtain a portrait representation capable of representing the current interest state of the user, and the preference distribution is modeled and intervened in the corresponding low-dimensional latent space. Finally, the expert hybrid decoder can accurately capture the data drift phenomenon caused by the dynamic evolution of the user interest. The specific purposes are as follows: Purpose one: effectively depict the user portrait. The application aims to integrate multiple source heterogeneous data of the user to construct a comprehensive feature vector as the basis of the user behavior portrait. The portrait not only includes the static attributes of the user, but more importantly, can capture the time dynamics and preference intensity in the behavior sequence, providing a solid data basis for predicting the evolution trend of the user interest.

[0017] Purpose two: avoid direct operation on the original complex graph data, and turn to model the distribution drift in the latent space which is easier to handle. The application does not directly modify the interaction relationship structure or features in the social network, but first compresses the original complex graph data into a low-dimensional latent space by using a user portrait encoder based on variational inference. In the space, the distribution drift is simulated and processed, which is more efficient and feasible than operating in the original data space.

[0018] Purpose three: generate more abundant and diverse user interest evolution trends by intervening in the latent distribution to improve the generalization ability of the social platform recommendation model. The application proposes a new "latent distribution extrapolation" strategy. By using a special intervention network, the probability distribution parameters (i.e. mean and standard deviation) of the user portrait in the latent space are systematically disturbed, thereby generating a series of new latent distributions as proxies for the possible evolution direction of the user's future interest. This way can simulate a wider range of distribution changes in a parameterized form, which has stronger coverage and flexibility than limited data enhancement rules.

[0019] Purpose four: use the expert hybrid decoder to specialize in processing different latent interest evolution patterns. For the generated multiple latent interest distributions, the application uses an expert hybrid decoder, which includes multiple parallel "expert" decoders. Each expert specializes in processing a specific latent interest distribution drift, thereby realizing fine modeling of the different interest evolution possibilities of the user.

[0020] As shown in Figure 1 The core idea of the application is to enhance the generalization ability of the model by simulating diversified latent distribution drift under the dynamic graph neural network framework, which specifically includes the following steps: Step 1: Collect and integrate multi-source user data from social media platforms to construct comprehensive features; the specific process is as follows: Step 1.1: The collected multi-source data specifically includes: static attribute data (such as user personal information) and dynamic interaction behavior data; among which, dynamic interaction behavior data includes behavior sequence data (such as posting, liking, forwarding, commenting, and collecting) and time dynamic information (such as timestamps of behavior occurrence). In social media platforms, user behavior is dynamic and diverse. To construct accurate user profiles, this invention encodes every interaction between the user and content. The core of this stage lies in transforming the user's behavior at a certain point in time (such as posting, liking, forwarding, commenting, and collecting) into a vector representation that can be understood by a machine learning model. That is, extracting behavioral and temporal features from dynamic interaction behavior data and integrating them with static attribute (user personal information) data to generate a comprehensive feature vector. Specifically, this invention formalizes an interaction into a triple. ,in Indicates user, Indicates the interaction target (post or comment, target user, etc.). This indicates the current moment in which the interaction occurs. To characterize the historical dependencies of the interaction, the current moment... Previous users The interaction target is denoted as At the current moment Previous interaction target Users who interact are denoted as To characterize this type of dynamic interaction, we first trace back the user's actions. At the present moment Previous interactions yielded a first-order historical interaction sequence. Then, the interaction target was traced back in the same way. At the present moment Previous interactions yielded a first-order historical interaction sequence. Considering the large scale of the interaction data, to avoid information redundancy, this invention sets a truncation strategy for historical interaction sequences, retaining only the most recent ones. One interaction record.

[0021] Step 1.2: Extract the user's or interaction target's own characteristics from static attribute data or dynamic interaction behavior data respectively; For users At the present moment Historical interaction sequence This includes Each interaction record corresponds to an interaction goal. The interaction target can be a post or comment, or it can be a target user. the history interaction sequence at the current time Also contains interaction records, each of which corresponds to a user In the feature construction process, the personal information of the user (such as the personal homepage text) is regarded as static attribute data, and the post, the comment and the homepage text of the target user belong to text information. In order to ensure the consistency of the feature extraction method, the application adopts a text analysis tool LIWC based on a dictionary to uniformly process the text content of the user and the interaction target. Specifically, first, the analysis text (including the personal homepage text of the user, the post or the comment text and the homepage text of the target user) is preprocessed, including word segmentation, removal of stop words and noise symbol operation. Subsequently, the vocabulary obtained after preprocessing is matched with the LIWC dictionary, the appearance frequency thereof in different psychological, language and semantic categories (such as emotion, cognition, sociality, time, etc.) is counted, and normalization processing is performed, so as to obtain a semantic feature vector. Finally, the semantic feature vector extracted from the personal information of the user is defined as the user the self feature at the current time ; the semantic feature vector extracted from the interaction target (the post or the comment, the target user, etc.) is defined as the self feature of the interaction target at the current time Each of the features contains vectors, corresponding to the length of the history interaction sequence and (that is, the interaction records correspond to vectors).

[0022] Step 1.3, One-Hot encoding is performed according to the interaction type to obtain the interaction feature; ​​​Interaction type refers to the different categories of interactive behaviors that occur between a user and the interaction target. The number of interaction categories may vary under different distribution offset scenarios. For example, in a typical scenario on a social platform, interaction types include posting, liking, forwarding, commenting, collecting, following, and unfollowing. Since interaction types are essentially discrete categorical data, directly inputting them into the model would lead to a lack of comparability between features. Therefore, this invention uses one-hot encoding for representation. In the one-hot encoding process, each interaction type is assigned a fixed feature position. When an interaction occurs, the corresponding position is set to 1, and all other positions are set to 0. For example, when the interaction type is "like," the encoding of the position corresponding to "like" is 1, while the positions of other interaction types are all 0, thus obtaining a vector representation with the same dimension as the number of interaction categories. In this way, the equidistance of different interaction types in the numerical space can be ensured, avoiding false ordering relationships between categories. Finally, at the current moment... The interactive features are defined as This feature vector can accurately characterize the types of interaction behaviors between the user and the interaction target, and provide input for subsequent feature fusion and modeling.

[0023] Step 1.4: Extract the temporal characteristics of users or interaction targets from dynamic interaction behavior data; For users At the present moment Historical interaction sequence , which includes the most recent Each interaction record corresponds to a historical timestamp. To characterize the temporal features of the interaction, this invention encodes each timestamp in the historical interaction sequence, thereby obtaining the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows: ; in, , All of these are trainable parameters; Encoding dimension for time; For the current moment With historical moments The time interval. Therefore, the user At the current time The time feature matrix can be represented as It consists of a historical interaction sequence In The time of each of the interaction records in the interaction target is encoded to obtain a time encoding vector.

[0024] Similarly, the time of each of the interaction records in the interaction target at the current time is encoded to obtain a time feature matrix of the interaction target at the current time . Thus, the time feature matrix of the interaction target at the current time is obtained by the time encoding vector of each of the interaction records in the corresponding historical interaction sequence , and the length of the historical interaction sequence is consistent. Step 1.5, in order to unify the dimensions of different features, the present application multiplies all the features of the user with the learnable parameter matrix to obtain the complete features of the user:

[0025] ; ; wherein, is the complete feature of the user at the current time ; , , , are all learnable parameters for aligning features of different dimensions; is the interaction feature at the current time .

[0026] Similarly, all the features of the interaction target are multiplied with the learnable parameter matrix to obtain the complete features of the interaction target: ; wherein, is the complete feature of the interaction target at the current time ; , , are all learnable parameters for aligning features of different dimensions; is the interaction feature at the current time .

[0027] Step 1.6, the complete features of the user and the complete features of the interaction target are spliced to obtain the comprehensive features: ; wherein, is the comprehensive feature at the current time ​​The comprehensive features fully characterize the interaction features between the user and the interaction target at that moment, providing input for subsequent user interest modeling and recommendation.

[0028] Step 2: Construct a user profile encoder based on variational inference to perform dynamic encoding and generate the original latent representation; then construct an accurate user profile. In the dynamic graph neural network framework, the user profiling encoder, combined with a Transformer-based multi-head attention mechanism, can effectively capture the sequential dependencies in user behavior patterns. Unlike traditional encoders that output deterministic vectors, the user profiling encoder of this invention does not output a deterministic vector, but rather the parameters (including mean and standard deviation) of a latent Gaussian distribution, which probabilistically models the user's current comprehensive preferences. This not only expresses the uncertainty of user interests but also provides a flexible latent space representation for the simulation and intervention of subsequent interest evolution. For example, if a user first browses digital products and then follows review videos related to those products, this continuity of interest represents an evolutionary path of user interests. To better capture the dynamic evolution of social networks, this invention addresses the sequential dependencies of each node (user) at each moment. or interactive target Initialize the current time. state vector and The state of nodes is updated through a recurrent neural network; this mechanism enables the model to remember the user's long-term historical behavior and connect new interactions with existing interests.

[0029] The specific working process of a user profile encoder based on variational inference is as follows: Step 2.1: First, for the current time... Comprehensive characteristics Multi-head attention encoding is performed because For users With the target of interaction By concatenating features, multi-dimensional dependencies between them can be captured: ; ; ; in, For the current moment Attention representation; This is a multi-head attention mechanism; , The first The, the One point of attention; Learnable parameters for the multi-head attention output layer; for attention mechanism; , are the learnable parameters for the th attention head query , key , value ; represents layer normalization; is the transpose symbol; is the dimension of the key vector for each attention head; is the number of attention heads; is the activation function; through multi-head attention, the user portrait encoder can capture the diversified relationship patterns between users and interaction targets in different representation subspaces in parallel, to improve the expression ability of dynamic graph neural networks.

[0030] Step 2.2, then, the attention representation at the current time is concatenated with the hidden state at the previous time to input into two independent feed-forward neural networks (FFN) to generate the mean and standard deviation of the Gaussian distribution, respectively: ; ; ; wherein, is the feed-forward neural network; is the input feature of the feed-forward neural network; , , , are all learnable parameters; is the hidden state at the previous time ; is the activation function; is the mean of the Gaussian distribution at the current time ; is the standard deviation of the Gaussian distribution at the current time ; is the feed-forward neural network for generating the mean; is the feed-forward neural network for generating the standard deviation; Step 2.3, the approximate posterior distribution obtained based on the variational inference is: ; wherein, represents the approximate posterior distribution of the latent representation; represents, given , , and approximate posterior distribution of the inferred latent representation; represents the interaction structure at the current time ; represents the interaction structure at the current time ; represents the Gaussian distribution; represents the diagonal covariance matrix.

[0031] To enable end-to-end training of the model by gradient descent, the present application employs a reparameterization method to sample the original latent representation from the above approximate posterior distribution: ; wherein, is a standard Gaussian noise, i.e. ; is a unit covariance matrix; is an element-wise multiplication.

[0032] Step 2.4, in order to further model the process of interest dynamic evolution over time, a gated recurrent unit is used to iteratively update the hidden state: ; wherein, and are the hidden states at the current time and the previous time ; represents the update function of the gated recurrent neural network. This design enables the model to associate the newly generated interactions with the existing interest patterns while maintaining long-term memory, thereby effectively capturing the dynamic evolution characteristics of the interaction structure in the social network.

[0033] The user portrait encoder adopts a probabilistic encoding method, and its output is the mean and the standard deviation of the Gaussian distribution at the current time . Then, a reparameterization method is used to sample an original latent distribution evolving over time from the parameters of the above Gaussian distribution, thereby providing greater flexibility and expressiveness for the subsequent latent distribution intervention stage.

[0034] Step 3, in order to simulate the diversified distribution shift of user preferences in social platforms, the present application constructs an intervention network for latent distribution extrapolation in the latent space generated by the user portrait encoder of the dynamic graph neural network based on variational inference; the intervention network is the core of simulating user interest evolution, which includes a set of learnable "perturbators" that perturb the original user latent preference distribution parameters (mean and standard deviation ) to generate multiple new, intervened latent representations These newly generated distributions are used as "proxies" for unknown interest shifts (e.g., interest broadening or interest migration) that the user might experience in the real world. The intervention network maximizes the diversity among the generated distributions through a specialized loss function to ensure that as wide a range of interest evolution scenarios as possible are simulated.

[0035] The present application extrapolates the original latent representation into a mixture of multiple intervened latent representations through an intervention network; The intervention network is composed of independent perturbers , the learnable parameters of the th perturber; where each perturber receives the original Gaussian distribution parameters and outputs a new set of Gaussian distribution parameters and samples the user's intervened latent representation from the above parameters using the reparameterization method: ; ; where , are the mean and standard deviation of the Gaussian distribution of the th perturber at the current time ; represents the latent representation sampled from the approximate posterior distribution intervened by the th perturber at the current time ; is the standard Gaussian noise, i.e. .

[0036] In implementation, the perturber uses two independent linear transformations to map the input , respectively: ; ; where , , , are the learnable parameters of the th perturber; is the activation function.

[0037] To ensure that the generated distribution has sufficient diversity to cover a variety of potential test distributions in the social platform (e.g. short-lived interest in a newly released movie, discussion triggered by a sudden news event), the training objective of the intervention network aims to maximize the variance between the sets of new Gaussian distribution parameters; the intervention loss is defined as follows: ; where, is the intervention loss; and are hyperparameters that control the weights of the mean and standard deviation diversity, respectively; is the variance calculation.

[0038] Step 4, construct an expert mixed decoder to decode and aggregate the original latent representation and the perturbed latent representations to generate the final recommendation result; Step 4.1, for the generated multiple potential interest distributions, the invention uses an expert mixed decoder for processing; The goal of the expert mixed decoder is to decode the latent representation generated by the user portrait encoder based on variational inference into a specialized user preference representation under the dynamic graph neural network framework; the first expert decoder serves as a reference expert to process the original, unperturbed latent representation representing the user's current core interest; the remaining expert decoders process the perturbed latent representations respectively, so each expert decoder corresponds to a perturber for processing the output of each perturber; the gating network calculates the weight of each expert decoder to determine the contribution of each expert decoder in the aggregation process of the final user representation; In the invention, there are expert decoders, of which the first expert decoder serves as a reference expert to decode the unperturbed original approximate posterior distribution This expert decoder focuses on learning the user's long-term, stable core interest; the remaining expert decoders decode the perturbed approximate posterior distributions These expert decoders are trained to adapt to different potential interest evolution patterns, e.g., users' short-lived interest in a certain hot topic; each expert decoder is a learnable nonlinear function ; To achieve adaptive selection among different experts, the gating network receives the intervened latent representation As input, the weights of expert decoders are computed; the introduction of noise helps to achieve more balanced expert utilization in training: ; where is the weight of the th expert decoder; , are learnable parameter matrices; and are activation functions; The final aggregated representation is obtained by weighted summation of the outputs of all expert decoders: ; where is the aggregated representation at the current time ; is the th expert decoder.

[0039] Since in the multi-source data fusion process in step 1, the input aggregated representation at the current time is the concatenation of the user and the interaction target features and models their relationship through an attention mechanism, therefore, in the expert mixture decoding stage, the user and the interaction target vectors need to be extracted from the aggregated representation and averaged as their final representations: ; ; where , are the final representations of the user and the interaction target at the current time ; is the averaging operation; and are the corresponding index lengths of the user and the interaction target at the current time .

[0040] Step 4.2, predicting with the final representation of the user and the interaction target to complete the recommendation task; In the recommendation task, by calculating the similarity between the user and the interaction target (or post) , the new content that the user may be interested in is predicted. Specifically, in order to judge whether the user and the interaction target will interact at the current time , the inner product of the final representation of the user and the final representation of the interaction target is calculated, normalized, and the interest probability of the user to the interaction target is obtained: ; Wherein, represents the interest probability of the user to the interaction target at the current time ; is an activation function; , are the final representations of the user and the interaction target at the current time ; Sort all interest probabilities in descending order, and output the interaction targets corresponding to the top interest probabilities as the final recommendation results.

[0041] Step 5, in order to obtain a core preference representation with stability to interest evolution, the training process of the dynamic graph neural network is designed as a double-layer optimization problem. Specifically, the goal of the outer optimization is to learn a representation that can robustly capture the user's core interest, and to minimize the composite training target (composed of the lower bound loss and the alignment loss) by updating the encoder and decoder parameters; while the goal of the inner optimization is to generate a diverse latent distribution, and to minimize its corresponding intervention loss by updating the parameters of the intervention network.

[0042] The composite training target contains two parts: the evidence lower bound loss (including the reconstruction loss of the interaction relationship structure and the KL divergence loss), and the alignment loss; the training optimization is performed by weighting the evidence lower bound loss and the alignment loss. This ensures that the model can accurately encode and reconstruct user behavior, and also maintains robustness when facing simulated distribution shifts, thereby improving the recommendation performance in real dynamic environments.

[0043] Step 5.1, the evidence lower bound loss is the core loss function of the user portrait encoder based on variational inference, used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: one is the reconstruction loss of the dynamic graph neural network reconstructing the interaction relationship , for measuring the accuracy of the reconstructed interaction relationship structure of the model for the user and the interaction target; the other is the KL divergence loss , for constraining the difference between the approximate posterior distribution corresponding to each perturber and its conditional prior : ; ; ; wherein, is the evidence lower bound loss, composed of the reconstruction loss of the interaction relationship structure and the KL divergence loss ; is the total number of time steps; is the number of perturbers and latent distributions; denotes the latent representation sampled from the approximate posterior distribution at the current time ; denotes the latent representation sampled from the approximate posterior distribution after intervention by the th perturber at the current time ; is the interaction relationship structure before the current time ; is the comprehensive feature before the current time ; is the conditional probability distribution of generating given ; denotes the conditional probability distribution of generating at the current time given ; denotes the conditional prior distribution defined based on at the current time ; denotes the approximate posterior distribution of the latent representation approximately inferred by the user portrait encoder at the current time given , and ; denotes the approximate posterior distribution of the latent representation approximately inferred by the th perturber at the current time given , and ; is the KL divergence; is the posterior distribution and the prior distribution KL divergence between them.

[0044] Step 5.2, Alignment loss is the key to ensure the model learns an invariant representation. This loss function aims to minimize the expected distance between the output representation of each "perturbed" expert and the output representation of the "reference" expert. This forces the model to keep its core user profile representation stable when facing various interest shifts: ; where, is the alignment loss; is the number of perturbers and latent distributions; , are the approximate posterior distributions of the first, the expert decoder, respectively; denotes the latent representation sampled from the approximate posterior distribution of the th perturber at the current time ; denotes the latent representation sampled from the approximate posterior distribution at the current time .

[0045] Step 5.3, The double-layer optimization strategy of the invention includes inner-layer optimization and outer-layer optimization, which are used to update two independent types of parameters under the dynamic graph neural network framework, where the inner-layer optimization updates the parameters of the intervention network , and the outer-layer optimization updates the parameters of the encoder and decoder .

[0046] Inner-layer optimization: For the fixed parameters of the main model (encoder, decoder, etc.) , the parameters of the intervention network are optimized, and the goal is to maximize the diversity of the generated distribution, constantly creating new and unpredictable interest changes for the decoder. The update formula is: ; where, is the optimal parameter of the intervention network; is the intervention loss.

[0047] Outer-layer optimization: Fix the optimal intervention network parameters , then optimize the parameters of the main model (encoder, decoder, etc.) , and the goal is to minimize the weighted sum of the evidence lower bound loss and the alignment loss. The update formula is: ; where, is the optimal parameter of the main model (encoder, decoder, etc.); is a hyperparameter that controls the alignment strength. reconstruction loss for the interaction structure; KL divergence loss; alignment loss.

[0048] The application carries out experiments in a variety of different distribution shift application scenarios to verify the feasibility and superiority of the method of the application in the scene of feature distribution and historical interaction data distribution shift. In the experiment process, the method proposed in the application (hereinafter referred to as MoVE) is compared with a variety of prior art schemes, including the empirical risk minimization method ERM, the invariant risk minimization method IRM, the dynamic graph attention network DIDA based on disentanglement intervention, the dynamic graph invariant learning method SILD, the dynamic graph attention method TGAT, the dynamic graph method TGN based on recurrent neural network, the method CAWN based on causal anonymous walk, the method DygFormer based on Transformer, and the dynamic graph method FreeDyg based on frequency domain enhancement. Finally, in the application scenario of user interest evolution and distribution shift, the experimental results of each method are comprehensively compared and analyzed, thereby further verifying the effectiveness and superiority of the method of the application in practical application.

[0049] The application first evaluates a variety of interaction behaviors of users in a social platform scenario, including posting and like requests on the Reddit platform within one month, and editing records on the Wikipedia platform. Specifically, the Reddit dataset contains 10984 nodes and 672447 interactions, and the Wikipedia dataset contains 9227 nodes and 157474 interactions. The experimental results are shown in Table 1, in which the best result is indicated in bold, and the suboptimal result is indicated in underline. In these two datasets, the feature distribution of the test set is significantly shifted. If the model cannot effectively utilize the invariant patterns across the distribution in the data, its performance on the test set will obviously decrease.

[0050] Table 1 shows that in the case of rapid migration of user interest from one topic circle to another, the application can maintain a high recommendation accuracy, while the comparative method has a significant performance decrease after the interest changes.

[0051] Table 1 Recommendation prediction results under user feature behavior distribution shift (average accuracy%) .

[0052] In order to further verify the ability of the application in dealing with out-of-distribution data, in addition to verification in the social platform scenario, the application also carries out experiments in the historical interaction data distribution shift scenario.

[0053] Specifically, the UN Trade dataset covers trade records between countries over the past 30 years, containing 299 nodes and 507497 interactions; the UN Vote dataset covers voting records between countries at the United Nations General Assembly over the past 70 years, containing 201 nodes and 1035742 interactions. Due to the continuous evolution of the geopolitical landscape, the distribution of these two datasets also changes significantly in different time periods.

[0054] The experiment adopts two settings: in direct push learning, the training phase, the model can access the test data in addition to the training data; in inductive learning, the training phase, the model cannot observe any samples of the test data and can only make inferences based on learned knowledge at test time, so this setting is more challenging. The experimental results are shown in Table 2, where the best results are marked in bold and the suboptimal results are marked in underlined.

[0055] Table 2 shows that the present application can effectively identify potential environmental changes and avoid over-reliance on false correlations, thereby maintaining consistent excellent recommendation performance in various dynamic environments.

[0056] Table 2 shows that the present application can effectively identify potential environmental changes and avoid over-reliance on false correlations, thereby maintaining consistent excellent recommendation performance in various dynamic environments. .

[0057] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A hybrid expert and dynamic graph neural network-based recommendation method against data drift, characterized in that, The dynamic graph neural network comprises a user portrait encoder and an expert mixed decoder; specifically comprising the following steps: Step 1, collecting and fusing multi-source data of users from a social platform to build comprehensive features; Step 2, constructing a user portrait encoder based on variational inference for dynamic encoding to generate original latent representations; Step 3, constructing an intervention network for extrapolating the original latent representations into a mixture of multiple intervened latent representations; Step 4, constructing an expert mixed decoder to decode and aggregate the original latent representations and the intervened latent representations to generate a final recommendation result; Step 5, designing a composite training target and a double-layer optimization strategy.

2. The method of claim 1, wherein, The specific process of step 1 is as follows: Step 1.1, the collected multi-source data includes static attribute data and dynamic interaction behavior data; wherein the static attribute data includes user personal information; the dynamic interaction behavior data includes behavior sequence data and time dynamic information; the behavior sequence data includes posting, liking, forwarding, commenting, and collecting; the time dynamic information includes the timestamp of the behavior occurrence; Step 1.2, extract the self-features of the user or the interaction target respectively; the interaction target is divided into two categories: one is a post or a comment, and the other is a target user; the user's personal information, posts, comments, and the target user's homepage text all belong to text information; the specific process of extracting self-features is as follows: first, pre-process the text, including word segmentation, removing stop words and noise symbols; then match the pre-processed words with the dictionary in the text analysis tool LIWC, count the frequency of each word in different psychological, linguistic and semantic categories, and normalize to obtain the corresponding semantic feature vector; define the semantic feature vector extracted from the user's personal information as the user's self-feature at the current time at the current time of the user ; define the semantic feature vector extracted from the interaction target as the self-feature of the interaction target at the current time of the interaction target ; Step 1.3, performing one-hot encoding according to the interaction type to obtain interaction features; the interaction types include posting, liking, forwarding, commenting, collecting, following, and unfollowing; in the one-hot encoding process, each interaction type is assigned a fixed feature position, and when a certain interaction occurs, the corresponding position is set to 1, and the remaining positions are set to 0; Step 1.4: Extract the temporal features of the user or interaction target; for users At the present moment Historical interaction sequence , which includes the most recent Each interaction record corresponds to a historical timestamp. Encode each timestamp in the historical interaction sequence to obtain the historical interaction sequence. Time feature matrix Specifically, the current time is calculated first. With historical moments time interval And it learns its periodic pattern through time encoding, with each time interval encoded as follows: ； wherein, , are trainable parameters; is a time encoding dimension; Likewise, the interaction target is encoded at the current time point, obtaining a time feature matrix of the historical interaction sequence at each timestamp ; Step 1.5, multiplying all features of the user with a learnable parameter matrix to obtain the complete features of the user: ； wherein, is the current time user 's complete features; , , are all learnable parameters; is the current time of interaction features; Step 1.6, concatenating the complete features of the user and the complete features of the interaction target to obtain comprehensive features: ； wherein, is the current time interaction target complete features; , , are learnable parameters; is the current time interaction features; In step 2, the specific working process of the user portrait encoder based on variational inference is as follows: ； wherein is the integrated feature of the current time instance.

3. The method of claim 1, wherein the method further comprises: Step 2.2, concatenating the attention representation at the current time and the hidden state at the previous time and inputting them into two independent feedforward neural networks to generate the mean and standard deviation of the Gaussian distribution, respectively: Step 2.1, on the current time instance the integrated features of the current time instance performing multi-head attention encoding: ； ； ； wherein, is the attention representation at the current time instant ; is the multi-head attention mechanism; , are the first , second attention heads, respectively; are the learnable parameters of the multi-head attention output layer; is the attention mechanism; , are the learnable parameters of the first attention head query , key , value , respectively; denotes layer normalization; is the transpose symbol; is the dimension of the key vector for each attention head; is the activation function; Step 2.3, obtaining the approximate posterior distribution based on variational inference: ； ； ； in, It is a feedforward neural network; These are the input features of the feedforward neural network; , , , All are learnable parameters; For the previous moment The hidden state; For activation functions; For the current moment The mean of a Gaussian distribution; For the current moment The standard deviation of the Gaussian distribution; A feedforward neural network for generating the mean; To generate a feedforward neural network for standard deviation; Step 2.4, iteratively updating the hidden state using a gated recurrent unit to model the dynamic evolution of interest over time: ； wherein, denotes an approximate posterior distribution of the latent representation approximated by the user profile encoder at the current time , and posterior distribution of the latent representation approximated by the user profile encoder at the current time from the approximate posterior distribution; denotes an interaction structure at the current time ; denotes a Gaussian distribution; denotes a diagonal covariance matrix; are sampled from the approximate posterior distribution using the reparameterization method, given by: ； wherein is a standard Gaussian noise, i.e. ; is a unit covariance matrix; is an element-wise multiplication; The training target of the intervention network is to maximize the variance between the parameters of each new Gaussian distribution, and the intervention loss calculation formula is defined as follows: ； where, and are the hidden states at the current time and the previous time respectively; denotes the update function of the gated recurrent unit.

4. The method of claim 3, wherein, The step 3, the intervening network consists of one independent perturbator , is the learnable parameter of the th perturbator; Each perturber receives the original Gaussian distribution parameters and outputs a new set of Gaussian distribution parameters and employs a reparameterization method to sample the user- intervened latent representation from the new Gaussian distribution parameters: ； ； wherein, , are the mean and standard deviation of the Gaussian distribution of the th disturbance; denotes the latent representation sampled from the approximate posterior distribution of the th disturbance after intervention. Finally, corresponding to the potential representation after the intervention ; Disturbance The input is mapped using two independent linear transformations: ​​ ； ； wherein, , , , are learnable parameters of the th perturber; is an activation function; The weighted sum of the outputs of all expert decoders is obtained as the final aggregated representation: ； where, is the intervention loss; and are hyperparameters, respectively controlling the mean and standard deviation of the diversity; is the variance computation.

5. The method of claim 4, wherein, The step 4, the expert mixed decoder contains a gating network and a parallel expert decoder; the first expert decoder processes the original latent representation as a reference model ; the rest of the expert decoders process the intervened latent representations respectively One expert decoder corresponds to one perturber; the gating network calculates the weight of each expert decoder, as follows: ； wherein, is the weight of the expert decoder; , are learnable parameter matrices; The inner product of the final representation of the user and the final representation of the interaction target is calculated and normalized to obtain the interest probability of the user to the interaction target: ； wherein, is the current time of aggregation; is the th expert decoder; In the expert mixing decoding phase, the vectors of the users and the interaction target are taken from the aggregated representation respectively and averaged as their final representation: ； ； wherein, , are the final representation of the current time user , interaction target ; is the average operation; and are the corresponding index lengths of the current time user and interaction target ; In step 5, the composite training target includes two parts: evidence lower bound loss and alignment loss; ； in, Indicates user At the present moment For the interaction target The probability of interest; For activation functions; Sort all interest probabilities in descending order, output the interaction target corresponding to the top interest probability as the final recommendation result.

6. The method of claim 5, wherein, The evidence lower bound loss is the loss function of the user portrait encoder based on variational inference, which is used to maximize the lower bound of the log-likelihood of the data; it consists of two parts: reconstruction loss of interaction relationship structure and KL divergence loss; the specific formula is as follows: The update formula of the inner layer optimization is as follows: ； ； ； wherein, is the evidence lower bound loss; is the reconstruction loss of the interaction structure; is the KL divergence loss; is the total number of time steps; is the current time the interaction structure before; is the comprehensive feature before the current time is the conditional probability distribution of , given to generate ; is the conditional prior distribution of , defined based on ; is the approximate posterior distribution of the latent representation , approximated inferred by the user profile encoder, given , and ; is the approximate posterior distribution of the latent representation , approximated inferred by the th perturber, given , and ; is the KL divergence; Alignment loss The formula is: ； in, , The first and the second, respectively. Approximate posterior distribution of an expert decoder; The double-layer optimization strategy comprises an inner-layer optimization and an outer-layer optimization . The inner-layer optimization is used to update parameters of the intervention network , and the outer-layer optimization is used to update parameters of the encoder and the decoder The update formula of the outer layer optimization is as follows: ； wherein, are the parameters of the optimal intervention network; ​ ； wherein, are parameters of the optimal encoder and decoder; is a hyperparameter that controls the strength of alignment.

Images