Out-of-distribution generalization method of dynamic graph neural network model and server
By performing feature modeling and decoupling at different time scales in the dynamic graph neural network model, and combining Fourier transform and cluster analysis, an auxiliary loss function and a multi-scale intervention risk loss function are constructed. This solves the problem of insufficient generalization of the dynamic graph neural network model in out-of-distribution scenarios and improves the model's adaptability and prediction accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2025-12-19
- Publication Date
- 2026-05-08
AI Technical Summary
Existing dynamic graph neural network models lack generalization ability when faced with differences between the distribution of test data and training data, making it difficult to effectively adapt to diverse real-world application scenarios.
By modeling the temporal features of dynamic graphs at different time scales, using Fourier transform to decouple features, separating environment-independent invariant features from environment-related changing features, and inferring environment labels through cluster analysis, an auxiliary loss function is constructed to guide model learning, and the model parameters are optimized by combining a lightweight neural network and a multi-scale intervention risk loss function.
It improves the generalization ability and stability of dynamic graph neural network models in out-of-distribution scenarios, and enhances their adaptability and prediction accuracy in unknown scenarios.
Smart Images
Figure CN121997976A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of electronic digital data processing technology, and in particular to a distributed outgeneralization method and server for a dynamic graph neural network model. Background Technology
[0002] Dynamic Graph Neural Networks (DyGNNs), as an important technology for processing time series graph data, aim to model the temporal evolution of the structure and node attributes of dynamic graphs, playing a key role in many fields such as social network analysis and financial transaction monitoring.
[0003] The relevant technical methods mainly employ two approaches to learn the evolutionary patterns of dynamic graphs: The first method uses graph neural networks to capture the spatial structure information of each time slice, and then combines it with sequence models such as recurrent neural networks or long short-term memory networks to model temporal dependencies. The second method treats graph changes as a continuous flow of events, and models them in the continuous time domain through temporal encoding functions or self-attention mechanisms. These methods identify and predict the evolutionary patterns of dynamic graph structures by learning the spatiotemporal feature patterns in the dynamic graph.
[0004] However, related technologies are mainly based on the independent and identically distributed (ICD) assumption, which assumes that the training and test data are independently sampled from the same distribution. In practical applications, the distribution of test data often comes from diverse sources, and the distribution differs significantly from the training dataset, resulting in low out-of-distribution generalization ability of the dynamic graph neural network models built by these technologies. Summary of the Invention
[0005] This application provides a method and server for out-generalization of dynamic graph neural network models to improve the out-generalization capability of dynamic graph neural network models.
[0006] Firstly, a method for out-of-distribution generalization of a dynamic graph neural network model is provided and applied to a server. This method includes: the server obtaining a temporal feature matrix from a dynamic graph snapshot sequence; the server performing temporal average pooling on the temporal feature matrix at progressively increasing preset time steps to obtain multi-scale feature representations at different time scales; the server converting the multi-scale feature representations into spectral domain features at different time scales using Fourier transform; the server processing the spectral domain features using a multilayer perceptron at a preset scale to obtain a soft mask matrix; the server calculating the spectral domain features and the soft mask matrix using a preset function set to obtain invariant and variable features at different time scales; the server using cluster analysis to use the cluster centers of the variable features as environment labels at different time scales; the server constructing an optimization objective function based on invariant features, environment labels, a preset auxiliary loss function, and a preset task loss function. The preset auxiliary loss function is used to eliminate residual environmental influences in invariant features at the same time scale as the environment labels, and the preset task loss function is used to measure the difference between the predictions made by the dynamic graph neural network model based on invariant features and the true labels; a smaller value indicates a more accurate prediction; and the server updating the network parameters of the dynamic graph neural network model based on the optimization objective function.
[0007] By employing the above technical solution, the server models the temporal features of the dynamic graph at different time scales and uses Fourier transform to decouple the features in the frequency domain, separating invariant features independent of the environment from variable features related to the environment. Furthermore, by clustering the variable features, potential environmental labels are inferred unsupervised, and these environmental labels are used to construct an auxiliary loss function to guide model learning. This reduces the residual environmental influence in the invariant features, enabling the model to focus on learning the stable patterns in the dynamic graph's evolution process. This, in turn, improves the generalization ability of the dynamic graph neural network model when facing unknown scenarios (i.e., out-of-distribution scenarios) with distributions different from the training data.
[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the server calculates the spectral domain features and the soft mask matrix using a preset function set to obtain invariant and variable features at different time scales. Specifically, this includes the following: the preset function set is: In the above function group, For invariant features at the s-th time scale; The change characteristics at the s-th time scale; This is the inverse Fourier transform; Features of the spectral domain; It is a soft mask matrix; This represents element-wise product.
[0009] By adopting the above technical solution, a decoupling method for invariant and variable features is provided through a preset function group.
[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the server constructing an optimization objective function based on invariant features, environmental labels, a preset auxiliary loss function, and a preset task loss function specifically includes: the preset auxiliary loss function is: In the above function, A pre-defined auxiliary loss function is used to maximize the independence between invariant features and environment labels, thereby eliminating residual environment-related information in invariant features; N is the total number of nodes in the dynamic graph; S is the total number of time scales; For the s-th time scale, the first... Invariant characteristics of each node; This refers to the environmental label at the s-th time scale; for and The cosine similarity.
[0011] By adopting the above technical solution and introducing a specific preset auxiliary loss function, the cosine similarity between invariant features and corresponding environmental labels is minimized, and residual information related to the environment in invariant features is actively penalized, thereby prompting the model to learn more generalizable invariant features and thus improving the stability of the dynamic graph neural network model in out-of-distribution scenarios.
[0012] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of the server constructing an optimization objective function based on invariant features, environment labels, a preset auxiliary loss function, and a preset task loss function, the method further includes: the server constructing a mutual information maximization loss function based on variable features and environment labels; the mutual information maximization loss function is: In the above function, The mutual information maximization loss function is used to maximize the correlation between change features and environmental labels at different scales; N is the total number of nodes in the dynamic graph; S is the total number of time scales. For the first The first time scale The changing characteristics of each node; For the first The first time scale The environment label of each node; Used for estimation and The mutual information score between them.
[0013] By adopting the above technical solution, the mutual information maximization loss function maximizes the correlation between the changing features and environmental labels at different scales, prompting the model to recognize that potential environmental factors have a common influence across scales, thereby learning a more consistent representation of environmental factors, which in turn improves the accuracy of environmental label inference and makes invariant and changing features more accurate.
[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after the server calculates the spectral domain features and soft mask matrix using a preset function set to obtain invariant and variable features at different time scales, the method further includes: the server estimating the contribution value of the invariant features at each time scale based on a preset lightweight neural network model; the server using the normalized contribution value as weight to perform weighted summation of the invariant features at each time scale to obtain fused invariant features.
[0015] By adopting the above technical solution, the importance weights of each scale are learned through a lightweight neural network. This allows for dynamic emphasis on time scales with richer information based on the characteristics of specific data and tasks, thereby improving the prediction accuracy of the dynamic graph neural network model.
[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the server constructing an optimization objective function based on invariant features, environmental labels, a preset auxiliary loss function, and a preset task loss function specifically includes: after the server performs a weighted summation of the preset auxiliary loss function, the mutual information maximization loss function, and the preset multi-scale intervention risk loss function based on preset hyperparameters, it sums the summation with the preset task loss function to obtain the optimization objective function; the preset multi-scale intervention risk loss function is: In the above function, As a multi-scale intervention loss function, by simulating scenarios where fluctuations arise from specific changing features, the supervised dynamic graph neural network model relies solely on fused invariant features to make predictions. ; The prediction results of the invariant feature classifier in the dynamic graph neural network model are based on the fusion of invariant features; Represents element-wise product; From the characteristics of change Specific change characteristics obtained from sampling; Potential interference values for specific change characteristics; To normalize potential interference values; A pre-defined multi-scale intervention risk loss function is used to measure the stability of the multi-scale intervention loss function.
[0017] By employing the aforementioned technical solution, a pre-defined multi-scale intervention risk loss function simulates random perturbations to changing features and requires the dynamic graph neural network model to maintain the stability of its prediction results under these perturbations (i.e., minimize the variance of the intervention loss). This forces the model to reduce its dependence on spurious correlations present in the changing features. This makes the relationship between the invariant features learned by the dynamic graph neural network model and the prediction results closer to a causal relationship, thus exhibiting stronger robustness when facing out-of-distribution data containing unknown spurious correlations.
[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the server updating the network parameters of the dynamic graph neural network model based on the optimization objective function specifically includes: the server using gradient descent to iteratively update the parameters of the graph neural network backbone network, the multilayer perceptron of a preset scale, the preset lightweight neural network model, and the invariant feature classifier in the dynamic graph neural network model based on the optimization objective function; the server calculating the value of the current optimization objective function after each iteration; when the value is within a preset minimum value range for a preset number of consecutive iterations, the server determines that the optimization objective function has converged; the server stopping updating the parameters and saving the latest parameters after the optimization objective function has converged.
[0019] By adopting the above technical solution, the network parameters of the dynamic graph neural network model are continuously optimized by optimizing the objective function, thereby improving the distributional out-generalization ability of the dynamic graph neural network model.
[0020] In a second aspect, embodiments of this application provide a server comprising: one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the server to perform the method described in the first aspect and any possible implementation thereof.
[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.
[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.
[0023] Understandably, the server provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.
[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. The server models the temporal features of the dynamic graph at different time scales and uses Fourier transform to decouple the features in the frequency domain, separating invariant features independent of the environment from variable features related to the environment. Furthermore, it infers potential environmental labels unsupervised by clustering the variable features and uses these labels to construct an auxiliary loss function to guide model learning. This reduces the residual environmental influence in the invariant features, allowing the model to focus on learning the stable patterns in the dynamic graph's evolution, thereby improving the generalization ability of the dynamic graph neural network model when facing unknown scenarios (i.e., out-of-distribution scenarios) with a different distribution than the training data.
[0025] 2. By introducing a specific pre-defined auxiliary loss function, the cosine similarity between invariant features and corresponding environmental labels is minimized, and residual information related to the environment in invariant features is actively penalized, thereby prompting the model to learn more generalizable invariant features and thus improving the stability of the dynamic graph neural network model in out-of-distribution scenarios.
[0026] 3. A pre-defined multi-scale intervention risk loss function simulates random perturbations to changing features and requires the dynamic graph neural network model to maintain the stability of its predictions under these perturbations (i.e., minimize the variance of the intervention loss). This forces the model to reduce its dependence on spurious correlations in changing features. This makes the relationship between the invariant features learned by the dynamic graph neural network model and the prediction results closer to a causal relationship, thus exhibiting stronger robustness when facing out-of-distribution data containing unknown spurious correlations. Attached Figure Description
[0027] Figure 1 This is a flowchart illustrating a distributed outgeneralization method for a dynamic graph neural network model in an embodiment of this application.
[0028] Figure 2 This is another flowchart illustrating a distributed outgeneralization method for a dynamic graph neural network model in an embodiment of this application.
[0029] Figure 3 This is a schematic diagram of the physical device structure of a server in an embodiment of this application. Detailed Implementation
[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.
[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.
[0032] This application provides a method and server for out-generalization of dynamic graph neural network models to improve the out-generalization capability of dynamic graph neural network models.
[0033] Please see Figure 1 This is a flowchart illustrating a distributed outgeneralization method for a dynamic graph neural network model in an embodiment of this application.
[0034] S101. The server obtains the time-series feature matrix from the dynamic graph snapshot sequence.
[0035] In this context, a dynamic graph snapshot sequence refers to a series of samples taken from a dynamic graph at continuous or discrete points in time. For example, a daily structural snapshot of a social network over T consecutive days, denoted as... and with The timestamp is represented in the form of , where T is the number of timestamps. For a set of nodes, The edge set is defined as follows. The temporal feature matrix is a three-dimensional matrix whose dimensions integrate time, nodes, and features, used to represent the feature evolution process of all nodes in the dynamic graph at all observation time points.
[0036] Specifically, the server first receives or loads a sequence of dynamic graph snapshots. Since the raw dynamic graph data cannot be directly processed by a neural network, it needs to be converted into a numerical feature representation. Therefore, the server uses a pre-defined graph neural network backbone, such as a graph convolutional network or a graph attention network, to process each time snapshot in the dynamic graph snapshot sequence. The process is then performed. For each snapshot, the graph neural network backbone generates a low-dimensional feature vector for each node in the graph by aggregating information from neighboring nodes. The node feature calculation process is as follows: ,in , indicating time hour The embedding representation of each node is given, where d is the dimension of the node embedding. The server then concatenates the node embeddings from each time step in chronological order to obtain the temporal feature matrix. To obtain the temporal evolution of node features in all time snapshots, where T is the total number of timestamps, N is the total number of nodes, and d is the dimension of each node's embedding.
[0037] S102. The server performs time-averaged pooling on the temporal feature matrix by gradually increasing the preset time step scale to obtain multi-scale feature representations at different time scales.
[0038] Temporal average pooling is a technique for downsampling time-series data along the time dimension. It compresses the data by calculating the average of feature vectors within a time window, thereby reducing the temporal resolution while preserving the main trends. The preset time step scale refers to different temporal granularities used to observe the evolution of a dynamic graph; smaller scales focus on short-term, high-frequency changes, while larger scales focus on long-term, low-frequency trends. Multi-scale feature representation refers to a series of feature sets extracted from the same dynamic graph sequence at multiple different time scales, with each set corresponding to a specific temporal observation granularity.
[0039] Specifically, this step involves obtaining the complete temporal feature matrix. Then it is executed. The server defines a series of scales. For each scale s, the server processes the original temporal feature matrix. Execute a window with a size of The time-average pooling operation is used. For example, at scale s=0, no pooling is performed, preserving the original fine-grained features; at scale s=1, every two consecutive time steps are merged into one, and its feature value is the average of the features of these two time steps; at scale s=2, every four time steps are merged into one. In this way, the server obtains multi-scale feature representations at different time scales. ,in Indicates the first The first scale A representation of d nodes across d dimensions. Smaller The value retains fine-grained short-term dynamic characteristics, while the larger value retains the fine-grained short-term dynamic characteristics. The value captures the long-term trend of coarse-grained changes.
[0040] S103. The server converts the multi-scale feature representation into spectral domain features at different time scales through Fourier transform.
[0041] The Fourier transform is a mathematical tool used to convert time series features from a time-domain representation to a frequency-domain representation. Spectral domain features refer to the representation of a signal in the frequency domain, where low-frequency components correspond to the smoothness and long-term trend of the signal, while high-frequency components correspond to the rapid and short-term fluctuations of the signal.
[0042] Specifically, this step involves processing the multi-scale feature representation obtained in S102. The purpose of this execution is to transform the feature evolution patterns in the time domain to the frequency domain for analysis, because in the frequency domain, low-frequency components representing stable trends and high-frequency components representing transient fluctuations are more easily distinguished. The transformation process of converting multi-scale feature representations into spectral domain features at different time scales using Fourier transform is as follows: ,in Describe the complex Fourier basis. This represents the number of frequency components, where T is the total number of time scales. For the first There are _t_i_t ... To use Fourier base Multiscale feature representation multiplied by the s-th time scale Spectral domain features were obtained. The tensor obtained from this process It includes node features in the spectral domain, with dimensions of frequency number K × number of nodes N × feature dimension d.
[0043] S104. The server processes the spectral domain features based on a multilayer perceptron with a preset scale to obtain a soft mask matrix.
[0044] Among them, the Multilayer Perceptron (MLP) is a feedforward artificial neural network composed of multiple fully connected layers, capable of learning nonlinear relationships in input data. A pre-scaled multilayer perceptron refers to an MLP independently configured for each time scale s, denoted as . The soft mask matrix is used to process spectral features at the s-th time scale. It is a matrix that matches the dimension of the spectral features, with element values between 0 and 1. It is used to weight each frequency component of the spectral features. A value close to 1 indicates that the frequency component is considered important (e.g., belonging to invariant features), while a value close to 0 indicates that the frequency component is considered unimportant or should be suppressed (e.g., belonging to variable features).
[0045] Specifically, this step involves obtaining spectral domain features. The subsequent execution aims to generate a weight for each node and each frequency component to determine whether the frequency component should be classified as an invariant or variable feature. For each time scale s, the server assigns the spectral domain features at that scale... (Usually, the real part and imaginary part are concatenated, or their modulus) are used as input and fed into the corresponding multilayer perceptron. This MLP learns the mapping from spectral domain features to their importance scores through a series of nonlinear transformations. The output of the MLP is then passed through a sigmoid activation function, which compresses the output values to the (0,1) interval, thereby generating the final soft mask matrix. The dimensions of this matrix match the frequency and node dimensions of the spectral domain features.
[0046] S105. The server calculates the spectral domain features and soft mask matrix using a preset function set to obtain invariant and variable features at different time scales.
[0047] The pre-defined function set refers to a fixed set of mathematical operations used to decompose spectral domain features into two parts based on the soft mask matrix. Invariant features are those that remain stable under different environments or distributions and are strongly correlated with the core prediction task; in the frequency domain, they typically correspond to low-frequency, stable signal components. Variable features are those that fluctuate significantly with environmental changes and may introduce spurious correlations; in the frequency domain, they typically correspond to high-frequency, volatile signal components.
[0048] Specifically, this step is performed after S104, and its purpose is to perform feature decoupling operations. For each scale s, the preset function set applied by the server is as follows: In this function group, For invariant features at the s-th time scale; The change characteristics at the s-th time scale; This is the inverse Fourier transform; Features of the spectral domain; It is a soft mask matrix; This represents element-wise multiplication. The preset function set is used to calculate the spectral domain features and the soft mask matrix, yielding invariant features at different time scales. and characteristics of change And denote the set of change characteristics across all time scales as .
[0049] S106. The server uses cluster analysis to identify cluster centers with changing characteristics as environmental labels at different time scales.
[0050] Specifically, the change characteristics are obtained on the S105 server. Execution occurs afterward. These change characteristics encode the dynamic graph. Environmental factors exist within the cluster. Since nodes in the same environment should share similar environmental factors, K-means clustering algorithm is used at each time scale to infer the potential environment and select cluster centers. It is considered an environmental label.
[0051] S107. The server constructs an optimization objective function based on invariant features, environment labels, preset auxiliary loss functions, and preset task loss functions.
[0052] In this context, the pre-defined auxiliary loss function refers to an additional loss term introduced besides the main task loss to guide the model in learning specific attributes. The pre-defined task loss function measures the difference between the model's predictions and the true labels, and is the main driving force for model optimization; examples include cross-entropy loss for classification tasks and mean squared error loss for regression tasks. The objective function is the overall function that needs to be minimized during model training.
[0053] Specifically, the preset auxiliary loss function is: In the above function, A pre-defined auxiliary loss function is used to maximize the irrelevance between invariant features and environment labels (achieved by minimizing the cosine similarity between them, since the loss function has a negative sign, it maximizes irrelevance), in order to eliminate residual environment-related information in invariant features; N is the total number of nodes in the dynamic graph; S is the total number of time scales; For the s-th time scale, the first... Invariant characteristics of each node; This refers to the environmental label at the s-th time scale; for and Cosine similarity. Preset task loss function. The final optimization objective function L is calculated based on predictions made from invariant features (which can be invariant features at a certain scale or invariant features fused from multiple scales) and the true label Y. In the form of, It is a hyperparameter that balances the importance of both.
[0054] S108. The server updates the network parameters of the dynamic graph neural network model based on the optimization objective function.
[0055] Among them, network parameters refer to all learnable variables in the dynamic graph neural network model, including the weights of the graph neural network backbone, the weights of MLPs at various scales, and the weights of the classifiers.
[0056] Specifically, in a training iteration, the server first performs a forward propagation process, from S101 to S107, to calculate the overall optimization objective function L. Then, the server uses a backpropagation algorithm to calculate the gradient of L with respect to all learnable parameters in the model. This gradient indicates the direction of parameter adjustment; adjusting the parameters along this direction minimizes the loss function most quickly. Finally, the server uses an optimizer (such as Adam) to update all network parameters based on the calculated gradients. This forward propagation-loss calculation-backpropagation-parameter update process is repeated many times until the optimization objective function converges to a sufficiently small stable value, or until a preset number of training iterations is reached, marking the completion of model training.
[0057] In the above embodiments, the server models the temporal features of the dynamic graph at different time scales and uses Fourier transform to decouple the features in the frequency domain, separating invariant features independent of the environment and variable features related to the environment. Furthermore, by clustering the variable features, potential environmental labels are inferred unsupervised, and these environmental labels are used to construct an auxiliary loss function to guide model learning. This reduces the residual environmental influence in the invariant features, allowing the model to focus on learning the stable patterns in the dynamic graph's evolution process. This, in turn, improves the generalization ability of the dynamic graph neural network model when facing unknown scenarios (i.e., out-of-distribution scenarios) with a different distribution than the training data.
[0058] However, dynamic graph features often exhibit scale dependence, and environmental factors typically influence dynamic graph patterns across scales. To better capture these effects, this invention utilizes environmental labels inferred at different time scales to enhance the model's ability to identify potential environmental factors and extract invariant features.
[0059] Please see Figure 2 This is another flowchart illustrating a distributed outgeneralization method for a dynamic graph neural network model in an embodiment of this application.
[0060] S201. The server obtains the time-series feature matrix from the dynamic graph snapshot sequence.
[0061] S202. The server performs time-averaged pooling on the temporal feature matrix by gradually increasing the preset time step scale to obtain multi-scale feature representations at different time scales.
[0062] S203. The server converts the multi-scale feature representation into spectral domain features at different time scales through Fourier transform.
[0063] S204. The server processes the spectral domain features based on a multilayer perceptron with a preset scale to obtain a soft mask matrix.
[0064] S205. The server calculates the spectral domain features and soft mask matrix using a preset function set to obtain invariant and variable features at different time scales.
[0065] Step S201 is similar to step S101, step S202 is similar to step S102, step S203 is similar to step S103, step S204 is similar to step S104, and step S205 is similar to step S105, so they will not be repeated here.
[0066] S206. The server estimates the contribution value of invariant features at various time scales based on a preset lightweight neural network model.
[0067] Here, the preset lightweight neural network model refers to a network with a simple structure and few parameters, such as a single-layer or two-layer multilayer perceptron (MLP), which is denoted as […]. Its function is to quickly assess the importance of input features. The contribution value is a scalar used to quantify the relative importance of a scale-invariant feature to the final prediction task across all time scales.
[0068] Specifically, this step obtains invariant features across all scales. The subsequent execution aims to enable the dynamic graph neural network model to determine which time-scale invariant features are present in the current task. More importantly, the server will retain its invariable characteristics. Input to a shared or scale-specific lightweight neural network The network outputs an unnormalized logarithmic score, i.e., the contribution value. This value reflects the strength of the predictive power that the model believes is contained in the invariant features of scale s.
[0069] S207. The server uses the normalized contribution value as the weight to perform a weighted summation of the invariant features at each time scale to obtain the fused invariant features.
[0070] The normalization process refers to converting the contribution value obtained from S206 into a probability distribution with a sum of 1.
[0071] Specifically, this step S206 is executed after the contribution value is obtained. The server collects the contribution values for all scales s. In this embodiment, it can be achieved through the formula: right Normalization is performed. Then, the server uses the normalized contribution value as weight to perform a weighted summation of the invariant features at each time scale to obtain the fused invariant features. .
[0072] S208. The server uses cluster analysis to identify cluster centers with changing characteristics as environmental labels at different time scales.
[0073] Step S208 is similar to step S106, and will not be described again here.
[0074] S209. The server constructs a mutual information maximization loss function based on the change characteristics and environmental labels.
[0075] Mutual information, a concept in information theory, measures the interdependence between two random variables. Higher mutual information indicates more information about one variable than the other. The mutual information maximization loss function is an auxiliary loss term designed to improve the interdependence between changing features and environmental labels across different time scales.
[0076] Specifically, this step is based on a core assumption: underlying environmental factors influence the patterns of dynamic graphs across scales. Therefore, environmental information at one time scale should be inferred from the changing characteristics at other time scales. To achieve this, the server constructs a mutual information maximization loss function. The mutual information maximization loss function is: In the above function, The mutual information maximization loss function is used to maximize the correlation between change features and environmental labels at different scales; N is the total number of nodes in the dynamic graph; S is the total number of time scales. For the first The first time scale The changing characteristics of each node; For the first The first time scale The environment label of each node; Used for estimation and The mutual information score between them. This mutual information maximization loss function enables the mutual information maximization loss function to more accurately capture and separate environmental factors across multiple scales.
[0077] S210. After the server performs a weighted summation of the preset auxiliary loss function, the mutual information maximization loss function, and the preset multi-scale intervention risk loss function based on preset hyperparameters, it sums the summation with the preset task loss function to obtain the optimized objective function.
[0078] Specifically, the pre-defined multi-scale intervention risk loss function is as follows: In the above function, As a multi-scale intervention loss function, by simulating scenarios where fluctuations arise from specific changing features, the supervised dynamic graph neural network model relies solely on fused invariant features to make predictions. ; The prediction results of the invariant feature classifier in the dynamic graph neural network model are based on the fusion of invariant features; Represents element-wise product; From the characteristics of change Specific change characteristics obtained from sampling; Potential interference values for specific change characteristics; To normalize potential interference values; A pre-defined multi-scale intervention risk loss function is used to measure the stability of the multi-scale intervention loss function.
[0079] The optimization objective function in this embodiment ,in, It represents the difference between the prediction made by the invariant feature classifier in the dynamic graph neural network model based on invariant features and the true label. The smaller the value, the more accurate the prediction. It is the mutual information maximization loss constructed in S209. It is a preset auxiliary loss function constructed in S107. It is the aforementioned pre-defined multi-scale intervention risk loss function. , and It is a hyperparameter used to balance various losses.
[0080] S211. The server updates the network parameters of the dynamic graph neural network model based on the optimization objective function.
[0081] Specifically, the execution flow of this step is similar to S108, but the objective function optimization is more complex. The server uses gradient descent (such as the Adam optimizer) to minimize the objective function L from S210. In each iteration, the gradient of L with respect to all learnable parameters is calculated via backpropagation. These parameters include those of the graph neural network backbone, the multilayer perceptron of a preset scale, the preset lightweight neural network model, and the invariant feature classifier. The server then updates these parameters based on the gradient. The server continuously monitors the value of the objective function L or its performance on the validation set. When the value of the objective function L remains stable within a preset minimum range for a preset number of iterations (e.g., 10), the server determines that the model has converged. At this point, the training process stops, and the current optimal network parameters are saved.
[0082] In the above embodiments, the mutual information maximization loss function maximizes the correlation between changing features and environmental labels at different scales, prompting the dynamic graph neural network model to recognize that potential environmental factors have a common influence across scales. This leads to a more consistent representation of environmental factors, thereby improving the accuracy of environmental label inference and making both invariant and changing features more accurate. Furthermore, a preset multi-scale intervention risk loss function simulates random perturbations to changing features, requiring the dynamic graph neural network model to maintain the stability of its prediction results under this perturbation (i.e., minimizing the variance of the intervention loss). This forces the model to reduce its dependence on spurious correlations in changing features. This makes the relationship between the invariant features learned by the dynamic graph neural network model and the prediction results closer to a causal relationship, thus exhibiting stronger robustness when facing out-of-distribution data containing unknown spurious correlations.
[0083] The above describes a method for out-generalization of a dynamic graph neural network model in an embodiment of this application. The following describes an exemplary server 300 provided in an embodiment of this application.
[0084] Figure 3 This is a schematic diagram of an exemplary hardware structure of a server 300 provided in an embodiment of this application. In some embodiments, the server 300 is a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements a distributed outgeneralization method for a dynamic graph neural network model according to an embodiment of this application.
[0085] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0086] In some embodiments of this application, a computer-readable storage medium is also provided, including instructions that, when executed on the server 300, cause the server 300 to perform a distributional outgeneralization method for a dynamic graph neural network model according to an embodiment of this application.
[0087] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
[0088] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".
[0089] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.
[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method for distributed external generalization of a dynamic graph neural network model, applied to a server, characterized in that, The method includes: The server obtains the temporal feature matrix from the dynamic graph snapshot sequence; The server performs a time-averaged pooling operation on the temporal feature matrix by gradually increasing the preset time step scale, thereby obtaining multi-scale feature representations at different time scales. The server converts the multi-scale feature representation into spectral domain features at different time scales using Fourier transform. The server processes the spectral domain features based on a multilayer perceptron with a preset scale to obtain a soft mask matrix; The server calculates the spectral domain features and the soft mask matrix using a preset set of functions to obtain invariant and variable features at different time scales; The server uses cluster analysis to identify the cluster centers of the changing characteristics as environmental labels at different time scales. The server constructs an optimization objective function based on the invariant features, the environment labels, a preset auxiliary loss function, and a preset task loss function. The preset auxiliary loss function is used to eliminate residual environmental influences in the invariant features at the same time scale as the environment labels. The preset task loss function is used to measure the difference between the prediction made by the dynamic graph neural network model based on the invariant features and the true labels. The smaller the value, the more accurate the prediction. The server updates the network parameters of the dynamic graph neural network model based on the optimization objective function.
2. The method according to claim 1, characterized in that, The server calculates the spectral domain features and the soft mask matrix using a preset function set to obtain invariant and variable features at different time scales. Specifically, this includes the following steps: The preset function set is as follows: In the above function group, the For invariant features at the s-th time scale; The The change characteristics at the s-th time scale; The This is the inverse Fourier transform; The The spectral domain features; The The soft mask matrix; The This represents element-wise product.
3. The method according to claim 1, characterized in that, The step of the server constructing an optimization objective function based on the invariant features, the environmental labels, the preset auxiliary loss function, and the preset task loss function specifically includes: The preset auxiliary loss function is: In the above function, the The preset auxiliary loss function is used to maximize the independence between the invariant features and the environment label, so as to eliminate residual environment-related information in the invariant features; N is the total number of nodes in the dynamic graph; S represents the total number of time scales; The For the s-th time scale, the first... Invariant characteristics of each node; The This refers to the environmental label at the s-th time scale; The For the With the The cosine similarity.
4. The method according to claim 1, characterized in that, Before the step where the server constructs an optimization objective function based on the invariant features, the environmental labels, the preset auxiliary loss function, and the preset task loss function, the method further includes: The server constructs a mutual information maximization loss function based on the change characteristics and the environmental labels; The mutual information maximization loss function is: In the above function, the The mutual information maximization loss function is used to maximize the correlation between the change features and the environmental labels at different scales. N is the total number of nodes in the dynamic graph; S represents the total number of time scales; The For the first The first time scale The changing characteristics of each node; The For the first The first time scale The environment label of each node; The Used to estimate the and stated The mutual information score between them.
5. The method according to claim 4, characterized in that, After the server calculates the spectral domain features and the soft mask matrix using a preset function set to obtain invariant and variable features at different time scales, the method further includes: The server estimates the contribution value of the invariant feature at various time scales based on a preset lightweight neural network model; The server uses the normalized contribution value as weight to perform a weighted summation of the invariant features at each time scale to obtain the fused invariant features.
6. The method according to claim 5, characterized in that, The step of the server constructing an optimization objective function based on the invariant features, the environmental labels, the preset auxiliary loss function, and the preset task loss function specifically includes: After the server performs a weighted summation of the preset auxiliary loss function, the mutual information maximization loss function, and the preset multi-scale intervention risk loss function based on preset hyperparameters, it sums the summation with the preset task loss function to obtain the optimization objective function. The preset multi-scale intervention risk loss function is: In the above function, the The loss function is a multi-scale intervention method. By simulating scenarios where the specific changing features cause fluctuations, the dynamic graph neural network model is supervised to make predictions based solely on the fused invariant features. ; The The prediction results of the invariant feature classifier in the dynamic graph neural network model based on the fused invariant features; The Represents element-wise product; The From the aforementioned change characteristics Specific change characteristics obtained from sampling; The The potential interference value for the specific change characteristic; The To normalize the potential interference values; The The preset multi-scale intervention risk loss function is used to measure the stability of the multi-scale intervention loss function.
7. The method according to claim 6, characterized in that, The step of the server updating the network parameters of the dynamic graph neural network model based on the optimization objective function specifically includes: The server uses gradient descent to iteratively update the parameters of the graph neural network backbone network, the multilayer perceptron of the preset scale, the preset lightweight neural network model, and the invariant feature classifier in the dynamic graph neural network model based on the optimization objective function; The server calculates the value of the current optimization objective function after each iteration; When the numerical value is within a preset minimum range for a preset number of consecutive iterations, the server determines that the optimization objective function has converged. The server stops updating the parameters and saves the latest parameters after the optimization objective function converges.
8. A server, characterized in that, The server includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the server to perform the method as described in any one of claims 1-7.
9. A computer program product containing instructions, characterized in that, When the computer program product is run on the server, the server performs the method as described in any one of claims 1-7.
10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the server, the server causes the server to perform the method as described in any one of claims 1-7.