A knowledge feedback-based dynamic graph learning optimization method based on parameter expansion
By constructing a parameter expansion and reverse knowledge feedback mechanism with a bidirectional knowledge loop structure, the problems of knowledge squeezing and causal association in dynamic graph learning are solved, achieving efficient and accurate dynamic graph learning and improving the model's performance on historical graphs.
Patent Information
- Application Number
- CN202611190288.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-06
- Publication Date
- 2026-11-10
AI Technical Summary
Existing dynamic graph learning methods suffer from high computational complexity and sensitivity to noise when dealing with large-scale sparse graphs. They are also prone to knowledge squeezing and catastrophic forgetting, neglect the potential of reverse knowledge feedback, and fail to effectively capture temporal causal relationships.
We adopt a knowledge feedback dynamic graph learning optimization method based on parameter expansion. By constructing a bidirectional knowledge loop structure through graph-aware parameter expansion strategy and reverse knowledge feedback mechanism, we solve the knowledge squeezing problem and model the temporal causal relationship between snapshots, thereby improving the accuracy of node representation.
It effectively alleviates knowledge crowding and catastrophic forgetting, improves the accuracy and efficiency of dynamic graph learning, and enhances the performance of the new model on historical graphs.
Smart Images

Figure CN123070177A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of dynamic graph learning, specifically involving a knowledge feedback-based dynamic graph learning optimization method based on parameter expansion. Background Technology
[0002] Dynamic graph learning (DGL), an important branch of graph representation learning, aims to model and analyze graph data with topological structures and node attributes that evolve over time. Because traditional static graph learning methods are difficult to adapt to dynamic scenarios, researchers have gradually shifted their focus to DGL methods, which can handle temporal dynamic characteristics, in recent years.
[0003] DGL methods are mainly divided into two categories: Continuous-Time Dynamic Graph (CTDG) learning methods and Discrete-Time Dynamic Graph (DTDG) learning methods. CTDG methods model the dynamic graph as a continuously evolving process over time, capturing real-time changes in nodes and boundaries through differential equations or time-point processes. The advantage of this type of method is its ability to characterize fine-grained dynamic behavior, making it suitable for high-frequency interaction scenarios. However, this method has high computational complexity, is sensitive to noise, and struggles to handle large-scale sparse graphs. In contrast, DTDG methods discretize the dynamic graph into a series of time snapshots, making them computationally efficient and easily scalable to large-scale sparse graphs. However, the limitation of DTDG methods lies in their reliance on snapshots to divide the time granularity, which may lead to the loss of short-term dynamic information.
[0004] Existing Dynamic Graphics (DGL) methods typically employ models with fixed parameter capacity to uniformly represent the temporal evolution of dynamic graphs. However, limited by this fixed parameter capacity, the incorporation of new graph knowledge can overwrite historical graph knowledge, leading to knowledge squeeze. Knowledge squeeze is essentially a manifestation of catastrophic forgetting in DGL. Specifically, due to the fixed model parameter capacity, new and historical graph knowledge conflict within the finite parameter space. Furthermore, various events in the dynamic graph, such as node addition and deletion, exhibit strong temporal causal relationships. While the strategy of directly freezing existing parameters and expanding with new parameters can address the knowledge squeeze problem, this strategy ignores the temporal logical relationships between these events, preventing the model from capturing these temporal causal connections.
[0005] Furthermore, existing DGL methods primarily focus on positive knowledge injection, i.e., leveraging historical graph knowledge to improve the model's performance on new graphs. However, these methods neglect the potential of reverse knowledge feedback. Reverse knowledge feedback refers to using knowledge from the new graph to reinforce the historical graph. Ignoring this will result in a unidirectional flow of knowledge, potentially leading to a decline in the new model's performance on historical graphs. Unlike continuous learning, which aims to avoid catastrophic forgetting (i.e., ensuring the new model doesn't lose learned historical graph knowledge to maintain its performance on historical graphs), reverse knowledge feedback not only requires the new model to retain acquired graph knowledge but also emphasizes using newly learned graph knowledge to improve its performance on historical graphs. Summary of the Invention
[0006] The purpose of this invention is to provide a knowledge feedback-based dynamic graph learning optimization method based on parameter expansion. This method not only solves the knowledge squeeze problem through graph-aware parameter expansion strategies, but also explicitly models the temporal causal relationships between snapshots, thereby improving the accuracy of dynamic node representation.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0008] A knowledge-feedback dynamic graph learning optimization method based on parameter expansion includes:
[0009] Obtain discrete-time dynamic graph samples, and divide each snapshot in the discrete-time dynamic graph samples into static subgraphs and dynamic subgraphs;
[0010] Freeze the current parameters of the graph neural network and initialize the new parameters of the graph neural network;
[0011] The static subgraph is encoded using the frozen current parameters, the dynamic subgraph is encoded using the new parameters, and the parameter expansion loss is calculated based on the encoded representation.
[0012] Causal labels are constructed based on adjacent snapshots, and causal score matrices are calculated based on the encoded representations of the target node and the encoded representations of the neighboring nodes. The causal loss is obtained by measuring the deviation between the causal labels and the causal score matrix.
[0013] Construct a meta-learning optimization objective, obtain the meta-loss, calculate the new snapshot gradient and meta-gradient based on the meta-learning optimization objective, and combine the new snapshot gradient and meta-gradient to update the parameters of the encoded dynamic subgraph in the graph neural network;
[0014] Based on the parameter expansion loss, causal loss, and meta-loss, the parameters of the encoded dynamic subgraph are updated again using the optimizer to obtain the updated parameters of the graph neural network.
[0015] The trained graph neural network is used to encode the discrete-time dynamic graph to be processed, and the snapshot of the future time is predicted and output.
[0016] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.
[0017] Preferably, for the snapshot at the current moment, the part of the snapshot at the current moment that has not changed compared with the snapshot at the previous moment is divided into a static subgraph, and the part derived from the process of evolving from the snapshot at the previous moment to the snapshot at the current moment is divided into a dynamic subgraph.
[0018] Preferably, the new parameters for initializing the graph neural network include:
[0019] Will Time-Static Subgraph parameters Expand to ,in express The parameters for positive knowledge injection at any given moment. express Momentary dynamic subgraph The parameters, during the training of the graph neural network Freezing means not participating in updates, but only being used for loss calculation during the forward propagation process.
[0020] Preferably, the parameter expansion loss is calculated as follows:
[0021]
[0022] In the formula, For parameter expansion loss, Indicates the current moment. Indicates the time index. Indicates using Static parameters frozen at all times right Dynamic subgraph of time Perform forward encoding calculation, Represents the cross-entropy loss function. Indicates using Dynamic parameters added at any time right Dynamic subgraph of time Perform forward encoding calculation, Indicates using Dynamic parameters added at any time right Forward encoding is performed on snapshots at time points. Indicates using Parameters of positive knowledge injection at any moment right Time-Static Subgraph Perform forward encoding calculation, Indicates using Parameters of positive knowledge injection at any moment right Momentary dynamic subgraph Perform forward encoding calculations.
[0023] Preferably, the step of constructing causal labels based on adjacent snapshots includes:
[0024] exist At any point in the snapshot Take node exist The neighborhood set at any given time ;
[0025] Pick Adjacency matrix at time step and Adjacency matrix at time step The difference is used as Edge change matrix at time step ,Pick Attribute matrix at time and Attribute matrix at time The difference is used as Attribute change matrix at time step ;
[0026] If the edge transformation matrix Middle node with neighboring nodes The connection state change between them is not zero, or the attribute change matrix Middle Neighbor Nodes If the attribute change is not zero, then... Neighbor nodes at time right Node of time The causal label is set to 1; otherwise, it is set to 0, resulting in... Each node of time The causal label vectors of all nodes in the snapshot are stacked as follows: Causal label matrix at time points , For nodes The number of.
[0027] Preferably, the causal loss is calculated as follows:
[0028]
[0029] In the formula, For causal loss, For nodes The number of Represents the causal label matrix Neighbor nodes at time right Node of time causal label, Represents the target node neighboring nodes exist Causal score at any moment For the Sigmoid function, Represents a node exist Encoded representation of time, Represents a node exist Encoded representation of time, This represents the matrix transpose operation.
[0030] Preferably, the construction of the meta-learning optimization objective and the resulting meta-loss include:
[0031] The optimization objectives for constructing meta-learning are as follows:
[0032]
[0033] In the formula, express Parameters of the graph neural network at time step Indicate using dummy variables right Time-Static Subgraph Perform forward encoding calculation, Indicate the use of dummy variables right Dynamic subgraph of time Perform forward encoding calculation, Indicates the meta-loss weights. Let represent the meta-loss, and let the meta-loss be expressed as follows:
[0034]
[0035] In the formula, Indicates the current moment. This indicates the use of dummy variables. Graph neural networks in the original The loss value calculated on the snapshot at time point. Indicates the use of history The graph neural network with parameters at time step in the original The loss value calculated on the snapshot at time point. Representing history Parameters for a snapshot at a given moment.
[0036] Preferably, the step of calculating the new snapshot gradient and the meta-gradient based on the meta-learning optimization objective, and combining the new snapshot gradient and the meta-gradient to update the parameters of the encoded dynamic subgraph in the graph neural network includes:
[0037] Initialize dummy variables for Parameters of positive knowledge injection at any moment ;
[0038] According to the meta-learning optimization objective Partially calculate the gradient of the new snapshot, based on the meta-learning optimization objective. Partially compute the meta-gradient;
[0039] The sum of the new snapshot gradient and the original gradient is taken as the total gradient;
[0040] Get parameters and The difference in total gradients, multiplied by a factor of two, is used as the updated value of the graph neural network. Parameters of a snapshot at any given time. The learning rate is the rate of reverse knowledge feedback.
[0041] To address the problems of existing technologies, this invention provides a dynamic graph learning method with backward knowledge feedback via parameter expansion (DBKF-PE), which constructs a bidirectional knowledge loop structure. First, to solve the knowledge compression problem, a novel graph-aware parameter expansion strategy is proposed: by injecting forward knowledge, a new set of parameters is constructed to learn new graph knowledge. Simultaneously, the parameters corresponding to historical graphs are frozen to ensure that historical graph knowledge is not overwritten. Building upon this, to overcome the limitation of direct parameter expansion in capturing temporal causal relationships, a causal auxiliary task is constructed, requiring the model to consider the causal relationship between the new graph and the previous time-series graph while learning new graph knowledge. Subsequently, based on the parameters generated during the forward knowledge injection process, a novel backward knowledge feedback mechanism (BKFM) is proposed. This mechanism constructs a meta-learning optimization objective by evaluating the performance difference between new and old parameters on the historical graph, and determines the optimization direction of the new parameters by calculating the meta-gradient, transforming DGL into a meta-learning optimization objective to improve the performance of the new model on the historical graph. Attached Figure Description
[0042] Figure 1 This is a structural block diagram of the knowledge feedback dynamic graph learning optimization method based on parameter expansion according to the present invention;
[0043] Figure 2 This is a flowchart of the knowledge feedback dynamic graph learning optimization method based on parameter expansion according to the present invention;
[0044] Figure 3 This is a schematic diagram of the graph sensing parameter extension strategy of the present invention;
[0045] Figure 4 This is a diagram showing the experimental results of reverse knowledge feedback on the node classification task of this invention;
[0046] Figure 5 This is a graph showing the experimental results of reverse knowledge feedback on the community detection task of this invention;
[0047] Figure 6 The experimental diagram shows the impact of the extended units of this invention on the experimental results.
[0048] Figure 7 The figure shows the effect of the original loss parameter on the experimental results. Detailed Implementation
[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.
[0051] Dynamic graphs are a core technology for modeling various temporal interaction systems in the real world. However, existing dynamic graph learning methods have significant shortcomings in representation capabilities, mainly due to the following two reasons: (1) The fixed parameter capacity of existing methods makes them prone to knowledge squeeze during continuous learning of new graph knowledge; (2) In addition, these methods follow the basic paradigm of learning new graph knowledge based on historical graph knowledge, but ignore the potential of reverse knowledge feedback in strengthening the learning of historical graph knowledge. To address the above problems, this invention proposes a knowledge feedback-based dynamic graph learning optimization method based on parameter expansion. This method achieves efficient and accurate representation of dynamic graphs by constructing a bidirectional knowledge loop structure. This loop structure consists of two core modules: forward knowledge injection and reverse knowledge feedback. Specifically, forward knowledge injection adopts a new graph-aware parameter expansion strategy, which learns new knowledge by adding new parameters while freezing the original parameters, fundamentally avoiding conflicts between new graph and historical graph knowledge in the parameter space. Meanwhile, reverse knowledge feedback is modeled as a meta-learning optimization objective. Through the designed reverse knowledge feedback mechanism, this invention can not only effectively alleviate the catastrophic forgetting problem, but also further improve the performance of the new model on historical graph tasks. Finally, large-scale experiments on six datasets verified that the DBKF-PE model of this invention significantly outperforms the current state-of-the-art baseline models.
[0052] This invention focuses on discrete-time dynamic graph learning. A dynamic graph can be represented as... ,in express A snapshot of the moment (image). Indicates the total number of snapshots. for The set of all nodes in a snapshot at any given moment. express The adjacency matrix at time ( (Represents the number of nodes, and the vector space). express Attribute matrix at time ( (Indicates the number of attributes).
[0053] The training objective of dynamic graph learning is to learn functions. : Utilize the past Information from the first snapshot is used to predict the... A snapshot (such as node classification and link prediction tasks), in which Indicates the dimension of the node.
[0054] The overall structure of the DBKF-PE model proposed in this invention is as follows: Figure 1 As shown. Specifically, the input to the DBKF-PE model is... The DTDG consists of several snapshots. Each snapshot is then divided into static and dynamic subgraphs. Next, through forward knowledge injection, the Graph Neural Network (GNN) learns the topological structure and attribute information of the static subgraph. The model learns the topological structure and attribute information of the dynamic subgraph by freezing the original parameters of the GNN and expanding them with new parameters. Furthermore, this invention proposes a causal guidance mechanism to capture temporal causal relationships in the dynamic graph. Simultaneously, based on reverse knowledge feedback, the model's learning of new snapshots not only retains knowledge of the historical graph but also improves the performance of the new model on historical graphs.
[0055] like Figure 2 As shown, this embodiment provides a knowledge feedback-based dynamic graph learning optimization method based on parameter expansion, which specifically includes the following steps:
[0056] Step 1: Obtain discrete-time dynamic graph samples and divide each snapshot in the discrete-time dynamic graph samples into static subgraphs and dynamic subgraphs.
[0057] In a dynamic graph, subgraph May be affected by subgraph The impact of this caused changes in its data structure (such as the addition of edges), from arrive Some subgraphs The changes remain unchanged. Therefore, when information is passed between snapshots, the model does not need to repeatedly learn information from unchanged subgraphs, thus improving parameter utilization efficiency. Changing subgraphs reflect the dynamic nature of snapshots. This embodiment will... The snapshot at time is divided into static subgraphs. (Corresponding invariant subgraph) and dynamic subgraph (Corresponding to the changing subgraph), as shown in formula (1) and Figure 1 As shown:
[0058] (1)
[0059] in, yes Evolved to The resulting subgraphs showing changes during the process. Note that... When performing graph partitioning, and They are exactly the same; both represent and Subgraphs that remain unchanged. But for When performing graph partitioning, i.e. , Moment and Moment There may be differences because... to Invariant subgraphs at time t, and to The invariant subgraphs at different times may not be consistent. Subsequent parameter usage will be based on the final partitioning result from the partitioning formula for analysis and calculation, for example... express Moment , rather than Moment Furthermore, to ensure that the contextual information of dynamic subgraphs is not lost during the graph partitioning phase, dynamic subgraphs not only include the nodes and edges that have changed, but also the nodes and edges of the second-order neighborhoods of the changed nodes. This operation also helps to preserve potential associations between nodes in the static subgraph and nodes in the dynamic subgraph, such as causal associations and temporal evolutionary associations.
[0060] Step 2: Freeze the current parameters of the graph neural network and initialize the new parameters of the graph neural network; use the frozen current parameters to encode the static subgraph, use the new parameters to encode the dynamic subgraph, and calculate the parameter expansion loss based on the encoded representation.
[0061] To address the knowledge squeeze problem, this embodiment proposes a novel graph-aware parameter expansion strategy, such as... Figure 3 As shown: A new set of parameters is trained to specifically learn the dynamic subgraph features of new snapshots, while the original snapshot parameters are frozen to prevent overwriting knowledge from historical graphs. Specifically, when a new graph pattern appears, the model adds a complete set of network parameters. The new graph pattern is learned, while the original parameters are frozen and stored as historical graph patterns. Furthermore, the new parameters need to be retrained on the new graph pattern to accurately represent it. This strategy ensures that the model's parameter capacity can grow over time (i.e., parameter expansion), thus avoiding knowledge conflicts between the new and historical graphs in the parameter space and effectively solving the knowledge squeeze problem. The result can be obtained through formula (2). static subgraph parameter.
[0062] (2)
[0063] in Represents the cross-entropy loss function. / / GNN learning / / The loss value at that time, express time The parameters.
[0064] Subsequently, this embodiment will use parameters Expand to (in express time The parameters, express The parameters for positive knowledge injection at any given moment. express time (parameters), thereby enhancing the DBKF-PE model's ability to learn new snapshots. The ability. Then freeze. To prevent overwriting historical knowledge. It is important to note that... It does not participate in updates and is only used for loss calculation during the forward propagation process. Based on this, the loss of the graph-aware parameter expansion strategy can be calculated using formula (3). .
[0065] (3)
[0066] This strategy minimizes the loss. optimization This enhances the model's ability to learn from both the original and new snapshots. The three sets of forward mapping symbols in formulas (2) and (3) have the following interpretations: This indicates that the static parameters are frozen at time t. Encoding the static subgraph at time t The reasoning output that solidifies static historical knowledge; Indicates the use of frozen static parameters right Momentary dynamic subgraph Perform forward computation to quantify the adaptation error of the old parameters to the new structure; Indicates use Add trainable dynamic parameters at any time Encode the current dynamic subgraph, specifically learning the structural change features added in the snapshot.
[0067] The specific execution steps of the graph-aware parameter expansion strategy are as follows:
[0068] (1) Split the current snapshot Static subgraph With dynamic subgraph ;
[0069] (2) Historical static parameters It does not participate in gradient updates and initializes training separately. Fit dynamic subgraph features;
[0070] (3) By using formula (2) to constrain static branch loss, the conflict between new and old knowledge is weakened;
[0071] (4) Combine the two sets of parameters to obtain the extended parameters. This process is then repeated cyclically with subsequent time-series snapshots.
[0072] Step 3: Construct causal labels based on adjacent snapshots, and calculate the causal score matrix based on the encoded representation of the target node and the encoded representation of the neighboring nodes. Measure the deviation between the causal labels and the causal score matrix to obtain the causal loss.
[0073] To capture the temporal causal relationships between snapshots, this embodiment proposes a novel causal guidance mechanism to consider state changes between nodes (addition of attributes or edges). In dynamic graphs, a node's state change is typically triggered by changes in its neighbors in previous snapshots. Specifically, in At any given time, for any node Define its in The neighborhood set at any given time Define the edge transformation matrix. and attribute change matrix This is used to capture the temporal causal relationship between snapshots, as shown in formula (4):
[0074] (4)
[0075] To identify nodes exist The reason for the change in state at any given time is attributed in this embodiment to Neighboring nodes that always satisfy any of the following conditions Changes:
[0076] Edge causality: Node and When the connection state between them changes, it is denoted as .
[0077] Attribute causality: Node When its own attributes change, it is denoted as (in (representing the zero vector).
[0078] Based on this, it can be... Each node of time Constructing causal label vectors As shown in formula (5):
[0079] (5)
[0080] in show Time-neighbor nodes ( Changes in the nodes may be related to the nodes. exist The potential causes of state changes at any given time. This embodiment stacks the causal label vectors of all nodes into... Causal label matrix at time points Based on this, a causal auxiliary task is designed: given the node representation of a new snapshot, predict its causal neighbor nodes from the previous snapshot. Specifically, for the target node... Graph neural networks need to predict their causal neighbors. exist Causal score at time This embodiment uses nodes. exist Representation of time With nodes exist Representation of time Calculate the causal score matrix As shown in equation (6):
[0081] (6)
[0082] in This represents the Sigmoid function. Binary cross-entropy loss is then used to measure its performance. and The deviation between them is shown in formula (7):
[0083] (7)
[0084] in This indicates the causal loss of the causal auxiliary task. This represents the matrix transpose operation. By minimizing this loss, DBKF-PE is driven to learn the state changes between nodes, thereby capturing the temporal causal relationships between snapshots.
[0085] Step 4: Construct a meta-learning optimization objective, obtain the meta-loss, calculate the new snapshot gradient and meta-gradient based on the meta-learning optimization objective, and combine the new snapshot gradient and meta-gradient to update the parameters of the encoded dynamic subgraph in the graph neural network.
[0086] The core of the BKFM method proposed in this embodiment lies in constructing a meta-learning optimization objective. This objective specifies that the direction of parameter updates should improve the performance of the new model across all previous snapshots. By optimizing this objective, we obtain the final updated parameters. The meta-learning optimization objective is shown in equation (8):
[0087] (8)
[0088] in Indicates the original loss. Indicates the meta-loss weights. This represents a dummy variable. This embodiment aims to use... The new model can outperform the old parameter model on the original snapshot. Therefore, Equation (9) is used to maximize the performance improvement.
[0089] (9)
[0090] in This represents the loss value calculated by the model on the original snapshot. express The original parameters on This represents the marginal hyperparameter. The goal of this embodiment is to make... The value tends towards negative values to improve performance. The optimization steps are as follows:
[0091] initialization: Initialize to .
[0092] Gradient calculation: The BKFM gradient consists of two parts, namely the gradient of the new snapshot ( ) and meta-gradient ( ).
[0093] Parameter update: The model performs the final update using combined gradients, as shown in Equation (10):
[0094] (10)
[0095] in This represents the learning rate of reverse knowledge feedback.
[0096] BKFM not only alleviates catastrophic forgetting but also continuously improves the model's performance on existing snapshots, thereby enhancing the model's representational capabilities.
[0097] Step 5: Based on the parameter expansion loss, causal loss, and meta-loss, use the optimizer to update the parameters of the encoded dynamic subgraph again, and obtain the updated parameters of the graph neural network.
[0098] The overall loss of DBKF-PE is shown in equation (11):
[0099] (11)
[0100] DBKF-PE minimizes Achieve effective dynamic graph learning and update model parameters using the Adam optimizer.
[0101] Furthermore, this embodiment equates minimizing the overall loss with maximizing the joint information objective to demonstrate the solid theoretical foundation of the proposed causal guidance mechanism, as shown in Equation (12):
[0102] (12)
[0103] in Indicates mutual information, express The node representation of time, express A matrix of real labels for each moment.
[0104] Subsequently, this embodiment uses the cross-entropy loss of node classification as an example to evaluate the loss of the main task ( The goal of minimizing the cross-entropy loss is to make the distribution predicted by the model more accurate. To approximate the true label distribution. In information theory, this objective is equivalent to maximizing... and Mutual information between Its definition is shown in equation (13):
[0105] (13)
[0106] in Represents the mathematical expectation. Represents a probability distribution. / express / Subvectors.
[0107] Next, this embodiment evaluates causal loss. It is equivalent to maximizing mutual information. The causal auxiliary task is modeled as a binary classification problem, therefore binary cross-entropy (BCE) is used as the loss function. (Single node pair) Causal loss As shown in equation (14):
[0108] (14)
[0109] in This represents the predicted probability (causal score). / express / time / The representation of . Based on this, it can be expressed according to equation (15). The causal loss of each node pair is averaged:
[0110] (15)
[0111] As can be seen from equation (15), under the assumption of independent and identically distributed loss, the BCE loss is essentially a negative log-likelihood function, and the mutual information of causal loss is shown in equation (16):
[0112] (16)
[0113] The second item express The entropy. Because of Regardless of model parameters, when optimizing model parameters It is a constant. Therefore, maximizing Equivalent to maximizing As shown in equation (17):
[0114] (17)
[0115] Given the monotonicity of the logarithmic function, maximizing the expected log-likelihood function is equivalent to minimizing its negative value, as shown in equation (18):
[0116] (18)
[0117] At this point, the right side of the equation represents the causal loss. The expected form of the result. Therefore, it can be proven that minimizing the overall loss is equivalent to maximizing the joint information objective.
[0118] Step 6: Encode the discrete-time dynamic graph to be processed using the trained graph neural network, and predict snapshots of future times. Based on the predicted future snapshots, downstream graph analysis tasks such as node classification and community detection can be further performed.
[0119] To visually demonstrate the advantages of this application, the following experimental analysis is conducted:
[0120] This experiment used six benchmark datasets to evaluate the performance of the DBKF-PE model and baseline methods of this invention in node classification and community detection.
[0121] (1) Experimental environment.
[0122] (1.1) Benchmark dataset.
[0123] This experiment evaluated the performance of the DBKF-PE model and its baseline methods using six benchmark datasets. The benchmark datasets include High School, Cell Phone, Cora, Citeseer, Photo, and DBLP. Relevant information about the benchmark datasets is shown in Table 1.
[0124] Table 1. Benchmark Dataset Information
[0125]
[0126] (1.2) Baseline method.
[0127] This experiment evaluates the performance of DBKF-PE by comparing it with current mainstream dynamic graph learning and continuous learning methods.
[0128] DGL methods include: EvolveGCN (Evolving Graph Convolutional Networks), DyGNN (Dynamic Graph Neural Network), DNE (Dynamic Network Embedding), DGCN (Dynamic Graph Convolutional Network), DG-Mamba (Dynamic Graph Mamba, a robust and efficient dynamic graph structure learning based on a selective state-space model), MSGCL (Multi-Snapshot Graph Contrastive Learning), CT-CAVAE (Continuous-Time Community-Aware Variational Autoencoder), and SFDyG (Snapshot Fusion Dynamic Graph Neural Network, a scalable discrete-time dynamic graph neural network based on input snapshot fusion).
[0129] Continuous learning methods include: ContinualGNN (Continuous Graph Neural Network), DiCGRL (Disentangle-based Continuous Graph Representation Learning), TWP (Temporal Weight Projection), PI-GNN (Parameter Isolation Graph Neural Network), PiECL (Physics-Informed Explainable Continual Learning), and GDDSG (Graph-Driven Dynamic Similarity Grouping).
[0130] (1.3) Implementation details.
[0131] The code environment for this experiment is Python 3.8, and DBKF-PE is trained using a server equipped with 512GB of memory, an x86_64 architecture processor, and an Ubuntu 20.04.06 LTS operating system. The graphics processor used in this experiment is an NVIDIA A800, and the graph neural network model employed is a two-layer GraphSAGE (SAmple and aggreGatE) model. and The values were set to 0.05 and 0.001 respectively. Furthermore, for node classification, this experiment strictly followed the experimental protocol of PI-GNN (Parameter Separated Graph Neural Network). For community detection, this experiment used the Louvain algorithm as the downstream model and followed the experimental protocol established by the Louvain algorithm.
[0132] (1.4) Evaluation indicators.
[0133] For node classification, this experiment uses Performance Mean (PM) and Forgetting Measure (FM) to evaluate the model's ability to mitigate catastrophic forgetting. A higher PM value indicates better model performance, while a higher FM value indicates a lower degree of forgetting.
[0134] In community detection, this experiment used Normalized Mutual Information (NMI) and Adjusted Rand Index (ARI) to evaluate the model's detection performance. Higher NMI and ARI values indicate better model performance.
[0135] (2) Experimental analysis of downstream tasks.
[0136] (2.1) Experimental analysis of node classification.
[0137] Table 2 (including a continuation of Table 2) shows the performance comparison between DBKF-PE and baseline methods in node classification tasks, with PM and FM (%) as evaluation metrics and corresponding standard deviations. As shown in Table 2, DBKF-PE outperforms the baseline model overall. Specifically, DBKF-PE performs best on the High School, Cell Phone, Cora, and DBLP datasets. This is mainly because DBKF-PE employs the graph-aware parameter extension strategy proposed in this experiment, enhancing the model's ability to learn various graph patterns by learning stable and dynamic subgraphs between different snapshots. Furthermore, DBKF-PE uses the causal guidance mechanism proposed in this experiment to capture the temporal evolution information of dynamic graphs, enabling fine-grained learning of temporal causal relationships between nodes and attributes. Notably, DBKF-PE's PM value on Photo is lower than that of SFDyG. This is mainly because SFDyG integrates multiple snapshots into a single temporal graph, thereby reducing the sparsity of the dynamic graph. Since I-GNN utilizes continuous learning to mitigate catastrophic forgetting, it reduces the probability of the model forgetting historical information; DBKF-PE's FM value on Citeseer is also lower than that of PI-GNN. The main reason is that PI-GNN uses a continuous learning mechanism to alleviate the problem of catastrophic forgetting, which helps to reduce the probability of the model forgetting historical information.
[0138] Table 2 Performance comparison of each model on the node classification task
[0139]
[0140] Continued from Table 2
[0141]
[0142] (2.2) Experimental analysis of community testing
[0143] Table 3 (including a continuation of Table 3) shows the performance comparison between DBKF-PE and baseline methods in community detection. As shown in Table 3, DBKF-PE outperforms the baseline model overall. Specifically, DBKF-PE performs best on the High School, Cell Phone, Photo, and DBLP datasets. Its principle is similar to node classification, both employing graph-aware parameter expansion strategies and causal guidance mechanisms to improve the accuracy of node representations. This allows similar nodes to cluster more effectively, forming stable communities. Notably, DBKF-PE's ARI value on Cora is lower than that of DG-Mamba. This is mainly because DG-Mamba leverages Mamba's temporal capture capabilities to enhance the consistency of nodes across all snapshots, thereby improving the accuracy of community detection. Furthermore, DBKF-PE's ARI value on Citeseer is lower than that of MSGCL. This is mainly because MSGCL utilizes graph contrastive learning to strengthen the dependencies between nodes and communities, thereby reducing errors in calculating node membership.
[0144] Table 3 Performance comparison of each model on the community detection task
[0145]
[0146] Continued from Table 3
[0147]
[0148] (3) Reverse knowledge feedback experiment.
[0149] To evaluate the model's ability to provide reverse knowledge feedback, this experiment uses a model trained on all snapshots to learn each original snapshot and records the experimental results, such as... Figure 4 and Figure 5 As shown in the figure, ACC represents accuracy.
[0150] from Figure 4 and Figure 5 As can be seen, DBKF-PE outperforms the baseline model in overall performance. The main reason is that DBKF-PE employs a reverse knowledge feedback mechanism, using knowledge from the new graph to enhance historical graph knowledge. This allows the new model to recognize graph patterns that the old model struggled to identify, thereby improving the new model's learning performance on historical graphs.
[0151] It is worth noting that, unlike continuous learning models (such as PI-GNN and GDDSG) that focus solely on mitigating catastrophic forgetting, DBKF-PE not only avoids the performance degradation of dynamic graph learning on historical snapshots but also achieves performance improvements. As shown in Table 2, GDDSG's FM value on Cora is greater than 0, indicating that no forgetting occurred. However, Figure 4 and Figure 5The results show that GDDSG outperforms DBKF-PE in both node classification and community detection tasks. This indicates that BKFM can both preserve knowledge and enhance it, thus improving the new model's ability to represent nodes in historical graphs after learning new graph knowledge.
[0152] (4) Ablation experiment.
[0153] To evaluate the contribution of each component in the DBKF-PE model to the overall performance, ablation experiments were conducted. The experimental results are shown in Tables 4 (including a continuation of Table 4) and 5. The evaluation metrics are PM and FM (%), with corresponding standard deviations. DBKF-PE w / o PE refers to the model obtained after removing the graph-aware parameter extension strategy from DBKF-PE. DBKF-PE w / o CG refers to the model obtained after removing the causal guidance mechanism from DBKF-PE. DBKF-PE w / o BKFM refers to the model obtained after removing BKFM from DBKF-PE.
[0154] Table 4 Ablation experiments on node classification tasks
[0155]
[0156] Continued from Table 4
[0157]
[0158] Table 5 Ablation experiments in community testing tasks
[0159]
[0160] As shown in Tables 4 and 5, compared to DBKF-PE, the performance degradation of DBKF-PE w / o PE is the most significant, followed by DBKF-PE w / o CG, and then DBKF-PE w / o BKFM. This indicates that the graph-aware parameter extension strategy has the most significant impact on DBKF-PE, and also proves that each component is an indispensable part of DBKF-PE.
[0161] (5) Parameter experiment.
[0162] This experiment takes community detection as an example, focusing on the expansion units (number of parameter expansions) and the meta-loss parameters. Parametric experiments were conducted on the HighSchool and Photo datasets. The experimental results are as follows: Figure 6 and Figure 7 As shown, the time unit is seconds.
[0163] like Figure 6As shown, different numbers of expansion units have different impacts on the performance of DBKF-PE. Increasing the number of expansion units does not improve the performance of DBKF-PE. On the contrary, it continuously increases the training time cost. Therefore, this experiment sets the number of expansion units to 12 to balance model performance and time cost.
[0164] like Figure 7 As shown, different The value has varying degrees of impact on the performance of DBKF-PE. When When set to 0.1, DBKF-PE achieves optimal NMI and ARI values on both the High School and Photo datasets. Therefore, the determined... The value is crucial for optimizing DBKF-PE.
[0165] (6) Model compression experiment.
[0166] The number of parameters of DBKF-PE is affected by The quantity has an impact. If A sharp increase in the number of parameters may lead to a sharp increase in the number of parameters in DBKF-PE. Therefore, this invention uses model distillation to address the parameter surge problem. Experimental results are listed in Tables 6 (including a continuation of Table 6) and 7, where the evaluation indicators are PM and FM (unit: %), along with their corresponding standard deviations.
[0167] Table 6 Experimental results of model compression on node classification tasks
[0168]
[0169] Continued from Table 6
[0170]
[0171] Table 7 Experimental results of model compression on the community detection task
[0172]
[0173] As shown in Tables 6 and 7, the performance of DBKF-PE did not decrease significantly after model distillation and remained at a relatively high level, indicating that DBKF-PE is still very suitable for processing dynamic graphs with a large number of snapshots.
[0174] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a knowledge feedback dynamic graph learning optimization method based on parameter expansion.
[0175] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.
[0176] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.
[0177] The above embodiments merely illustrate several implementation methods of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.
Claims
1. A knowledge feedback-based dynamic graph learning optimization method based on parameter expansion, characterized in that, include: Obtain discrete-time dynamic graph samples, and divide each snapshot in the discrete-time dynamic graph samples into static subgraphs and dynamic subgraphs; Freeze the current parameters of the graph neural network and initialize the new parameters of the graph neural network; The static subgraph is encoded using the frozen current parameters, the dynamic subgraph is encoded using the new parameters, and the parameter expansion loss is calculated based on the encoded representation. Causal labels are constructed based on adjacent snapshots, and causal score matrices are calculated based on the encoded representations of the target node and the encoded representations of the neighboring nodes. The causal loss is obtained by measuring the deviation between the causal labels and the causal score matrix. Construct a meta-learning optimization objective, obtain the meta-loss, calculate the new snapshot gradient and meta-gradient based on the meta-learning optimization objective, and combine the new snapshot gradient and meta-gradient to update the parameters of the encoded dynamic subgraph in the graph neural network; Based on the parameter expansion loss, causal loss, and meta-loss, the parameters of the encoded dynamic subgraph are updated again using the optimizer to obtain the updated parameters of the graph neural network. The trained graph neural network is used to encode the discrete-time dynamic graph to be processed, and the snapshot of the future time is predicted and output.
2. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, For the snapshot at the current moment, the part of the snapshot at the current moment that has not changed compared to the snapshot at the previous moment is divided into a static subgraph, and the part that is derived from the process of evolving from the snapshot at the previous moment to the snapshot at the current moment is divided into a dynamic subgraph.
3. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, The new parameters for initializing the graph neural network include: Will Time-Static Subgraph parameters Expand to ,in express The parameters for positive knowledge injection at any given moment. express Momentary dynamic subgraph The parameters, during the training of the graph neural network Freezing means not participating in updates, but only being used for loss calculation during the forward propagation process.
4. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, The parameter expansion loss is calculated as follows: In the formula, For parameter expansion loss, Indicates the current moment. Indicates the time index. Indicates using Static parameters frozen at all times right Dynamic subgraph of time Perform forward encoding calculation, Represents the cross-entropy loss function. Indicates using Dynamic parameters added at any time right Dynamic subgraph of time Perform forward encoding calculation, Indicates using Dynamic parameters added at any time right Forward encoding is performed on snapshots at time points. Indicates using Parameters of positive knowledge injection at any moment right Time-Static Subgraph Perform forward encoding calculation, Indicates using Parameters of positive knowledge injection at any moment right Momentary dynamic subgraph Perform forward encoding calculations.
5. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, The construction of causal labels based on adjacent snapshots includes: exist At any point in the snapshot Take node exist The neighborhood set at any given time ; Pick Adjacency matrix at time step and Adjacency matrix at time step The difference is used as Edge change matrix at time step ,Pick Attribute matrix at time and Attribute matrix at time The difference is used as Attribute change matrix at time step ; If the edge transformation matrix Middle node with neighboring nodes The connection state change between them is not zero, or the attribute change matrix Middle Neighbor Nodes If the attribute change is not zero, then... Neighbor nodes at time right Node of time The causal label is set to 1; otherwise, it is set to 0, resulting in... Each node of time The causal label vectors of all nodes in the snapshot are stacked as follows: Causal label matrix at time points , For nodes The number of.
6. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, The causal loss is calculated as follows: In the formula, For causal loss, For nodes The number of Represents the causal label matrix Neighbor nodes at time right Node of time causal label, Represents the target node neighboring nodes exist Causal score at any moment For the Sigmoid function, Represents a node exist Encoded representation of time, Represents a node exist Encoded representation of time, This represents the matrix transpose operation.
7. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 1, characterized in that, The construction of the meta-learning optimization objective, resulting in the meta-loss, includes: The optimization objectives for constructing meta-learning are as follows: In the formula, express Parameters of the graph neural network at time step Indicate the use of dummy variables right Time-Static Subgraph Perform forward encoding calculation, Indicate the use of dummy variables right Dynamic subgraph of time Perform forward encoding calculation, Indicates the meta-loss weights. Let represent the meta-loss, and let the meta-loss be expressed as follows: In the formula, Indicates the current moment. This indicates the use of dummy variables. Graph neural networks in the original The loss value calculated on the snapshot at time point. Indicates the use of history The graph neural network with parameters at time step in the original The loss value calculated on the snapshot at time point. Representing history Parameters for a snapshot at a given moment.
8. The knowledge feedback-based dynamic graph learning optimization method based on parameter expansion according to claim 7, characterized in that, The step of calculating the new snapshot gradient and meta gradient based on the meta-learning optimization objective, and combining the new snapshot gradient and meta gradient to update the parameters of the encoded dynamic subgraph in the graph neural network, includes: Initialize dummy variables for Parameters of positive knowledge injection at any moment ; According to the meta-learning optimization objective Partially calculate the gradient of the new snapshot, based on the meta-learning optimization objective. Partially compute the meta-gradient; The sum of the new snapshot gradient and the original gradient is taken as the total gradient; Get parameters and The difference in total gradients, multiplied by a factor of two, is used as the updated value of the graph neural network. Parameters of a snapshot at any given time. The learning rate is the rate of reverse knowledge feedback.
