A Dynamic Graph Representation Learning Method Based on Self-Supervised Learning

Abstract

The present invention relates to a dynamic graph representation learning method based on self-supervised learning, including: sampling the dynamic graph to obtain a spatio-temporal subgraph, a non-temporal subgraph, and a non-spatial subgraph respectively; using a spatio-temporal weight encoder to obtain node-level and graph-level representations of the three subgraphs respectively; designing spatio-temporal contrastive learning and defining a total loss function from the perspectives of time and space respectively; training the spatio-temporal weight encoder through the total loss function, and finally sending all the nodes in the dynamic graph into the trained spatio-temporal weight encoder to obtain node representations of the dynamic graph, and applying them to tasks such as classification, recommendation, and link prediction in application scenarios of dynamic graph model classes. The present invention integrates the time and space information of the dynamic graph, enhancing the performance of the dynamic graph representation method; having good interpretability, verifiability, representation performance, and transferability. And the present invention can be applied to an e-commerce network in reality to predict the products that users may purchase.

Description

Technical Field

[0001] The present invention belongs to the field of graph representation learning, and particularly relates to a dynamic graph representation learning method based on self-supervised learning. Background Art

[0002] With the explosive growth of graph-structured related data, more and more scholars have joined the research in the field of graph representation learning. However, most existing supervised or semi-supervised graph representation learning methods rely heavily on data labels. On the one hand, collecting data labels will cost a lot of manpower and material resources. On the other hand, the generalization ability and robustness of the model trained using data labels are poor. This brings great trouble to the current research. In recent years, there have been many studies on self-supervised graph representation learning to solve the above problems.

[0003] The idea of self-supervised graph representation learning is to alleviate the dependence of the model on data labels by designing a specific task, and further improve the generalization ability and robustness of the model. For example: The DGI model, as the first self-supervised graph representation learning method, generates node representations by maximizing the mutual information between node representations and global graph representations. MVGRL is a self-supervised graph representation model from a multi-view perspective, and the goal of this model is to maximize the node representations in the original graph and the node representations in the diffused graph. The Sub-Con model maximizes the mutual information between node representations and subgraph representations to learn node representations. Different from previous works is the Grace model, which mainly focuses on node-level consistency. It generates new perspectives through a perturbation function and learns node representations by maximizing the representations of the same node under different views.

[0004] However, existing self-supervised graph representation learning models ignore the evolution process on the graph. Specifically, existing self-supervised graph representation learning models discard the time information on the graph to simplify the operation, which will cause the model to fail to capture the evolution process on the graph, further resulting in calculation errors. Through research, we found that there are very few related studies on learning dynamic graph representations based on self-supervised methods.

[0005] The method of the present invention incorporates the idea of self-supervised learning, that is, to study a dynamic graph representation learning method based on self-supervised learning. Its overall idea is to define a contrastive learning task that takes into account both time and space, and enables the model to complete this task to capture the time and space features of the dynamic graph. Specifically, the method of the present invention first defines three subgraphs, named spatiotemporal subgraph, non-temporal subgraph, and non-spatial subgraph respectively. In addition, the present invention includes a spatiotemporal weight encoder, which can aggregate the time and space information of the subgraphs. Finally, we apply it to the e-commerce scenario to predict the products that users may purchase. Summary of the Invention

[0006] The present invention proposes a dynamic graph representation learning method based on self-supervised learning. First, the method takes any node in the graph as the central node and performs spatio-temporal random walks and random walks respectively. The sampled subgraphs are defined as spatio-temporal subgraphs and non-temporal subgraphs respectively. In addition, spatio-temporal random walks are performed with any other node as the central node, and the sampled subgraph is called a non-spatial subgraph. Secondly, a spatio-temporal weight encoder is used to capture the time and space information in the graph to obtain the node representations and subgraph representations of the corresponding nodes. Finally, a contrastive learning method is used to maximize the mutual information between the node representations and subgraph representations to learn the node representations. The specific technical solutions are as follows:

[0007] A dynamic graph representation learning method based on self-supervised learning includes the following steps:

[0008] S1. Take any node in the graph as the central node and sample spatio-temporal subgraphs, non-temporal subgraphs and non-spatial subgraphs respectively;

[0009] S2. Use a spatio-temporal weight encoder to capture the time and space information in the graph to obtain the node representations of the corresponding nodes and the graph-level representations of the subgraphs;

[0010] S3. Learn the node representations by maximizing the mutual information between the node representations and the graph-level representations of the subgraphs through a contrastive learning method, and define the total loss function from the perspectives of time and space respectively;

[0011] S4. Complete the training of the spatio-temporal weight encoder through the total loss function, and finally send all the nodes in the dynamic graph into the trained spatio-temporal weight encoder to obtain the node representations of the dynamic graph;

[0012] On the basis of the above technical solutions, the specific steps of step S1 are as follows: Given a dynamic graph , where represents the set of edges, is a series of sets of edges. There is an edge in the graph indicating that node and node generate an edge at moment, represents a function that can map each edge to a timestamp. The present invention hopes to collect a spatio-temporal subgraph centered on node in the dynamic graph .

[0013] For the sampling of the spatio-temporal subgraph , it is mainly divided into the following two steps: initial edge selection and spatio-temporal random walk.

[0014] In this step of initial edge selection, taking node Sampling is performed for the central node. Sampling includes unbiased sampling and biased sampling.

[0015] For unbiased sampling, define all the edge sets around node as , and use a sampling method with a uniform distribution to sample the initial edges. The probability that edge is selected is defined as:

[0016] ;

[0017] For biased sampling, it is recognized that all the edges around node have different probabilities of being selected. Biased sampling assigns different sampling probabilities to the edges according to the time when the edges appear. Specifically, the probability that edge is selected is defined as:

[0018] ;

[0019] Among them, represents any edge connected to the central node , is the time corresponding to edge , represents the time corresponding to edge , and is the minimum time associated with the edges in the dynamic graph. Biased sampling encourages sampling of the later-appearing edges.

[0020] The main task completed in this step of spatio-temporal random walk is to select the next node. The next node can be selected from the spatio-temporal neighbor set . The spatio-temporal neighbor set can be defined as:

[0021] ;

[0022] Among them, represents the spatio-temporal neighbor set of node at time , represents the node adjacent to node , represents the time on the edge between node and node .

[0023] Similarly, the present invention also has two spatio-temporal random walk strategies, which can be defined as biased spatio-temporal random walk and unbiased spatio-temporal random walk.

[0024] For unbiased spatio-temporal random walk, given any edge , node At time the spatio-temporal neighbors are selected with the following probabilities:

[0025] ;

[0026] For a biased spatio-temporal random walk, the probability that node selects its spatio-temporal neighbor at time is:

[0027] ;

[0028] Wherein, represents any spatio-temporal neighbor of node at time , is the time corresponding to node , is the time corresponding to node .

[0029] In addition, we introduce the idea of random walk, that is, sampling without considering time factors during sampling. Taking node as the central node to perform a random walk to sample an a-temporal subgraph . And taking node (where node and node are different nodes) as the central node to perform a spatio-temporal random walk to obtain a non-spatial subgraph .

[0030] Based on the above technical solution, the specific steps of step S2 are: by using a spatio-temporal weight encoder to obtain the representations of all nodes.

[0031] Specifically, after obtaining the spatio-temporal subgraph , the spatio-temporal weight encoder is used to output the representation of each node in the spatio-temporal subgraph. The main goal of the spatio-temporal weight encoder is to avoid the so-called representation obsolescence problem. The spatio-temporal weight encoder includes a time aggregator and a space aggregator.

[0032] For the time aggregator, when an event involving the node itself occurs, the temporal representation of node at time is expressed as:

[0033] ;

[0034] Wherein, LSTM represents the LSTM model, represents node and The interaction features between at the moment is the time when the node was last updated.

[0035] For the spatial aggregator, this component is used to merge the representation information of spatio-temporal neighbors, and it can assign different weights according to the different appearance times of neighbor nodes. Specifically, the spatial representation of node at time is defined as follows:

[0036] ;

[0037] where represents the node connected to node represents the time encoding for calculating the time interval between the current time and time is the temporal representation of node at moment is a learnable parameter.

[0038] Finally, we obtain the node representation of node at time by aggregating the temporal representation and the spatial representation :

[0039] ;

[0040] where is a learnable parameter.

[0041] It should be noted that the graph-level representation of the spatio-temporal subgraph is obtained through the readout function.

[0042] Similarly, through the spatio-temporal weight encoder, the graph-level representation of the non-temporal subgraph and the graph-level representation of the non-spatial subgraph can be obtained.

[0043] The specific steps of step S3 are as follows: By comparing positive and negative samples, the spatio-temporal weight encoder is trained. The specific contrast learning objective is to distinguish spatio-temporal subgraphs, non-temporal subgraphs, and non-spatial subgraphs. For the selection of positive samples, the node representation of the spatio-temporal subgraph and the graph-level representation of the spatio-temporal subgraph are used to form positive samples.

[0044] Secondly, the present invention uses the non-temporal subgraph​​​ Graph-level representation and spatio-temporal subgraph node representation constitute a time negative sample, called a time negative example. The loss function of the time negative sample is defined as:

[0045] ;

[0046] wherein, represents the sigmoid function, is the margin value, represents the expectation function, is the graph-level representation of the spatio-temporal subgraph, is the node node representation in the spatio-temporal subgraph, is the graph-level representation of the non-temporal subgraph.

[0047] In addition, the graph-level representation of the non-spatial subgraph and the spatio-temporal subgraph node representation constitute a spatial negative sample. The loss function of the spatial negative sample is defined as:

[0048] ;

[0049] wherein, is the sigmoid function, is the margin value, represents the expectation function, is the graph-level representation of the spatio-temporal subgraph, is the node node representation in the spatio-temporal subgraph, is the graph-level representation of the non-spatial subgraph.

[0050] Therefore, the total loss function of the spatio-temporal weight encoder is:

[0051] ;

[0052] where is the balance parameter of the total loss function.

[0053] Advantages of the present invention:

[0054] In order to more efficiently mine the time and space information contained in the dynamic graph, this application intends to fuse the time and space information on the graph during the dynamic graph representation learning process, and proposes a dynamic graph representation learning method based on self-supervised learning. The method has good interpretability and excellent representation performance, and has the following advantages:

[0055] (1) High interpretability: The present invention samples the dynamic graph to obtain a spatio-temporal subgraph, a non-temporal subgraph, and a non-spatial subgraph respectively. Based on these three subgraphs, a contrastive learning task is designed to capture the time and space information. This solution has good interpretability.

[0056] (2) Strong transferability: The spatio-temporal sampling graph method described in the present invention is relatively independent, and the generated subgraphs can be used as auxiliary graphs for other dynamic graph representation learning methods, and it has strong transferability.

[0057] (3) Excellent representation performance: The present invention is used to capture the time and space information on the dynamic graph, and constructs classical link prediction tasks and node classification tasks in the e-commerce prediction scenario. Through experiments, it is found that the present invention greatly improves the performance of tasks such as classification, recommendation, and prediction in the e-commerce application scenario of graph models. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention has the following drawings:

[0059] Figure 1 It is a schematic diagram of a dynamic graph representation learning framework based on self-supervised learning;

[0060] Figure 2 It is a schematic diagram of a spatio-temporal weight encoder. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The present invention will be further described in detail below with reference to the drawings.

[0062] The present invention proposes a dynamic graph representation learning method based on self-supervised learning (hereinafter referred to as the method), as Figure 1 shown.

[0063] Our invention mainly takes the e-commerce field as the background, with products and users as nodes in the graph, and the interactions between users and products as edges in the graph. Specific interactions can be understood as the following operations: browsing, collecting, and purchasing. Finally, our invention will obtain the feature descriptions of users and products, and predict the products that users may purchase based on these features.

[0064] First, the method takes any node in the graph as the central node, and performs spatio-temporal random walks and random walks respectively. The sampled subgraphs are defined as spatio-temporal subgraphs and non-temporal subgraphs respectively. In addition, taking any other node as the central node to perform spatio-temporal random walks, the sampled subgraph is called a non-spatial subgraph; secondly, using the spatio-temporal weight encoder to capture the time and space information in the graph to obtain the node representations and subgraph representations of the corresponding nodes; finally, through the contrastive learning method, the mutual information between the node representations and subgraph representations is maximized to learn the representations of the nodes. The specific technical solutions are as follows:

[0065] (1) Generate subgraphs

[0066] As shown Figure 1 in the given dynamic graph , where represents the set of edges, is a set of a series of edges, and there exists an edge in the graph indicating that node and node generate an edge at time. represents a function that can map each edge to a timestamp. The present invention hopes to collect a spatio-temporal subgraph centered on node in the dynamic graph .

[0067] For the sampling of the spatio-temporal subgraph , it is mainly divided into the following two steps: initial edge selection and spatio-temporal random walk.

[0068] In the step of initial edge selection, sampling is performed with node as the central node. There are two ways to select this method, one is called unbiased sampling and the other is called biased sampling.

[0069] In the step of initial edge selection, sampling is performed with node as the central node. Sampling includes unbiased sampling and biased sampling.

[0070] For unbiased sampling, define the set of all edges around node as , and use a uniform distribution sampling method to sample the initial edge. The probability that edge is selected is defined as:

[0071] ;

[0072] For biased sampling, it is considered that all edges around node are selected with different probabilities. Biased sampling will assign different sampling probabilities to edges according to the time when the edges appear. Specifically, the probability that edge is selected is defined as:

[0073] ;

[0074] Among them, represents any edge connected to the central node , is the time corresponding to edge , represents the time corresponding to edge while is the minimum time associated with an edge in a dynamic graph. Biased sampling encourages sampling of later-occurring edges.

[0075] The main task accomplished in this step of spatio-temporal random walk is to select the next node. The next node can be selected from the spatio-temporal neighbor set as follows. The spatio-temporal neighbor set can be defined as:

[0076] ;

[0077] where represents the spatio-temporal neighbor set of node at time , represents the nodes adjacent to node , represents the time on the edge between node and node .

[0078] Similarly, the present invention also has two spatio-temporal random walk strategies, which can be defined as biased spatio-temporal random walk and unbiased spatio-temporal random walk.

[0079] For unbiased spatio-temporal random walk, given any edge , the probability of selecting the spatio-temporal neighbor of node at time is as follows:

[0080] ;

[0081] For biased spatio-temporal random walk, the probability of selecting the spatio-temporal neighbor of node at time is:

[0082] ;

[0083] where represents any spatio-temporal neighbor of node at time , is the time corresponding to node , is the time corresponding to node .

[0084] In addition, we borrow the idea of random walk, that is, sampling without considering time factors during sampling. Taking node as the central node to perform random walk to sample the non-temporal subgraph . And taking node (where node And the node Taking different nodes (as the central node) for spatio-temporal random walk to obtain a non-spatial subgraph .

[0085] (2)Spatio-temporal weight encoder

[0086] As Figure 2 shown, in order to capture the temporal and spatial representations of nodes, the present invention designs a spatio-temporal weight encoder. Specifically, after obtaining , the spatio-temporal weight encoder is used to output the representation of each node in the spatio-temporal subgraph. The main goal of the spatio-temporal weight encoder is to avoid the so-called representation obsolescence problem. It consists of two components, called the temporal aggregator and the spatial aggregator.

[0087] For the temporal aggregator, when an event involving the node itself occurs, the node at time The temporal representation is expressed as:

[0088] ;

[0089] Among them, LSTM represents the LSTM model, represents the interaction feature between node and at time , is the time when node was last updated.

[0090] For the spatial aggregator, this component is used to merge the representation information of spatio-temporal neighbors, and it can assign different weights according to the different appearance times of neighbor nodes. Specifically, the spatial representation of node at time is defined as follows:

[0091] ;

[0092] Among them, represents the node connected to node represents calculating the time encoding of the distance between the current time and time , is the temporal representation of node at time, is a learnable parameter.

[0093] Finally, we aggregate the temporal representation and the spatial representation Obtain nodes At time representation :

[0094] ;

[0095] Among them, are learnable parameters. It should be noted that the graph-level representation of the spatio-temporal subgraph is obtained through the readout function.

[0096] Similarly, through the spatio-temporal weight encoder, the graph-level representation of the non-temporal subgraph can be obtained as well as the graph-level representation of the non-spatial subgraph .

[0097] (3) Spatio-temporal contrast learning

[0098] By comparing positive and negative samples, the spatio-temporal weight encoder is trained. The specific contrast learning objective is to distinguish spatio-temporal subgraphs, non-temporal subgraphs, and non-spatial subgraphs. For the selection of positive samples, the present invention constructs positive samples from the central node representation of the spatio-temporal subgraph and the graph-level representation of the spatio-temporal subgraph.

[0099] Secondly, the present invention uses the graph-level representation of the non-temporal subgraph and the central node representation of the spatio-temporal subgraph to form a time negative sample, called a time negative example. The loss function of the time negative sample is defined as:

[0100] ;

[0101] Among them, represents the sigmoid function, is the margin value, represents the expectation function, is the graph-level representation of the spatio-temporal subgraph, is the representation of the node in the spatio-temporal subgraph, is the graph-level representation of the non-temporal subgraph.

[0102] In addition, the graph representation of the non-spatial subgraph and the central node representation of the spatio-temporal subgraph form a space negative sample. The loss function of the space negative sample is defined as:

[0103] ;​​​​

[0104] Among them, is the sigmoid function, is the marginal value, represents the expected function, is the graph-level representation of the spatio-temporal subgraph, is for the node in the spatio-temporal subgraph, is the graph-level representation of the non-spatial subgraph. Therefore, the total loss function of the spatio-temporal weight encoder is:

[0105] ;

[0106] Among them is the balance parameter of the total loss function.

[0107] S4. Complete the training of the spatio-temporal weight encoder through the total loss function. Finally, send all the nodes in the dynamic graph into the trained spatio-temporal weight encoder to obtain the node representations of the dynamic graph. We apply it to the e-commerce scenario to predict the products that users may purchase.

[0108] 1. In order to depict the time and space information of the dynamic graph and model the original graph more efficiently, the present invention proposes to use the idea of contrastive learning. By sampling subgraphs on the dynamic graph as auxiliary graphs, the time and space information in the dynamic graph representation learning process is further enriched, and it has good interpretability.

[0109] 2. Aiming at the characteristic that the nodes and edges in the subgraph are dynamically changing, the present invention proposes a spatio-temporal weight encoder to capture the time and space features in the dynamic graph.

[0110] 3. The spatio-temporal sampling graph method described in the present invention is relatively independent, and the generated subgraph can be used as an auxiliary graph for other dynamic graph representation learning methods, and has strong transferability.

[0111] 4. The present invention is used to capture the time and space information on the dynamic graph. Through experiments, it is found that the present invention can be applied to the e-commerce scenario to predict the products purchased by users.

[0112] The examples in the present invention are only for clearly illustrating the present invention and are not intended to limit the implementation manner of the present invention. For those of ordinary skill in the art, based on the above description, other different forms of changes or modifications can be made. It is impossible to list all the implementation manners here. Any obvious changes or modifications derived from the technical solutions of the present invention still fall within the protection scope of the present invention.

[0113] The content not detailed in this specification belongs to the prior art well-known to those skilled in the art.

Claims

1. A dynamic graph representation learning method based on self-supervised learning, characterized in that, It includes the following steps: S1. Take products and users in the e-commerce scenario as nodes in the graph. Take any node in the graph as the central node, and sample the spatio-temporal subgraph, non-temporal subgraph, and non-spatial subgraph respectively. The interaction between users and products is the edge in the graph. Take any node in the graph as the central node, and perform spatio-temporal random walk and random walk respectively. The sampled subgraphs are defined as the spatio-temporal subgraph and non-temporal subgraph respectively. Perform spatio-temporal random walk with any other node as the central node, and the sampled subgraph is called the non-spatial subgraph; S2. Use the spatio-temporal weight encoder to capture the time and space information in the graph, obtain the node representation of the corresponding node and the graph-level representation of the subgraph, and obtain the feature descriptions of users and products; S3. Use the contrastive learning method to maximize the interaction information between the node representation and the graph-level representation of the subgraph to learn the node representation. The specific interactions are browsing, collecting, and purchasing. Define the total loss function from the perspectives of time and space respectively; S4. Complete the training of the spatio-temporal weight encoder through the total loss function. Finally, send all the nodes in the dynamic graph into the trained spatio-temporal weight encoder to obtain the node representation of the dynamic graph. Predict the products that the user may purchase according to this node representation; The specific steps of step S2 are as follows: The spatio-temporal weight encoder includes a time aggregator and a space aggregator; For the time aggregator, when an event involving the node itself occurs, the node at time temporal representation is represented as: ; Among them, LSTM is denoted as the LSTM model, denotes a node and the interaction feature between at time is the time when node was last updated; The spatial aggregator is used to merge the representation information of spatio-temporal neighbors and can assign different weights according to the different appearance times of neighbor nodes; nodes at time spatial representation is defined as follows: ; Among them, represents the node connected to the node , represents the calculation of the current time and the time spacing time code, is the node at the timing representation at the moment, is a learnable parameter; Finally, by aggregating the temporal representation and the spatial representation the node in time is obtained : ; Among them, are learnable parameters; Acyclic subgraph Graph-level representation And acyclic subgraph Graph-level representation Obtained by a spatio-temporal weight encoder; graph-level representation of spatio-temporal subgraphs Obtained by a readout function.

2. The dynamic graph representation learning method based on self-supervised learning according to claim 1, wherein The specific steps of step S1 are: Given a dynamic graph , where represents the set of edges, is a set of a series of edges. There exists an edge in the graph, which represents that node and node generate an edge at time. represents a function that can map each edge to a timestamp; Sampling of the spatio-temporal subgraph includes the following two steps: initial edge selection and spatio-temporal random walk.

3. The dynamic graph representation learning method based on self-supervised learning according to claim 2, characterized in that: In the initial edge selection step, sampling is performed with node as the central node; the sampling includes unbiased sampling and biased sampling; For unbiased sampling, define all the edge sets around the node as , and sample the initial edges using a sampling method with a uniform distribution; Edge The probability of being selected is defined as: ; For biased sampling, the identified nodes All the edges around have different probabilities of being selected; biased sampling assigns different sampling probabilities to edges according to the time when the edges appear; specifically, for edge The probability of being selected is defined as: ; Among them, represents any edge connected to the central node connected, is the edge corresponding time, represents the edge corresponding time, is the minimum time associated with the edges in the dynamic graph.

4. The dynamic graph representation learning method based on self-supervised learning according to claim 2, wherein: The main task of the spatio-temporal random walk step is to select the next node; the next node is selected from the spatio-temporal neighbor set ; the spatio-temporal neighbor set is defined as: ; Among them, represents the set of spatio-temporal neighbors of node at time ; represents the node adjacent to node ; represents the time on the edge between node and node . The spatio-temporal random walk includes a biased spatio-temporal random walk and an unbiased spatio-temporal random walk; For an unbiased spatio-temporal random walk, given any edge , the node at time has the following probability of being selected as a spatio-temporal neighbor : ; For the biased spatio-temporal random walk, the node at time in the spatio-temporal neighborhood is selected with a probability of: ; Among them, represents the node at time any spatio-temporal neighbor of, is the node corresponding time, is the node corresponding time.

5. The method for dynamic graph representation learning based on self-supervised learning according to claim 2, wherein: Referring to the idea of random walk, sampling is performed without considering time factors during sampling; Random walk is performed with the node as the central node to sample the non-temporal subgraph ; and random walk is performed with the node as the central node for spatio-temporal random walk to obtain the non-spatial subgraph .

6. The dynamic graph representation learning method based on self-supervised learning according to claim 2, wherein The specific steps of step S3 are as follows: Train the spatio-temporal weight encoder by comparing positive and negative samples. The contrastive learning objective is to distinguish the spatio-temporal subgraph, non-temporal subgraph, and non-spatial subgraph. For the selection of positive samples, the node representation of the spatio-temporal subgraph and the graph-level representation of the spatio-temporal subgraph are used to form positive samples; The non-temporal subgraph at the graph level representation and the node representations of the spatio-temporal subgraph are used to form time negative samples, and the loss function of the time negative samples is defined as: ​ ; Among them, represents the sigmoid function, is the marginal value, represents the expectation function, is the graph-level representation of the spatio-temporal subgraph, is for the nodes in the spatio-temporal subgraph node representation, is the graph-level representation of the non-temporal subgraph; The non-spatial subgraph at the graph level representation and the node representation of the spatio-temporal subgraph are used to form spatial negative samples; the loss function of the spatial negative samples is defined as: ​ ; Among them, is the sigmoid function, is the marginal value, represents the expected function, is the graph-level representation of the spatio-temporal subgraph, is for the node in the spatio-temporal subgraph is the graph-level representation of the non-spatial subgraph; The total loss function of the spatio-temporal weight encoder is: ; Among them is the balance parameter of the total loss function.

Images