A high-order dependency embedding method and system for traffic sequence data

Abstract

The present invention discloses a high-order dependency embedding method and system for traffic sequence data, comprising: constructing a high-order network using traffic sequence data, performing random walks on the high-order network to obtain an initial embedding representation, optimizing node embedding representation through multi-level contrastive learning, aggregating high-order nodes, and obtaining a final embedding representation of first-order nodes; constructing a high-order network through high-order dependencies, capturing complex high-order dependencies in traffic trajectory data, and more comprehensively reflecting potential patterns and relationships in the traffic network; optimizing node embedding representation through multi-level contrastive learning, making full use of information of first-order networks and high-order networks, and generating node embeddings with stronger expression ability and robustness; adopting a self-supervised learning method, training by utilizing the intrinsic structural information of the data, and reducing dependence on a large amount of labeled data; and being applicable to various traffic trajectory data, having wide applicability and good expansibility.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic network analysis, and in particular relates to a high-order dependency embedding method and system for traffic sequence data. Background Art

[0002] The analysis and optimization of transportation networks are of great significance in urban transportation planning, logistics management, and transportation system monitoring. Transportation networks are composed of nodes (such as transportation hubs such as airports and ports) and edges (such as transportation connections such as flight routes and air routes), which are interwoven into a complex network structure. In-depth analysis of the structure and characteristics of transportation networks can provide important guidance for optimizing transportation routes, improving transportation efficiency, and reducing operating costs.

[0003] Network embedding methods aim to map nodes in a network into a low-dimensional space while retaining their proximity based on the network topology. Traditional first-order networks capture pairwise relationships based on the Markov property, but have limitations in dealing with multivariate interactions in complex networks. Sequential data, as a common form of data in transportation networks, not only reveals the order of interactions between nodes, but also implies higher-order dependencies, which are often ignored by traditional first-order network embedding methods. In transportation networks, the relationships between nodes often have complex high-order dependencies, such as flight transfer paths, multi-port stopover routes for ships, etc. These high-order dependencies are crucial for a deep understanding of the operating mechanism and optimization strategies of transportation networks.

[0004] However, existing technologies have many deficiencies in dealing with these high-order dependencies and are unable to fully mine and utilize implicit information in traffic networks. In addition, most graph neural network-based methods rely on a large amount of labeled data for supervised learning, which is a significant challenge in the field of traffic network analysis. In practical applications, the acquisition of labeled data is often costly and scarce, which limits the widespread application of these methods in traffic network analysis.

[0005] In summary, existing methods are inadequate in handling high-order dependencies in traffic networks and relying on large amounts of labeled data, which results in limited performance of the generated node embedding representations in complex network analysis tasks.

[0006] Publication No. CN117789470A discloses a traffic flow prediction method based on future knowledge differential transformation neural network, which solves the problem of difficulty in mining spatiotemporal patterns between different dates in adjacent time domains and future time domains in the prior art, and does not involve related technologies such as high dependency relationships in traffic networks. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention provides a high-order dependency embedding method and system for traffic sequence data, which constructs a high-order network through high-order dependencies and optimizes the node embedding representation by multi-level contrastive learning, so that the final node embedding representation is more expressive and robust.

[0008] The technical solution of the present invention is achieved in this way:

[0009] A high-order dependency embedding method for traffic sequence data includes the following steps:

[0010] S1. Obtain the sequence data of traffic trajectories and build a high-order network. The specific method is as follows:

[0011] Obtain the sequence data of traffic trajectories, construct a first-order network through the sequence data, obtain the node transition probability on the first-order network, and obtain the transition probability distribution of the first-order node; then calculate the transition probability distribution of higher-order nodes; use the KL divergence to compare the difference in the transition probability distribution of two adjacent order nodes, and identify high-order dependencies by setting a dynamic threshold and comparing it with the KL divergence;

[0012] According to the high-order dependencies, high-order nodes are created, and the newly created high-order nodes are connected through edges to form a high-order network;

[0013] S2: Based on the high-order network constructed in S1, random walks are performed on the high-order network to obtain the initial node embedding representation;

[0014] S3. Perform multi-level contrastive learning in high-order networks to optimize node embedding representation; the specific method is:

[0015] Construct positive example pairs and negative example pairs in the high-order network, and construct positive example pairs and negative example pairs in the first-order network; then, based on the positive example pairs, negative example pairs and loss functions in the corresponding hierarchical networks, respectively optimize the high-order nodes in the high-order network and the first-order nodes in the first-order network to obtain optimized node embedding representations;

[0016] S4. Aggregate the embedding representations of the first-order nodes and the higher-order nodes in the high-order network to obtain the node embedding representation of the first-order nodes.

[0017] Preferably, the previous node of the node is the predecessor node, and the subsequent node of the node is the successor node; in a high-order network, the constructed positive example pair is a high-order network positive example pair, and the constructed negative example pair is a high-order network negative example pair; in a first-order network, the constructed positive example pair is a first-order network positive example pair, and the constructed negative example pair is a first-order network negative example pair.

[0018] Preferably, in S1, the sequence data of the traffic trajectory is obtained to identify the high-order dependency relationship; specifically,

[0019] S1.1. Setting according to the sequence data of traffic trajectory is a discrete random variable sequence, and a first-order network following the Markov property is established by the discrete random variable sequence, that is,

[0020] ;

[0021] Among them, p represents the transition probability of the node, X represents the data in the discrete random variable sequence, t represents different moments, and s represents different first-order nodes in the first-order network;

[0022] S1.2. Calculate the first-order node S according to the first-order network established in S1.1. t The transition probability to the next node in a first-order network is:

[0023] ;

[0024] in, Represents the first-order node S t The transition probability in a first-order network is Represents the first-order node S t The weight of the directed edge to node S, i is the first-order node S t The set of nodes that can be directly reached by the directed edges of ;

[0025] Calculate the transition probability of a high-order node to the next node in a high-order network:

[0026] ;

[0027] in, Represents a high-order node Transition probabilities in high-order networks, Represents a high-order node The weight of the directed edge to node S, i is a high-order node The set of nodes that can be directly reached by the directed edges of ;

[0028] Specifically, when calculating the transition probability from a first-order node or a high-order node to the next node, the calculation method is the same. Therefore, i is not distinguished here and does not affect the understanding of the overall technical solution.

[0029] S1.3, according to the transition probability of all nodes obtained in S1.2, set the k-order high-order node The transition probability distribution of walking to the next node in the traffic trajectory is , where S k represents a high-order node of order k; d k represents the corresponding transition probability distribution, p Krepresents the transition probability, N represents the number of nodes connected to the high-order node in the high-order network;

[0030] If in a high-order network, there are k+1-order high-order nodes , which corresponds to the probability distribution ;

[0031] According to the probability distribution, the KL divergence between the high-order node and the first-order node transfer probability distribution is calculated, and the d k and d k+1 The difference between the two transition probability distributions:

[0032] Set a dynamic threshold by comparing the KL divergence with the dynamic threshold, that is:

[0033] If D kl is greater than the dynamic threshold, the high-order node is introduced. kl If it is less than the dynamic threshold, the high-order node will not be introduced;

[0034] S1.4, connect the introduced high-order node with its previous and subsequent nodes through edges, generate new edges, and delete the edge connecting the original first-order node to its previous first-order node; remove the nodes without incoming edges;

[0035] S1.5. Continue to adjust the successor nodes of the introduced high-order nodes by comparing the KL divergence and the dynamic threshold size to build a high-order network; if the successor node depends on the current high-order node, re-establish the connection.

[0036] Specifically, the connections between nodes in the network are represented by edges, which represent direct relationships or interactions between nodes, such as a ship moving from port 1 to port 2.

[0037] However, in some complex systems, the relationship between nodes is not only determined by the current node, but may also be affected by several previous nodes. For example, the next stop of a ship may not only depend on the port it is currently in, but also on the ports it has visited before. In this case, the traditional first-order network model is not enough because it only considers the influence of the current node and ignores the deeper dependencies.

[0038] Identify high-order dependencies and find out which paths are dependent from the sequence data by comparing KL divergence. High-order nodes represent high-order dependencies.

[0039] If the KL divergence is greater than the threshold, it means that adding the high-order node will have a significant impact on the probability distribution of the node's transition state. For example, if the KL divergence is greater than the threshold in the path from A to B, it means that in some cases, a step before A will affect the decision from A to B, then this path is a high-order dependency.

[0040] Create high-order nodes. Once such high-order dependencies are discovered, new nodes, namely high-order nodes, will be created. These high-order nodes not only represent the current position or state, but also include the important steps before. For example, originally there was only the node "Port 2" in the network, but now there may be "Port 2|Port 1", which means that the ship that came from Port 1 before has arrived at Port 2, such a new node.

[0041] Connect nodes, connect these newly created nodes through edges based on the newly identified high-order dependencies. In this process, they will ensure that these newly connected edges reflect the dependency patterns in the actual data.

[0042] In S1.3, in order to prevent over-capturing of dependencies in the network, a maximum order needs to be set. The high-order network is built on the basis of the first-order network, and the high-order dependencies in the data are checked by counting the frequency of occurrence of the path composed of the current node and its predecessor node in the sequence data.

[0043] Preferably, the specific method for calculating KL divergence according to probability distribution is:

[0044] ;

[0045] in is the KL divergence.

[0046] Specifically, KL divergence uses Kullback-Leibler divergence to compare the differences between probability distributions to determine whether high-order dependencies are preserved.

[0047] Preferably, the dynamic threshold is defined as follows:

[0048] ;

[0049] Among them, Ext represents one of the nodes, Support represents the in-degree of node Ext, is a tunable hyperparameter.

[0050] Specifically, in order to ensure that only significant high-order nodes are retained, a dynamic threshold is set to compare with the KL divergence. This threshold can effectively ensure that only high-order dependencies that fully satisfy the high-order dependencies will be retained in the high-order network.

[0051] Preferably, in S2, performing a random walk on a high-order network specifically includes:

[0052] By simulating the wandering process of random walkers in a high-order network, a node sequence is obtained, which includes the structural information of the high-order network.

[0053] The structural information of the high-order network is encoded by the Skip-Gram model to obtain the initial node embedding representation.

[0054] Specifically, the random walk strategy is introduced into the high-order network constructed in the previous step. By traversing nodes of various orders on the high-order network, the local and global information of the network is captured. Random walk is a commonly used network embedding method. By simulating the wandering process of random walkers in the network, a series of node sequences can be obtained. These node sequences contain the structural information of the network and can be used for subsequent node embedding representation learning.

[0055] The SG model is the Skip-Gram model, which uses the Skip-Gram model to encode these valid information and obtain the initial node embedding representation. This model is a word embedding-based model that can learn the low-dimensional vector representation of the node by maximizing the probability of nodes appearing around a given node. This representation method has good generalization ability and can be used for various network analysis tasks.

[0056] Preferably, in S3, high-order and first-order multi-level comparative learning is performed on the node embedding representation in the high-order network to obtain an optimized node embedding representation; specifically, the step includes:

[0057] S3.1. At the level of high-order networks, the high-order nodes and their predecessor nodes in the node sequence obtained by random walk are designated as positive example pairs of the high-order network; for each high-order node, its non-neighbor nodes are randomly sampled to obtain negative example nodes, and the high-order node and its negative example nodes are designated as negative example pairs;

[0058] Specifically, on a high-order network, the negative example pairs obtained by sampling non-adjacent nodes may include multiple negative example nodes.

[0059] Through contrastive learning, the similarity of positive pairs is maximized and the similarity of negative pairs is minimized. At the level of this high-order network, the loss function of contrastive learning is as follows:

[0060] ;

[0061] in, is a high-level loss function, X is a set of high-order nodes in a high-order network, and x i is x j Positive node, x k is a negative example node, f(xi ) represents x i Node embedding representation; sim is the similarity function, which is calculated using dot product;

[0062] S3.2. At the level of the first-order network, the first-order nodes connected by directed edges constitute node pairs. The node pairs are sorted according to the weights of the directed edges, and the nodes ranked in the top 25%-35% of the weight ranking are designated as the first-order network positive example pairs;

[0063] On the first-order network, randomly sample its non-neighbor nodes to obtain the first-order network negative example pairs;

[0064] The optimized node embedding of the first-order network obtained by the loss function is expressed as follows:

[0065] ;

[0066] in, is a first-order loss function, Z is a set of first-order nodes in a first-order network, z i Yes j Positive node, z k is a negative example node, f(z i ) indicates z i The embedding vector of

[0067] S3.3. The loss function of the entire model is obtained by using the contrast learning mechanism of the high-order network and the first-order network, and by accumulating their corresponding loss functions, the overall loss function of the model is obtained:

[0068] ;

[0069] in, Represents the overall loss function. The model is optimized by minimizing the overall loss function to obtain the optimized node embedding representation.

[0070] Specifically, multi-level contrastive learning is divided into two levels, namely the high-order network level and the first-order network level. The two are parallel in order to construct the loss function of the entire model, thereby optimizing the initial node embedding representation obtained earlier. The embedding is mainly optimized by continuously minimizing the loss function L.

[0071] The core of contrastive learning is to construct positive and negative pairs. The formulas at both levels are the same, both are contrastive learning formulas, the main difference is the choice of constructing positive and negative pairs.

[0072] Positive example pairs, such as in the sequence data of traffic trajectories, , where x j is a high-order node. Then it and its predecessor node x iTogether they form a positive pair. These positive pairs show structural similarities on high-order networks, which are essential for learning high-quality node embedding representations. For each high-order node, negative nodes are obtained by randomly sampling non-adjacent nodes. Then, the InfoNCE loss function is used to maximize the similarity of positive pairs and minimize the similarity of negative pairs, thereby optimizing the node embedding representation.

[0073] However, high-order networks may ignore the connections between high-order nodes of different orders belonging to the same first-order node. However, these high-order relationships are also important for learning the adjacency information of nodes. Therefore, in addition to contrastive learning on high-order networks, it is also necessary to introduce contrastive learning on first-order networks to capture these neighbor relationships. In contrastive learning at the first-order network level, node pairs consisting of edge-connected nodes are sorted by the weights of directed edges. Node pairs with higher weight rankings are regarded as positive pairs, and negative pairs are also obtained by sampling non-neighbor nodes.

[0074] The final loss function of the entire model is obtained by accumulating the contrastive learning loss functions of the two levels. By minimizing this total loss function, the node embedding representations of the high-order network level and the first-order network level can be optimized at the same time, thereby obtaining a more accurate and robust node embedding representation.

[0075] Preferably, S4 specifically includes assigning weights according to the frequency of occurrence of each high-order node, multiplying the embedding representation of the high-order node by its corresponding weight, and then performing weighted summation on the weighted node embedding representations in the high-order network to obtain the final embedding representation of the first-order node.

[0076] Specifically, since what is ultimately needed is the embedded representation of the first-order nodes, and the high-order network contains nodes of different orders, an effective aggregation mechanism is needed to integrate the information of these high-order nodes into the first-order nodes.

[0077] A frequency-based weighted aggregation mechanism is introduced in S4. Specifically, weights are assigned according to the frequency of occurrence of each high-order node, and the number of occurrences of nodes of different orders belonging to the same first-order node is counted. Frequently appearing nodes will be given higher weights because they are more critical in the network. Finally, the weights of these nodes are used to perform weighted aggregation on the node embedding representations: that is, by multiplying the embedding representations of the high-order nodes with their corresponding weights and then summing them, the final embedding representation of the first-order nodes is obtained. This weighted aggregation method not only integrates the information in the high-order network, but also retains the high-order dependency characteristics unique to each high-order node, making the final first-order node embedding representation more expressive and robust.

[0078] Preferably, the high-order network includes high-order nodes and first-order nodes.

[0079] A high-order dependency embedding system for traffic sequence data is applied in a high-order dependency embedding method for traffic sequence data as described in any one of the above.

[0080] Compared with the prior art, the present invention achieves the following beneficial effects:

[0081] 1. The present invention provides a method and system for embedding high-order dependencies for traffic sequence data. According to the sequence data of traffic trajectories, a high-order network is constructed: by constructing a high-order network, the present invention can capture the complex high-order dependencies in traffic trajectory data. This method does not only rely on the traditional Markov property, but by introducing high-order nodes and high-order edges, it can more comprehensively reflect the potential patterns and relationships in the traffic network.

[0082] 2. Optimizing node embedding representation using multi-level contrastive learning: The present invention uses a multi-level contrastive learning mechanism to fully utilize the information of the first-order network and the high-order network, and the generated node embedding has stronger expressiveness and robustness.

[0083] 3. The present invention adopts a self-supervised learning method, which reduces the reliance on a large amount of labeled data by using the intrinsic structural information of the data for training. This is of great significance in practical application scenarios where data labeling is expensive or difficult to obtain.

[0084] 4. The present invention is applicable to various traffic trajectory data, including flight data, shipping data, and simulation data, etc. Moreover, the present method can also be further applied to any sequence data, and has wide applicability and good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 is a flow chart of a high-order dependency embedding method for traffic sequence data provided in an embodiment of the present invention;

[0086] Figure 2 It is one of the schematic diagrams of constructing a high-order network in a high-order dependency embedding method for traffic sequence data provided in an embodiment of the present invention;

[0087] Figure 3 This is the second schematic diagram of constructing a high-order network in a high-order dependency embedding method for traffic sequence data provided in an embodiment of the present invention;

[0088] Figure 4 It is a schematic diagram of a weighted aggregation mechanism of a high-order dependency embedding method for traffic sequence data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0090] Example

[0091] like Figures 1 to 4 , a high-order dependency embedding method for traffic sequence data, including the following steps:

[0092] S1. Obtain the sequence data of traffic trajectories and build a high-order network. The specific method is as follows:

[0093] Obtain the sequence data of traffic trajectories, construct a first-order network through the sequence data, obtain the node transition probability on the first-order network, and obtain the transition probability distribution of the first-order node; then calculate the transition probability distribution of higher-order nodes; use the KL divergence to compare the difference in the transition probability distribution of two adjacent order nodes, and identify high-order dependencies by setting a dynamic threshold and comparing it with the KL divergence;

[0094] The previous node of a node is the predecessor node, and the subsequent node of the node is the successor node; in a high-order network, the constructed positive example pairs are high-order network positive example pairs, and the constructed negative example pairs are high-order network negative example pairs; in a first-order network, the constructed positive example pairs are first-order network positive example pairs, and the constructed negative example pairs are first-order network negative example pairs.

[0095] According to the high-order dependency, high-order nodes are created, and the newly created high-order nodes are connected through edges to form a high-order network; specifically,

[0096] S1.1. Setting according to the sequence data of traffic trajectory is a discrete random variable sequence, and a first-order network following the Markov property is established by the discrete random variable sequence, that is,

[0097] ;

[0098] Among them, p represents the transition probability of the node, X represents the data in the discrete random variable sequence, t represents different moments, and s represents different first-order nodes in the first-order network;

[0099] S1.2. Calculate the first-order node S according to the first-order network established in S1.1. t The transition probability to the next node in a first-order network is:

[0100] ;

[0101] in, Represents the first-order node S t The transition probability in a first-order network is Represents the first-order node S t The weight of the directed edge to node S, i is the first-order node S t The set of nodes that can be directly reached by the directed edges of ;

[0102] Calculate the transition probability of a high-order node to the next node in a high-order network:

[0103] ;

[0104] in, Represents a high-order node Transition probabilities in high-order networks, Represents a high-order node The weight of the directed edge to node S, i is a high-order node The set of nodes that can be directly reached by the directed edges of ;

[0105] Specifically, when calculating the transition probability from a first-order node or a high-order node to the next node, the calculation method is the same. Therefore, i is not distinguished here and does not affect the understanding of the overall technical solution.

[0106] S1.3, according to the transition probability of all nodes obtained in S1.2, set the k-order high-order node The transition probability distribution of walking to the next node in the traffic trajectory is , where S k represents a high-order node of order k; d k represents the corresponding transition probability distribution, p K represents the transition probability, N represents the number of nodes connected to the high-order node in the high-order network;

[0107] If in a high-order network, there are k+1-order high-order nodes , which corresponds to the probability distribution ;

[0108] According to the probability distribution, the KL divergence between the high-order node and the first-order node transfer probability distribution is calculated, and the d k and d k+1 The difference between the two transition probability distributions:

[0109] ;

[0110] in is the KL divergence.

[0111] Specifically, KL divergence uses Kullback-Leibler divergence to compare the differences between probability distributions to determine whether high-order dependencies are preserved.

[0112] Set a dynamic threshold, which is defined as follows:

[0113] ;

[0114] Among them, Ext represents one of the nodes, Support represents the in-degree of node Ext, is a tunable hyperparameter.

[0115] Specifically, in order to ensure that only significant high-order nodes are retained, a dynamic threshold is set to compare with the KL divergence. This threshold can effectively ensure that only high-order dependencies that fully satisfy the high-order dependencies will be retained in the high-order network.

[0116] By comparing the KL divergence with the dynamic threshold, that is:

[0117] If D kl is greater than the dynamic threshold, the high-order node is introduced. kl If it is less than the dynamic threshold, the high-order node will not be introduced;

[0118] S1.4, connect the introduced high-order node with its previous and subsequent nodes through edges, generate new edges, and delete the edge connecting the original first-order node to its previous first-order node; remove the nodes without incoming edges;

[0119] S1.5. Continue to adjust the successor nodes of the introduced high-order nodes by comparing the KL divergence and the dynamic threshold size to build a high-order network; if the successor node depends on the current high-order node, re-establish the connection.

[0120] The high-order network includes high-order nodes and first-order nodes.

[0121] Specifically, the connections between nodes in the network are represented by edges, which represent direct relationships or interactions between nodes, such as a ship moving from port 1 to port 2.

[0122] However, in some complex systems, the relationship between nodes is not only determined by the current node, but may also be affected by several previous nodes. For example, the next stop of a ship may not only depend on the port it is currently in, but also on the ports it has visited before. In this case, the traditional first-order network model is not enough because it only considers the influence of the current node and ignores the deeper dependencies.

[0123] Identify high-order dependencies and find out which paths are dependent from the sequence data by comparing KL divergence. High-order nodes represent high-order dependencies.

[0124] If the KL divergence is greater than the threshold, it means that adding the high-order node will have a significant impact on the probability distribution of the node's transition state. For example, if the KL divergence is greater than the threshold in the path from A to B, it means that in some cases, a step before A will affect the decision from A to B, then this path is a high-order dependency.

[0125] Create high-order nodes. Once such high-order dependencies are discovered, new nodes, namely high-order nodes, will be created. These high-order nodes not only represent the current position or state, but also include the important steps before. For example, originally there was only the node "Port 2" in the network, but now there may be "Port 2|Port 1", which means that the ship that came from Port 1 before has arrived at Port 2, such a new node.

[0126] Connect nodes, connect these newly created nodes through edges based on the newly identified high-order dependencies. In this process, they will ensure that these newly connected edges reflect the dependency patterns in the actual data.

[0127] In S1.3, in order to prevent over-capturing of dependencies in the network, a maximum order needs to be set. The high-order network is built on the basis of the first-order network, and the high-order dependencies in the data are checked by counting the frequency of occurrence of the path composed of the current node and its predecessor node in the sequence data.

[0128] according to Figure 2 As shown, this is the construction process of a small high-order network.

[0129] The calculation results show that node C has two second-order dependencies, one from node A and one from node B. Therefore, we introduce two second-order nodes C|A and C|B in FON, and the corresponding edges, namely C|A points to D and C|B points to E. Since the predecessor node of C|A is A, we introduce an edge from A to C|A and delete the edge from A to C. Similarly, perform the same operation on node C|B. Then, since node C no longer has any incoming edges at this time, we delete the node and its outgoing edges. Finally, we get Figure 2 The high-order network shown. This completes the construction process of the high-order network.

[0130] This embodiment also provides another situation in the process of constructing a high-order network.

[0131] according to Figure 2 and Figure 3, suppose another case, the threshold is set higher. Since there are only two AC paths in the sequence data, and three BC paths. At this time, the high-order dependency of the nodes of C|A is not significant enough, that is, the KL divergence is less than the threshold, while the KL divergence of C|B is greater than the threshold. Then C|A in the graph will not be added, and the C node will not be deleted, but will be retained. The resulting image is as follows Figure 3 There are only four edges.

[0132] S2, based on the high-order network constructed in S1, perform random walks on the high-order network to obtain the initial node embedding representation; specifically,

[0133] By simulating the wandering process of random walkers in a high-order network, a node sequence is obtained, which includes the structural information of the high-order network.

[0134] The structural information of the high-order network is encoded by the Skip-Gram model to obtain the initial node embedding representation.

[0135] Specifically, the random walk strategy is introduced into the high-order network constructed in the previous step. By traversing nodes of various orders on the high-order network, the local and global information of the network is captured. Random walk is a commonly used network embedding method. By simulating the wandering process of random walkers in the network, a series of node sequences can be obtained. These node sequences contain the structural information of the network and can be used for subsequent node embedding representation learning.

[0136] The SG model is the Skip-Gram model, which uses the Skip-Gram model to encode these valid information and obtain the initial node embedding representation. This model is a word embedding-based model that can learn the low-dimensional vector representation of the node by maximizing the probability of nodes appearing around a given node. This representation method has good generalization ability and can be used for various network analysis tasks.

[0137] S3. Perform multi-level contrastive learning in high-order networks to optimize node embedding representation; the specific method is:

[0138] Construct positive example pairs and negative example pairs in the high-order network, and construct positive example pairs and negative example pairs in the first-order network; then, based on the positive example pairs, negative example pairs and loss functions in the corresponding hierarchical networks, respectively optimize the high-order nodes in the high-order network and the first-order nodes in the first-order network to obtain optimized node embedding representations;

[0139] In S3, high-order and first-order multi-level comparative learning is performed on the node embedding representation in the high-order network to obtain an optimized node embedding representation; specifically, it includes:

[0140] S3.1. At the level of high-order networks, the high-order nodes and their predecessor nodes in the node sequence obtained by random walk are designated as positive example pairs of the high-order network; for each high-order node, its non-neighbor nodes are randomly sampled to obtain negative example nodes, and the high-order node and its negative example nodes are designated as negative example pairs;

[0141] Specifically, on a high-order network, the negative example pairs obtained by sampling non-adjacent nodes may include multiple negative example nodes.

[0142] Through contrastive learning, the similarity of positive pairs is maximized and the similarity of negative pairs is minimized. At the level of this high-order network, the loss function of contrastive learning is as follows:

[0143] ;

[0144] in, is a high-level loss function, X is a set of high-order nodes in a high-order network, and x i is x j Positive node, x k is a negative example node, f(x i ) represents x i Node embedding representation; sim is the similarity function, which is calculated using dot product;

[0145] S3.2. At the level of the first-order network, the first-order nodes connected by directed edges constitute node pairs. The node pairs are sorted according to the weights of the directed edges, and the nodes ranked in the top 25%-35% of the weight ranking are designated as the first-order network positive example pairs;

[0146] Preferably, the nodes ranked in the top 30% by weight are designated as first-order network positive example pairs;

[0147] On the first-order network, randomly sample its non-neighbor nodes to obtain the first-order network negative example pairs;

[0148] The optimized node embedding of the first-order network obtained by the loss function is expressed as follows:

[0149] ;

[0150] in, is a first-order loss function, Z is a set of first-order nodes in a first-order network, z i Yes j Positive node, z k is a negative example node, f(z i ) indicates z i The embedding vector of

[0151] S3.3. The loss function of the entire model is obtained by using the contrast learning mechanism of the high-order network and the first-order network, and by accumulating their corresponding loss functions, the overall loss function of the model is obtained:

[0152] ;

[0153] in, Represents the overall loss function. The model is optimized by minimizing the overall loss function to obtain the optimized node embedding representation.

[0154] Specifically, multi-level contrastive learning is divided into two levels, namely the high-order network level and the first-order network level. The two are parallel in order to construct the loss function of the entire model, thereby optimizing the initial node embedding representation obtained earlier. The embedding is mainly optimized by continuously minimizing the loss function L.

[0155] The core of contrastive learning is to construct positive and negative pairs. The formulas at both levels are the same, both are contrastive learning formulas, the main difference is the choice of constructing positive and negative pairs.

[0156] Positive example pairs, such as in the sequence data of traffic trajectories, , where x j is a high-order node. Then it and its predecessor node x i Together they form a positive pair. These positive pairs show structural similarities on high-order networks, which are essential for learning high-quality node embedding representations. For each high-order node, negative nodes are obtained by randomly sampling non-adjacent nodes. Then, the InfoNCE loss function is used to maximize the similarity of positive pairs and minimize the similarity of negative pairs, thereby optimizing the node embedding representation.

[0157] However, high-order networks may ignore the connections between high-order nodes of different orders belonging to the same first-order node. However, these high-order relationships are also important for learning the adjacency information of nodes. Therefore, in addition to contrastive learning on high-order networks, it is also necessary to introduce contrastive learning on first-order networks to capture these neighbor relationships. In contrastive learning at the first-order network level, node pairs consisting of edge-connected nodes are sorted by the weights of directed edges. Node pairs with higher weight rankings are regarded as positive pairs, and negative pairs are also obtained by sampling non-neighbor nodes.

[0158] The final loss function of the entire model is obtained by accumulating the contrastive learning loss functions of the two levels. By minimizing this total loss function, the node embedding representations of the high-order network level and the first-order network level can be optimized at the same time, thereby obtaining a more accurate and robust node embedding representation.

[0159] S4. Aggregate the embedding representations of the first-order nodes and the higher-order nodes in the higher-order network to obtain the node embedding representation of the first-order nodes. Specifically,

[0160] According to the frequency of occurrence of each high-order node, a weight is assigned, the embedding representation of the high-order node is multiplied by its corresponding weight, and then the weighted node embedding representations in the high-order network are weighted summed to obtain the final embedding representation of the first-order node.

[0161] Specifically, since what is ultimately needed is the embedded representation of the first-order nodes, and the high-order network contains nodes of different orders, an effective aggregation mechanism is needed to integrate the information of these high-order nodes into the first-order nodes.

[0162] like Figure 4 , a frequency-based weighted aggregation mechanism is introduced in S4. Specifically, weights are assigned according to the frequency of occurrence of each high-order node, and the number of occurrences of nodes of different orders belonging to the same first-order node is counted. Frequently appearing nodes will be assigned higher weights because they are more critical in the network. Finally, the node embedding representations are weighted aggregated using the weights of these nodes: that is, by multiplying the embedding representations of high-order nodes with their corresponding weights and then summing them, the final embedding representation of the first-order nodes is obtained. This weighted aggregation method not only integrates the information in the high-order network, but also retains the high-order dependency characteristics unique to each high-order node, so that the final first-order node embedding representation is more expressive and robust.

[0163] A high-order dependency embedding system for traffic sequence data is applied in the high-order dependency embedding method for traffic sequence data as described above.

[0164] This method is applicable to the sequence data of any traffic trajectory. In order to verify the effectiveness of the technical solution proposed by the present invention, a series of experiments were conducted. The effectiveness of the method was verified by experiments. Here, a link prediction experiment (a general experiment of node embedding) was conducted. That is, the node embedding is used to predict the connection relationship of the edge. In other words, taking shipping as an example, it is to predict which port the ship at port 1 is most likely to go to next.

[0165] The experiment uses two real traffic sequence datasets and one simulated dataset. These datasets contain rich traffic trajectory information, covering different types of traffic entities (such as airports, ports, etc.) and the connection relationships between them. Among them, the flight dataset contains passenger flight itineraries in the United States in 2019, which comes from the U.S. Bureau of Transportation Statistics database. Nodes represent airport locations and edges represent passengers' travel between airports. The shipping dataset includes global shipping trajectories, with nodes representing ports and edges representing ship routes between these ports. The ship simulation dataset simulates the shipping movements of 100,000 ships sailing in a grid of 100 ports, totaling 10 million times. Table 1 shows the statistical information of the dataset.

[0166] Table 1:

[0167]

[0168] Link prediction experiments were conducted on these three datasets. For the link prediction task, the area under the precision-recall curve (AUPRC) was used as the evaluation metric. Table 2 shows the link prediction experimental results.

[0169] Table 2:

[0170]

[0171] The experimental results show the excellent performance of the proposed solution on the three traffic trajectory datasets, achieving the best results on all three datasets. This shows that the proposed method accurately captures the high-order dependencies in the traffic trajectory and uses it to generate a highly expressive node embedding representation. This proves the effectiveness of the present invention in predicting itineraries (ship routes / passenger flights, etc.) in traffic networks.

[0172] The flight data set verifies the performance of the present invention in flight path prediction. The shipping data set contains global shipping trajectory data, and the ability of the present invention in ship route prediction is verified using this data set. The simulated ship data set provides a controlled experimental environment for further verifying the effectiveness and robustness of the present invention.

[0173] According to the disclosure and teaching of the above description, those skilled in the art to which the present invention belongs may also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although some specific terms are used in this specification, these terms are only for the convenience of description and do not constitute any limitation to the present invention.

Claims

1. A high-order dependency embedding method for traffic sequence data, characterized in that: The following steps are involved: S1. Obtain the sequence data of traffic trajectories and build a high-order network. The specific method is as follows: Obtain the sequence data of traffic trajectories, construct a first-order network through the sequence data, obtain the node transition probability on the first-order network, and obtain the transition probability distribution of the first-order node; then calculate the transition probability distribution of higher-order nodes; use the KL divergence to compare the difference in the transition probability distribution of two adjacent order nodes, and identify high-order dependencies by setting a dynamic threshold and comparing it with the KL divergence; According to the high-order dependencies, high-order nodes are created, and the newly created high-order nodes are connected through edges to form a high-order network; S2: Based on the high-order network constructed in S1, random walks are performed on the high-order network to obtain the initial node embedding representation; S3. Perform multi-level contrastive learning in high-order networks to optimize node embedding representation; the specific method is: Construct positive example pairs and negative example pairs in the high-order network, and construct positive example pairs and negative example pairs in the first-order network; then, based on the positive example pairs, negative example pairs and loss functions in the corresponding hierarchical networks, respectively optimize the high-order nodes in the high-order network and the first-order nodes in the first-order network to obtain optimized node embedding representations; S4. Aggregate the embedding representations of the first-order nodes and the higher-order nodes in the high-order network to obtain the node embedding representation of the first-order nodes.

2. According to claim 1, a high-order dependency embedding method for traffic sequence data is characterized in that: The previous node of the node is the predecessor node, and the subsequent node of the node is the successor node; in a high-order network, the constructed positive example pairs are high-order network positive example pairs, and the constructed negative example pairs are high-order network negative example pairs; in a first-order network, the constructed positive example pairs are first-order network positive example pairs, and the constructed negative example pairs are first-order network negative example pairs.

3. The high-order dependency embedding method for traffic sequence data according to claim 2 is characterized in that: In S1, the sequence data of traffic trajectories is obtained to identify high-order dependencies; specifically, S1.

1. Setting according to the sequence data of traffic trajectory is a discrete random variable sequence, and a first-order network following the Markov property is established by the discrete random variable sequence, that is, ; Among them, p represents the transition probability of the node, X represents the data in the discrete random variable sequence, t represents different moments, and s represents different first-order nodes in the first-order network; S1.

2. Calculate the first-order node S according to the first-order network established in S1.

1. t The transition probability to the next node in a first-order network is: ； in, Represents the first-order node S t The transition probability in a first-order network is Represents the first-order node S t The weight of the directed edge to node S, j is the first-order node S t The set of nodes that can be directly reached by the directed edges of ; Calculate the transition probability of a high-order node to the next node in a high-order network: ； in, Represents a high-order node Transition probabilities in high-order networks, Represents a high-order node The weight of the directed edge to node S, i is a high-order node The set of nodes that can be directly reached by the directed edges of ; S1.3, according to the transition probability of all nodes obtained in S1.2, set the k-order high-order node The transition probability distribution of walking to the next node in the traffic trajectory is , where S k represents a high-order node of order k; d k represents the corresponding transition probability distribution, p K represents the transition probability, N represents the number of nodes connected to the high-order node in the high-order network; If in a high-order network, there are k+1-order high-order nodes , which corresponds to the probability distribution ; According to the probability distribution, the KL divergence between the high-order node and the first-order node transfer probability distribution is calculated, and the d k and d k+1 The difference between the two transition probability distributions: Set a dynamic threshold by comparing the KL divergence with the dynamic threshold, that is: If D KL is greater than the dynamic threshold, the high-order node is introduced. KL If it is less than the dynamic threshold, the high-order node will not be introduced; S1.4, connect the introduced high-order node with its previous and subsequent nodes through edges, generate new edges, and delete the edge connecting the original first-order node to its previous first-order node; remove the nodes without incoming edges; S1.

5. Continue to adjust the successor nodes of the introduced high-order nodes by comparing the KL divergence and the dynamic threshold size to build a high-order network; if the successor node depends on the current high-order node, re-establish the connection.

4. The high-order dependency embedding method for traffic sequence data according to claim 3 is characterized in that: The specific method for calculating KL divergence based on probability distribution is: ； Where D KL is the KL divergence.

5. The high-order dependency embedding method for traffic sequence data according to claim 3 is characterized in that: The dynamic threshold is defined as follows: ； Among them, Ext represents one of the nodes, Support represents the in-degree of node Ext, is an adjustable hyperparameter; k+1 represents the order.

6. The high-order dependency embedding method for traffic sequence data according to claim 1 is characterized in that: In S2, a random walk is performed on a high-order network, specifically including: By simulating the wandering process of random walkers in a high-order network, a node sequence is obtained, which includes the structural information of the high-order network. The structural information of the high-order network is encoded by the Skip-Gram model to obtain the initial node embedding representation.

7. The high-order dependency embedding method for traffic sequence data according to claim 2 is characterized in that: In S3, high-order and first-order multi-level comparative learning is performed on the node embedding representation in the high-order network to obtain an optimized node embedding representation; specifically, it includes: S3.

1. At the level of high-order networks, the high-order nodes and their predecessor nodes in the node sequence obtained by random walk are designated as positive example pairs of the high-order network; for each high-order node, its non-neighbor nodes are randomly sampled to obtain negative example nodes, and the high-order node and its negative example nodes are designated as negative example pairs; Through contrastive learning, the similarity of positive pairs is maximized and the similarity of negative pairs is minimized. At the level of this high-order network, the loss function of contrastive learning is as follows: ； in, is a high-order loss function, X is a set of high-order nodes in a high-order network, and x i is x j Positive node, x k is a negative example node, f(x i ) represents x i Node embedding representation; sim is the similarity function, which is calculated using dot product; S3.

2. At the level of the first-order network, the first-order nodes connected by directed edges constitute node pairs. The node pairs are sorted according to the weights of the directed edges, and the nodes ranked in the top 25-35% of the weight ranking are designated as the first-order network positive example pairs; On the first-order network, randomly sample its non-neighbor nodes to obtain the first-order network negative example pairs; The loss function of the first level is as follows: ； in, is a first-order loss function, Z is a set of first-order nodes in a first-order network, z u Yes v Positive node, z m is a negative example node, f(z u ) indicates z u Node embedding representation of ; S3.

3. The loss function of the entire model is obtained by using the contrast learning mechanism of the high-order network and the first-order network, and by accumulating their corresponding loss functions, the overall loss function of the model is obtained: ； in, Represents the overall loss function. The model is optimized by minimizing the overall loss function to obtain the optimized node embedding representation.

8. The high-order dependency embedding method for traffic sequence data according to claim 1 is characterized in that: The S4 specifically includes assigning weights according to the frequency of occurrence of each high-order node, multiplying the embedding representation of the high-order node by its corresponding weight, and then performing weighted summation on the weighted node embedding representations in the high-order network to obtain the final embedding representation of the first-order node.

9. A high-order dependency embedding system for traffic sequence data, characterized in that: The method is applied in a high-order dependency embedding method for traffic sequence data as described in any one of claims 1 to 8.

Images