Traffic status prediction method and device

By integrating the characteristics of the main transportation network and the auxiliary transportation network, predicting the traffic state, the problem of insufficient prediction accuracy in the prior art is solved, and higher prediction accuracy is achieved.

CN118692243BActive Publication Date: 2024-12-03HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411186889.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-03
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

The existing traffic state prediction methods are difficult to accurately predict the future traffic state of nodes in the traffic network, especially when considering the relationship between the main traffic network and the auxiliary traffic network, the prediction accuracy is insufficient.

Method used

By obtaining the autocorrelation characteristics in the primary traffic network and the cross-correlation characteristics in the secondary traffic network, feature fusion is performed to predict traffic states. The specific steps include: obtaining autocorrelation features based on the first connectivity relationship between the primary node and the target node data, obtaining cross-correlation features based on the second connectivity relationship between the primary node and the secondary node and the target node data, and passing the features The fusion layer performs weighted summing processing to obtain fusion-related features to predict the traffic state of the next prediction cycle.

Benefits of technology

The accuracy of traffic state prediction is improved, and the traffic state of nodes in the traffic network can be predicted more accurately while taking into account the relationship between the main traffic network and the auxiliary traffic network.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118692243B_ABST
    Figure CN118692243B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a method and device for predicting traffic status, which relates to the field of intelligent transportation technology. The specific scheme is: obtaining the autocorrelation characteristics of each main node in the main traffic network; obtaining the cross-correlation characteristics of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary traffic network and the target node data of each auxiliary node; mixing and fusing the autocorrelation characteristics and cross-correlation characteristics of each main node to obtain the fused correlation characteristics of each main node; predicting the traffic status of each main node in the next prediction cycle based on the fused correlation characteristics of each main node. The traffic status prediction scheme provided by the embodiment of the present application can not only predict the traffic status of nodes in the traffic network, but also improve the prediction accuracy compared to predicting the traffic status by only considering the correlation relationship between the nodes of the traffic network itself.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method and device for predicting traffic status. Background Art

[0002] As cities continue to expand, it becomes increasingly important to ensure smooth traffic networks in cities. In order to help managers better manage traffic, the traffic management platform hopes to be able to predict the traffic status of each node in the traffic network, so as to discover in advance the nodes in the traffic network that may have the risk of traffic congestion in the future, so that managers can make better overall arrangements to alleviate congestion and ensure smooth traffic.

[0003] Therefore, a traffic state prediction scheme is needed to predict the traffic state of nodes in the traffic network. Summary of the invention

[0004] The purpose of the embodiment of the present application is to provide a traffic state prediction method and device to predict the traffic state of nodes in a traffic network. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a traffic state prediction method, the method comprising:

[0006] Obtaining the autocorrelation characteristics of each main node in the main transportation network, wherein the autocorrelation characteristics of each main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, wherein the target node data includes: node data of the node in a current prediction period and at least one adjacent historical period before the current prediction period;

[0007] Based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node, the mutual correlation characteristics of each main node are obtained, wherein the main transportation network and the auxiliary transportation network are heterogeneous networks;

[0008] The autocorrelation features and cross-correlation features of each main node are mixed and fused to obtain the fused correlation features of each main node;

[0009] Based on the fusion-related features of each main node, the traffic status of each main node in the next prediction cycle is predicted.

[0010] In one embodiment of the present application, the autocorrelation features and cross-correlation features of each master node are mixed and fused to obtain the fused correlation features of each master node, including:

[0011] The autocorrelation features and cross-correlation features of each main node are processed in the following manner to obtain the fused correlation features of each main node:

[0012] The autocorrelation feature and the cross-correlation feature of the main node are concatenated to obtain the first intermediate feature;

[0013] Performing a linear transformation on the intermediate feature to obtain a second intermediate feature;

[0014] Based on the second intermediate feature, obtaining weight coefficients of the autocorrelation feature and the cross-correlation feature of the master node;

[0015] Based on the obtained weight coefficient, the autocorrelation features and cross-correlation features of the main node are weighted and summed to obtain the fused correlation features of the main node.

[0016] In one embodiment of the present application, the obtaining of the mutual correlation features of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node includes:

[0017] According to the target node data of each auxiliary node in the auxiliary transportation network, the time domain characteristics of each auxiliary node are obtained;

[0018] Based on the second connectivity relationship between each main node and each auxiliary node, a nonlinear transformation is performed on the time domain features of each auxiliary node to obtain the cross-correlation features of each main node.

[0019] In one embodiment of the present application, the nonlinear transformation of the time domain features of each auxiliary node based on the second connectivity relationship between each primary node and each auxiliary node to obtain the mutual correlation features of each primary node includes:

[0020] Based on the third intermediate feature, a time domain feature of each auxiliary node is linearly transformed to obtain a fourth intermediate feature, wherein the third intermediate feature is a feature obtained by normalizing elements in an adjacency matrix in units of preset processing units, the adjacency matrix represents a second connectivity relationship between each main node and each auxiliary node, and the processing unit is a matrix row or a matrix column;

[0021] The fourth intermediate feature is subjected to nonlinear transformation to obtain the cross-correlation feature of each main node.

[0022] In one embodiment of the present application, the step of obtaining the autocorrelation characteristics of each main node in the main traffic network includes:

[0023] According to the target node data of each main node in the main transportation network, the time domain characteristics of each main node are obtained;

[0024] Based on the first connectivity relationship, a nonlinear transformation is performed on the time domain features of each master node to obtain the autocorrelation features of each master node.

[0025] In one embodiment of the present application, when the target node data of the node to be processed includes multimodal data, the time domain features of the node to be processed are obtained in the following manner, wherein the node to be processed is: a primary node or a secondary node:

[0026] Normalizing each modality data in the target node data of the node to be processed to obtain first intermediate data corresponding to each modality;

[0027] Mapping the first intermediate data corresponding to each modality to the same embedding space respectively to obtain the second intermediate data corresponding to each modality;

[0028] splicing the second intermediate data corresponding to each modality to obtain multimodal embedded data of the node to be processed;

[0029] Perform time domain feature extraction on the multimodal embedded data to obtain the time domain features of the node to be processed.

[0030] In one embodiment of the present application, mapping the first intermediate data corresponding to each modality to the same embedding space to obtain the second intermediate data corresponding to each modality includes:

[0031] According to the feature extraction method corresponding to each modality, feature extraction is performed on the first intermediate data corresponding to each modality respectively to obtain the data features corresponding to each modality as the second intermediate data corresponding to each modality in the same embedding space.

[0032] In one embodiment of the present application, the predicting of the traffic status of each master node in the next prediction period based on the fusion-related features of each master node includes:

[0033] For each main node, a linear transformation is performed on the fusion-related features of the main node to obtain a predicted value representing the traffic conditions of the main node in the next prediction period.

[0034] In one embodiment of the present application, the step of obtaining the autocorrelation characteristics of each main node in the main traffic network includes:

[0035] Input the target node data of each main node in the main traffic network into the first data embedding layer in the pre-trained traffic state prediction model, so that the first data embedding layer normalizes the modal data in the target node data of each main node, and maps the normalized data corresponding to each modality to the same embedding space, wherein the traffic state prediction model further includes a first splicing layer, a first time convolution layer, a graph convolution layer, a second data embedding layer, a second splicing layer, a second time convolution layer, an interactive information convolution layer, a feature fusion layer and a linear output layer;

[0036] Inputting the output result of the first data embedding layer into the first splicing layer, so that the first splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedded data of each master node;

[0037] Inputting the multimodal embedded data of each master node into the first time convolution layer, so that the first time convolution layer extracts time domain features from the multimodal embedded data of each master node to obtain time domain features of each master node;

[0038] Inputting the time domain features of each master node into the graph convolution layer, so that the graph convolution layer performs nonlinear transformation on the time domain features of each master node to obtain the autocorrelation features of each master node;

[0039] The obtaining of the mutual correlation features of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node includes:

[0040] Inputting the target node data of each auxiliary node in the auxiliary traffic network into the second data embedding layer, so that the second data embedding layer normalizes each modality data in the target node data of each auxiliary node, and maps the normalized data corresponding to each modality to the same embedding space;

[0041] Inputting the output result of the second data embedding layer into the second splicing layer, so that the second splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedding data of each auxiliary node;

[0042] Inputting the multimodal embedding data of each auxiliary node into the second temporal convolutional layer, so that the second temporal convolutional layer extracts time domain features from the multimodal embedding data of each auxiliary node to obtain time domain features of each auxiliary node;

[0043] Inputting the time domain features of each auxiliary node into the mutual information convolution layer, so that the mutual information convolution layer performs nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node;

[0044] The step of mixing and fusing the autocorrelation features and the cross-correlation features of each master node to obtain the fused correlation features of each master node includes:

[0045] Inputting the autocorrelation features and cross-correlation features of each main node into the feature fusion layer, so that the feature fusion layer mixes and fuses the autocorrelation features and cross-correlation features of each main node to obtain fused correlation features of each main node;

[0046] The predicting of the traffic status of each main node in the next prediction cycle based on the fusion-related features of each main node includes:

[0047] The fusion-related features of each main node are input into the linear output layer, so that the linear output layer performs a linear transformation on the fusion-related features of each main node to obtain a predicted value representing the traffic conditions of each main node in the next prediction period.

[0048] In one embodiment of the present application, the interactive information convolution layer is constructed based on the Inter-GCN network obtained by deforming the graphics processing network GCN;

[0049] and / or

[0050] The feature fusion layer is constructed based on the attention aggregation weight network.

[0051] In a second aspect, an embodiment of the present application provides a traffic state prediction device, the device comprising:

[0052] A first obtaining module is used to obtain the autocorrelation characteristics of each main node in the main traffic network, wherein the autocorrelation characteristics of each main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, and the target node data includes: node data of the node in the current prediction period and at least one adjacent historical period before the current prediction period;

[0053] a second obtaining module, configured to obtain the mutual correlation characteristics of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node, wherein the main transportation network and the auxiliary transportation network are heterogeneous networks;

[0054] The feature fusion module is used to mix and fuse the autocorrelation features and cross-correlation features of each main node to obtain the fused correlation features of each main node;

[0055] The state prediction module is used to predict the traffic state of each main node in the next prediction cycle based on the fusion-related features of each main node.

[0056] In one embodiment of the present application, the feature fusion module is specifically used to: process the autocorrelation features and cross-correlation features of each main node in the following manner to obtain the fused related features of each main node: concatenate the autocorrelation features and cross-correlation features of the main node to obtain a first intermediate feature; perform a linear transformation on the intermediate feature to obtain a second intermediate feature; based on the second intermediate feature, obtain the weight coefficient of the autocorrelation features and cross-correlation features of the main node; based on the obtained weight coefficient, perform weighted summation processing on the autocorrelation features and cross-correlation features of the main node to obtain the fused related features of the main node.

[0057] In one embodiment of the present application, the second acquisition module includes: a first acquisition submodule, used to obtain the time domain characteristics of each auxiliary node according to the target node data of each auxiliary node in the auxiliary transportation network; a first transformation submodule, used to perform nonlinear transformation on the time domain characteristics of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node, so as to obtain the mutual correlation characteristics of each main node.

[0058] In one embodiment of the present application, the first transformation submodule is specifically used to: perform a linear transformation on the time domain features of each auxiliary node based on the third intermediate feature to obtain a fourth intermediate feature, wherein the third intermediate feature is: a feature obtained by normalizing the elements in the adjacency matrix based on a preset processing unit, the adjacency matrix represents the second connectivity relationship between each main node and each auxiliary node, and the processing unit is a matrix row or a matrix column; perform a nonlinear transformation on the fourth intermediate feature to obtain a mutual correlation feature of each main node.

[0059] In one embodiment of the present application, the first acquisition module is specifically used to: obtain the time domain characteristics of each main node according to the target node data of each main node in the main transportation network; based on the first connectivity relationship, perform nonlinear transformation on the time domain characteristics of each main node to obtain the autocorrelation characteristics of each main node.

[0060] In one embodiment of the present application, when the target node data of the node to be processed contains multimodal data, the time domain features of the node to be processed are obtained in the following manner, wherein the node to be processed is: a main node or an auxiliary node: normalizing each modal data in the target node data of the node to be processed respectively to obtain the first intermediate data corresponding to each modality; mapping the first intermediate data corresponding to each modality to the same embedding space respectively to obtain the second intermediate data corresponding to each modality; splicing the second intermediate data corresponding to each modality to obtain the multimodal embedded data of the node to be processed; performing time domain feature extraction on the multimodal embedded data to obtain the time domain features of the node to be processed.

[0061] In one embodiment of the present application, the first intermediate data corresponding to each modality are respectively mapped to the same embedding space to obtain the second intermediate data corresponding to each modality, including: according to the feature extraction method corresponding to each modality, feature extraction is performed on the first intermediate data corresponding to each modality to obtain the data features corresponding to each modality as the second intermediate data corresponding to each modality in the same embedding space.

[0062] In one embodiment of the present application, the state prediction module is specifically used to: for each main node, perform a linear transformation on the fusion-related features of the main node to obtain a predicted value representing the traffic condition of the main node in the next prediction period.

[0063] In one embodiment of the present application, the first obtaining module is specifically used to:

[0064] Input the target node data of each main node in the main traffic network into the first data embedding layer in the pre-trained traffic state prediction model, so that the first data embedding layer normalizes the modal data in the target node data of each main node, and maps the normalized data corresponding to each modality to the same embedding space, wherein the traffic state prediction model further includes a first splicing layer, a first time convolution layer, a graph convolution layer, a second data embedding layer, a second splicing layer, a second time convolution layer, an interactive information convolution layer, a feature fusion layer and a linear output layer;

[0065] Inputting the output result of the first data embedding layer into the first splicing layer, so that the first splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedded data of each master node;

[0066] Inputting the multimodal embedded data of each master node into the first time convolution layer, so that the first time convolution layer extracts time domain features from the multimodal embedded data of each master node to obtain time domain features of each master node;

[0067] Inputting the time domain features of each master node into the graph convolution layer, so that the graph convolution layer performs nonlinear transformation on the time domain features of each master node to obtain the autocorrelation features of each master node;

[0068] The second obtaining module is specifically used for:

[0069] Inputting the target node data of each auxiliary node in the auxiliary traffic network into the second data embedding layer, so that the second data embedding layer normalizes each modality data in the target node data of each auxiliary node, and maps the normalized data corresponding to each modality to the same embedding space;

[0070] Inputting the output result of the second data embedding layer into the second splicing layer, so that the second splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedding data of each auxiliary node;

[0071] Inputting the multimodal embedding data of each auxiliary node into the second temporal convolutional layer, so that the second temporal convolutional layer extracts time domain features from the multimodal embedding data of each auxiliary node to obtain time domain features of each auxiliary node;

[0072] Inputting the time domain features of each auxiliary node into the mutual information convolution layer, so that the mutual information convolution layer performs nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node;

[0073] The feature fusion module is specifically used for:

[0074] Inputting the autocorrelation features and cross-correlation features of each main node into the feature fusion layer, so that the feature fusion layer mixes and fuses the autocorrelation features and cross-correlation features of each main node to obtain fused correlation features of each main node;

[0075] The state prediction module is specifically used for:

[0076] The fusion-related features of each main node are input into the linear output layer, so that the linear output layer performs a linear transformation on the fusion-related features of each main node to obtain a predicted value representing the traffic conditions of each main node in the next prediction period.

[0077] In one embodiment of the present application, the interactive information convolution layer is constructed based on the Inter-GCN network obtained by deforming the graphics processing network GCN;

[0078] and / or

[0079] The feature fusion layer is constructed based on the attention aggregation weight network.

[0080] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0081] Memory, used to store computer programs;

[0082] The processor is used to implement the method described in the first aspect when executing the program stored in the memory.

[0083] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0084] It can be seen from the above that when the scheme provided by the embodiment of the present application is used to predict the traffic state, the autocorrelation characteristics of the main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, taking into account the association relationship between the nodes of the main traffic network itself, and the cross-correlation characteristics are obtained based on the second connectivity relationship between the main node and the auxiliary node and the target node data of each auxiliary node, taking into account the association relationship between the main node of the main traffic network and the auxiliary node of the auxiliary traffic network. In this way, the autocorrelation characteristics and cross-correlation characteristics of each main node are mixed and fused to obtain the fused correlation characteristics of each main node. The fused correlation characteristics can be regarded as the characteristics of the main node obtained under the condition of comprehensive consideration of the above two association relationships. In this way, predicting the traffic state based on the fused correlation characteristics can also make accurate predictions under the condition of comprehensive consideration of the above two association relationships. It can be seen that the traffic state prediction scheme provided by the embodiment of the present application can not only realize the prediction of the traffic state of the nodes in the traffic network, but also, compared with only considering the association relationship between the nodes of the traffic network itself to predict the traffic state, this scheme can also improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0086] Figure 1 A schematic diagram of the structure of a network node provided in an embodiment of the present application;

[0087] Figure 2 A schematic diagram of a flow chart of a first traffic state prediction method provided in an embodiment of the present application;

[0088] Figure 3 A flow chart of a second traffic state prediction method provided in an embodiment of the present application;

[0089] Figure 4 A schematic diagram of the structure of a traffic state prediction model provided in an embodiment of the present application;

[0090] Figure 5 A schematic diagram of the structure of a traffic status prediction device provided in an embodiment of the present application;

[0091] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0093] First, the transportation network mentioned in the solution provided in the embodiment of the present application is introduced.

[0094] For example, Figure 1 A schematic diagram of the structure of a transportation network. Figure 1 There are three network nodes p1, p2, and p3, among which there is a path from p1 to p2 and a path from p2 to p1 between p1 and p2, there is a path from p2 to p3 between p2 and p3, and there is no path between p1 and p3. The connectivity relationship between these three nodes indicates that there is a path from p1 to p2, a path from p2 to p1, and a path from p2 to p3, but there is no path from p3 to p2, from p1 to p3, and from p3 to p1.

[0095] In one embodiment of the present application, the above-mentioned connectivity relationship can be represented in the form of an adjacency matrix, and the transportation network can be represented by a network directed graph G including network nodes N, a path set L between network nodes, and an adjacency matrix A, G=(N, L, A).

[0096] For the adjacency matrix, one of the rows and columns of the adjacency matrix can represent the starting point of the path, and the other can represent the end point of the path.

[0097] When rows represent the starting points of paths and columns represent the end points of paths, the element value of the matrix element in the adjacency matrix represents whether there is a path from the network node corresponding to the row where the matrix element is located to the network node corresponding to the column where the matrix element is located. If the matrix element is 1, it means that there is a path, and if the matrix element is 0, it means that there is no path.

[0098] For example, for the above Figure 1For the three network nodes shown, there is a path from p1 to p2, which means that the element value of the matrix element in the first row and second column of the adjacency matrix is ​​1. There is a path from p2 to p1, which means that the element value of the matrix element in the second row and first column of the adjacency matrix is ​​1. There is a path from p2 to p3, which means that the element value of the matrix element in the second row and third column of the adjacency matrix is ​​1. There is no path from p3 to p2, which means that the element value of the matrix element in the third row and second column of the adjacency matrix is ​​0. There is no path from p1 to p3, which means that the element value of the matrix element in the first row and third column of the adjacency matrix is ​​0. There is no path from p3 to p1, which means that the element value of the matrix element in the third row and first column of the adjacency matrix is ​​0.

[0099] In addition, since there is no path pointing from a network node to itself, the element value of the matrix elements with the same number of rows and columns in the adjacency matrix is ​​0.

[0100] In summary, the connectivity relationship between the three network nodes can be expressed as follows:

[0101] ;

[0102] When columns represent path starting points and rows represent path end points, the element value of a matrix element in the adjacency matrix represents whether there is a path from the network node corresponding to the column where the matrix element is located to the network node corresponding to the row where the matrix element is located.

[0103] Next, a traffic status prediction method and device provided in the embodiments of the present application are described one by one.

[0104] See also Figure 2 , a flow chart of a traffic status prediction method is provided. In this embodiment, the method includes the following steps S201-S204.

[0105] Step S201: Obtain the autocorrelation characteristics of each main node in the main transportation network.

[0106] The main traffic network is the traffic network where the network nodes to be predicted are located, and the network nodes included in the main traffic network are called main nodes.

[0107] For example, a city's transportation network may include a bus stop network, a subway station network, and a road network fork network. If the traffic status of a bus stop needs to be predicted, the bus stop network is the main transportation network, and each bus stop contained in the bus stop network is the main node.

[0108] The autocorrelation feature of each master node is obtained based on the first connectivity relationship between the master nodes and the target node data of each master node.

[0109] If there is a path between two network nodes, it means that the two network nodes are connected. If there is no path, it means that the two network nodes are not connected. The connectivity relationship represents whether there is a path between the two network nodes, or whether they are connected.

[0110] The first connectivity relationship indicates whether any two nodes in each master node are connected.

[0111] The target node data includes: node data of the node in the current prediction period and at least one adjacent historical period before the current prediction period.

[0112] The at least one adjacent historical period may be one or more historical periods adjacent to the current prediction period.

[0113] The above-mentioned node data refers to the data generated at the network nodes. The network nodes of different transportation networks are different, and the data generated may also be different. For example, for a bus stop network, the node data of the bus stop can be the bus traffic volume, passenger flow, departure volume, etc.; for a road network fork network, the node data of the road intersection can be the image data of the camera installed at the intersection, the point cloud data collected by the radar, the vehicle quantity data obtained by the coil detection, etc.

[0114] In addition, the node data may be data generated periodically on the network nodes. For example, for a bus stop, the node data may be the vehicle flow and passenger flow every half hour.

[0115] Specifically, the node data of network nodes in the traffic network can be generated periodically. Every time a network node generates a node data, the traffic management platform can synchronously obtain the node data. In this way, when the traffic management platform needs to make a traffic status prediction, the traffic management platform can obtain the autocorrelation characteristics of each main node based on the node data of the current prediction period of each main node at the current moment and at least one adjacent historical period before the current prediction period, as well as the first connectivity relationship obtained in advance.

[0116] For example, when the traffic management platform receives a prediction instruction from the manager, the traffic management platform can respond to the prediction instruction and obtain the autocorrelation characteristics of each main node based on the node data of the current prediction period in which each main node is at the current moment and at least one adjacent historical period before the current prediction period, as well as the pre-obtained first connectivity relationship.

[0117] For another example, the traffic management platform can synchronously obtain the node data generated by each main node once, and use the obtained node data as the node data of each main node in the current prediction period, and obtain the autocorrelation characteristics of each main node based on the node data and the node data of at least one adjacent historical period before it, as well as the above-mentioned first connectivity relationship.

[0118] Based on the target node data of each master node and the above-mentioned first connectivity relationship, the autocorrelation characteristics of each master node can be obtained by any one of the following two implementation methods.

[0119] In the first implementation method, the time domain characteristics of each main node can be obtained according to the target node data of each main node in the main traffic network; based on the first connectivity relationship, the time domain characteristics of each main node are nonlinearly transformed to obtain the autocorrelation characteristics of each main node.

[0120] For each master node, the target node data of the master node includes: the node data of the master node in the current prediction cycle and at least one adjacent historical cycle before the current prediction cycle. It can be seen that the target node data of the master node includes node data of at least two adjacent cycles, and the node data of adjacent cycles are correlated in the time domain. Therefore, the time domain feature extraction can be performed on the target node data of the master node to obtain the time domain feature of the master node. After the time domain feature extraction is performed on the target node data of each master node, the time domain feature of each master node can be obtained.

[0121] For example, the target node data of each master node can be input into TCN (Temporal Convolutional Network), and TCN can be used to perform time domain convolution processing on the target node data of each master node to obtain the time domain characteristics of each master node. After TCN performs time domain convolution processing on the target node data of each master node, it outputs the time domain characteristics of each master node.

[0122] After the time domain features of each master node are obtained, the time domain features of each master node may be nonlinearly transformed based on the first connectivity relationship to obtain the autocorrelation features of each master node.

[0123] Since the first connectivity relationship is the connectivity relationship between each main node, for each main node, the first connectivity relationship includes the connectivity relationship between the main node and the associated nodes that have a path with the main node. Therefore, based on the first connectivity relationship, the time domain characteristics of each main node are nonlinearly transformed, and the obtained autocorrelation characteristics of each main node include: the main relationship characteristics that characterize the association relationship between each main node and the associated nodes of the main node.

[0124] Specifically, the parameter representing the first connectivity relationship may be multiplied by the time domain feature of each master node, and then the multiplication result may be subjected to nonlinear transformation to obtain the autocorrelation feature of each master node.

[0125] For example, the first connectivity relationship can be represented in the form of an adjacency matrix. The adjacency matrix representing the first connectivity relationship is multiplied with the time domain features of each main node, and the multiplication result is nonlinearly transformed to obtain the autocorrelation features of each main node.

[0126] In one embodiment of the present application, the autocorrelation characteristics of each master node can be obtained by the following expression:

[0127] ;

[0128] in, Represents the autocorrelation characteristics of each master node, represents a nonlinear transformation operation, represents the first connectivity relationship, Represents the time domain characteristics of each master node, represents the first preset parameter, Indicates the second preset parameter.

[0129] Included in the master relationship feature is: a master relationship feature that characterizes the relationship between each master node and the master node associated with the master node. ,in, is an integer between 1 and m, where m is the total number of master nodes included in the main transportation network, and the associated master nodes of each master node are other master nodes that have paths with the master node.

[0130] The above expression can also be understood as: using the first connectivity relationship as the convolution kernel, first convolving the time domain features of each main node, then linearly transforming the features obtained by the convolution, and finally performing a nonlinear transformation on the features after the linear transformation to obtain the autocorrelation features of each main node.

[0131] The above main relationship characteristics can be expressed as follows:

[0132] ;

[0133] in, represents the main relationship feature corresponding to the i-th main node, It represents the master relationship feature between the i-th master node and the j-th associated master node of the master node, and k is the total number of associated master nodes with the i-th master node.

[0134] In another embodiment of the present application, based on the first connectivity relationship, the nonlinear transformation of the time domain features of each master node can be implemented using a pre-trained first feature transformation model. After obtaining the time domain features of each master node, the time domain features of each master node can be input into the first feature transformation model to obtain the autocorrelation features of each master node output by the model.

[0135] In addition, the above , It can also be determined based on the model parameters in the first feature transformation model.

[0136] In this implementation, the time domain characteristics of each main node are obtained according to the target node data of each main node in the main traffic network, and then the time domain characteristics of each main node are nonlinearly transformed based on the first connectivity relationship to obtain the autocorrelation characteristics of each main node. In this way, the autocorrelation characteristics of each main node not only reflect the connectivity between the main nodes, but also reflect the time domain correlation of the node data of each main node in different periods, so that the obtained autocorrelation characteristics are more accurate. In this way, predicting the traffic status based on the autocorrelation characteristics can improve the prediction accuracy.

[0137] In the second implementation method, a feature extraction model for extracting autocorrelation features of network nodes in a traffic network can be pre-trained. When the traffic management platform needs to predict traffic status, the target node data of each main node is input into the feature extraction model, so that the feature extraction model performs model inference based on the above-mentioned first connectivity relationship and the input target node data, and outputs the autocorrelation features of each main node.

[0138] In one embodiment of the present application, when using a feature extraction model to obtain the autocorrelation features of each main node, the traffic management platform can synchronously obtain the node data generated by each main node once, and input the obtained node data into the feature extraction model, so that the feature extraction model uses the newly input node data as the node data of the current prediction period and the previously input node data as the node data of the adjacent historical period before the current prediction period to perform model inference, and output the autocorrelation features of each main node.

[0139] Step S202: based on the second connectivity relationship between each primary node and each secondary node in the secondary transportation network and the target node data of each secondary node, the mutual correlation characteristics of each primary node are obtained.

[0140] Among them, the auxiliary transportation network is other transportation networks besides the main transportation network, which is used to assist in predicting the traffic status of each main node in the main transportation network. The network nodes included in the auxiliary transportation network are called auxiliary nodes.

[0141] For example, for the bus stop network, subway station network, and road network bifurcation network, if the bus stop network is the main transportation network, then at least one of the remaining subway station networks and road network bifurcation networks can be used as an auxiliary transportation network. If the subway station network is used as the auxiliary transportation network, then each subway station contained in the subway station network is an auxiliary node.

[0142] The primary transportation network and the secondary transportation network are heterogeneous networks, that is, the primary transportation network and the secondary transportation network are different transportation networks. For example, the primary transportation network is a bus station network, and the secondary transportation network is a subway station network.

[0143] The second connectivity relationship indicates whether the primary node and the secondary node are connected. If there is a path between a primary node and a secondary node, it means that the primary node and the secondary node are connected. If not, it means that the primary node and the secondary node are not connected. For an introduction to the second connectivity relationship, please refer to the above-mentioned concept explanation, which will not be repeated here.

[0144] There may be one or more auxiliary transportation networks. For each auxiliary transportation network, the mutual correlation characteristics of each main node corresponding to the auxiliary transportation network may be obtained based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node.

[0145] The method of obtaining the mutual correlation characteristics of each main node based on the second connectivity relationship and the target node data of each auxiliary node is similar to the method of obtaining the autocorrelation characteristics of each main node based on the first connectivity relationship and the target node data of each main node in the above step S201. Any one of the following two implementation methods can be used to obtain the mutual correlation characteristics of each main node.

[0146] In the first implementation method, the time domain characteristics of each auxiliary node can be obtained according to the target node data of each auxiliary node in the auxiliary transportation network; based on the second connectivity relationship, the time domain characteristics of each auxiliary node are nonlinearly transformed to obtain the transformed characteristics as the cross-correlation characteristics of each auxiliary node.

[0147] For each auxiliary node, the target node data of the auxiliary node includes: the node data of the auxiliary node in the current prediction cycle and at least one adjacent historical cycle before the current prediction cycle. It can be seen that the target node data of the auxiliary node includes node data of at least two adjacent cycles, and the node data of adjacent cycles are correlated in the time domain. Therefore, the target node data of the auxiliary node can be extracted in the time domain to obtain the time domain features of the auxiliary node. After the target node data of each auxiliary node is extracted in the time domain, the time domain features of each auxiliary node can be obtained.

[0148] After the time domain features of each auxiliary node are obtained, the time domain features of each auxiliary node may be nonlinearly transformed based on the second connectivity relationship to obtain the cross-correlation features of each main node.

[0149] Since the second connectivity relationship is the connectivity relationship between each main node and each auxiliary node, for each auxiliary node, the second connectivity relationship includes the connectivity relationship between the auxiliary node and the main node with which there is a path between the auxiliary node. Therefore, based on the second connectivity relationship, the time domain features of each auxiliary node are nonlinearly transformed, and the obtained mutual correlation features of each main node include: auxiliary relationship features that characterize the association relationship between each main node and the auxiliary nodes associated with the main node.

[0150] Specifically, the parameter representing the second connectivity relationship may be multiplied by the time domain feature of each auxiliary node, and then a nonlinear transformation may be performed on the multiplication result to obtain the cross-correlation feature of each main node.

[0151] For example, the second connectivity relationship can be represented in the form of an adjacency matrix. The adjacency matrix representing the second connectivity relationship is multiplied with the time domain features of each auxiliary node, and a nonlinear transformation is performed on the multiplication result to obtain the mutual correlation features of each main node.

[0152] This method can also obtain more accurate cross-correlation features, so that predicting traffic conditions based on cross-correlation features can improve prediction accuracy.

[0153] In the second implementation method, a feature extraction model for extracting the mutual correlation features of network nodes in the traffic network can be pre-trained. When the traffic management platform needs to predict the traffic status, the target node data of each auxiliary node is input into the feature extraction model, so that the feature extraction model performs model inference based on the above-mentioned second connectivity relationship and the input target node data, and outputs the model inference results as the mutual correlation features of each main node.

[0154] Step S203: The autocorrelation features and cross-correlation features of each master node are mixed and fused to obtain fused correlation features of each master node.

[0155] Specifically, the autocorrelation features and cross-correlation features of each master node may be mixed and fused by any of the following five implementation methods.

[0156] In the first implementation, the weight coefficients of the autocorrelation features and cross-correlation features of each master node can be obtained respectively, so as to perform weighted summation processing on the autocorrelation features and cross-correlation features of each master node, and obtain the weighted summation result as the fusion correlation feature of each master node.

[0157] Among them, the weight coefficients of the autocorrelation features and cross-correlation features of each master node can be pre-set or determined by the methods mentioned in subsequent embodiments. For details, please refer to the subsequent embodiments for obtaining the fusion correlation features of each master node, which will not be described in detail here.

[0158] In the second implementation method, the autocorrelation features and cross-correlation features of each master node can be input into a pre-trained fully connected network to perform full-connection processing on the autocorrelation features and cross-correlation features of each master node to obtain the output results of the fully connected network as the fused correlation features of each master node.

[0159] In a third implementation manner, for each main node, the autocorrelation feature and the cross-correlation feature of the main node may be connected according to a feature dimension to obtain a connected feature as a fused correlation feature of the main node.

[0160] If the size of the autocorrelation feature of the main node in the depth dimension is p, and the size of the cross-correlation feature of the main node in the depth dimension is q, then the autocorrelation feature and the cross-correlation feature of the main node are connected according to the depth dimension to obtain the fused correlation feature of the main node, and the size of the obtained fused correlation feature in the depth dimension is p+q.

[0161] In one embodiment of the present application, when the autocorrelation features and cross-correlation features of the main node are connected according to a feature dimension to obtain a fused related feature, a concat operation can be used to splice the autocorrelation features and cross-correlation features of the main node on a feature dimension, and the spliced ​​features are used as the fused related features.

[0162] In the fourth implementation method, the features can be represented in the form of vectors, where the autocorrelation features in vector form are called autocorrelation feature vectors, and the cross-correlation features in vector form are called cross-correlation feature vectors. When the autocorrelation features and cross-correlation features of the main node are mixed and fused, the autocorrelation feature vectors and cross-correlation feature vectors of the main node can be combined into a feature complex vector as the fused correlation features of the main node.

[0163] For example, the characteristic complex vector z=x+iy, where x is the autocorrelation characteristic vector of the main node, y is the mutual correlation characteristic vector of the main node, and i is an imaginary unit.

[0164] In one embodiment of the present application, when combining the autocorrelation feature vector and the cross-correlation feature vector of the master node into a feature complex vector, an add operation can be used to combine the autocorrelation feature vector and the cross-correlation feature vector of the master node to obtain a feature complex vector.

[0165] In the fifth implementation method, other feature fusion methods can be used, such as weighted fusion, feature fusion based on attention mechanism, etc., to mix and fuse the autocorrelation features and cross-correlation features of each main node to obtain the fused correlation features of each main node.

[0166] In one embodiment of the present application, when there are multiple auxiliary transportation networks, the autocorrelation features of each main node and the corresponding cross-correlation features of each auxiliary transportation network may be mixed and fused to obtain the fused correlation features of each main node.

[0167] Step S204: predicting the traffic status of each main node in the next prediction cycle based on the fusion-related features of each main node.

[0168] The traffic status of a node can be expressed in many forms. For example, the traffic status of a node can be represented by two states: a smooth state and a congested state. For another example, the traffic status of a node can be represented by a state parameter that characterizes the degree of traffic congestion of the node. The larger the parameter value, the more congested the node is, and the smaller the parameter value, the more unobstructed the node is.

[0169] Specifically, based on the fusion-related features of each master node, any one of the following two implementation methods can be used to predict the traffic status of each master node in the next prediction cycle.

[0170] In the first implementation, for each main node, a linear transformation is performed on the fusion-related features of the main node to obtain a predicted value representing the traffic conditions of the main node in the next prediction period.

[0171] Specifically, the fusion-related features of the main node can be linearly transformed according to preset linear transformation parameters to obtain a linear transformation result as a predicted value of the traffic state of the main node in the next prediction period.

[0172] Since the fused related features of the main node not only fuse the autocorrelation features and cross-correlation features of the main node, but are also obtained based on the node data of multiple periods of the main node and the auxiliary node, the fused related features of the main node can reflect the association relationship between the main node and other main nodes and auxiliary nodes associated with it, and can also reflect the time domain correlation of the node data of multiple periods of the main node. Therefore, performing linear transformation on the fused related features of the main node can accurately obtain the predicted value and improve the prediction accuracy.

[0173] In the second implementation, the fusion-related features of each main node can be input into a pre-trained prediction model to obtain the traffic status of each main node in the next prediction cycle output by the prediction model.

[0174] It can be seen from the above that when the scheme provided by the embodiment of the present application is used to predict the traffic state, the autocorrelation characteristics of the main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, taking into account the association relationship between the nodes of the main traffic network itself, and the cross-correlation characteristics are obtained based on the second connectivity relationship between the main node and the auxiliary node and the target node data of each auxiliary node, taking into account the association relationship between the main node of the main traffic network and the auxiliary node of the auxiliary traffic network. In this way, the autocorrelation characteristics and cross-correlation characteristics of each main node are mixed and fused to obtain the fused correlation characteristics of each main node. The fused correlation characteristics can be regarded as the characteristics of the main node obtained under the condition of comprehensive consideration of the above two association relationships. In this way, predicting the traffic state based on the fused correlation characteristics can also make accurate predictions under the condition of comprehensive consideration of the above two association relationships. It can be seen that the traffic state prediction scheme provided by the embodiment of the present application can not only realize the prediction of the traffic state of the nodes in the traffic network, but also, compared with only considering the association relationship between the nodes of the traffic network itself to predict the traffic state, this scheme can also improve the prediction accuracy.

[0175] In the first implementation method of obtaining the mutual correlation characteristics of each main node mentioned in the above step S202, based on the second connectivity relationship, the method mentioned in the following embodiment one or two can be used to perform nonlinear transformation on the time domain characteristics of each auxiliary node to obtain the mutual correlation characteristics of each main node.

[0176] Embodiment 1: Based on the third intermediate feature, a linear transformation is performed on the time domain feature of each auxiliary node to obtain a fourth intermediate feature; and a nonlinear transformation is performed on the fourth intermediate feature to obtain a cross-correlation feature of each main node.

[0177] Among them, the third intermediate feature is: a feature obtained by normalizing the elements in the adjacency matrix with a preset processing unit as a unit, the adjacency matrix represents the second connectivity relationship between each main node and each auxiliary node, and the processing unit is a matrix row or a matrix column.

[0178] It can be seen from the above statement that when obtaining the mutual correlation features of each main node, it is necessary to first normalize the elements in the adjacency matrix representing the second connectivity relationship to obtain the third intermediate feature, and then obtain the fourth intermediate feature based on the third intermediate feature, and then obtain the mutual correlation features of each main node based on the fourth intermediate feature.

[0179] The adjacency matrix representing the second connectivity relationship is described below.

[0180] For the adjacency matrix representing the second connectivity relationship, when the rows of the adjacency matrix represent the starting points of the paths and the columns represent the end points of the paths, each row of the adjacency matrix may correspond to each primary node, each column may correspond to each secondary node, and the element value of the matrix element in the adjacency matrix represents whether there is a path from the primary node corresponding to the row where the matrix element is located to the secondary node corresponding to the column where the matrix element is located. In this case, the above-mentioned processing unit may be a matrix row.

[0181] In the case where the columns of the adjacency matrix represent the starting points of the paths and the rows represent the end points of the paths, the columns of the adjacency matrix may correspond to the primary nodes, the rows may correspond to the secondary nodes, and the element value of the matrix element in the adjacency matrix represents whether there is a path from the primary node corresponding to the column where the matrix element is located to the secondary node corresponding to the row where the matrix element is located. In this case, the above-mentioned processing unit may be a matrix column.

[0182] The following takes the processing unit as an example of a matrix behavior to normalize the elements in the adjacency matrix representing the second connectivity relationship to illustrate two implementation methods of obtaining the third intermediate feature.

[0183] In the first implementation method, for each row of the adjacency matrix, the sum of the element values ​​of the elements in the row can be calculated, and then the element value of each element in the row is divided by the calculated sum of the element values ​​to obtain a new element value for each element, thereby obtaining a new adjacency matrix as the third intermediate feature.

[0184] For example, the third intermediate feature can be obtained by the following expression:

[0185] ;

[0186] in, represents the adjacency matrix representing the second connectivity relationship, represents the operation of finding the sum of the elements in each row of a matrix, Represents the third intermediate feature.

[0187] In the second implementation method, the adjacency matrix can be split according to the matrix rows to obtain the row matrices corresponding to each row, each row matrix is ​​globally normalized, and the processed row matrices are spliced ​​to obtain the spliced ​​matrix as the third intermediate feature.

[0188] The specific implementation method of obtaining the fourth intermediate feature based on the third intermediate feature is described below.

[0189] After obtaining the third intermediate feature according to any one of the above two implementation methods and obtaining the time domain features of each auxiliary node, the third intermediate feature can be multiplied by the time domain features of each auxiliary node, and then the multiplication result can be linearly transformed to obtain the fourth intermediate feature.

[0190] The specific implementation method of obtaining the mutual correlation feature of each master node according to the fourth intermediate feature is described below.

[0191] After obtaining the fourth intermediate feature, a nonlinear transformation may be performed on the fourth intermediate feature to obtain a transformation result as a cross-correlation feature of each main node.

[0192] In one embodiment of the present application, the mutual correlation characteristics of each master node can be obtained by the following expression:

[0193] ;

[0194] in, represents the cross-correlation characteristics of each master node, represents the time domain characteristics of each auxiliary node, represents the third preset parameter, Indicates the fourth preset parameter.

[0195] Included in: auxiliary relationship features that characterize the relationship between each primary node and the secondary nodes associated with the primary node ,in, is an integer between 1 and m, where m is the total number of master nodes included in the main transportation network, and the associated slave nodes of each master node are slave nodes that have paths with the master node.

[0196] The above auxiliary relationship characteristics can be expressed as follows:

[0197] ;

[0198] in, represents the auxiliary relationship feature corresponding to the ith primary node, represents the secondary relationship feature between the i-th primary node and the h-th associated secondary node of the primary node, and u is the total number of associated secondary nodes with the i-th primary node.

[0199] Embodiment 2: The mutual correlation features of each main node are obtained through a pre-trained second feature transformation model.

[0200] Specifically, after obtaining the time domain features of each auxiliary node, the time domain features of each auxiliary node are input into the second feature transformation model, so that the feature transformation model performs a linear transformation on the time domain features of each auxiliary node based on the third intermediate features to obtain the fourth intermediate features, and performs a nonlinear transformation on the fourth intermediate features to output the cross-correlation features of each main node.

[0201] In addition, the above , It can also be determined based on the model parameters in the second feature transformation model.

[0202] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic conditions, the third intermediate feature is a feature obtained by normalizing the adjacency matrix. In this way, various processing is performed on the time domain features of each auxiliary node based on the third intermediate feature, which can speed up the acquisition of the cross-correlation features of each main node, thereby improving the prediction efficiency.

[0203] The target node data of the primary node and the secondary node may be single-mode data or multi-mode data. For example, the target node data of a road intersection may be one or more of image data, point cloud data, and vehicle quantity data.

[0204] When obtaining the time domain characteristics of the primary node and the secondary node, when the target node data of the primary node and the secondary node are single-modal data, the method mentioned in the aforementioned method embodiment can be directly adopted; when the target node data of the primary node and the secondary node are multi-modal data, the time domain characteristics of the primary node and the secondary node can be obtained by the method provided in the following embodiment.

[0205] In one embodiment of the present application, the time domain features of the node to be processed are obtained according to steps 1 to 4 in the following manner, wherein the node to be processed is a primary node or a secondary node:

[0206] Step 1: Normalize each modality data in the target node data of the node to be processed to obtain the first intermediate data corresponding to each modality.

[0207] The target node data of the node to be processed includes data of each modality, and the data of each modality is data of the current prediction period and at least one adjacent period before the current prediction period.

[0208] For each mode, the data of the mode in the target node data of the node to be processed can be normalized to obtain the data corresponding to the mode. After performing the same processing on various modes, the first intermediate data corresponding to various modes can be obtained.

[0209] Step 2: Map the first intermediate data corresponding to each modality to the same embedding space to obtain the second intermediate data corresponding to each modality.

[0210] Each mode of the node data can be known in advance. Therefore, for each mode, a mapping method for the mode can be used to process the first intermediate data corresponding to the mode to obtain the second intermediate data corresponding to the mode.

[0211] Specifically, any one of the following two implementation methods may be used to process the first intermediate data corresponding to each modality.

[0212] In the first implementation method, feature extraction can be performed on the first intermediate data corresponding to each modality according to the feature extraction method corresponding to each modality to obtain data features corresponding to each modality as the second intermediate data corresponding to each modality in the same embedding space.

[0213] For example, for the image data modality, an image feature extraction method is used to extract features from the first intermediate data of the image data modality to obtain image features;

[0214] For the text data modality, a text feature extraction method is used to extract features from the first intermediate data of the text data modality to obtain text features;

[0215] For other modalities, a feature extraction method corresponding to the other modality is adopted to perform feature extraction on the first intermediate data of the other modality to obtain features corresponding to the other modality.

[0216] According to the feature extraction method corresponding to each modality, feature extraction is performed on the first intermediate data corresponding to each modality to obtain the data features corresponding to each modality. The data features corresponding to each modality can accurately represent the second intermediate data corresponding to each modality, so that the second intermediate data can be used to predict traffic conditions, which can improve the prediction accuracy.

[0217] In the second implementation method, the mapping rules between the data of each modality and the data in the same embedding space can be set in advance. In this way, after obtaining the first intermediate data corresponding to each modality, the first intermediate data corresponding to each modality can be mapped to the same embedding space according to the mapping rules corresponding to each modality.

[0218] Step 3: Concatenate the second intermediate data corresponding to each modality to obtain multimodal embedded data of the node to be processed.

[0219] Specifically, the second intermediate data corresponding to each modality can be input into a fully connected network for connection, and the data output by the fully connected network can be obtained as multimodal embedding data of the node to be processed.

[0220] The node to be processed can be a primary node or a secondary node. The multimodal embedded data of each primary node or each secondary node can be uniformly expressed as ,in, Represents data, Indicates the total number of cycles in which the data in the target node data resides. Indicates all primary nodes or all secondary nodes, Represents multimodality.

[0221] Step 4: Extract time domain features of multimodal embedded data to obtain the time domain features of the nodes to be processed.

[0222] In the first implementation, time convolution processing can be performed on the multimodal embedded data to obtain the time domain features of the nodes to be processed.

[0223] For example, the following expression is used to extract time-domain features from the multimodal embedding data of the processing node:

[0224] ;

[0225] in, represents the time domain characteristics of the i-th node to be processed, represents the temporal convolution processing, Represents the multimodal embedding data of the i-th node to be processed from T periods before the current prediction period t to the current prediction period t.

[0226] Above , Indicates the deep hidden dimension to which the features extracted from the time domain feature belong. The time domain features of each main node or each auxiliary node can be expressed as .

[0227] In the second implementation, the multimodal embedded data can be input into TCN to obtain the features of TCN output as the time domain features of the node to be processed.

[0228] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic status, traffic status prediction is performed based on target node data containing multimodal data, and the data used for prediction is relatively rich, which can improve the prediction accuracy.

[0229] In one embodiment of the present application, see Figure 3 , shows the prediction process of traffic status prediction using multimodal data of network nodes in three types of traffic networks.

[0230] The number of the above-mentioned transportation networks may also be other numbers besides three, which is not limited in the embodiments of the present application.

[0231] Figure 3 In the figure, modality 1 data, modality 2 data, and modality 3 data are multimodal data of network nodes in one transportation network, modality 4 data and modality 5 data are multimodal data of network nodes in another transportation network, and modality 6 data are multimodal data of network nodes in the last transportation network.

[0232] Feature fusion is a method of performing data mapping, splicing, and feature extraction on multimodal data as mentioned above. When a traffic network corresponds to only one mode, time domain features can be directly extracted from the data to obtain time domain features.

[0233] Model 1, Model 2, and Model 3 are all used to perform nonlinear transformation on the time domain characteristics of the network nodes in their respective corresponding transportation networks, and obtain the correlation characteristics of the main nodes of the main transportation networks in the three transportation networks, such as autocorrelation characteristics or cross-correlation characteristics.

[0234] Model fusion refers to the fusion of the model output results of Model 1, Model 2, and Model 3, that is, the fusion of the three correlation features of the main node, and predicting the traffic status of the main node in the next prediction cycle based on the fused features, and obtaining the predicted value representing the traffic status of the main node in the next prediction cycle as the output result.

[0235] The first implementation method of the hybrid fusion mentioned in the above step S203 is described below.

[0236] In one embodiment of the present application, the autocorrelation feature and the cross-correlation feature of each master node may be processed in the following manner to obtain the fused correlation feature of each master node:

[0237] The autocorrelation features and cross-correlation features of the main node are concatenated to obtain the first intermediate feature; the intermediate feature is linearly transformed to obtain the second intermediate feature; based on the second intermediate feature, the weight coefficient of the autocorrelation features and cross-correlation features of the main node is obtained; based on the obtained weight coefficient, the autocorrelation features and cross-correlation features of the main node are weighted to obtain the fused correlation features of the main node.

[0238] Specifically, the cross-correlation feature can be added to the tail of the autocorrelation feature of the main node to obtain the first intermediate feature of the main node, or the autocorrelation feature and the cross-correlation feature of the main node are input into the fully connected network to obtain the first intermediate feature output by the fully connected network.

[0239] After obtaining the first intermediate feature, the first intermediate feature may be linearly transformed according to preset linear transformation parameters to obtain the second intermediate feature.

[0240] Based on the second intermediate feature, any one of the following three implementation methods may be used to obtain the weight coefficients of the autocorrelation feature and the cross-correlation feature of the master node.

[0241] In the first implementation, the second intermediate feature may be regularized to obtain a regularized result, which is used as a weight coefficient of the autocorrelation feature and the cross-correlation feature of the main node.

[0242] In the second implementation, the second intermediate feature may be normalized to obtain a normalized result, which is used as a weight coefficient of the autocorrelation feature and the cross-correlation feature of the main node.

[0243] For example, the weight coefficients of the autocorrelation feature and the cross-correlation feature of the master node can be obtained by the following expression:

[0244] ;

[0245] in, Represents the weight coefficient of the autocorrelation feature and cross-correlation feature of the i-th main node, represents normalization processing, Represents a splicing operation, represents the autocorrelation feature of the ith primary node, that is, the primary relationship feature corresponding to the ith primary node mentioned above, represents the mutual correlation feature of the ith primary node, that is, the auxiliary relationship feature corresponding to the ith primary node mentioned above, represents the fifth preset parameter, Represents the sixth prediction parameter.

[0246] In the third implementation, the autocorrelation features and cross-correlation features of the main node can be input into a pre-trained attention weight network to obtain the weight coefficients of the autocorrelation features and cross-correlation features of the main node output by the attention weight network.

[0247] After obtaining the weight coefficients of the autocorrelation features and the cross-correlation features of the main node, the autocorrelation features and the cross-correlation features of the main node can be weighted to obtain the fused correlation features of the main node.

[0248] For example, the autocorrelation features and cross-correlation features of the main node are weighted by the following expression:

[0249] ;

[0250] in, represents the fusion-related features of the i-th master node, Indicates the autocorrelation feature and cross-correlation feature of the i-th main node Features, Indicates the autocorrelation feature and cross-correlation feature of the i-th main node The weight coefficient of a feature.

[0251] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic status, the weight coefficients of the autocorrelation characteristics and cross-correlation characteristics of the main node are obtained through a series of processing, and the autocorrelation characteristics and cross-correlation characteristics of the main node are weighted based on the obtained weight coefficients. The fused related features of the main node can be accurately obtained, so that traffic status prediction based on the fused related features of the main node can improve prediction accuracy.

[0252] In one embodiment of the present application, see Figure 4 , provides a structural diagram of a traffic state prediction model. Figure 4It can be seen that the traffic state prediction model includes a first data embedding layer, a first splicing layer, a first time convolution layer, a graph convolution layer, a second data embedding layer, a second splicing layer, a second time convolution layer, an interactive information convolution layer, a feature fusion layer and a linear output layer.

[0253] Among them, the first data embedding layer is used to normalize the modal data in the target node data of the master node, and map the normalized data corresponding to each modality to the same embedding space.

[0254] The first splicing layer is used to splice the mapped data corresponding to each modality and output the multimodal embedded data of the main node.

[0255] The first time convolution layer is used to extract time domain features from the multimodal embedded data of the master node and output the time domain features of the master node. The first time convolution layer can be constructed based on TCN.

[0256] The graph convolution layer is used to perform nonlinear transformation on the time domain features of the main node based on the first connectivity relationship and output the autocorrelation features of the main node. The graph convolution layer can be constructed based on GCN (Graph Convolutional Network).

[0257] The second data embedding layer is used to normalize the data of each modality in the target node data of the auxiliary node, and map the normalized data corresponding to each modality to the same embedding space.

[0258] The second concatenation layer is used to concatenate the mapped data corresponding to each modality and output the multimodal embedding data of the auxiliary node.

[0259] The second time convolution layer is used to extract time domain features from the multimodal embedded data of the auxiliary node and output the time domain features of the auxiliary node. The second time convolution layer can also be constructed based on TCN.

[0260] The interactive information convolution layer is used to perform nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node, and output the mutual correlation features of each main node. The interactive information convolution layer can also be constructed by deforming the GCN to obtain the Inter-GCN. GCN is used to perform nonlinear transformation on the time domain features of the main node based on the first connectivity relationship between each main node, and the deformed Inter-GCN is used to perform nonlinear transformation on the time domain features of the auxiliary node based on the second connectivity relationship between each main node and each auxiliary node. The interactive information convolution layer is constructed based on the Inter-GCN. In the process of traffic status prediction, it can accurately output the mutual correlation features of each main node, and then perform traffic status prediction based on the mutual correlation features of each main node, which can improve the prediction accuracy.

[0261] The feature fusion layer is used to mix and fuse the autocorrelation features and cross-correlation features of each main node, and output the fused correlation features of each main node.

[0262] The feature fusion layer can be constructed based on the Attention aggregation weight network. In this case, the feature fusion layer can accurately assign weights to autocorrelation features and cross-correlation features based on the attention mechanism, perform weighted summation of autocorrelation features and cross-correlation features, and achieve mixed fusion of autocorrelation features and cross-correlation features, and accurately obtain the fusion-related features of each main node, and then predict the traffic status based on the fusion-related features, which can improve the prediction accuracy.

[0263] The linear output layer is used to perform linear transformation on the fusion-related features of each main node and output the predicted value representing the traffic conditions of each main node in the next prediction period.

[0264] The following is combined with the above Figure 4 The traffic state prediction model shown is used to illustrate the traffic state prediction process.

[0265] After obtaining the target node data of each main node in the main traffic network and the target node data of each auxiliary node in the auxiliary traffic network, the target node data of each main node is input into the first data embedding layer. After the first data embedding layer, the first splicing layer, the first time convolution layer and the graph convolution layer perform data processing in sequence, the graph convolution layer outputs the autocorrelation features of each main node; the target node data of each auxiliary node is input into the second data embedding layer. After the second data embedding layer, the second splicing layer, the second time convolution layer and the interactive information convolution layer perform data processing in sequence, the interactive information convolution layer outputs the mutual correlation features of each main node; the feature fusion layer obtains the autocorrelation features output by the graph convolution layer and the mutual correlation features output by the interactive information convolution layer, mixes and fuses these two features, and outputs the fusion-related features of each main node; the linear output layer obtains the fusion-related features output by the feature fusion layer, performs linear transformation on the fusion-related features, and outputs the predicted value representing the traffic conditions of each main node in the next prediction period.

[0266] In this solution, the above-mentioned traffic status prediction model can accurately predict the traffic status of network nodes, and the output of each network layer in the model has a specific meaning. For example, the multimodal embedded data output by the first data embedding layer or the second data embedding layer is a collection of data corresponding to each mode in the same embedding space, the time domain features of the nodes output by the first time convolution layer or the second time convolution layer represent the time domain correlation between the node data of different periods of the nodes, the autocorrelation features of the main node output by the graph convolution layer represent the association relationship between the main node and other main nodes, the mutual correlation features of the main node output by the interactive information convolution layer represent the association relationship between the main node and the auxiliary node, the fusion correlation features of the main node output by the feature fusion layer are the features obtained by combining the above two association relationships, and the linear output layer directly outputs the predicted value. Therefore, the traffic status prediction model provided by the embodiment of the present application can improve the interpretability of the model.

[0267] In one embodiment of the present application, when training the above-mentioned traffic status prediction model, any historical period acquired by the traffic management platform can be used as the current prediction period to obtain sample target node data of the main node and the auxiliary node, and the state value representing the traffic state of the main node in the next period of the selected historical period can be used as the true value of the main node, and the model training can be performed using back propagation and gradient descent method.

[0268] During the training process, the following expression can be used as the loss function to calculate the loss value:

[0269] ;

[0270] in, represents the loss value, F represents the total number of master nodes, The output of the model is The predicted value of the traffic status of the next prediction cycle for the master node, Indicates The real value of the traffic status of the next prediction cycle is represented by the master node, t represents the current prediction cycle, represents the L2 regularization term, Represents the preset loss function parameters.

[0271] During the training process, when the number of training iterations reaches a preset threshold or the difference in loss values ​​during multiple consecutive iterations is within a preset range, the model training is terminated.

[0272] Corresponding to the above-mentioned traffic status prediction method, the embodiment of the present application also provides a traffic status prediction device.

[0273] In one embodiment of the present application, see Figure 5, a schematic diagram of the structure of a traffic state prediction device is provided. In this embodiment, the device includes:

[0274] The first obtaining module 501 is used to obtain the autocorrelation characteristics of each main node in the main traffic network, wherein the autocorrelation characteristics of each main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, and the target node data includes: the node data in the current prediction period and at least one adjacent historical period before the current prediction period;

[0275] A second obtaining module 502 is used to obtain the mutual correlation characteristics of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node, wherein the main transportation network and the auxiliary transportation network are heterogeneous networks;

[0276] The feature fusion module 503 is used to mix and fuse the autocorrelation features and cross-correlation features of each main node to obtain the fused correlation features of each main node;

[0277] The state prediction module 504 is used to predict the traffic state of each main node in the next prediction cycle based on the fusion-related features of each main node.

[0278] It can be seen from the above that when the scheme provided by the embodiment of the present application is used to predict the traffic state, the autocorrelation characteristics of the main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, taking into account the association relationship between the nodes of the main traffic network itself, and the cross-correlation characteristics are obtained based on the second connectivity relationship between the main node and the auxiliary node and the target node data of each auxiliary node, taking into account the association relationship between the main node of the main traffic network and the auxiliary node of the auxiliary traffic network. In this way, the autocorrelation characteristics and cross-correlation characteristics of each main node are mixed and fused to obtain the fused correlation characteristics of each main node. The fused correlation characteristics can be regarded as the characteristics of the main node obtained under the condition of comprehensive consideration of the above two association relationships. In this way, predicting the traffic state based on the fused correlation characteristics can also make accurate predictions under the condition of comprehensive consideration of the above two association relationships. It can be seen that the traffic state prediction scheme provided by the embodiment of the present application can not only realize the prediction of the traffic state of the nodes in the traffic network, but also, compared with only considering the association relationship between the nodes of the traffic network itself to predict the traffic state, this scheme can also improve the prediction accuracy.

[0279] In one embodiment of the present application, the feature fusion module 503 is specifically used to:

[0280] The autocorrelation features and cross-correlation features of each main node are processed in the following manner to obtain the fused correlation features of each main node:

[0281] The autocorrelation feature and the cross-correlation feature of the main node are concatenated to obtain the first intermediate feature;

[0282] Performing a linear transformation on the intermediate feature to obtain a second intermediate feature;

[0283] Based on the second intermediate feature, obtaining weight coefficients of the autocorrelation feature and the cross-correlation feature of the master node;

[0284] Based on the obtained weight coefficient, the autocorrelation features and cross-correlation features of the main node are weighted and summed to obtain the fused correlation features of the main node.

[0285] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic status, the weight coefficients of the autocorrelation characteristics and cross-correlation characteristics of the main node are obtained through a series of processing, and the autocorrelation characteristics and cross-correlation characteristics of the main node are weighted based on the obtained weight coefficients. The fused related features of the main node can be accurately obtained, so that traffic status prediction based on the fused related features of the main node can improve prediction accuracy.

[0286] In one embodiment of the present application, the second obtaining module 502 includes:

[0287] The first acquisition submodule is used to obtain the time domain characteristics of each auxiliary node according to the target node data of each auxiliary node in the auxiliary traffic network;

[0288] The first transformation submodule is used to perform nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node.

[0289] In this solution, the above method can be used to obtain more accurate cross-correlation features, so that traffic status can be predicted based on the cross-correlation features, which can improve prediction accuracy.

[0290] In one embodiment of the present application, the first transformation submodule is specifically used to:

[0291] Based on the third intermediate feature, a time domain feature of each auxiliary node is linearly transformed to obtain a fourth intermediate feature, wherein the third intermediate feature is a feature obtained by normalizing elements in an adjacency matrix in units of preset processing units, the adjacency matrix represents a second connectivity relationship between each main node and each auxiliary node, and the processing unit is a matrix row or a matrix column;

[0292] The fourth intermediate feature is subjected to nonlinear transformation to obtain the cross-correlation feature of each main node.

[0293] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic conditions, the third intermediate feature is a feature obtained by normalizing the adjacency matrix. In this way, various processing is performed on the time domain features of each auxiliary node based on the third intermediate feature, which can speed up the acquisition of the cross-correlation features of each main node, thereby improving the prediction efficiency.

[0294] In one embodiment of the present application, the first obtaining module 501 is specifically used to:

[0295] According to the target node data of each main node in the main transportation network, the time domain characteristics of each main node are obtained;

[0296] Based on the first connectivity relationship, a nonlinear transformation is performed on the time domain features of each master node to obtain the autocorrelation features of each master node.

[0297] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic status, the time domain characteristics of each main node are obtained based on the target node data of each main node in the main traffic network, and then the time domain characteristics of each main node are nonlinearly transformed based on the first connectivity relationship to obtain the autocorrelation characteristics of each main node. In this way, the autocorrelation characteristics of each main node not only reflect the connectivity between the main nodes, but also reflect the time domain correlation of the node data of each main node in different periods, so that the obtained autocorrelation characteristics are more accurate. In this way, predicting traffic status based on autocorrelation characteristics can improve prediction accuracy.

[0298] In one embodiment of the present application, when the target node data of the node to be processed includes multimodal data, the time domain features of the node to be processed are obtained in the following manner, wherein the node to be processed is: a primary node or a secondary node:

[0299] Normalizing each modality data in the target node data of the node to be processed to obtain first intermediate data corresponding to each modality;

[0300] Mapping the first intermediate data corresponding to each modality to the same embedding space respectively to obtain the second intermediate data corresponding to each modality;

[0301] splicing the second intermediate data corresponding to each modality to obtain multimodal embedded data of the node to be processed;

[0302] Perform time domain feature extraction on the multimodal embedded data to obtain the time domain features of the node to be processed.

[0303] It can be seen from the above that when the solution provided in the embodiment of the present application is used to predict traffic status, traffic status prediction is performed based on target node data containing multimodal data, and the data used for prediction is relatively rich, which can improve the prediction accuracy.

[0304] In one embodiment of the present application, mapping the first intermediate data corresponding to each modality to the same embedding space to obtain the second intermediate data corresponding to each modality includes:

[0305] According to the feature extraction method corresponding to each modality, feature extraction is performed on the first intermediate data corresponding to each modality respectively to obtain the data features corresponding to each modality as the second intermediate data corresponding to each modality in the same embedding space.

[0306] In this solution, feature extraction is performed on the first intermediate data corresponding to each modality according to the feature extraction method corresponding to each modality to obtain data features corresponding to each modality. The data features corresponding to each modality can accurately represent the second intermediate data corresponding to each modality, so that the second intermediate data can be used to predict traffic conditions, which can improve prediction accuracy.

[0307] In one embodiment of the present application, the state prediction module 504 is specifically used to:

[0308] For each main node, a linear transformation is performed on the fusion-related features of the main node to obtain a predicted value representing the traffic conditions of the main node in the next prediction period.

[0309] In this scheme, since the fused related features of the main node not only fuse the autocorrelation features and cross-correlation features of the main node, but are also obtained based on the node data of multiple periods of the main node and the auxiliary node, the fused related features of the main node can reflect the association relationship between the main node and other main nodes and auxiliary nodes associated with it, and can also reflect the time domain correlation of the node data of multiple periods of the main node. Therefore, linear transformation of the fused related features of the main node can accurately obtain the predicted value and improve the prediction accuracy.

[0310] In one embodiment of the present application, the first obtaining module 501 is specifically used to:

[0311] Input the target node data of each main node in the main traffic network into the first data embedding layer in the pre-trained traffic state prediction model, so that the first data embedding layer normalizes the modal data in the target node data of each main node, and maps the normalized data corresponding to each modality to the same embedding space, wherein the traffic state prediction model further includes a first splicing layer, a first time convolution layer, a graph convolution layer, a second data embedding layer, a second splicing layer, a second time convolution layer, an interactive information convolution layer, a feature fusion layer and a linear output layer;

[0312] Inputting the output result of the first data embedding layer into the first splicing layer, so that the first splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedded data of each master node;

[0313] Inputting the multimodal embedded data of each master node into the first time convolution layer, so that the first time convolution layer extracts time domain features from the multimodal embedded data of each master node to obtain time domain features of each master node;

[0314] Inputting the time domain features of each master node into the graph convolution layer, so that the graph convolution layer performs nonlinear transformation on the time domain features of each master node to obtain the autocorrelation features of each master node;

[0315] The second obtaining module 502 is specifically used for:

[0316] Inputting the target node data of each auxiliary node in the auxiliary traffic network into the second data embedding layer, so that the second data embedding layer normalizes each modality data in the target node data of each auxiliary node, and maps the normalized data corresponding to each modality to the same embedding space;

[0317] Inputting the output result of the second data embedding layer into the second splicing layer, so that the second splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedding data of each auxiliary node;

[0318] Inputting the multimodal embedding data of each auxiliary node into the second temporal convolutional layer, so that the second temporal convolutional layer extracts time domain features from the multimodal embedding data of each auxiliary node to obtain time domain features of each auxiliary node;

[0319] Inputting the time domain features of each auxiliary node into the mutual information convolution layer, so that the mutual information convolution layer performs nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node;

[0320] The feature fusion module 503 is specifically used for:

[0321] Inputting the autocorrelation features and cross-correlation features of each main node into the feature fusion layer, so that the feature fusion layer mixes and fuses the autocorrelation features and cross-correlation features of each main node to obtain fused correlation features of each main node;

[0322] The state prediction module 504 is specifically used for:

[0323] The fusion-related features of each main node are input into the linear output layer, so that the linear output layer performs a linear transformation on the fusion-related features of each main node to obtain a predicted value representing the traffic conditions of each main node in the next prediction period.

[0324] In this solution, the above-mentioned traffic status prediction model can accurately predict the traffic status of network nodes, and the output of each network layer in the model has a specific meaning. For example, the multimodal embedded data output by the first data embedding layer or the second data embedding layer is a collection of data corresponding to each mode in the same embedding space, the time domain features of the nodes output by the first time convolution layer or the second time convolution layer represent the time domain correlation between the node data of different periods of the nodes, the autocorrelation features of the main node output by the graph convolution layer represent the association relationship between the main node and other main nodes, the mutual correlation features of the main node output by the interactive information convolution layer represent the association relationship between the main node and the auxiliary node, the fusion correlation features of the main node output by the feature fusion layer are the features obtained by combining the above two association relationships, and the linear output layer directly outputs the predicted value. Therefore, the traffic status prediction model provided by the embodiment of the present application can improve the interpretability of the model.

[0325] In one embodiment of the present application, the interactive information convolution layer is constructed based on an Inter-GCN network obtained by deforming a graphics processing network GCN.

[0326] In this scheme, GCN is used to perform nonlinear transformation on the time domain features of the main nodes based on the first connectivity relationship between the main nodes, and the deformed Inter-GCN is used to perform nonlinear transformation on the time domain features of the auxiliary nodes based on the second connectivity relationship between the main nodes and the auxiliary nodes. The interactive information convolution layer is constructed based on Inter-GCN, which can accurately output the mutual correlation features of each main node during the traffic status prediction process, and then perform traffic status prediction based on the mutual correlation features of each main node, which can improve the prediction accuracy.

[0327] In one embodiment of the present application, the feature fusion layer is constructed based on the Attention aggregation weight network.

[0328] In this scheme, the feature fusion layer can accurately assign weights to autocorrelation features and cross-correlation features based on the attention mechanism, perform weighted summation of autocorrelation features and cross-correlation features, and realize mixed fusion of autocorrelation features and cross-correlation features. It can accurately obtain the fused related features of each main node, and then predict the traffic status based on the fused related features, which can improve the prediction accuracy.

[0329] The present application also provides an electronic device, such as Figure 6 As shown, including:

[0330] Memory 601, used for storing computer programs;

[0331] The processor 602 is used to execute the program stored in the memory 601 to implement the following steps:

[0332] Obtaining the autocorrelation characteristics of each main node in the main transportation network, wherein the autocorrelation characteristics of each main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, wherein the target node data includes: node data of the node in a current prediction period and at least one adjacent historical period before the current prediction period;

[0333] Based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node, the mutual correlation characteristics of each main node are obtained, wherein the main transportation network and the auxiliary transportation network are heterogeneous networks;

[0334] The autocorrelation features and cross-correlation features of each main node are mixed and fused to obtain the fused correlation features of each main node;

[0335] Based on the fusion-related features of each main node, the traffic status of each main node in the next prediction cycle is predicted.

[0336] The description of the processor 602 executing the program stored in the memory 601 to implement other traffic status prediction functions can be found in the aforementioned method embodiment, which will not be repeated here.

[0337] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 communicate with each other via the communication bus.

[0338] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0339] The communication interface is used for communication between the above electronic device and other devices.

[0340] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0341] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0342] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned traffic status prediction methods are implemented.

[0343] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the traffic status prediction methods in the above embodiments.

[0344] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0345] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0346] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0347] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A traffic state prediction method, characterized in that: The method comprises: Obtaining the autocorrelation characteristics of each main node in the main traffic network, wherein the autocorrelation characteristics of each main node are obtained based on the first connectivity relationship between the main nodes and the target node data of each main node, the target node data including: node data of the node in the current prediction period and at least one adjacent historical period before the current prediction period, and the first connectivity relationship characterizes whether the nodes in each main node are connected; Based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node, the cross-correlation characteristics of each main node are obtained, wherein the main transportation network and the auxiliary transportation network are heterogeneous networks, and the second connectivity relationship represents whether the main node and the auxiliary node are connected; The autocorrelation features and cross-correlation features of each main node are mixed and fused to obtain the fused correlation features of each main node; Predict the traffic status of each main node in the next prediction cycle based on the fusion related features of each main node; The obtaining of the mutual correlation features of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node includes: According to the target node data of each auxiliary node in the auxiliary transportation network, the time domain characteristics of each auxiliary node are obtained; Based on the second connectivity relationship between each main node and each auxiliary node, a nonlinear transformation is performed on the time domain features of each auxiliary node to obtain the cross-correlation features of each main node; The step of obtaining the autocorrelation characteristics of each main node in the main traffic network includes: According to the target node data of each main node in the main transportation network, the time domain characteristics of each main node are obtained; Based on the first connectivity relationship, a nonlinear transformation is performed on the time domain features of each main node to obtain the autocorrelation features of each main node; In the case where the target node data of the node to be processed includes multimodal data, the time domain features of the node to be processed are obtained in the following manner, wherein the node to be processed is: a primary node or a secondary node: Normalizing each modality data in the target node data of the node to be processed to obtain first intermediate data corresponding to each modality; Mapping the first intermediate data corresponding to each modality to the same embedding space respectively to obtain the second intermediate data corresponding to each modality; splicing the second intermediate data corresponding to each modality to obtain multimodal embedded data of the node to be processed; Perform time domain feature extraction on the multimodal embedded data to obtain the time domain features of the node to be processed.

2. The method according to claim 1, characterized in that The step of mixing and fusing the autocorrelation features and the cross-correlation features of each master node to obtain the fused correlation features of each master node includes: The autocorrelation features and cross-correlation features of each main node are processed in the following manner to obtain the fused correlation features of each main node: The autocorrelation feature and the cross-correlation feature of the main node are concatenated to obtain the first intermediate feature; Performing a linear transformation on the intermediate feature to obtain a second intermediate feature; Based on the second intermediate feature, obtaining weight coefficients of the autocorrelation feature and the cross-correlation feature of the master node; Based on the obtained weight coefficient, the autocorrelation features and cross-correlation features of the main node are weighted and summed to obtain the fused correlation features of the main node.

3. The method according to claim 1, characterized in that The nonlinear transformation of the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node includes: Based on the third intermediate feature, a time domain feature of each auxiliary node is linearly transformed to obtain a fourth intermediate feature, wherein the third intermediate feature is a feature obtained by normalizing elements in an adjacency matrix in units of preset processing units, the adjacency matrix represents a second connectivity relationship between each main node and each auxiliary node, and the processing unit is a matrix row or a matrix column; The fourth intermediate feature is subjected to nonlinear transformation to obtain the cross-correlation feature of each main node.

4. The method according to claim 1, characterized in that: The first intermediate data corresponding to each modality is mapped to the same embedding space to obtain the second intermediate data corresponding to each modality, including: According to the feature extraction method corresponding to each modality, feature extraction is performed on the first intermediate data corresponding to each modality respectively to obtain the data features corresponding to each modality as the second intermediate data corresponding to each modality in the same embedding space.

5. The method according to any one of claims 1 to 4, characterized in that The predicting of the traffic status of each main node in the next prediction cycle based on the fusion-related features of each main node includes: For each main node, a linear transformation is performed on the fusion-related features of the main node to obtain a predicted value representing the traffic conditions of the main node in the next prediction period.

6. The method according to claim 1, characterized in that The step of obtaining the autocorrelation characteristics of each main node in the main traffic network includes: Input the target node data of each main node in the main traffic network into the first data embedding layer in the pre-trained traffic state prediction model, so that the first data embedding layer normalizes the modal data in the target node data of each main node, and maps the normalized data corresponding to each modality to the same embedding space, wherein the traffic state prediction model further includes a first splicing layer, a first time convolution layer, a graph convolution layer, a second data embedding layer, a second splicing layer, a second time convolution layer, an interactive information convolution layer, a feature fusion layer and a linear output layer; Inputting the output result of the first data embedding layer into the first splicing layer, so that the first splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedded data of each master node; Inputting the multimodal embedded data of each master node into the first time convolution layer, so that the first time convolution layer extracts time domain features from the multimodal embedded data of each master node to obtain time domain features of each master node; Inputting the time domain features of each master node into the graph convolution layer, so that the graph convolution layer performs nonlinear transformation on the time domain features of each master node to obtain the autocorrelation features of each master node; The obtaining of the mutual correlation features of each main node based on the second connectivity relationship between each main node and each auxiliary node in the auxiliary transportation network and the target node data of each auxiliary node includes: Inputting the target node data of each auxiliary node in the auxiliary traffic network into the second data embedding layer, so that the second data embedding layer normalizes each modality data in the target node data of each auxiliary node, and maps the normalized data corresponding to each modality to the same embedding space; Inputting the output result of the second data embedding layer into the second splicing layer, so that the second splicing layer splices the mapped data corresponding to each modality to obtain multimodal embedding data of each auxiliary node; Inputting the multimodal embedding data of each auxiliary node into the second temporal convolutional layer, so that the second temporal convolutional layer extracts time domain features from the multimodal embedding data of each auxiliary node to obtain time domain features of each auxiliary node; Inputting the time domain features of each auxiliary node into the mutual information convolution layer, so that the mutual information convolution layer performs nonlinear transformation on the time domain features of each auxiliary node based on the second connectivity relationship between each main node and each auxiliary node to obtain the mutual correlation features of each main node; The step of mixing and fusing the autocorrelation features and the cross-correlation features of each master node to obtain the fused correlation features of each master node includes: Inputting the autocorrelation features and cross-correlation features of each main node into the feature fusion layer, so that the feature fusion layer mixes and fuses the autocorrelation features and cross-correlation features of each main node to obtain fused correlation features of each main node; The predicting of the traffic status of each main node in the next prediction cycle based on the fusion-related features of each main node includes: The fusion-related features of each main node are input into the linear output layer, so that the linear output layer performs a linear transformation on the fusion-related features of each main node to obtain a predicted value representing the traffic conditions of each main node in the next prediction period.

7. The method according to claim 6, characterized in that The interactive information convolution layer is constructed based on the Inter-GCN network obtained by deforming the graphics processing network GCN; and / or The feature fusion layer is constructed based on the attention aggregation weight network.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.