Traffic State Prediction and Tracing Method, Device and Equipment Based on Multimodal Data

Abstract

The present invention provides a traffic state prediction and traceability method, device, and equipment based on multi-modal data, which can be applied to the fields of spatio-temporal prediction and traffic planning and management. The method includes: obtaining a topological graph representing road connection relationships and multi-modal traffic sequence data for a target historical period, where the topological graph includes road nodes and edge relationships, and the road nodes represent roads; using a graph neural network to process each traffic sequence data and the topological graph to obtain traffic spatio-temporal features; processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain a congestion prediction result, where the congestion prediction result represents whether a road node is congested; processing the congestion prediction result based on the Shapley value algorithm to obtain a traceability result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

Description

Technical Field

[0001] The present invention relates to the fields of spatio-temporal prediction, traffic planning, and management, and specifically relates to a traffic state prediction and traceability method, device, and equipment based on multi-modal data. Background Art

[0002] With the acceleration of the urbanization process, traffic management has become an increasingly prominent challenge. Traditional traffic congestion prediction or traceability analysis has a small investigation scope and a small data capacity, and cannot comprehensively reflect the comprehensiveness and complexity of the traffic system, making it difficult to cope with the situation of multiple factors superimposed such as sudden accidents. Summary of the Invention

[0003] In view of the above problems, the present invention provides a traffic state prediction and traceability method, device, and equipment based on multi-modal data.

[0004] According to the first aspect of the present invention, a traffic state prediction and traceability method based on multi-modal data is provided, including: obtaining a topological graph representing road connection relationships, and obtaining multi-modal traffic sequence data for a target historical period, where the topological graph includes road nodes and edge relationships, and the road nodes represent roads; using a graph neural network to process each traffic sequence data and the topological graph to obtain traffic spatio-temporal features; processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain a congestion prediction result, where the congestion prediction result represents whether a road node is congested; processing the congestion prediction result based on the Shapley value algorithm to obtain a traceability result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0005] According to an embodiment of the present invention, processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain a congestion prediction result includes: processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain an attention fusion feature; using a fully connected neural network to process the attention fusion feature to obtain a predicted traffic flow velocity of the road node, where the predicted traffic flow velocity represents the average driving speed of all vehicles predicted on the road node; determining the congestion prediction result according to the predicted traffic flow velocity.

[0006] According to an embodiment of the present invention, determining the congestion prediction result according to the predicted traffic flow velocity includes: in the case where the predicted traffic flow velocity is less than a preset threshold, determining the congestion prediction result as a first prediction result representing congestion.

[0007] According to an embodiment of the present invention, the multi-modal traffic sequence data includes at least one of the following: traffic event text data, image and video data, holiday attribute data, traffic control data.

[0008] According to an embodiment of the present invention, each traffic sequence data and topological graph are processed using a graph neural network to obtain traffic spatio-temporal features, including: processing traffic event text data and topological graph using a first graph neural network to obtain first traffic spatio-temporal features; processing image and video data and topological graph using a second graph neural network to obtain second traffic spatio-temporal features; processing holiday attribute data and topological graph using a third graph neural network to obtain third traffic spatio-temporal features; processing traffic control data and topological graph using a fourth graph neural network to obtain fourth traffic spatio-temporal features.

[0009] According to an embodiment of the present invention, the congestion prediction result is processed based on the Shapley value algorithm to obtain a traceability result, including: based on the congestion prediction result, determining congested road nodes that are already congested from the topological graph; determining adjacent road nodes adjacent to the congested road nodes based on edge relationships; obtaining the predicted traffic flow velocity and the actual traffic flow velocity of the congested road nodes and the adjacent road nodes respectively; for each target road node, processing the predicted traffic flow velocity and the actual traffic flow velocity of the target road node based on a traffic flow velocity scoring function to obtain a predicted scoring result and an actual scoring result, where the target road nodes include congested road nodes and adjacent road nodes; subtracting the actual scoring result from the predicted scoring result to obtain the target marginal contribution; processing the target marginal contribution and the total number of participants based on the Shapley value algorithm to obtain the target Shapley value, where the total number of participants is the product of the total number of modalities and the number of target road nodes, and the total number of modalities is the number of modalities of multi-modal traffic sequence data, and the target Shapley value represents the contribution value of the traffic sequence data of the target road node to the occurrence of congestion; subtracting the average Shapley value from the target Shapley value to obtain the congestion contribution amount of the traffic sequence data of the target road node, where the average Shapley value is obtained based on multiple target Shapley values corresponding to multiple target road nodes; determining the traceability result based on the congestion contribution amount.

[0010] According to an embodiment of the present invention, determining the traceability result based on the congestion contribution amount includes: determining the congestion contribution amounts corresponding to the traffic sequence data of multiple target road nodes respectively; determining the target congestion contribution amount with the largest congestion contribution amount from multiple congestion contribution amounts; and determining the traffic sequence data of the target road node corresponding to the target congestion contribution amount as the traceability result.

[0011] The second aspect of the present invention provides a traffic state prediction and traceability device based on multi-modal data, including: an acquisition module for acquiring a topological graph representing road connection relationships and multi-modal traffic sequence data for a target historical period, where the topological graph includes road nodes and edge relationships, and the road nodes represent roads; a traffic spatio-temporal feature obtaining module for using a graph neural network to process each traffic sequence data and the topological graph to obtain traffic spatio-temporal features; a congestion prediction result obtaining module for obtaining a congestion prediction result based on a cross-attention network processing the traffic spatio-temporal features corresponding to each of the multiple traffic sequence data, where the congestion prediction result represents whether a road node is congested; a traceability result obtaining module for obtaining a traceability result based on the Shapley value algorithm processing the congestion prediction result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0012] The third aspect of the present invention provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, where the above one or more processors execute the above one or more computer programs to implement the steps of the above method.

[0013] The fourth aspect of the present invention further provides a computer program product, including a computer program or instruction, where the above computer program or instruction implements the steps of the above method when executed by a processor.

[0014] According to the embodiments of the present invention, by acquiring a topological graph representing road connection relationships and multi-modal traffic sequence data for a target historical period, the traffic sequence data such as weather conditions, road accidents, and traffic signal states, etc., can provide more comprehensive information for the system, bringing more practical traffic auxiliary decision-making capabilities. The traffic sequence data and the topological graph can be processed by a graph neural network to obtain traffic spatio-temporal features; the traffic spatio-temporal features corresponding to each of the multiple traffic sequence data can be processed based on a cross-attention network to obtain a congestion prediction result, where the congestion prediction result represents whether a road node is congested. By using a graph neural network and a cross-attention network to process multi-modal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby realizing accurate traffic state prediction. The congestion prediction result can be processed based on the Shapley value algorithm to obtain a traceability result. By effectively integrating multi-modal traffic sequence data, accurate traffic state prediction can be realized, and a comprehensive traceability analysis of congestion events can be carried out, thereby assisting traffic management departments in decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Through the following description of the embodiments of the present invention with reference to the accompanying drawings, the above content and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0016] Figure 1Schematically shows an application scenario diagram of a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention;

[0017] Figure 2 Schematically shows a flowchart of a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention;

[0018] Figure 3 Schematically shows a schematic diagram of traffic state prediction based on multi-modal data according to an embodiment of the present invention;

[0019] Figure 4 Schematically shows a flowchart of a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention;

[0020] Figure 5 Schematically shows a schematic diagram of congestion traceability based on multi-modal data according to an embodiment of the present invention;

[0021] Figure 6 Schematically shows a structural block diagram of a traffic state prediction and traceability device based on multi-modal data according to an embodiment of the present invention;

[0022] Figure 7 Schematically shows a block diagram of an electronic device suitable for implementing a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention. Detailed Embodiments

[0023] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present invention. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0024] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning that those skilled in the art usually understand this expression (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0027] To solve the problem of urban traffic congestion, traditional traffic congestion prediction or traceability systems mainly rely on single, data-based data, such as traffic flow, speed data, etc., and cannot comprehensively reflect the comprehensiveness and complexity of the traffic system, and it is difficult to handle the situation of multi-factor superposition such as weather changes and sudden accidents. The development of multi-modal data technology brings new ideas. By collecting data on various factors affecting traffic, such as weather conditions, road accidents, and traffic signal states, etc., it can provide more comprehensive information for the system and bring more practical traffic auxiliary decision-making capabilities. However, for the processing and analysis of traffic multi-modal data, there are still various challenges such as the fusion problem brought by data heterogeneity and the latency problem of traffic anomalies, and simple traffic flow prediction cannot meet the needs of urban management, and it is necessary to further analyze the causes of congestion in order to take targeted measures.

[0028] In view of this, the present invention provides a traffic state prediction and traceability method based on multi-modal data, a traffic state prediction and traceability device, and a device based on multi-modal data. The method includes: obtaining a topological graph representing road connection relationships, and obtaining multi-modal traffic sequence data for a target historical period, wherein the topological graph includes road nodes and edge relationships, and the road nodes represent roads; using a graph neural network to process each traffic sequence data and the topological graph to obtain traffic spatio-temporal features; based on a cross-attention network to process the traffic spatio-temporal features corresponding to each of the multiple traffic sequence data to obtain a congestion prediction result, the congestion prediction result representing whether a road node is congested; based on the Shapley value algorithm to process the congestion prediction result to obtain a traceability result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0029] It should be noted that the traffic state prediction and traceability method based on multi-modal data and the traffic state prediction and traceability device based on multi-modal data provided by the present invention can be used in the field of spatio-temporal prediction, and can also be used in any field other than the spatio-temporal prediction field, such as the field of urban traffic control. Therefore, the application fields of the traffic state prediction and traceability method based on multi-modal data and the traffic state prediction and traceability device based on multi-modal data provided by the present invention are not limited.

[0030] In the technical solution of the present invention, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, adopt necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In the scenario of making automated decisions using personal information, the methods, devices, and systems provided by the embodiments of the present invention all provide corresponding operation entrances for users to choose to agree or refuse the results of automated decisions; if the user chooses to refuse, the expert decision-making process will be entered. The expression "automated decision" here refers to the activity of automatically analyzing and evaluating an individual's behavior habits, hobbies, or economic, health, credit status, etc. through a computer program and making decisions. The expression "expert decision" here refers to the activity of making decisions by personnel who are engaged in work in a specific field, have specialized experience, knowledge, and skills, and have reached a certain professional level.

[0032] Figure 1 The application scenario diagram of the traffic state prediction and traceability method based on multi-modal data according to the embodiment of the present invention is schematically shown.

[0033] As Figure 1 shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0034] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, desktop computers, and the like.

[0036] The server 105 may be a server that provides various services, such as a background management server (only for example) that supports the websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0037] It should be noted that the traffic state prediction and traceability method based on multimodal data provided by the embodiments of the present invention can generally be executed by the server 105. Correspondingly, the traffic state prediction and traceability device based on multimodal data provided by the embodiments of the present invention can generally be set in the server 105. The traffic state prediction and traceability method based on multimodal data provided by the embodiments of the present invention can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the traffic state prediction and traceability device based on multimodal data provided by the embodiments of the present invention can also be set in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0038] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0039] Figure 2 is merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0040] As Figure 2 shown, the traffic state prediction and traceability method based on multimodal data in this embodiment includes operations S210 to S240, and this traffic state prediction and traceability method based on multimodal data can be executed by an electronic device.

[0041] In operation S210, obtain a topological graph representing the road connection relationship, and obtain multimodal traffic sequence data for a target historical period.

[0042] In operation S220, each traffic sequence data and the topological graph are processed using a graph neural network to obtain traffic spatio-temporal features.

[0043] In operation S230, based on a cross-attention network, the traffic spatio-temporal features corresponding to multiple traffic sequence data are processed to obtain a congestion prediction result, which characterizes whether a road node is congested.

[0044] In operation S240, based on the Shapley value algorithm, the congestion prediction result is processed to obtain a traceability result.

[0045] According to an embodiment of the present invention, a topological graph representing road connection relationships can be obtained, and multi-modal traffic sequence data for a target historical period can be obtained. The topological graph can include road nodes and edge relationships, and the road nodes represent roads. The multi-modal traffic sequence data can include time series, map information, event logs, etc., but is not limited thereto. The embodiments of the present invention do not limit the types of traffic sequence data. The traffic sequence data can contain data sampled at each timestamp of a target historical moment, and each sampling includes information of modes, and each mode corresponds to records. The topological graph consists of a node set, an edge set, and an attribute matrix, providing the geographical connection relationships between

[0046] nodes on the map. According to an embodiment of the present invention, the multi-modal traffic monitoring information consists of multi-modal traffic sequence data and map information Specifically, the multi-modal sequence data is sampled once at each timestamp from to moments, where contains information of modes, and each mode contains records of the number of map nodes where is the number of map nodes ( ). The map edge set is where represents the adjacency matrix of the topological graph . The topological graph provides the geographical connection relationships between nodes on the map.

[0047] According to an embodiment of the present invention, each traffic sequence data and topological graph can be processed by a graph neural network to obtain traffic spatio-temporal features. The traffic sequence data of each modality can be processed by a dedicated graph neural network, combined with the topological graph to capture the time-varying features of nodes and the mutual influence between nodes.

[0048] According to an embodiment of the present invention, the traffic spatio-temporal features of different modalities are integrated through a cross-attention network to improve the performance of multi-modal learning. Specifically, the traffic spatio-temporal features corresponding to multiple traffic sequence data can be processed based on the cross-attention network to obtain a congestion prediction result, and the congestion prediction result characterizes whether a congestion occurs at a road node.

[0049] According to an embodiment of the present invention, the traffic congestion prediction task can be described as given , predict in the future at time whether a congestion event will occur within a period of time steps, that is, predict the value of

[0050] ;

[0051] The traffic congestion traceability task can be described as given , , if it is predicted that a congestion will occur in the future at time within time steps, that is, calculate the contribution of each traffic sequence data of each road to this congestion, for the modality of the contribution of road node to the congestion is denoted as .

[0052] According to an embodiment of the present invention, the traceability result is obtained based on the Shapley value algorithm processing the congestion prediction result. The Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result. The Shapley value algorithm can quantitatively analyze the contribution of each modality of each road node in the multi-modal traffic sequence data to the congestion, so as to identify which modalities and road nodes have the greatest impact on the traffic congestion prediction result.

[0053] According to an embodiment of the present invention, by obtaining a topological graph representing road connection relationships and multi-modal traffic sequence data for a target historical period, such as weather conditions, road accidents, and traffic signal states, the traffic sequence data can provide more comprehensive information for the system, bringing more practical traffic assistance decision-making capabilities. The traffic sequence data and the topological graph can be processed using a graph neural network to obtain traffic spatio-temporal features; the traffic spatio-temporal features corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain a congestion prediction result, where the congestion prediction result indicates whether a road node is congested. By using a graph neural network and a cross-attention network to process multi-modal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby achieving accurate traffic state prediction. The congestion prediction result can be processed based on the Shapley value algorithm to obtain a traceability result. By effectively integrating multi-modal traffic sequence data, accurate traffic state prediction can be achieved, and a comprehensive traceability analysis of congestion events can be performed, thereby assisting traffic management departments in decision-making.

[0054] According to an embodiment of the present invention, processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain a congestion prediction result includes: processing the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain an attention fusion feature; processing the attention fusion feature using a fully connected neural network to obtain a predicted traffic flow rate of a road node, where the predicted traffic flow rate represents the average driving speed of all predicted vehicles on the road node; and determining the congestion prediction result according to the predicted traffic flow rate.

[0055] According to an embodiment of the present invention, the traffic spatio-temporal features corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain an attention fusion feature, thereby improving the performance of multi-modal learning; the attention fusion feature can be processed using a fully connected neural network to obtain a predicted traffic flow rate of a road node, where the predicted traffic flow rate represents the average driving speed of all predicted vehicles on the road node; and the congestion prediction result can be determined according to the predicted traffic flow rate.

[0056] According to an embodiment of the present invention, using a graph neural network and a cross-attention network to process multi-modal data can solve the fusion problem caused by data heterogeneity, thereby achieving accurate traffic state prediction.

[0057] According to an embodiment of the present invention, determining the congestion prediction result according to the predicted traffic flow rate includes: when the predicted traffic flow rate is less than a preset threshold, determining the congestion prediction result as a first prediction result indicating congestion.

[0058] According to an embodiment of the present invention, when the predicted traffic flow velocity is less than a preset threshold, it can be determined that the congestion prediction result is the first prediction result indicating congestion. The predicted traffic flow velocity can also be converted into a congestion prediction result through a piecewise function. For example, when the predicted traffic flow velocity is in the interval of [0, 20), the congestion prediction result is congestion; when the predicted traffic flow velocity is in the interval of [20, 40), the congestion prediction result is possible congestion; when the predicted traffic flow velocity is in the interval of [40, 60], the congestion prediction result is no congestion, and the speed limit of this road is 60.

[0059] According to an embodiment of the present invention, the technology of predicting congestion by setting a preset flow velocity threshold can realize early warning and active control of traffic congestion, improve traffic management efficiency, and optimize resource allocation.

[0060] Figure 3 A schematic diagram of traffic state prediction based on multi-modal data according to an embodiment of the present invention is schematically shown.

[0061] As Figure 3 shown, each graph neural network inputs traffic sequence data of a specific modality and a topological graph to obtain traffic spatio-temporal features. Among them, the encoding of the graph neural network is . Based on the cross-attention network, traffic spatio-temporal features corresponding to multiple traffic sequence data are processed to obtain attention fusion features. Among them, the data processing process of the cross-attention network is . The attention fusion features are processed by a fully connected neural network to obtain the predicted traffic flow velocity of the road nodes. The encoding after the fusion of multi-modal sequence data will be mapped to the predicted traffic flow velocity through a simple fully connected neural network . According to the predicted traffic flow velocity, the congestion prediction result is determined . The predicted traffic flow velocity can be mapped to a congestion prediction result according to a simple piecewise function .

[0062] According to an embodiment of the present invention, the multi-modal traffic sequence data includes at least one of the following: traffic event text data, image and video data, holiday attribute data, traffic control data.

[0063] According to an embodiment of the present invention, the traffic event text data may include a timestamp, a date, and abnormal events, etc. The abnormal events may include traffic accidents, construction activities, and weather conditions. The image and video data may include camera monitoring data, in-vehicle camera data, etc. The holiday attribute data may include large-scale events, holidays, etc. The traffic control data may include traffic signal timing data, speed limits, and lane allocation data, etc. Additionally, the multi-modal traffic sequence data may further include sensor data, time series data, and so on, but is not limited thereto. The embodiments of the present invention do not list all the types of multi-modal traffic sequence data one by one.

[0064] According to an embodiment of the present invention, each traffic sequence data and the topological graph are processed using a graph neural network to obtain traffic spatio-temporal features, including: processing the traffic event text data and the topological graph using a first graph neural network to obtain a first traffic spatio-temporal feature; processing the image and video data and the topological graph using a second graph neural network to obtain a second traffic spatio-temporal feature; processing the holiday attribute data and the topological graph using a third graph neural network to obtain a third traffic spatio-temporal feature; processing the traffic control data and the topological graph using a fourth graph neural network to obtain a fourth traffic spatio-temporal feature.

[0065] According to an embodiment of the present invention, the traffic event text data and the topological graph can be processed using a first graph neural network to obtain a first traffic spatio-temporal feature, the image and video data and the topological graph can be processed using a second graph neural network to obtain a second traffic spatio-temporal feature, the holiday attribute data and the topological graph can be processed using a third graph neural network to obtain a third traffic spatio-temporal feature, and the traffic control data and the topological graph can be processed using a fourth graph neural network to obtain a fourth traffic spatio-temporal feature.

[0066] According to an embodiment of the present invention, each modality of traffic sequence data can be processed through a dedicated graph neural network and a topological graph, combining spatial and temporal information, thereby capturing the characteristics of nodes changing over time and describing the connection relationships between nodes, learning the states of nodes at different time points, and being able to utilize the graph structure information to capture the mutual influences between nodes, thereby improving the accuracy of traffic state prediction.

[0067] Figure 4 A flowchart of a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention is schematically shown.

[0068] As Figure 4 shown, the traffic state prediction and traceability method based on multi-modal data of this embodiment includes operations S410 to S470, and the traffic state prediction and traceability method based on multi-modal data can be executed by an electronic device.

[0069] In operation S410, based on the congestion prediction result, the congested road nodes that have already become congested are determined from the topological graph.

[0070] In operation S420, adjacent road nodes adjacent to the congested road node are determined based on the edge relationship.

[0071] In operation S430, the predicted traffic flow velocity and the actual traffic flow velocity of the congested road node and the adjacent road nodes are obtained respectively.

[0072] In operation S440, for each target road node, the predicted traffic flow velocity and the actual traffic flow velocity of the target road node are processed based on the traffic flow velocity scoring function to obtain a predicted scoring result and an actual scoring result. The target road nodes include the congested road node and the adjacent road nodes.

[0073] In operation S450, the predicted scoring result is subtracted from the actual scoring result to obtain the target marginal contribution.

[0074] In operation S460, the target marginal contribution and the total number of participants are processed based on the Shapley value algorithm to obtain the target Shapley value.

[0075] In operation S470, the target Shapley value is subtracted from the average Shapley value to obtain the congestion contribution amount of the traffic sequence data of the target road node.

[0076] According to an embodiment of the present invention, based on the congestion prediction result, the congested road nodes that are already congested can be determined from the topological graph. Adjacent road nodes adjacent to the congested road node are determined based on the edge relationship.

[0077] According to an embodiment of the present invention, the predicted traffic flow velocity and the actual traffic flow velocity of the congested road node and the adjacent road nodes can be obtained respectively.

[0078] According to an embodiment of the present invention, the target road nodes may include the congested road node and the adjacent road nodes. For each target road node, the predicted traffic flow velocity of the target road node is processed based on the traffic flow velocity scoring function to obtain a predicted scoring result. The actual traffic flow velocity is processed based on the traffic flow velocity scoring function to obtain an actual scoring result.

[0079] According to an embodiment of the present invention, a traffic flow velocity scoring function is defined . Input the predicted traffic flow velocity , and output the predicted scoring result. It has the characteristic that the more serious the traffic congestion is, the larger the output of is.

[0080] According to an embodiment of the present invention, the target marginal contribution can be expressed as the difference when the data at the road node maintains the true value compared with the case where the data at the road node takes the predicted result value. Subtracting the true scoring result from the predicted scoring result gives the target marginal contribution, and the formula is as shown in (1).

[0081] (1)

[0082] Among them, is the traffic sequence data of the i-th mode, refers to the j-th road node, represents the contribution of the traffic sequence data of the i-th mode at the j-th road node to congestion. represents when taking the predicted result the predicted scoring result, represents when taking the true value the true scoring result.

[0083] According to an embodiment of the present invention, the Shapley value algorithm can be used to fairly distribute the total benefits generated in the cooperation to each participant. The target Shapley value characterizes the contribution value of the traffic sequence data of the target road node to the occurrence of congestion. Based on the Shapley value algorithm, processing the target marginal contribution and the total number of participants gives the target Shapley value, as shown in formula (2).

[0084] (2)

[0085] Among them, is the target Shapley value of the traffic sequence data of the i-th mode at the j-th road node, is the set of dimensions of all modes, is the subset the number of participants in, is the total number of participants, that is, the product between the total number of modes and the number of target road nodes. The participants can represent the traffic sequence data of each mode of each road node, and the total number of modes is the number of modes of the multi-modal traffic sequence data.

[0086] According to an embodiment of the present invention, subtracting the average Shapley value from the target Shapley value gives the congestion contribution amount of the traffic sequence data of the target road node, as shown in formula (3).

[0087] (3)

[0088] Among them, is the average Shapley value, represents the congestion contribution amount of the traffic sequence data of the i-th mode at the j-th road node.

[0089] According to an embodiment of the present invention, the traceability result can be determined based on the congestion contribution amount.

[0090] According to an embodiment of the present invention, by calculating the contribution of the traffic sequence data of each road node to the congestion, a quantitative method is used to more intuitively identify which modes and roads have the greatest impact on the traffic congestion prediction result, thereby assisting the traffic management department in making decisions.

[0091] Figure 5 Schematically shows a schematic diagram of congestion traceability based on multi-modal data according to an embodiment of the present invention.

[0092] As Figure 5 shown, based on the traffic state prediction method, multi-modal traffic sequence data [X modality(1) , X modality(2) , …, X modality(m) is processed to obtain the predicted traffic flow velocity . The true traffic flow velocity corresponding to the traffic sequence data of each road node is . Based on the scorer Scorer, the predicted traffic flow velocity and the true traffic flow velocity can be processed, and the target marginal contribution (i.e., Shapley value) needs to be calculated. The average Shapley value can be calculated according to the target Shapley values corresponding to multiple road nodes. For the traffic sequence data of each road node, subtracting the average Shapley value from the corresponding target Shapley value can obtain the congestion contribution amount of the traffic sequence data of the road node. Finally, the congestion contribution amounts corresponding to the traffic sequence data of all road nodes are obtained [X modality(1) : score S1, X modality(2) : score S2……X modality(n) : score S n . Sorting the multiple congestion contribution amounts from large to small, the traffic sequence data of the road nodes corresponding to the top K congestion contribution amounts can be used as the root cause of congestion (traceability result). According to an embodiment of the present invention, determining the traceability result based on the congestion contribution amount includes: determining the congestion contribution amounts corresponding to the traffic sequence data of multiple target road nodes; determining the target congestion contribution amount with the largest congestion contribution amount from the multiple congestion contribution amounts; and determining the traffic sequence data of the target road node corresponding to the target congestion contribution amount as the traceability result.

[0093] According to an embodiment of the present invention, the congestion contribution amount corresponding to the traffic sequence data of multiple target road nodes can be determined, the target congestion contribution amount with the largest congestion contribution amount can be determined from the multiple congestion contribution amounts, and the traffic sequence data of the target road node corresponding to the target congestion contribution amount can be determined as the tracing result. For example, the target road nodes are A and B, and there are a total of two modalities of traffic sequence data, X and Y. The congestion contribution amount of AX is 0.5, the congestion contribution amount of AY is 0.2, the congestion contribution amount of BX is 0.8, and the congestion contribution amount of BY is 0.15. It can be seen that the congestion contribution amount of BX is the largest. Therefore, the root cause of the congestion of the congested road node is the X traffic sequence data of the B target road node. The traffic sequence data X of the target road node B corresponding to the target congestion contribution amount of 0.8 can be determined as the tracing result.

[0094] According to an embodiment of the present invention, based on the obtained tracing result, the congestion source can be accurately located, and problems of congestion of congested road nodes can be solved by optimizing signal light control and traffic diversion, etc., further relieving traffic pressure and improving traffic management efficiency.

[0095] Figure 6 The structural block diagram of a traffic state prediction and tracing device based on multi-modal data according to an embodiment of the present invention is schematically shown.

[0096] As Figure 6 shown, the traffic state prediction and tracing device 600 based on multi-modal data of this embodiment includes an acquisition module 610, a traffic spatio-temporal feature obtaining module 620, a congestion prediction result obtaining module 630, and a tracing result obtaining module 660.

[0097] The acquisition module 610 is used to acquire a topological graph representing road connection relationships and acquire multi-modal traffic sequence data of a target historical period, where the topological graph includes road nodes and edge relationships, and the road nodes represent roads.

[0098] The traffic spatio-temporal feature obtaining module 620 is used to process each traffic sequence data and the topological graph by using a graph neural network to obtain traffic spatio-temporal features.

[0099] The congestion prediction result obtaining module 630 is used to process the traffic spatio-temporal features corresponding to multiple traffic sequence data based on a cross-attention network to obtain a congestion prediction result, and the congestion prediction result represents whether a road node is congested.

[0100] The tracing result obtaining module 640 is used to process the congestion prediction result based on the Shapley value algorithm to obtain a tracing result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0101] According to an embodiment of the present invention, by obtaining a topological graph representing road connection relationships and multi-modal traffic sequence data for a target historical period, such as weather conditions, road accidents, and traffic signal states, the traffic sequence data can provide more comprehensive information for the system, bringing more practical traffic assistance decision-making capabilities. The graph neural network can be used to process the traffic sequence data and the topological graph to obtain traffic spatio-temporal features. Based on the cross-attention network, the traffic spatio-temporal features corresponding to multiple traffic sequence data can be processed to obtain congestion prediction results, which represent whether a road node is congested. By using the graph neural network and the cross-attention network to process multi-modal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby achieving accurate traffic state prediction. Based on the Shapley value algorithm, the congestion prediction results can be processed to obtain a traceability result. By effectively integrating multi-modal traffic sequence data, accurate traffic state prediction can be achieved, and a comprehensive traceability analysis of congestion events can be carried out, thereby assisting traffic management departments in decision-making.

[0102] According to an embodiment of the present invention, the congestion prediction result obtaining module 630 includes: an attention fusion feature obtaining sub-module, a predicted traffic flow velocity obtaining sub-module, and a congestion prediction result determining sub-module.

[0103] The attention fusion feature obtaining sub-module is configured to process the traffic spatio-temporal features corresponding to multiple traffic sequence data based on the cross-attention network to obtain attention fusion features.

[0104] The predicted traffic flow velocity obtaining sub-module is configured to process the attention fusion features by using a fully connected neural network to obtain the predicted traffic flow velocity of the road node, and the predicted traffic flow velocity represents the average driving speed of all vehicles predicted on the road node.

[0105] The congestion prediction result determining sub-module is configured to determine the congestion prediction result according to the predicted traffic flow velocity.

[0106] According to an embodiment of the present invention, the congestion prediction result determining sub-module includes: a first prediction result determining unit.

[0107] The first prediction result determining unit is configured to determine the congestion prediction result as a first prediction result indicating congestion when the predicted traffic flow velocity is less than a preset threshold.

[0108] According to an embodiment of the present invention, the multi-modal traffic sequence data includes at least one of the following:

[0109] Traffic event text data, image and video data, holiday attribute data, traffic control data.

[0110] According to an embodiment of the present invention, the traffic spatio-temporal feature obtaining module 620 includes: a first traffic spatio-temporal feature sub-module, a second traffic spatio-temporal feature sub-module, a third traffic spatio-temporal feature sub-module, and a fourth traffic spatio-temporal feature sub-module.

[0111] The first traffic spatio-temporal feature sub-module is configured to process traffic event text data and a topology graph by using a first graph neural network to obtain first traffic spatio-temporal features.

[0112] The second traffic spatio-temporal feature sub-module is configured to process image and video data and a topology graph by using a second graph neural network to obtain second traffic spatio-temporal features.

[0113] The third traffic spatio-temporal feature sub-module is configured to process holiday attribute data and a topology graph by using a third graph neural network to obtain third traffic spatio-temporal features.

[0114] The fourth traffic spatio-temporal feature sub-module is configured to process traffic control data and a topology graph by using a fourth graph neural network to obtain fourth traffic spatio-temporal features.

[0115] According to an embodiment of the present invention, the traceability result obtaining module 640 includes: a congested road node determination sub-module, an adjacent road node determination sub-module, a traffic flow velocity acquisition sub-module, a scoring result obtaining sub-module, a target marginal contribution obtaining sub-module, a target Shapley value obtaining sub-module, a congestion contribution amount obtaining sub-module, and a traceability result determination sub-module.

[0116] The congested road node determination sub-module is configured to determine congested road nodes that are already congested from the topology graph based on the congestion prediction result.

[0117] The adjacent road node determination sub-module is configured to determine adjacent road nodes adjacent to the congested road nodes based on the edge relationship.

[0118] The traffic flow velocity acquisition sub-module is configured to acquire the predicted traffic flow velocity and the actual traffic flow velocity of the congested road nodes and the adjacent road nodes respectively.

[0119] The scoring result obtaining sub-module is configured to process the predicted traffic flow velocity and the actual traffic flow velocity of each target road node based on a traffic flow velocity scoring function to obtain a predicted scoring result and an actual scoring result, where the target road nodes include the congested road nodes and the adjacent road nodes.

[0120] The target marginal contribution obtaining sub-module is configured to subtract the actual scoring result from the predicted scoring result to obtain a target marginal contribution.

[0121] The target Shapley value obtaining sub-module is used to process the target marginal contribution and the total number of participants based on the Shapley value algorithm to obtain the target Shapley value. The total number of participants is the product between the total number of modes and the number of target road nodes. The total number of modes is the number of modes of the multi-modal traffic sequence data. The target Shapley value represents the contribution value of the traffic sequence data of the target road node to congestion occurrence.

[0122] The congestion contribution amount obtaining sub-module is used to subtract the average Shapley value from the target Shapley value to obtain the congestion contribution amount of the traffic sequence data of the target road node. The average Shapley value is obtained based on multiple target Shapley values corresponding to multiple target road nodes.

[0123] The traceability result determining sub-module is used to determine the traceability result based on the congestion contribution amount.

[0124] According to an embodiment of the present invention, the traceability result determining sub-module includes: a congestion contribution amount determining unit, a target congestion contribution amount determining unit, and a traceability result determining unit.

[0125] The congestion contribution amount determining unit is used to determine the congestion contribution amount corresponding to the traffic sequence data of each of the multiple target road nodes.

[0126] The target congestion contribution amount determining unit is used to determine the target congestion contribution amount with the largest congestion contribution amount from the multiple congestion contribution amounts.

[0127] The traceability result determining unit is used to determine the traffic sequence data of the target road node corresponding to the target congestion contribution amount as the traceability result.

[0128] According to an embodiment of the present invention, any multiple of the acquisition module 610, the traffic spatio-temporal feature obtaining module 620, the congestion prediction result obtaining module 630, and the traceability result obtaining module can be combined and implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 610, the traffic spatio-temporal feature obtaining module 620, the congestion prediction result obtaining module 630, and the traceability result obtaining module can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable means such as integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the acquisition module 610, the traffic spatio-temporal feature obtaining module 620, the congestion prediction result obtaining module 630, and the traceability result obtaining module can be at least partially implemented as a computer program module, and when the computer program module is run, corresponding functions can be executed.

[0129] Figure 7 FIG. schematically shows a block diagram of an electronic device suitable for implementing a traffic state prediction and traceability method based on multi-modal data according to an embodiment of the present invention.

[0130] As Figure 7 shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random access memory (RAM) 703. The processor 701 can include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 701 can also include on-board memory for caching purposes. The processor 701 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0131] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present invention by executing the programs in the ROM 702 and / or the RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to the embodiments of the present invention by executing the programs stored in the one or more memories.

[0132] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, and the input / output (I / O) interface 705 is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. A driver 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the driver 710 as needed so that a computer program read from it can be installed into the storage part 708 as needed.

[0133] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0134] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or apparatus. For example, according to an embodiment of the present invention, the computer-readable storage medium may include one or more memories other than the ROM 702 and / or RAM 703 and / or ROM 702 and RAM 703 described above.

[0135] An embodiment of the present invention further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the traffic state prediction and traceability method based on multimodal data provided by the embodiments of the present invention.

[0136] When the computer program is executed by the processor 701, it executes the above functions defined in the system / apparatus of the embodiments of the present invention. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0137] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 709, and / or be installed from the removable medium 711. The program code contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or be installed from the removable medium 711. When the computer program is executed by the processor 701, it executes the above functions defined in the system of the embodiments of the present invention. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0139] According to an embodiment of the present invention, program code for executing the computer program provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming languages include, but are not limited to, programming languages such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0142] The above describes the embodiments of the present invention. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present invention. Although the embodiments are described separately above, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A traffic status prediction and tracing method based on multimodal data, characterized in that: include: Acquire a topological map representing road connection relationships, and acquire multimodal traffic sequence data for a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; Using a graph neural network to process each of the traffic sequence data and the topological map to obtain traffic spatiotemporal characteristics; Processing the traffic spatiotemporal features corresponding to each of the plurality of traffic sequence data based on a cross attention network to obtain a congestion prediction result, wherein the congestion prediction result indicates whether congestion occurs at the road node; The congestion prediction result is processed based on a Shapley value algorithm to obtain a tracing result, wherein the Shapley value algorithm is used to calculate a contribution value of each traffic sequence data to the congestion prediction result.

2. The method according to claim 1, characterized in that The cross-attention network-based processing of the traffic spatiotemporal features corresponding to the plurality of traffic sequence data to obtain a congestion prediction result includes: Processing the traffic spatiotemporal features corresponding to the plurality of traffic sequence data respectively based on a cross attention network to obtain an attention fusion feature; Processing the attention fusion feature using a fully connected neural network to obtain a predicted traffic flow speed of the road node, wherein the predicted traffic flow speed represents an average travel speed of all vehicles predicted on the road node; The congestion prediction result is determined according to the predicted traffic flow speed.

3. The method according to claim 2, characterized in that Determining the congestion prediction result according to the predicted traffic flow speed includes: When the predicted traffic flow speed is less than a preset threshold, the congestion prediction result is determined to be a first prediction result indicating the occurrence of congestion.

4. The method according to claim 1, characterized in that The multimodal traffic sequence data includes at least one of the following: Traffic incident text data, image and video data, holiday attribute data, and traffic control data.

5. The method according to claim 4, characterized in that The method of using a graph neural network to process each of the traffic sequence data and the topological graph to obtain traffic spatiotemporal features includes: Processing the traffic event text data and the topological graph using a first graph neural network to obtain a first traffic spatiotemporal feature; Processing the image video data and the topological map using a second graph neural network to obtain a second traffic spatiotemporal feature; Processing the holiday attribute data and the topological graph using a third graph neural network to obtain a third traffic spatiotemporal feature; The traffic control data and the topological map are processed using a fourth graph neural network to obtain a fourth traffic spatiotemporal feature.

6. The method according to claim 2, characterized in that The congestion prediction result is processed based on the Shapley value algorithm to obtain a tracing result, including: Based on the congestion prediction result, determining a congested road node that is already congested from the topological map; Determine an adjacent road node adjacent to the congested road node based on the edge relationship; Obtaining the predicted traffic flow speed and the actual traffic flow speed of the congested road node and the adjacent road node respectively; For each target road node, the predicted traffic flow speed and the actual traffic flow speed of the target road node are processed based on the traffic flow speed scoring function to obtain a predicted scoring result and an actual scoring result, wherein the target road node includes the congested road node and the adjacent road node; Subtract the actual scoring result from the predicted scoring result to obtain a target marginal contribution; The target marginal contribution and the total number of participants are processed based on the Shapley value algorithm to obtain a target Shapley value, wherein the total number of participants is the product of the total number of modes and the number of target road nodes, the total number of modes is the number of modes of the multi-modal traffic sequence data, and the target Shapley value represents the contribution value of the traffic sequence data of the target road node to the occurrence of congestion; subtracting the average Shapley value from the target Shapley value to obtain a congestion contribution of the traffic sequence data of the target road node, wherein the average Shapley value is obtained based on a plurality of target Shapley values ​​corresponding to a plurality of the target road nodes; and The tracing result is determined based on the congestion contribution.

7. The method according to claim 6, characterized in that Determining the tracing result based on the congestion contribution includes: Determine the congestion contribution amount corresponding to each of the traffic sequence data of the plurality of target road nodes; Determining a target congestion contribution amount having the largest congestion contribution amount from the plurality of congestion contributions; and The traffic sequence data of the target road node corresponding to the target congestion contribution is determined as the tracing result.

8. A traffic status prediction and tracing device based on multimodal data, characterized in that: include: An acquisition module, used to acquire a topological map representing road connection relationships, and to acquire multimodal traffic sequence data of a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; A traffic spatiotemporal feature acquisition module, used to process each of the traffic sequence data and the topological graph using a graph neural network to obtain traffic spatiotemporal features; A congestion prediction result obtaining module, used for processing the traffic spatiotemporal features corresponding to each of the plurality of traffic sequence data based on a cross attention network to obtain a congestion prediction result, wherein the congestion prediction result indicates whether congestion occurs at the road node; The tracing result obtaining module is used to process the congestion prediction result based on the Shapley value algorithm to obtain the tracing result. The Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images