Traffic state prediction and tracing method, device and equipment based on multi-modal data

Abstract

The invention provides a traffic state prediction and traceability method, device and equipment based on multi-modal data, which can be applied to the field of space-time prediction and traffic planning and management. The method comprises the steps that a topological graph representing a road connection relation and multi-mode traffic sequence data of a target historical time period are acquired, the topological graph comprises road nodes and edge relations, and the road nodes represent roads; processing each traffic sequence data and the topological graph by using a graph neural network to obtain traffic spatial-temporal characteristics; the traffic spatial-temporal features corresponding to the traffic sequence data are processed based on a cross attention network, a congestion prediction result is obtained, and the congestion prediction result represents whether the road nodes are congested or not; and processing the congestion prediction result based on a Shapley value algorithm to obtain a traceability result, the Shapley value algorithm being used for calculating a contribution value of each traffic sequence data to the congestion prediction result.

Description

Technical Field

[0001] The present invention relates to the field of spatiotemporal prediction and traffic planning and management, and in particular to a method, device and equipment for traffic state prediction and tracing based on multimodal data. Background Art

[0002] With the acceleration of urbanization, traffic management has become an increasingly significant challenge. Traditional traffic congestion prediction or traceability analysis has a small scope and small data capacity. It cannot fully reflect the comprehensiveness and complexity of the transportation system, and it is difficult to deal with situations where multiple factors are superimposed, such as sudden accidents. Summary of the invention

[0003] In view of the above problems, the present invention provides a method, device and equipment for traffic status prediction and tracing based on multimodal data.

[0004] According to a first aspect of the present invention, a method for traffic status prediction and tracing based on multimodal data is provided, comprising: obtaining a topological map representing road connection relationships, and obtaining multimodal traffic sequence data for a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; using a graph neural network to process each traffic sequence data and the topological map to obtain traffic spatiotemporal characteristics; processing the traffic spatiotemporal characteristics corresponding to each of a plurality of traffic sequence data based on a cross-attention network to obtain a congestion prediction result, the congestion prediction result representing whether congestion occurs at a road node; processing the congestion prediction result based on a Shapley value algorithm to obtain a tracing result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0005] According to an embodiment of the present invention, a congestion prediction result is obtained by processing the traffic spatiotemporal features corresponding to each of a plurality of traffic sequence data based on a cross-attention network, including: obtaining an attention fusion feature by processing the traffic spatiotemporal features corresponding to each of a plurality of traffic sequence data based on a cross-attention network; obtaining a predicted traffic flow speed of a road node by processing the attention fusion feature using a fully connected neural network, the predicted traffic flow speed representing the average driving speed of all vehicles predicted on the road node; and determining a congestion prediction result based on the predicted traffic flow speed.

[0006] According to an embodiment of the present invention, determining a congestion prediction result based on the predicted traffic flow speed includes: when the predicted traffic flow speed is less than a preset threshold, determining the congestion prediction result as a first prediction result indicating the occurrence of congestion.

[0007] According to an embodiment of the present invention, the multimodal traffic sequence data includes at least one of the following: traffic event text data, image and video data, holiday attribute data, and traffic control data.

[0008] According to an embodiment of the present invention, a graph neural network is used to process each traffic sequence data and topological map to obtain traffic spatiotemporal features, including: using a first graph neural network to process traffic event text data and topological maps to obtain first traffic spatiotemporal features; using a second graph neural network to process image video data and topological maps to obtain second traffic spatiotemporal features; using a third graph neural network to process holiday attribute data and topological maps to obtain third traffic spatiotemporal features; using a fourth graph neural network to process traffic control data and topological maps to obtain fourth traffic spatiotemporal features.

[0009] According to an embodiment of the present invention, a congestion prediction result is processed based on a Shapley value algorithm to obtain a tracing result, including: determining a congested road node that is already congested from a topological map based on the congestion prediction result; determining an adjacent road node adjacent to the congested road node based on an edge relationship; obtaining a predicted traffic flow speed and a real traffic flow speed for each of the congested road node and the adjacent road node; for each target road node, processing the predicted traffic flow speed and the real traffic flow speed of the target road node based on a traffic flow speed scoring function to obtain a predicted scoring result and a real scoring result, wherein the target road node includes the congested road node and the adjacent road node; and subtracting the predicted scoring result from the real scoring result. The target marginal contribution is obtained by dividing the results; the target marginal contribution and the total number of participants are processed based on the Shapley value algorithm to obtain the target Shapley value, wherein the total number of participants is the product of the total number of modes and the number of target road nodes, the total number of modes is the number of modes of multi-modal traffic sequence data, and the target Shapley value represents the contribution of the traffic sequence data of the target road node to the occurrence of congestion; the target Shapley value is subtracted from the average Shapley value to obtain the congestion contribution of the traffic sequence data of the target road node, wherein the average Shapley value is obtained based on multiple target Shapley values ​​corresponding to multiple target road nodes; the tracing result is determined based on the congestion contribution.

[0010] According to an embodiment of the present invention, determining the tracing result based on the congestion contribution includes: determining the congestion contribution corresponding to each of the traffic sequence data of multiple target road nodes; determining the target congestion contribution with the largest congestion contribution from the multiple congestion contribution; and determining the traffic sequence data of the target road node corresponding to the target congestion contribution as the tracing result.

[0011] The second aspect of the present invention provides a traffic status prediction and tracing device based on multimodal data, including: an acquisition module, used to obtain a topological map representing road connection relationships, and to obtain multimodal traffic sequence data of a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; a traffic spatiotemporal feature acquisition module, used to process each traffic sequence data and topological map using a graph neural network to obtain traffic spatiotemporal features; a congestion prediction result acquisition module, used to process the traffic spatiotemporal features corresponding to each of a plurality of traffic sequence data based on a cross-attention network to obtain a congestion prediction result, and the congestion prediction result represents whether congestion occurs at a road node; a tracing result acquisition module, used to process the congestion prediction result based on a Shapley value algorithm to obtain a tracing result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0012] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0013] The fourth aspect of the present invention further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0014] According to an embodiment of the present invention, by obtaining a topological map representing the road connection relationship and multimodal traffic sequence data of a target historical period, traffic sequence data such as weather conditions, road accidents and traffic signal status, etc., can provide the system with more comprehensive information and bring more practical traffic auxiliary decision-making capabilities. The traffic sequence data and topological map can be processed by a graph neural network to obtain traffic spatiotemporal characteristics; the traffic spatiotemporal characteristics corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain a congestion prediction result, which represents whether a road node is congested. By using a graph neural network and a cross-attention network to process multimodal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby achieving accurate traffic state prediction. The congestion prediction result can be processed based on the Shapley value algorithm to obtain a traceability result. By effectively integrating multimodal traffic sequence data, accurate traffic state prediction can be achieved, and a comprehensive traceability analysis of congestion events can be performed, thereby assisting the decision-making of traffic management departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1A diagram schematically shows an application scenario of a traffic state prediction and tracing method based on multimodal data according to an embodiment of the present invention;

[0017] Figure 2 A flowchart of a traffic state prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown;

[0018] Figure 3 A schematic diagram schematically shows a traffic state prediction based on multimodal data according to an embodiment of the present invention;

[0019] Figure 4 A flowchart of a traffic state prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown;

[0020] Figure 5 A schematic diagram of congestion tracing based on multimodal data according to an embodiment of the present invention is schematically shown;

[0021] Figure 6 Schematically shows a structural block diagram of a traffic state prediction and source tracing device based on multimodal data according to an embodiment of the present invention;

[0022] Figure 7 A block diagram of an electronic device suitable for implementing a traffic status prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0023] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0024] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0025] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0027] In order to solve the problem of urban traffic congestion, traditional traffic congestion prediction or tracing systems mainly rely on single, digitized data, such as traffic volume and speed data, which cannot fully reflect the comprehensiveness and complexity of the traffic system, and it is difficult to cope with the superposition of multiple factors such as weather changes and sudden accidents. The development of multimodal data technology has brought new ideas for this. By collecting data on various factors affecting traffic, such as weather conditions, road accidents and traffic signal status, it can provide the system with more comprehensive information and bring more practical traffic auxiliary decision-making capabilities. However, the processing and analysis of multimodal traffic data still faces many challenges such as fusion problems caused by data heterogeneity and delay problems caused by abnormal traffic conditions. In addition, simple traffic prediction cannot meet the needs of urban management, and further analysis of the causes of congestion is needed in order to take targeted measures.

[0028] In view of this, the present invention provides a method for predicting and tracing traffic status based on multimodal data, and a device and equipment for predicting and tracing traffic status based on multimodal data. The method includes: obtaining a topological map representing road connection relationships, and obtaining multimodal traffic sequence data of a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; using a graph neural network to process each traffic sequence data and topological map to obtain traffic spatiotemporal characteristics; based on a cross-attention network, processing the traffic spatiotemporal characteristics corresponding to each of the multiple traffic sequence data to obtain a congestion prediction result, and the congestion prediction result represents whether congestion occurs at the road node; processing the congestion prediction result based on a Shapley value algorithm to obtain a tracing result, and the Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0029] It should be noted that the traffic status prediction and tracing method based on multimodal data and the traffic status prediction and tracing device based on multimodal data provided by the present invention can be used in the field of spatiotemporal prediction, and can also be used in any field other than the field of spatiotemporal prediction, such as the field of urban traffic control. Therefore, the application field of the traffic status prediction and tracing method based on multimodal data and the traffic status prediction and tracing device based on multimodal data provided by the present invention is not limited.

[0030] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0032] Figure 1 The application scenario diagram of the traffic status prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown.

[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0034] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0036] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0037] It should be noted that the traffic state prediction and tracing method based on multimodal data provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the traffic state prediction and tracing device based on multimodal data provided in the embodiment of the present invention can generally be set in the server 105. The traffic state prediction and tracing method based on multimodal data provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the traffic state prediction and tracing device based on multimodal data provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0039] Figure 2 The flowchart of the traffic status prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown.

[0040] like Figure 2 As shown, the traffic status prediction and tracing method based on multimodal data of this embodiment includes operations S210 to S240, and the traffic status prediction and tracing method based on multimodal data can be executed by an electronic device.

[0041] In operation S210 , a topological map representing a road connection relationship is obtained, and multi-modal traffic sequence data of a target historical period is obtained.

[0042] In operation S220, each traffic sequence data and topology map is processed using a graph neural network to obtain traffic spatiotemporal characteristics.

[0043] In operation S230, traffic spatiotemporal features corresponding to each of the plurality of traffic sequence data are processed based on the cross attention network to obtain a congestion prediction result, which indicates whether congestion occurs at the road node.

[0044] In operation S240, the congestion prediction result is processed based on the Shapley value algorithm to obtain a tracing result.

[0045] According to an embodiment of the present invention, a topological map representing road connection relationships can be obtained, and multimodal traffic sequence data of a target historical period can be obtained. The topological map may include road nodes and edge relationships, and the road nodes represent roads. Multimodal traffic sequence data may include time series, map information, event logs, etc., but are not limited thereto. The embodiment of the present invention does not limit the type of traffic sequence data. The traffic sequence data may include data sampled at each timestamp at the target historical moment, and each sample includes Information of each mode, each mode corresponds to The topological map consists of a node set, an edge set, and an attribute matrix, providing The geographical connection relationship between nodes.

[0046] According to an embodiment of the present invention, the multimodal traffic monitoring information is composed of multimodal traffic sequence data. And map information Composition, specifically, multimodal sequence data from to The time is sampled once per timestamp, where Included The information of each mode contains the number of map nodes. records, is the number of modes. Topological diagram Describes the map information, the node set is ,in is the number of map nodes ( ), the map edge set is , Representation topology The adjacency matrix of . Topological graph Map provided The geographical connections between nodes.

[0047] According to an embodiment of the present invention, a graph neural network can be used to process each traffic sequence data and topology map to obtain traffic spatiotemporal characteristics. The traffic sequence data of each modality can be processed by a dedicated graph neural network, combined with the topology map to capture the characteristics of nodes changing over time and the mutual influence between nodes.

[0048] According to an embodiment of the present invention, the traffic spatiotemporal features of different modes are integrated through a cross-attention network to improve the performance of multimodal learning. Specifically, the traffic spatiotemporal features corresponding to each of the multiple traffic sequence data can be processed based on the cross-attention network to obtain a congestion prediction result, which indicates whether congestion occurs at a road node.

[0049] According to an embodiment of the present invention, the traffic congestion prediction task can be described as given ,predict Moment of the future Will there be a congestion event within a certain period of time, that is, prediction The value is expressed as follows:

[0050] ;

[0051] The traffic congestion tracing task can be described as given , If the prediction Moment of the future Congestion occurs in the time step, that is, , then calculate The contribution of each traffic sequence data of each road to the congestion, Modal Road Node The contribution to congestion is .

[0052] According to an embodiment of the present invention, the congestion prediction result is processed based on the Shapley value algorithm to obtain a traceability result. The Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result. The Shapley value algorithm can quantitatively analyze the contribution of each mode and each road node in the multimodal traffic sequence data to congestion, thereby identifying which modes and road nodes have the greatest impact on the traffic congestion prediction result.

[0053] According to an embodiment of the present invention, by obtaining a topological map representing the road connection relationship and multimodal traffic sequence data of a target historical period, traffic sequence data such as weather conditions, road accidents and traffic signal status, etc., can provide the system with more comprehensive information and bring more practical traffic auxiliary decision-making capabilities. The traffic sequence data and topological map can be processed by a graph neural network to obtain traffic spatiotemporal characteristics; the traffic spatiotemporal characteristics corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain a congestion prediction result, which represents whether a road node is congested. By using a graph neural network and a cross-attention network to process multimodal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby achieving accurate traffic state prediction. The congestion prediction result can be processed based on the Shapley value algorithm to obtain a traceability result. By effectively integrating multimodal traffic sequence data, accurate traffic state prediction can be achieved, and a comprehensive traceability analysis of congestion events can be performed, thereby assisting the decision-making of traffic management departments.

[0054] According to an embodiment of the present invention, a congestion prediction result is obtained by processing the traffic spatiotemporal features corresponding to each of a plurality of traffic sequence data based on a cross-attention network, including: obtaining an attention fusion feature by processing the traffic spatiotemporal features corresponding to each of a plurality of traffic sequence data based on a cross-attention network; obtaining a predicted traffic flow speed of a road node by processing the attention fusion feature using a fully connected neural network, the predicted traffic flow speed representing the average driving speed of all vehicles predicted on the road node; and determining a congestion prediction result based on the predicted traffic flow speed.

[0055] According to an embodiment of the present invention, the traffic spatiotemporal features corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain attention fusion features, thereby improving the performance of multimodal learning; the attention fusion features can be processed using a fully connected neural network to obtain the predicted traffic flow speed of the road node, and the predicted traffic flow speed represents the average driving speed of all vehicles predicted on the road node; the congestion prediction result can be determined based on the predicted traffic flow speed.

[0056] According to an embodiment of the present invention, the use of graph neural networks and cross-attention networks to process multimodal data can solve the fusion problem caused by data heterogeneity, thereby achieving accurate traffic status prediction.

[0057] According to an embodiment of the present invention, determining a congestion prediction result based on the predicted traffic flow speed includes: when the predicted traffic flow speed is less than a preset threshold, determining the congestion prediction result as a first prediction result indicating the occurrence of congestion.

[0058] According to an embodiment of the present invention, when the predicted traffic flow speed is less than a preset threshold, the congestion prediction result can be determined as a first prediction result indicating the occurrence of congestion. The predicted traffic flow speed can also be converted into a congestion prediction result through a piecewise function. For example, when the predicted traffic flow speed is in the interval of [0, 20), the congestion prediction result is that congestion occurs, when the predicted traffic flow speed is in the interval of [20, 40), the congestion prediction result is that congestion may occur, and when the predicted traffic flow speed is in the interval of [40, 60], the congestion prediction result is that congestion does not occur, and the speed limit of the road is 60.

[0059] According to an embodiment of the present invention, the technology of predicting congestion by setting a preset flow rate threshold can achieve early warning and active control of traffic congestion, improve traffic management efficiency, and optimize resource allocation.

[0060] Figure 3 A schematic diagram of traffic state prediction based on multimodal data according to an embodiment of the present invention is schematically shown.

[0061] like Figure 3 As shown in Figure 2, each graph neural network inputs traffic sequence data and topology maps of a specific mode to obtain traffic spatiotemporal features, where the encoding of the graph neural network is Based on the cross attention network, the traffic spatiotemporal features corresponding to multiple traffic sequence data are processed to obtain the attention fusion features. The data processing process of the cross attention network is as follows: The fully connected neural network is used to process the attention fusion features to obtain the predicted traffic flow speed of the road node and the encoding after the fusion of multimodal sequence data. It will be mapped to predict traffic flow speed through a simple fully connected neural network . Determine the congestion prediction results based on the predicted traffic flow speed The predicted traffic flow speed can be mapped to the congestion prediction result according to a simple piecewise function .

[0062] According to an embodiment of the present invention, the multimodal traffic sequence data includes at least one of the following: traffic event text data, image and video data, holiday attribute data, and traffic control data.

[0063] According to an embodiment of the present invention, traffic event text data may include timestamps, dates, and abnormal events, etc., and abnormal events may include traffic accidents, construction activities, and weather conditions. Image video data may include camera monitoring data and vehicle-mounted camera data, etc., holiday attribute data may include large-scale activities, holidays, etc., and traffic control data may include traffic signal timing data, speed limit and lane allocation data, etc. In addition, multimodal traffic sequence data may also include sensor data, time series data, etc., but is not limited to this. The embodiments of the present invention do not list the types of multimodal traffic sequence data one by one.

[0064] According to an embodiment of the present invention, a graph neural network is used to process each traffic sequence data and topological map to obtain traffic spatiotemporal features, including: using a first graph neural network to process traffic event text data and topological maps to obtain first traffic spatiotemporal features; using a second graph neural network to process image video data and topological maps to obtain second traffic spatiotemporal features; using a third graph neural network to process holiday attribute data and topological maps to obtain third traffic spatiotemporal features; using a fourth graph neural network to process traffic control data and topological maps to obtain fourth traffic spatiotemporal features.

[0065] According to an embodiment of the present invention, a first graph neural network can be used to process traffic event text data and a topological map to obtain a first traffic spatiotemporal feature, a second graph neural network can be used to process image video data and a topological map to obtain a second traffic spatiotemporal feature, a third graph neural network can be used to process holiday attribute data and a topological map to obtain a third traffic spatiotemporal feature, and a fourth graph neural network can be used to process traffic control data and a topological map to obtain a fourth traffic spatiotemporal feature.

[0066] According to an embodiment of the present invention, the traffic sequence data of each modality can be processed by a special graph neural network and topological graph, combining spatial and temporal information, so as to capture the characteristics of nodes changing over time and describe the connection relationship between nodes, learn the status of nodes at different time points, and use graph structure information to capture the mutual influence between nodes, thereby improving the accuracy of traffic status prediction.

[0067] Figure 4 The flowchart of the traffic status prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown.

[0068] like Figure 4 As shown, the traffic status prediction and tracing method based on multimodal data of this embodiment includes operations S410 to S470, and the traffic status prediction and tracing method based on multimodal data can be executed by an electronic device.

[0069] In operation S410 , based on the congestion prediction result, a congested road node that is already congested is determined from a topological map.

[0070] In operation S420 , adjacent road nodes adjacent to the congested road node are determined based on the edge relationship.

[0071] In operation S430 , the predicted traffic flow speed and the actual traffic flow speed of each of the congested road node and the adjacent road node are acquired.

[0072] In operation S440, for each target road node, the predicted traffic flow speed and the actual traffic flow speed of the target road node are processed based on the traffic flow speed scoring function to obtain a predicted scoring result and an actual scoring result. The target road node includes a congested road node and an adjacent road node.

[0073] In operation S450 , the predicted scoring result is subtracted from the actual scoring result to obtain the target marginal contribution.

[0074] In operation S460, the target marginal contribution and the total number of participants are processed based on a Shapley value algorithm to obtain a target Shapley value.

[0075] In operation S470 , the average Shapley value is subtracted from the target Shapley value to obtain a congestion contribution amount of the traffic sequence data of the target road node.

[0076] According to an embodiment of the present invention, based on the congestion prediction result, a congested road node that is already congested can be determined from the topological map. Adjacent road nodes adjacent to the congested road node can be determined based on the edge relationship.

[0077] According to the embodiment of the present invention, the predicted traffic flow speed and the actual traffic flow speed of each of the congested road node and the adjacent road node can be obtained.

[0078] According to an embodiment of the present invention, the target road node may include a congested road node and an adjacent road node. For each target road node, the predicted traffic flow speed of the target road node is processed based on the traffic flow speed scoring function to obtain a predicted scoring result. The actual traffic flow speed is processed based on the traffic flow speed scoring function to obtain an actual scoring result.

[0079] According to an embodiment of the present invention, a traffic flow rate scoring function is defined: . Enter predicted traffic speed , output the predicted scoring results. The characteristic is that the more serious the traffic congestion, the The larger the output.

[0080] According to an embodiment of the present invention, the target marginal contribution can be expressed as the difference between the data of the road node and the predicted result value when the data of the road node maintains the true value. The predicted score result is subtracted from the true score result to obtain the target marginal contribution, as shown in formula (1).

[0081] (1)

[0082] in, is the traffic sequence data of mode i, refers to the j road node, Represents the contribution of mode i and j road node to congestion. express Get prediction results The predicted score result at express Take the true value The actual rating results.

[0083] According to an embodiment of the present invention, the Shapley value algorithm can be used to fairly distribute the total benefits generated in the cooperation to each participant. The target Shapley value represents the contribution value of the traffic sequence data of the target road node to the occurrence of congestion. Based on the Shapley value algorithm, the target marginal contribution and the total number of participants are processed to obtain the target Shapley value, as shown in formula (2).

[0084] (2)

[0085] in, is the target Shapley value of the i-mode j road node, is the set of dimensions of all modes, Is a subset The number of participants in It is the total number of participants, that is, the product of the total number of modes and the number of target road nodes. Participants can represent the traffic sequence data of each mode of each road node, and the total number of modes is the number of modes of multi-modal traffic sequence data.

[0086] According to an embodiment of the present invention, the target Shapley value is subtracted from the average Shapley value to obtain the congestion contribution of the traffic sequence data of the target road node, as shown in formula (3).

[0087] (3)

[0088] in, is the average Shapley value, Represents the congestion contribution of the i-mode j road node.

[0089] According to an embodiment of the present invention, the tracing result may be determined based on the congestion contribution amount.

[0090] According to an embodiment of the present invention, by calculating the contribution of the traffic sequence data of each road node to the congestion, a quantitative method is used to more intuitively identify which modes and roads have the greatest impact on the traffic congestion prediction results, thereby assisting the traffic management department in making decisions.

[0091] Figure 5 A schematic diagram of congestion tracing based on multimodal data according to an embodiment of the present invention is schematically shown.

[0092] like Figure 5 As shown in the figure, the traffic state prediction method is used to process multimodal traffic sequence data[X modality(1) , X modality(2) , …, X modality(m) ], and get the predicted traffic flow speed The actual traffic flow speed corresponding to the traffic sequence data of each road node is . Based on the scorer Scorer, the predicted traffic flow speed and the actual traffic flow speed need to be processed, and the target marginal contribution (i.e. Shapley value) needs to be calculated. The average Shapley value can be calculated based on the target Shapley values ​​corresponding to multiple road nodes. For the traffic sequence data of each road node, the congestion contribution of the traffic sequence data of the road node can be obtained by subtracting the corresponding target Shapley value from the average Shapley value. Finally, the congestion contribution corresponding to the traffic sequence data of all road nodes is obtained [X modality(1) :score S1、X modality(2) :score S2……X modality(n) :score S n ]. The multiple congestion contributions are sorted from large to small, and the traffic sequence data of the road nodes corresponding to the first K congestion contributions can be used as the root cause of the congestion (tracing result). According to an embodiment of the present invention, determining the tracing result based on the congestion contribution includes: determining the congestion contribution corresponding to each of the traffic sequence data of multiple target road nodes; determining the target congestion contribution with the largest congestion contribution from the multiple congestion contributions; and determining the traffic sequence data of the target road node corresponding to the target congestion contribution as the tracing result.

[0093] According to an embodiment of the present invention, the congestion contribution corresponding to each of the traffic sequence data of multiple target road nodes can be determined, and the target congestion contribution with the largest congestion contribution can be determined from the multiple congestion contributions, and the traffic sequence data of the target road node corresponding to the target congestion contribution can be determined as the tracing result. For example, the target road nodes are A and B, and there are 2 modal traffic sequence data X and Y. The congestion contribution of AX is 0.5, the congestion contribution of AY is 0.2, the congestion contribution of BX is 0.8, and the congestion contribution of BY is 0.15. It can be seen that the congestion contribution of BX is the largest. Therefore, the root cause of the congestion of the congested road node is the X traffic sequence data of the B target road node. The traffic sequence data X of the target road node B corresponding to the target congestion contribution of 0.8 can be determined as the tracing result.

[0094] According to the embodiments of the present invention, the source of congestion can be accurately located based on the tracing results obtained, and the problem of congestion at congested road nodes can be solved by optimizing signal light control and traffic diversion, thereby further alleviating traffic pressure and improving traffic management efficiency.

[0095] Figure 6 The structural block diagram of a traffic status prediction and tracing device based on multimodal data according to an embodiment of the present invention is schematically shown.

[0096] like Figure 6 As shown, the traffic state prediction and tracing device 600 based on multimodal data of this embodiment includes an acquisition module 610, a traffic spatiotemporal feature acquisition module 620, a congestion prediction result acquisition module 630, and a tracing result acquisition module 660.

[0097] The acquisition module 610 is used to acquire a topological map representing road connection relationships and to acquire multimodal traffic sequence data of a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads.

[0098] The traffic spatiotemporal feature acquisition module 620 is used to process each traffic sequence data and topology map using a graph neural network to obtain traffic spatiotemporal features.

[0099] The congestion prediction result obtaining module 630 is used to process the traffic spatiotemporal characteristics corresponding to each of the multiple traffic sequence data based on the cross attention network to obtain a congestion prediction result, which represents whether congestion occurs at the road node.

[0100] The traceability result obtaining module 640 is used to process the congestion prediction result based on the Shapley value algorithm to obtain the traceability result. The Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

[0101] According to an embodiment of the present invention, by obtaining a topological map representing the road connection relationship and multimodal traffic sequence data of a target historical period, traffic sequence data such as weather conditions, road accidents and traffic signal status, etc., can provide the system with more comprehensive information and bring more practical traffic auxiliary decision-making capabilities. The traffic sequence data and topological map can be processed by a graph neural network to obtain traffic spatiotemporal characteristics; the traffic spatiotemporal characteristics corresponding to multiple traffic sequence data can be processed based on a cross-attention network to obtain a congestion prediction result, which represents whether a road node is congested. By using a graph neural network and a cross-attention network to process multimodal traffic sequence data, the fusion problem caused by data heterogeneity can be solved, thereby achieving accurate traffic state prediction. The congestion prediction result can be processed based on the Shapley value algorithm to obtain a traceability result. By effectively integrating multimodal traffic sequence data, accurate traffic state prediction can be achieved, and a comprehensive traceability analysis of congestion events can be performed, thereby assisting the decision-making of traffic management departments.

[0102] According to an embodiment of the present invention, the congestion prediction result obtaining module 630 includes: an attention fusion feature obtaining submodule, a predicted traffic flow speed obtaining submodule and a congestion prediction result determining submodule.

[0103] The attention fusion feature acquisition submodule is used to process the traffic spatiotemporal features corresponding to multiple traffic sequence data based on the cross attention network to obtain the attention fusion features.

[0104] The predicted traffic flow speed is obtained by the submodule, which is used to use the fully connected neural network to process the attention fusion features to obtain the predicted traffic flow speed of the road node. The predicted traffic flow speed represents the average driving speed of all vehicles predicted on the road node.

[0105] The congestion prediction result determination submodule is used to determine the congestion prediction result according to the predicted traffic flow speed.

[0106] According to an embodiment of the present invention, the congestion prediction result determination submodule includes: a first prediction result determination unit.

[0107] The first prediction result determining unit is used to determine the congestion prediction result as a first prediction result indicating the occurrence of congestion when the predicted traffic flow speed is less than a preset threshold.

[0108] According to an embodiment of the present invention, the multimodal traffic sequence data includes at least one of the following:

[0109] Traffic incident text data, image and video data, holiday attribute data, and traffic control data.

[0110] According to an embodiment of the present invention, the traffic spatiotemporal feature obtaining module 620 includes: a first traffic spatiotemporal feature submodule, a second traffic spatiotemporal feature submodule, a third traffic spatiotemporal feature submodule and a fourth traffic spatiotemporal feature submodule.

[0111] The first traffic spatiotemporal feature submodule is used to process traffic event text data and topological graphs using a first graph neural network to obtain first traffic spatiotemporal features.

[0112] The second traffic spatiotemporal feature submodule is used to process the image video data and the topological map using the second graph neural network to obtain the second traffic spatiotemporal feature.

[0113] The third traffic spatiotemporal feature submodule is used to process holiday attribute data and topological graphs using the third graph neural network to obtain the third traffic spatiotemporal features.

[0114] The fourth traffic spatiotemporal feature submodule is used to process traffic control data and topological graphs using a fourth graph neural network to obtain fourth traffic spatiotemporal features.

[0115] According to an embodiment of the present invention, the tracing result obtaining module 640 includes: a congested road node determination submodule, an adjacent road node determination submodule, a traffic flow speed acquisition submodule, a scoring result obtaining submodule, a target marginal contribution obtaining submodule, a target Shapley value obtaining submodule, a congestion contribution amount obtaining submodule and a tracing result determination submodule.

[0116] The congested road node determination submodule is used to determine the congested road nodes that are already congested from the topological map based on the congestion prediction result.

[0117] The adjacent road node determination submodule is used to determine the adjacent road nodes adjacent to the congested road node based on the edge relationship.

[0118] The traffic flow speed acquisition submodule is used to obtain the predicted traffic flow speed and the actual traffic flow speed of the congested road node and the adjacent road node respectively.

[0119] The scoring result obtaining submodule is used to process the predicted traffic flow speed and the actual traffic flow speed of each target road node based on the traffic flow speed scoring function to obtain the predicted scoring result and the actual scoring result. The target road nodes include congested road nodes and adjacent road nodes.

[0120] The target marginal contribution obtaining submodule is used to obtain the target marginal contribution by subtracting the actual scoring result from the predicted scoring result.

[0121] The target Shapley value obtaining submodule is used to process the target marginal contribution and the total number of participants based on the Shapley value algorithm to obtain the target Shapley value, wherein the total number of participants is the product of the total number of modes and the number of target road nodes, the total number of modes is the number of modes of multi-modal traffic sequence data, and the target Shapley value represents the contribution value of the traffic sequence data of the target road node to the occurrence of congestion.

[0122] The congestion contribution obtaining submodule is used to obtain the congestion contribution of the traffic sequence data of the target road node by subtracting the average Shapley value from the target Shapley value, wherein the average Shapley value is obtained based on multiple target Shapley values ​​corresponding to multiple target road nodes.

[0123] The tracing result determination submodule is used to determine the tracing result based on the congestion contribution.

[0124] According to an embodiment of the present invention, the tracing result determination submodule includes: a congestion contribution determination unit, a target congestion contribution determination unit and a tracing result determination unit.

[0125] The congestion contribution determination unit is used to determine the congestion contribution corresponding to each of the traffic sequence data of a plurality of target road nodes.

[0126] The target congestion contribution determination unit is used to determine a target congestion contribution with the largest congestion contribution from a plurality of congestion contributions.

[0127] The tracing result determination unit is used to determine the traffic sequence data of the target road node corresponding to the target congestion contribution as the tracing result.

[0128] According to an embodiment of the present invention, any multiple modules among the acquisition module 610, the traffic spatiotemporal feature acquisition module 620, the congestion prediction result acquisition module 630, and the traceability result acquisition module can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 610, the traffic spatiotemporal feature acquisition module 620, the congestion prediction result acquisition module 630, and the traceability result acquisition module can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 610, the traffic spatiotemporal characteristics acquisition module 620, the congestion prediction result acquisition module 630, and the tracing result acquisition module can be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function can be executed.

[0129] Figure 7 A block diagram of an electronic device suitable for implementing a traffic status prediction and tracing method based on multimodal data according to an embodiment of the present invention is schematically shown.

[0130] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0131] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0132] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0133] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0134] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0135] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the traffic state prediction and tracing method based on multimodal data provided by the embodiment of the present invention.

[0136] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 701. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0137] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0139] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0140] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0142] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A traffic status prediction and tracing method based on multimodal data, characterized in that: include: Acquire a topological map representing road connection relationships, and acquire multimodal traffic sequence data for a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; Using a graph neural network to process each of the traffic sequence data and the topological map to obtain traffic spatiotemporal characteristics; Processing the traffic spatiotemporal features corresponding to each of the plurality of traffic sequence data based on a cross attention network to obtain a congestion prediction result, wherein the congestion prediction result indicates whether congestion occurs at the road node; The congestion prediction result is processed based on a Shapley value algorithm to obtain a tracing result, wherein the Shapley value algorithm is used to calculate a contribution value of each traffic sequence data to the congestion prediction result.

2. The method according to claim 1, characterized in that The cross-attention network-based processing of the traffic spatiotemporal features corresponding to the plurality of traffic sequence data to obtain a congestion prediction result includes: Processing the traffic spatiotemporal features corresponding to the plurality of traffic sequence data respectively based on a cross attention network to obtain an attention fusion feature; Processing the attention fusion feature using a fully connected neural network to obtain a predicted traffic flow speed of the road node, wherein the predicted traffic flow speed represents an average travel speed of all vehicles predicted on the road node; The congestion prediction result is determined according to the predicted traffic flow speed.

3. The method according to claim 2, characterized in that Determining the congestion prediction result according to the predicted traffic flow speed includes: When the predicted traffic flow speed is less than a preset threshold, the congestion prediction result is determined to be a first prediction result indicating the occurrence of congestion.

4. The method according to claim 1, characterized in that: The multimodal traffic sequence data includes at least one of the following: Traffic incident text data, image and video data, holiday attribute data, and traffic control data.

5. The method according to claim 4, characterized in that The method of using a graph neural network to process each of the traffic sequence data and the topological graph to obtain traffic spatiotemporal features includes: Processing the traffic event text data and the topological graph using a first graph neural network to obtain a first traffic spatiotemporal feature; Processing the image video data and the topological map using a second graph neural network to obtain a second traffic spatiotemporal feature; Processing the holiday attribute data and the topological graph using a third graph neural network to obtain a third traffic spatiotemporal feature; The traffic control data and the topological map are processed using a fourth graph neural network to obtain a fourth traffic spatiotemporal feature.

6. The method according to claim 2, characterized in that The congestion prediction result is processed based on the Shapley value algorithm to obtain a tracing result, including: Based on the congestion prediction result, determining a congested road node that is already congested from the topological map; Determine an adjacent road node adjacent to the congested road node based on the edge relationship; Obtaining the predicted traffic flow speed and the actual traffic flow speed of the congested road node and the adjacent road node respectively; For each target road node, the predicted traffic flow speed and the actual traffic flow speed of the target road node are processed based on the traffic flow speed scoring function to obtain a predicted scoring result and an actual scoring result, wherein the target road node includes the congested road node and the adjacent road node; Subtract the actual scoring result from the predicted scoring result to obtain a target marginal contribution; The target marginal contribution and the total number of participants are processed based on the Shapley value algorithm to obtain a target Shapley value, wherein the total number of participants is the product of the total number of modes and the number of target road nodes, the total number of modes is the number of modes of the multi-modal traffic sequence data, and the target Shapley value represents the contribution value of the traffic sequence data of the target road node to the occurrence of congestion; subtracting the average Shapley value from the target Shapley value to obtain a congestion contribution of the traffic sequence data of the target road node, wherein the average Shapley value is obtained based on a plurality of target Shapley values ​​corresponding to a plurality of the target road nodes; and The tracing result is determined based on the congestion contribution.

7. The method according to claim 6, characterized in that Determining the tracing result based on the congestion contribution includes: Determine the congestion contribution amount corresponding to each of the traffic sequence data of the plurality of target road nodes; Determining a target congestion contribution amount having the largest congestion contribution amount from the plurality of congestion contributions; and The traffic sequence data of the target road node corresponding to the target congestion contribution is determined as the tracing result.

8. A traffic status prediction and tracing device based on multimodal data, characterized in that: include: An acquisition module, used to acquire a topological map representing road connection relationships, and to acquire multimodal traffic sequence data of a target historical period, wherein the topological map includes road nodes and edge relationships, and the road nodes represent roads; A traffic spatiotemporal feature acquisition module, used to process each of the traffic sequence data and the topological graph using a graph neural network to obtain traffic spatiotemporal features; A congestion prediction result obtaining module, used for processing the traffic spatiotemporal features corresponding to each of the plurality of traffic sequence data based on a cross attention network to obtain a congestion prediction result, wherein the congestion prediction result indicates whether congestion occurs at the road node; The tracing result obtaining module is used to process the congestion prediction result based on the Shapley value algorithm to obtain the tracing result. The Shapley value algorithm is used to calculate the contribution value of each traffic sequence data to the congestion prediction result.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images