Method, system, computer program product and computer readable medium for traffic prediction

Abstract

The invention related to a method for traffic flow prediction, comprising the steps of receiving traffic-related information from a static unit (20) and / or a dynamic unit (22), performing traffic flow prediction by a prediction unit (60), wherein at least one future state of traffic is estimated based on the received traffic-related information, and sharing the estimated at least one future state, by a prediction sharing module (58). The method is characterized by creating a road graph representation comprising nodes (14, 150, 150') and edges (16) between the nodes (14, 150, 150'), wherein the road graph representation has a lane-level resolution, assigning the received traffic-related information to a corresponding lane before performing the traffic flow prediction step to generate a lane-matched traffic-related information, if the received traffic-related information is not yet a lane-matched traffic-related information, performing the traffic flow prediction step by deep learning based on the road graph representation and on the lane-matched traffic-related information. The invention further relates to a system, a computer program product and a computer readable medium for implementing the method.

Description

[0001] METHOD, SYSTEM, COMPUTER PROGRAM PRODUCT AND COMPUTER READABLE MEDIUM FOR TRAFFIC PREDICTION

[0002] TECHNICAL FIELD

[0003] The invention relates to a method and a system for traffic prediction. The invention relates also to a computer program product and computer readable medium implementing the method.

[0004] Due to the global industry trends of automotive, connected mobility and to the widespread use of loT (Internet-of-Things) devices, there are thousands of entities that are in a communication connection with each other in an urban street environment, and the number of entities changes in a dynamical manner. Managing such a large number of entities can be a difficult task, however, the communication connection between the entities allows for developing new systems and methods in the field.

[0005] Intelligent Transportation System (ITS) is an indispensable part of smart cities, i.e. , areas that typically uses different types of electronic methods and sensors to collect specific data. Traffic prediction is an important component of ITS and smart cities. Accurate traffic prediction is essential to many real-world applications. It is connected to several smart city use-cases, such as Green Light Optimal Speed Advisory (GLOSA) or Traffic Lights Control (TLC), which are often driven by V2X (Vehicle-to-Everything) messaging technology.

[0006] For instance, traffic flow prediction can help cities alleviate congestion; car-hailing demand prediction can prompt car-sharing companies to pre-allocate cars to high- demand regions. Another example is model-based accident detection a.k.a. anomaly detection, where the deviance from the predicted values of traffic parameters from the real measured data may be used to evaluate if the difference is outside of a predefined confidence region.

[0007] BACKGROUND ART

[0008] A general traffic flow prediction task is to provide information about a difference of a speed and / or a volume between a current and future time instances at measurement points over an input road network. Due to its large-scale nature, and complex, highly dynamic behaviour, traffic prediction is a challenging problem that is traditionally dealt with by means of modelbased and data-driven approaches. As traffic flow can change swiftly and abruptly, approximating this problem by using e.g., the model-based Average Kinematic Wave principle cannot always be satisfactory.

[0009] Many recent State-of-the-art (SOTA) models utilize neural-network structures to deal with complex data. Some of these use Graph Neural Networks, or attention mechanisms to have the best performance. Although there are a variety of approaches attempted, most of them focus on the mathematically formulated vanilla traffic prediction problem whose main goal is to reach accuracy as high as possible.

[0010] A paper of Peter Ldvei et al. “Extending Graph Convolutional Network With Attention Mechanism To Estimate Traffic Flow Information In Real-time Smart City Environments” (28th ITS World Congress, Los Angeles, September 18-22, 2022, Paper ID 1228410) focuses on analysing various state of the art (SOTA) Deep Learning methods for Traffic Flow Prediction and suggests a novel model architecture for that by combining two SOTA models into one. The paper solely aims to solve the underlying mathematical problem, that is, to make predictions with the highest accuracy.

[0011] Another instance is Traffic4cast, which also has been organizing traffic prediction- related competitions annually. Traffic4cast aims to predict travel time and congestion levels on a node and super segment (a set of interconnected road segments) level. Consequently, the problem formulation is almost the same, competitors suggest similar deep learning methods to solve this challenge in the best way possible.

[0012] Another example is Google Maps ETA (estimated time of arrival) API, which provides an estimation of arrival time. Google (and DeepMind) formulates once again the problem in a similar fashion, dividing the road into (super)segments that consist of edges and interconnected nodes. Then the model utilizes spatio-temporal embeddings along with Graph Neural Network model architecture to make the predictions. Across the prior art, the general problem formulation is similar; however, these models belong to one of the two data collection technique categories mentioned above, i.e., GPS-based or infrastructure based, which leads to limitations. For example, as these solutions only utilize speed or travel time as an input, they are not fit for making lane-level predictions.

[0013] There are some prior art documents that adopt CV-based data collection. For example, Ranwa Al Mallah et al. “Prediction of Traffic Flow via Connected Vehicles” aims to predict traffic over segments of a road network via CV and V2X-based data collection. However, the method described in this paper is not adapted for large- scale graph-based prediction, but only uses a vanilla neural network. Furthermore, it does not make predictions on a lane-level.

[0014] There are other prior art solutions focusing on the vanilla graph-based traffic prediction task.

[0015] US 2021 / 0064999 A1 discloses a multi-scale multi-granularity spatial-temporal traffic volume prediction. It uses spatial proximity (distance between points), transition proximity (some sort of OD flow between points), spatial adjacency (on a regional level), uses different scales (site, region level), and different time encodings (for week, day, hour) and an attention mechanism based thereon.

[0016] US 2021 / 0233391 A1 discloses a method and device for predicting traffic flow or travel time period. It uses navigation-flow data (based on navigation route data), which can be useful for traffic flow prediction. It interprets the prediction and history horizons window independently, i.e., it first calculates for a first window and then inputs its hidden state to a second window. The method is not suitable for use in real-time applications.

[0017] US 9,207,098 B2 discloses a short-term travel-time prediction modelling augmented with radar-based precipitation predictions. The predictions are calculated via vanilla ML regression model based on local information. The calculations do not utilize network-level connections, thus are unable to make predictions on a large-scale road network. US 2021 / 0302960 A1 relates to vehicles and discloses a method for generating driving recommendations. The road is divided into lane-level cells, and driving recommendations are based on a predicted future status of the lane-level cells. Driving manoeuvres such as changing lanes, braking and accelerating are also considered in the model. The vehicle can communicate with network nodes such as road side units (RSUs), users’ mobile devices and other vehicles, in the latter case direct vehicle-to-vehicle (V2V) communication can be used. In a high speed highway environment a more coarse resolution is applied with a longitudinal length of 5 m per cell. In contrast, in an urban setting a more granular cell size is applied, e.g., a cell size having a longitudinal length of 0.5 m to 1 m per cell.

[0018] US 2020 / 0145875 A1 discloses a wireless communication method for connected vehicles. The document uses lane-level accuracy positioning and considers expected future behaviour of neighbouring vehicles.

[0019] US 2017 / 0193384 A1 discloses a system and method for determining driver intention at traffic intersections for automotive crash avoidance. It relates to V2I, V2V, V2X technology, and uses it to determine path history and path prediction to determine traffic flow and direction. A turn signal activation or braking data can also be used as an input.

[0020] US 2020 / 0096359 A1 discloses generation of real-time HD maps using wireless data of a remote vehicle. It considers positions with a lane-level accuracy and also tracks lane-changes. V2X wireless messaging can be used as a wireless communication with the remote vehicle.

[0021] US 2019 / 0311614 A1 relates to connected vehicles and discloses real-time traffic monitoring with them. The document discloses estimating traffic status in lane-level; however, the prediction area is very limited in space, i.e., it can not be used for estimating traffic over an extended route network.

[0022] CN 107564281 A relates to a WIFI-based macro traffic flow algorithm. According to the algorithm, a prediction device system is deployed along a road, which acquires broadcast type data packets through a wireless passive sensing mode, screen out data packets carrying mobile terminal device ID information. The broadcast type data packets are randomly sent by mobile terminal devices to an ambient environment; and macro traffic flow is evaluated and predicted through data analysis of time-stamped data packets. The method can only be used where the prediction device system is deployed.

[0023] CN 117671954 A relates to an urban traffic flow prediction method and system based on graph knowledge distillation. The technical solution can record adjacent association information between road sections, and can expresses a spatial structure relationship between the road sections.

[0024] The technical solution in US 11 ,223,969 B2 includes a traffic prediction block and the document also discloses that, e.g., via V2V communication, information can be exchanged about a lane change or an emergency braking. The document discloses that predictions are made along a route.

[0025] US 2021 / 0394797 A1 also mentions macro-level traffic flow prediction, i.e., prediction for an entire road network (e.g., prediction of potential congestion, prediction of potential incidents, prediction of network traffic demand, prediction of network status, prediction of network travel time). The time scale at a macroscopic level is longer than 1 s is relevant to tasks such as route computing.

[0026] US 2022 / 0319309 A1 relates to a method for traffic state estimation of a road network based on a plurality of vehicles including probe vehicles and non-probe vehicles. The method includes receiving probe vehicle data from the probe vehicles within a communication range of a host vehicle, and the method also includes spatially and temporally associating the probe vehicle data to lane level cells of the road network, and identifying empty lane level cells of the road network where the probe vehicle data is unavailable. A traffic density value for the road network is calculated based on the probe vehicle data and the estimated non-probe vehicle data.

[0027] CN 111523706 A relates to a cross-section lane level short-term traffic flow prediction method based on a deep learning combined model. CN 110414707 A discloses a computer implemented method for lane-level hazard prediction, e.g., lane closures, damaged vehicles, collisions, and / or debris on a road.

[0028] DESCRIPTION OF THE INVENTION

[0029] The primary object of the invention is to provide a method and a system for traffic prediction, which is free of the disadvantages of prior art approaches to the greatest possible extent.

[0030] Traffic flow prediction (TFP) provides predictions about the future traffic states (e.g., speed, volume, occupancy, density) based on recent traffic measurements at sampling points over an input road network. In some cases traffic flow prediction can also be used to predict a volume of traffic, congestion levels, or an estimated travel time between two points; however, all of these parameters can be derived from the traffic speed.. To provide relevant information for connected use-cases, such as emergency vehicle rerouting or traffic light control, one needs to predict accurately the traffic flow in a larger, interconnected road structure, wherein the road area comprises numerous points of interest at which one would like to know certain characteristics, such as speed, occupancy, and / or volume, of the passing by traffic. For this, one needs to define points of interest at which the predictions are to be made and also the connectivity between the points of interest is needed to be defined. The defined points of interest can be points in which relevant measurement data can be provided (measurement points), such as positions corresponding to loop detectors or any other fixed sensors, i.e. , fixed road-side cameras, LIDARs, etc. Preferably, the predictions for the defined points of interest are made all at once, i.e., simultaneously. Preferably, the predictions are made for future discrete timestamps (time instances).

[0031] Accurate representation of the traffic state is a vital asset for this task, thus collecting and providing rich and reliable data as an input has an utmost importance. However, most of the current systems and solutions know in the art either require the existence of a sampling infrastructure deployed for this specific purpose, or use discretized pre-processed information, which may not be available in many places. For example, there are areas or countries where there has been no need for such a service before, thus the required sampling infrastructure is not available, and even if such infrastructure exists, the deployment of it may be costly.

[0032] Usually, the sampling infrastructure does not have sufficient computational capacity to process the collected information, thus, application of a sampling infrastructure requires internet connectivity (such as LTE) to forward the collected information from the sampling infrastructure to a processing system, that has a sufficient computational capacity. As an example, the processing system can be implemented in a fog or in a cloud. Furthermore, the sampling infrastructure is prone to technical failures, such as power outages (e.g., all the sampling infrastructure in an area shuts down), which can cause a processing system to lose its primary input data source for a potentially significant amount of time.

[0033] An other group of prior art solutions uses GPS-based information coming from an end UE, such as smartphones, however these on the one hand may be less accurate, as one needs to measure estimated speed / volume based on raw position information, and on the other hand rely heavily on extensive mobile network coverage.

[0034] In conclusion, the known prior art solutions do not provide predictions on a lanelevel basis, but also mostly utilize raw speed or volume measurements as an input, and as a result, the accuracy might be lower in some cases due to insufficient information quantity or because the versatility of such application options may be compromised, e.g., making predictions on side-level only. These solutions also have limited coverage, meaning that the prediction service may only be provided where GPS data collection or sampling infrastructure is present.

[0035] Therefore, the objective of the present invention is to provide a method and a system for traffic prediction that aims to accurately predict future traffic states with high availability, low latency, and extended applicability

[0036] A further object of the method and the system according to the invention is to accurately predict wide-scale lane-wise traffic states in real-time with general availability. Furthermore, the objective of the invention is to provide a non-transitory computer program product for implementing the steps of the method according to the invention on one or more computers and a non-transitory computer readable medium comprising instructions for carrying out the steps of the method on one or more computers.

[0037] The objectives of the invention can be achieved by the method according to claim 1 . The objectives of the invention can be further achieved by the system according to claim 12, a non-transitory computer program product according to claim 18, and by the non-transitory computer readable medium according to claim 19. Preferred embodiments of the invention are defined in the dependent claims.

[0038] The main advantage of the system according to the invention compared to prior art approaches is that the method and the system according to the invention allows for macroscopic traffic modelling and prediction, i.e., predictions are not limited to a small local area around a single vehicle or a single intersection, but predictions are made for a larger, macroscopic area with multiple interconnecting roads.

[0039] A further advantage of the method and the system according to the invention is that it allows for predicting the traffic with a lane-level accuracy, compared to side-level accuracy, where in the latter case predictions are made for each side of the road having opposite direction of traffic, but the possible lanes on each side are not distinguished or treated separately.

[0040] An other advantage of the method and the system according to the invention is that it enables several new smart city use-cases related to traffic safety and efficiency, such as traffic re-routing and traffic light control, in which latency is a critical factor.

[0041] The method and the system according to the invention can receive real-time traffic information, based on which almost real-time traffic predictions can be made, i.e., the predictions can be made within a few seconds or within a few minutes at most.

[0042] Certain embodiments of the invention can adapt to changes of the road network, wherein the changes can be due to actual changes on the structure of the roads, e.g., when new lines or junctions are added to existing roads, or when predictions are made for a different road network segment, i.e., for a different region. It has been recognised that by utilizing transfer learning, changes in the road topology does not require re-training the whole system, which makes the system more effective and versatile.

[0043] Other embodiments of the invention can integrate data from existing sampling infrastructure, e.g., roadside infrastructure (such as an RSU, a loop detector, a camera, etc.), and possible sampling inefficiencies on the lane level can also be identified. For example, malfunctions of a sampling infrastructure, even if it only affects a single lane, can be identified and then fixed for contributing to a better prediction and safer traffic. Some possible malfunctions that can be identified via the system according to the invention are listed below:

[0044] - an RSU cannot collect information efficiently from a lane due to reduced coverage caused by an issue with its radio antenna,

[0045] - a loop detector's inductive system fails to function for a plurality of lanes,

[0046] - a camera's vision is partially (e.g., for one of the lanes) obscured by an object.

[0047] The method and system according to the invention allows for not only realizing a transport infrastructure to its fullest potential from a traffic prediction point of view, but it can aid the development and maintenance of its equipment as well. For instance, a node-wise prediction accuracy of a model can be analysed and spots of improvement (e.g., spots that has a significantly lower accuracy than the average) can also be identified, and based on this information equipment replacement or equipment deployment can be initiated that later further improves the accuracy of the prediction and improves safety.

[0048] BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Preferred embodiments of the invention are described below by way of example with reference to the following drawings, where

[0050] Fig. 1A shows an exemplary road structure with measurement points as nodes,

[0051] Fig. 1 B shows the road structure of Fig. 1 with graph edges between the nodes,

[0052] Fig. 2 shows measured speed at a measurement point of Fig. 1 A within a day and predicted speed at the same measurement point, Fig. 3 illustrates an exemplary implementation of the method according to the invention for predicting traffic flow,

[0053] Fig. 4 is a block diagram of a preferred implementation of the system according to the invention,

[0054] Fig. 5 is a block diagram of a preferred implementation of a connected traffic orchestrator unit,

[0055] Fig. 6 shows a vehicle’s trajectory association with the measurement points, Fig. 7 is a process diagram for prediction in a preferred implementation of an inference module, and

[0056] Fig. 8 is an illustration of the domain transformation of the lane-level traffic prediction.

[0057] MODES FOR CARRYING OUT THE INVENTION

[0058] This invention relates to a method and a system for traffic prediction that aims to accurately predict future traffic states with high availability, with low latency, and extended applicability.

[0059] The method according to the invention is for traffic flow prediction, and it comprises a step of receiving traffic-related information from a static unit and / or a dynamic unit, wherein the static unit is preferably a road-side unit, a loop detector or any other static infrastructure sensor adapted to record a passing-by traffic on a lane-level, and the dynamic unit is preferably a vehicle, a smartphone, or an on-board unit of a vehicle, etc. The traffic-related information is preferably extracted from a message, e.g., a V2X message, received from the static unit and / or the dynamic unit. The traffic-related information received through a message is preferably aggregated and associated with an element of the road graph representation, i.e., with a node or an edge. The traffic-related information is preferably one or more of the following: a speed of traffic, an occupancy, a volume of traffic, a density of traffic, a point-of- interest, a location of an event, weather data, or road work. Even more preferably, the traffic-related information further includes at least one of the following: a turn signal status of a vehicle, a lane change request of a vehicle, an emergency brake activation of a vehicle. The method according to the invention further comprises a step of performing traffic flow prediction by a prediction unit, wherein at least one future state of traffic is estimated based on the received traffic-related information, and sharing the estimated at least one future state, by a prediction sharing module. The estimated at least one future state is preferably shared with a connected traffic orchestrator unit, and / or with a traffic management centre (TMC), and / or as part of a V2X message.

[0060] The method according to the invention further includes creating a road graph representation comprising nodes and edges between the nodes, wherein the road graph representation has a lane-level resolution.

[0061] The method according to the invention optionally includes a step of assigning the received traffic-related information to a corresponding lane before performing the traffic flow prediction step to generate a lane-matched traffic-related information. If the received traffic-related information is already a lane-matched traffic-related information, this optional step does not need to be performed.

[0062] The traffic flow prediction step is performed by deep learning based on the road graph representation and on the lane-matched traffic-related information, therefore a lane-level prediction can be generated on the road graph representation.

[0063] The step of creating the road graph representation having a lane-level resolution is preferably performed by the following steps:

[0064] - defining the nodes of the road graph representation based on locations of static units, and / or based on a predefined map, and / or based on predefined locations, and

[0065] - defining the edges between the nodes based on physical connectivity of the nodes or based on a logical connectivity of the nodes, wherein physical connectivity is preferably determined between the nodes based on a physical distance between the nodes or based on map-based information. The method according to the invention preferably further comprises a step of updating the road graph representation via transfer learning to a new road graph representation, wherein transfer learning is preferably performed by the following steps:

[0066] - applying a trained graph neural network having a first layer, a last layer and one or more intermediate layers between the first layer and the last layer, wherein the trained graph neural network is trained for a road graph,

[0067] - removing the first layer and the last layer of the trained graph neural network,

[0068] - freezing all the parameters of the one or more intermediate layers, and

[0069] - re-training the graph neural network in an E2E fashion on the new road graph representation to create a new first layer and a new last layer.

[0070] Preferably, the step of assigning the received traffic-related information to a corresponding lane is performed by point-based map-matching and / or by trajectorybased map-matching.

[0071] The invention provides a traffic flow prediction method and system that is preferably V2X-assisted and aims to accurately predict future traffic states with high availability and low latency. The invention allows a macroscopic evaluation of traffic, meaning that it can work on large areas of road networks. The method and system can cooperate with RSUs, OBUs, and the system comprises a prediction unit (that may or may not be an RSU itself). In a preferred implementation of the method according to the invention, RSUs gather V2X messages, such as CAMS or BSMs, from OBUs in the vicinity (within the RSU’s range, which requires a sufficient level of RSU coverage in the area of interest) that contain positional and speed information as well as auxiliary characteristics, such as a turn signal status, a lane change request, or an emergency brake activation. Then, if this information is not yet lane-matched (map matched with lane-level precision), then either the OBU (before sending out the data), the RSU, or the prediction unit can perform map matching to assign the information to specific lanes. If the V2X coverage is low, for this purpose, loop detectors may also be used. Preferably, lane-matched information is passed to the prediction unit, which may run on, for instance, one of the following environments: an RSU; a MEC application on a MEC server; or a central instance, such as a virtual cloud environment.

[0072] The prediction unit can also create a (preferably local static) road graph representation with lane-level details. For this, it first defines a road graph topology. The definition is based either on a location of proximal loop detectors or via a generation module (that, for instance, defines these points in an equally distributed way from a distance point of view). Then, an adjacency matrix is assigned to these nodes either via information from an HD map provider or via any road distance calculator provider, e.g., Open Route Service API (which calculates the physical, route-based distance between two given locations).

[0073] If the road graph is not known before, i.e., new, then the inference module can employ a transfer learning technique to retrain its prediction parameter values. This way, the method according to the invention can adjust to any graph topology. Transfer learning can be implemented via re-training the first and last layers of the neural network in an E2E fashion while freezing the other parameters. The inference module is responsible for calculating the future traffic states for a time horizon or a series of time horizons (e.g., from 5 minutes up to 180 minutes ahead, with 5-minute steps).

[0074] According to the method, the predictions can be shared via a prediction sharing API, for example, in one of the following ways:

[0075] - to any RSU, so that it can provide its own traffic service to its listeners, e.g., other OBUs,

[0076] -as part of a V2X message (e.g., a DENM message) with a predefined frequency (e.g., 1 minute), or

[0077] - directly to a TMC that can utilize this information to provide traffic analytics to traffic engineers.

[0078] An inventive advantage according to the method and system of the invention is that lane-level traffic prediction can be made on a macroscopic, i.e., on a large, interconnected road graph (representing the problem as a point-wise prediction problem) by utilizing various kinds of inputs along with common speed information coming from various sources, such as roadside sampling infrastructure or C-ITS stations. The invention can combine CV trajectory-based inputs with special V2X- based inputs and utilize them to the widely studied (e.g., Traffic4cast, Google, etc.) graph-based traffic prediction task with lane-level granularity.

[0079] The method according to the invention can utilize V2X data (and its special, unique fields) as input., aside Besides “conventional” speed or volume information, the invention allows use of additional data fields that may be introduced via V2X processing, such as lane change or harsh braking frequency. Application of these additional information can increase the accuracy and reliability of the predictions.

[0080] The method according to the invention can assign received inputs to lanes via map matching, i.e. , lane-matching (if no loop detectors are available or if the received inputs are not already map-matched). Lane-level inputs introduce richer, more granular inputs, allowing the traffic prediction task to be modelled on a finer level. Most of the existing methods can only perform side-level predictions (with one measurement representing all the lanes combined). For lane-level predictions (as output), lane-level traffic flow (as input) is necessary; however, current macroscopic models that predict on a lane-level assume the presence of loop detectors (that provide the lane-level information directly), which is not the case for V2X-based inputs by default. If the inputs are lane-matched, then additional external lane-level information, such as (virtual) lane closures or the presence of priority lanes (typically in the US), can be utilized as well.

[0081] Embodiments of the method translate the continuous domain (space) into a discrete domain. Like GPS-based methods, V2X messages are transmitted frequently (e.g., sampled every 100 ms or seconds); however, as mentioned already, the efficient formulation of the problem requires a discrete road graph representation, thus this invention provides a method that allows to gather and aggregate these V2X messages.

[0082] An efficient way to model macroscopic traffic prediction is via Graph Neural Networks (GNN) - especially via Graph Convolutional Networks (GCN), which is a special GNN - along with Attention Mechanisms. For GCNs and Attention, the underlying model is tied to the specific dimensions of the input. For instance, a GCN trained on a specific road graph may not be reused directly on another one, even if the difference is as small as a single node. The method according to the invention provides a way to reapply a well-trained model to a new graph, via transfer learning. If a GNN is trained on a graph that preferably represents a larger region (collection of interconnected road graphs) in both size and nature, then, for each specific road graph within that larger region, the first and last layers of the network can be removed, and the intermediate parameters are to be frozen, and the new first and last layers (that are specific to the corresponding new graph) can be retrained in an E2E fashion. This improves the applicability of such models drastically, as it is not requiring to collect a vast amount of input data (which is usually the case for deep learning-based methods) for each graph. Additionally, graph-specific auxiliary information may be used as well to aid this process.

[0083] Lane-level predicted traffic state greatly enhances Advanced Traffic Management’s (ATM) capabilities (for instance, by making certain busy lanes chargeable).

[0084] The advantages of the method and system according to the invention include the following:

[0085] The method and system according to the invention does not rely on reliable network coverage (aside from the V2X infrastructure itself), i.e. , it may be used in the middle of a desert, as the data collection process is V2X-based. The RSUs could, for instance, download a HD map information beforehand from a provider before installation, thus eliminating the need for further requests. Should the area have sufficient network coverage, this surface may also be deployed on MEC servers.

[0086] The method associates measured speed and other traffic-related characteristics to lanes, making the input richer and more granular.

[0087] Prediction are performed on a lane-level, meaning that the output will also be associated with specific lanes. The method according to the invention employs, beyond conventional speed and other externally accessible information, the use of V2X-specific data fields (mentioned above) that may be acquired only via V2X communication.

[0088] Figs. 1 A and 1 B show an exemplary road structure on which the method and system according to the invention can be applied. Fig. 1 A also shows a typical area for traffic flow prediction according to short-term predictions of prior art solutions. Considering an ego-vehicle 10 traveling on the road, short-term predictions are usually made for a prediction area 12, denoted by a rectangle.

[0089] Contrary to the typical prior approaches, the invention allows for predictions for a much larger area than the prediction area 12, as depicted it in Fig. 1A. For the purpose of the invention large road structures, macroscopic regions can also be applied, i.e. , not only a part of a city (e.g. a close surrounding of an intersection), but whole cities and also interconnecting roads between the cities. For calculation purposes a map is preferably converted into a graph having nodes 14 and edges 16. The nodes 14 are measurement points (or points of interests), wherein traffic related data is collected or available. For example, an actual speed of individual vehicles or an average speed of the passing traffic can be measured in a measurement point. Measurement of other data, such as weather data (contributing to visibility) and road surface data (contributing to surface conditions) can further increase the quality of the traffic predictions. Further data can also taken into consideration for traffic predictions, e.g., points of interest (POI) such as stadiums, racing tracks or event data (football match), which also affect traffic comditions at a certain location in certain time windows. Furthermore, travel-related data received from V2X-capable entities can also be used as “measurements”, in which case V2X data can be aggregated and / or associated with the measurement points, which is discussed in more detail in connection with Fig. 6.

[0090] Fig. 1 B shows the graph of the road structure of Fig. 1 A in more detail, i.e., including relevant edges 16 between the nodes 14. An edge 16 represents a connection between two nodes 14 (measurement points) based on their physical distance. Fig. 2 shows (with dashed lines) actual measured speed of traffic in one of the measurement points of Fig. 1A. Fig. 2 also shows a prediction (denoted by a continuous line), wherein the prediction is calculated 60 min ahead of the measurements. A simplified way how predictions are made is shown in Fig. 3.

[0091] Fig. 3 illustrates the basics on the predictions performed by the method and system according to the invention. First, traffic-related data, such as speed is collected in the measurement points. Between the measurement points, data is interpolated based on the collected data in the measurement points. According to the example of Fig. 3, data collected in the last 60 minutes is used as a basis, i.e. , as a dynamic information. This is extended by recurring patterns (time information) and structural connections (road graph topology), and based on this data, a series of predictions are performed, preferably in 5 minutes time-steps up to 3 hours (180 minutes). This means, that the speed of traffic is calculated based on past measurements for future time instances. Predictions are preferably made for each measurement points (prediction points) in the region or in a relevance area, and predictions can be interpolated between the locations of the measurement points (prediction points). Traffic-related parameters to be predicted can be for example: speed, occupancy, traffic volume, etc.

[0092] Preferably, predictions for each measurement points (prediction points) are made simultaneously, and predictions are made for future discrete timestamps.

[0093] In this specification, speed will be used as a parameter to be estimated and predicted, as this parameter relates closely to the flow of the traffic. It is noted that other traffic descriptors (such as volume, occupancy, density, flow, etc.) can also be used as these offer similar ways to estimate other parameters, such as congestion. For instance, a way to estimate speed-based congestion can be based on a ratio of current / predicted speed versus a free-flow speed of a road section, whereas density-based congestion could be described with a ratio of current / predicted volume versus no volume. A combination of other traffic descriptors (e.g., speed and volume) can also be used.

[0094] Fig. 4 shows a block diagram of a preferred implementation of the system according to the invention, comprising a prediction unit 60 as its main component. The arrows between the blocks represent a preferred flow of data or information, however in some cases data, information or requests can be transmitted in the opposite direction as well. In some cases, more connection channels can exist between blocks of the prediction unit 60 than as depicted in Fig. 4. The individual blocks can be implemented as different units (e.g., either physical components of a system or as software components), but in certain embodiments some of these can also be implemented as jointly, in which case one physical or software component can have multiple functions. The blocks having a dashed outline denote optional components, and dashed arrows denote optional connection between certain blocks.

[0095] The prediction unit 60 is preferably configured to provide predicted traffic parameters based on inputs, e.g., measured data of one or more relevant traffic parameters. The traffic parameters can be speed, volume, occupancy, or density, etc.

[0096] An input of the prediction unit 60 can be provided by a static unit 20 and / or a dynamic unit 22. As an example, a static unit 20 can be a sensor or detector at a fixed location, such as a loop detector, a camera, a lidar, etc. A static unit 20 can be mounted on a roadside unit (RSU). Preferably, the static unit 20 provides traffic- related data with a lane-level accuracy, e.g., as a loop detector arranged in a lane, or a camera or lidar facing (or monitoring) a single lane, providing lane-level traffic- related input (measurement data) for the prediction unit 60. It is also possible, that a static unit 20 provides aggregated data (aggregated measurements) with lanelevel granularity based on a snapshot of the passing-by vehicles (e.g., speed and / or volume). However, a static unit 20 is less fit for providing information about a lane change, as such an event can be detected with multiple snapshots or with multiple static units 20 arranged spaced apart along a road section.

[0097] As an example, a dynamic unit 22 can be a vehicle or a device located in a vehicle, e.g., a smartphone in a vehicle, an onboard computer (onboard unit, OBU) of a vehicle, etc. A dynamic unit 22 is preferably adapted to measure and record data related to vehicle dynamics and / or close-range traffic events related to the vehicle. As an example, a device like an OBU can record granular events (even a lanechange) from a V2X message 23. Preferably, the device of the dynamic unit 22 is capable of recording its own position by any known technology, e.g., GPS (GNSS with or without RTK, RTCM), GLONASS, cellular-based, etc.

[0098] Preferably, the dynamic unit 22 is able to share its recorded measurements and events via a connectivity module (preferably part of the dynamic unit 22), and it is even more preferred, if the dynamic unit shares its measurements periodically. For such information sharing any known connectivity technology can be used, e.g., cellular (4G / 5G / 6G... ), Wi-Fi, V2X (C-V2X, DSRC), or Bluetooth, etc.

[0099] Due to the different capabilities of a static unit 20 and a dynamic unit 22, different processing is needed for the different inputs before their provided data can be utilised for prediction. It is also possible that multiple static units 20 and multiple dynamic units 22 are connected to the prediction unit 60 as data sources, which on the one hand can improve the accuracy of the prediction, but on the other hand requires different data processing for different data sources.

[0100] A dynamic unit 22 can be configured to provide raw measurement data or a V2X message containing traffic-related information. It is also to be noted, that some static units 20 (e.g., RSUs) can also share V2X messages 23 containing traffic-related data.

[0101] The prediction unit 60 of Fig. 4 comprises a CV (collected vehicle) data ingestion module 24, preferably for collecting and processing microscopic traffic parameters (for example, dynamics, such as speed, and / or events, such as a lane change) along with position information from the dynamic unit 22 or multiple dynamic units 22.

[0102] The data collection process performed by the CV data ingestion module 24 may involve additional, intermediate units and / or steps, not depicted on Fig. 4. For example, an RSU can provide further data to the CV data ingestion module 24 to be processed, e.g., an RSU can add further data or information to a V2X message forwarded by the RSU.

[0103] The CV data ingestion module 24 can accommodate various technologies, processing V2X messages 23 is just a possible implementation. The CV data ingestion module 24 preferably collects data of all the traversing vehicles which are equipped with connectivity modules, preferably Furthermore, the CV data ingestion module 24 collects the information emitted by the dynamic units 22.

[0104] Dynamic units 22 can share information regularly or irregularly. Therefore, as an intermediate step, pre-processing can be applied by one or more receiving devices, wherein the receiving devices can forward the information to the prediction unit 60. The prediction unit 60 can then process this information and apply any necessary aggregation (e.g., sequence generation by a sequence generator, or trajectory association). Any known aggregation technique can be used, for example, the ones that are common in stream processing, e.g. time window-based, or identification (ID)-based aggregation techniques.

[0105] From the information collected by the CV data ingestion module 24, data relating to the dynamic of a vehicle and any events that happened in a data collection time window can also be extracted. In some cases, data shared by a dynamic unit 22 is received by a static unit (e.g. RSU, MEC with V2X stack, etc.) and then the information is forwarded from the static unit 20 to the CV data ingestion module 24. The static units 20 can complete the information received from the dynamic units 22 with further information or can structure or embed the information received, therefore it can be easier to extract relevant information by the CV data ingestion module 24. For example, specific information can be embedded into specific fields of a V2X message 23, therefore such specific information can be extracted directly from a specific message field or part of a generation rule / logic.

[0106] In order to effectively use the information extracted from the dynamic units 22 by the CV data ingestion module 24, the information preferably needs to be map-matched. If the information originating from a dynamic unit 22 is already map-matched, then this information can be passed to a trajectory association module 32, which will be discussed in more detail below.

[0107] If the information extracted in the CV data ingestion module 24 is not yet map- matched, then preferably map-matching is performed, e.g., trajectory-based or point-based map matching. Point-based map matching is a simpler, less effective approach, preferably performed by a point-based map matching module 30. The point-based map matching module 30 matches each individual measurement (e.g., position, heading, etc.) to a specific lane. The road structure, including the position and properties of the lanes are provided by a map provider.

[0108] An other possible approach to perform map-matching is trajectory-based, preferably performed by a trajectory-based map matching module 28. In order to perform trajectory-based map matching, a sequence of measurements needs to be created, preferable by a sequence generator module 26. The sequence generator module 26 preferably organises the collected, extracted information received from the CV data ingestion module 24 into a series of measurements (preferably between measurement points / nodes of the graph), because, for example, vehicles as dynamic units 22 can provide measurement data / information not only in the nodes 14 of the road graph, but also between nodes 14.

[0109] Measurement data that is associated with a sequence is tied to the dynamic unit 22 that provided the measurement data (e.g., sent V2X messages 23 containing the measurement data). The connection between each sequence and its respective dynamic unit 22 is preferably based on IDs, wherein an ID is defined by a connectivity module in its own way. For instance, a V2X connectivity module may use pseudo IDs that they are only static for a predefined amount of time for security reasons, which means that if a new pseudo ID is assigned, a new sequence is defined.

[0110] The trajectory-based map matching module 28 preferably performs map matching by assigning each vehicle to a specific lane based on position information of the corresponding sequence.

[0111] Once map-matched (more preferably lane-matched) measurement data / information is available, it is passed to a trajectory association module 32, which aggregates the sampled measurements for all vehicles (e.g., an average speed of vehicles). As measurement data can be received from locations different from the locations of the nodes 14, measurement data is preferably assigned to a node 14, e.g., based on a smallest distance, i.e. , a node 14 closest to the location of a specific measurement data, or every measurement data can be assigned to a node 14 if the measurement data was taken within a predefined range of the node 14 (e.g., within a 100-meter radius of a node 14).

[0112] In cases of sequences, aggregation can be performed by first grouping the sequences spatially, e.g., by assigning the individual point-based measurements or the individual trajectories to a single graph element (a node 14 or an edge 16).

[0113] Once the sequences are grouped spatially, they need to be grouped further and be assigned to time windows, e.g., spatially grouped measurements are assigned to the same "bucket" if the measurements happened during the same (e.g., 5-minute) window. The time windows can be defined based on a sampling frequency. Within a time window the information is preferably aggregated (e.g., averaged), resulting in a spatio-temporal aggregated sequence. The goal is to create a similar structure as it would have been received from the static units 20, e.g., just like loop detectors provide snapshots of an average speed of the passing-by traffic in a last X minute. It allows that a traffic characteristics aggregator 38 can combine measurements from different origins.

[0114] An aggregated sequence is assigned to a corresponding element or elements of the graph (a node 14 or an edge 16). For example, speed measurements can be assigned to both edges 16 and nodes 14, while for some other types of measurements (e.g., lane change frequency between two nodes 14), it is more meaningful to associate with an edge 16 only.

[0115] Measurement data or information received from static units 20 is preferably fed into a point-wise association module 36 that is adapted to assign the measurement data of the static units 20 to the road graph defined by a road graph generator 54.

[0116] The prediction unit 60 preferably includes a data signal association module 34 that is adapted to collect information about dynamic units 22 via the static units 20. A static unit 20 (e.g., an RSU) can detect and track the movement of a dynamic unit 22 periodically via certain measurement technologies (e.g., via Bluetooth). By extracting some characteristics (e.g., MAC address, signal strength, power, etc.) of a device (such as Bluetooth headset, mobile phones, etc.) in the dynamic unit 22, the static unit 20 is capable of assuming and estimating an elapsed time for a tracked vehicle (dynamic unit 22) between two static units 20, and thus an average speed and / or travel time can be derived / estimated. Using an average of such values, data signal association module 34 can provide information (e.g., speed information) of the underlying traffic between locations of two static units 20, or between a series of locations (i.e. , an edge 16, or edges 16 in the graph). The data signal association module 34 this way can assign values to the edges 16 of the graph, after which - preferably with an isomorphic transformation - the values can be assigned to the nodes 14 instead, similar to the other modules or blocks of the prediction unit 60.

[0117] The prediction unit 60 can further include a traffic characteristics aggregator 38 that is adapted to aggregate measurements coming from both static units 20 and dynamic units 22, by applying data transformation functions, if necessary, e.g., in case two input sources (static units 20 and dynamic units 22) have a different coordinate system, unit scale, etc. Data transformation functions of the traffic characteristics aggregator 38 can include calculating an average, a weighted average or normalizing.

[0118] The prediction unit 60 needs a graph of the road structure to calculate the predictions, therefore the prediction unit 60 preferably includes a graph definition generator 46 that is responsible for defining the nodes 14 of the graph. The nodes 14 of the graph can be arbitrarily defined, or be based on already available information, such as actual positions of the static units 20 (static unit locations 44), if static units 20 are present in a given road area. If a predefined road network (e.g. provided by a map provider) is provided, it can also serve as a basis for defining the nodes 14 of the graph.

[0119] The prediction unit 60 preferably further includes a graph associativity generator 48 that is adapted to establish a physical connectivity between the nodes 14 of the graph, based on which the edges 16 of the graph can be determined. The physical connectivity can be determined by calculating a route-based distance (a distance along the route) between the nodes 14 or an explicit road network definition can be given by a map provider, preferably in a form of a high-definition (HD) map 42. Furthermore, the prediction unit 60 preferably includes a graph topology generator module 50 that is adapted to establish a logical connectivity between the nodes 14 of the graph (i.e. , the actual edges 16). Although the physical connectivity calculated by the graph associativity generator 48 can be used to determine the edges 16 as it can also reflect the underlying connectivity of the nodes 14, the logical connectivity can be different. The physical connectivity and the logical connectivity can be different, if for example the edges 16 determined on the basis of physical connectivity (physical edges) are too frequent (the density of their spatial distribution is skewed), or too rare, or if some regions are under or overrepresented. These examples call for re-defining the edges 16 between the nodes, e.g., deleting edges 16 if otherwise the edges 16 are too dense, or adding edges 16 where the nodes 14 are sparsely connected in a region. The graph (road topology) can be stored, for example, as a sparse adjacency matrix.

[0120] The prediction unit 60 preferably includes a road graph generator 54, which is adapted to handle region-wide / area-level specific road graphs. The road graph generator 54 preferably can perform a domain transfer between graphs via a transfer learning module 52. Furthermore, the road graph generator 54 is preferably adapted to generate or update graph topology for new or modified areas (i.e., if new road segments or connections are built, or roadworks temporarily disable one or more edges 16 or lanes).

[0121] The transfer learning module 52 is adapted to transfer a domain and model knowledge between areas and road graphs. In some cases, map data can be available in multiple parts (domains), e.g., for different countries, states or regions. However, vehicles can move between different domains (move between different countries, states, etc.). In such cases, a continuity of the graph and the prediction still needs to be maintained, therefore regional (domain-based) parameters need to be transferred, which requires a model-independent or domain-dependent transfer learning.

[0122] It is possible that a region can be divided into multiple subregions, and these subregions share specific parameters originating from a parent region. It is also possible that there are general rules / parameters that apply to multiple, non- intersecting regions equally. Such parameters can include driving and traffic behaviour aspects / patterns as well as regional regulations, habits, or road design features.

[0123] Predictions are preferably performed by an inference module 40 of the prediction unit. The predictive model of the inference module 40 can learn certain general structural components inside the model (e.g., by training for achieving the highest level of accuracy) that can be independent from the topology of the road graph. This knowledge can also be transferred to other parts of a road graph.

[0124] With the transfer learning module 52, model parameters can be transferred between models, functioning on different graphs. This can be applied to methods that incorporate the structure of an underlying road graph explicitly in their calculations. Amongst others, certain machine learning-based methods and, in particular, specific deep-learning-based methods (such as GCN-s) may be adopters of such a modeldependent and domain-independent transfer learning technique.

[0125] The inference module 40 of the prediction unit 60 is adapted to perform predictions based on the inputs (lane-matched traffic info 39) coming from the dynamic units 22 and static units 20 as well as other external data sources. As already discussed, the lane-matched traffic info 39 can be already pre-processed by different blocks (components) of the prediction unit 60. The prediction process as performed by the inference module 40 is further discussed in connection with Fig. 7.

[0126] Preferably, the inference module 40 performs predictions for elements of a graph, i.e. , for nodes'! 4 and / or edges 16. Said graph may or may not be equal to the road graph given as input to the inference module 40.

[0127] For instance, if static units 20 are rare and consequently underrepresenting the underlying traffic, then adding intermediate prediction points (supported by the input coming from the dynamic units 22) is reasonable and can be carried out by the inference module 40.

[0128] On the other hand, if a road graph includes more elements than the measurements static units 20 and dynamic units 22 can provide, interpolated predictions based on the graph’s structure can also be performed. Preferably, the prediction module 60 includes a feedback module 56 that can be used as a feedback loop for the road graph generator 54. In particular, unbalanced prediction inaccuracies (e.g., if some part of the road graph performs worse than the average from an accuracy point of view) may be a good indicator that the road graph (either its edge topology or node layout) is poorly designed / functioning for the prediction task. For instance, this may serve as an input on where to put additional static units 20 (based or not based on dynamic units 22).

[0129] Preferably, the feedback module 56 can directly communicate with not only the inference module 40, but also with the traffic characteristic aggregator 38, the road graph generator 54, and / or the graph topology module 50.

[0130] The inference module 40 is preferably connected to a prediction sharing module 58 that is adapted to distribute prediction information. Predictions can be distributed for example via V2X messages 62, via a connected traffic orchestrator unit 64 (see Fig. 5 for more details), or via a traffic management centre (TMC) 66 for traffic management purposes.

[0131] Fig. 5 shows a block diagram of a preferred implementation of a connected traffic orchestrator unit 64. The connected traffic orchestrator unit 64 can be implemented in the same place as the prediction unit 60, or can be implemented in a different place. For example, the prediction unit 60 may run in a cloud, whereas the connected traffic orchestrator unit 64 may be a collection of applications and services running on the MEC.

[0132] The connected traffic orchestrator unit 64 preferably includes a predicted traffic state evaluator 84 that is adapted to evaluate a predicted state of the traffic (predicted traffic information 82), such as identifying congestion levels across a prediction area, calculate predicted travel times, etc Afterwards, based on these predicted states, for congested segments or areas, connected traffic orchestrator unit 64 may identify which graph elements (based on information coming from the prediction unit 60) will have the largest contribution to create such predicted congested states (predicted congestion propagation). This is desired in congestion prevention, as the congested state of a road section is usually created from a different road section. The connected traffic orchestrator unit 64 can assign proactive properties to certain areas / elements of the map (e.g. a lane), such as a lane change penalty zone, a congested lane, etc. These assigned properties can be also added to distributed map format sharing services, such as NDS.

[0133] The connected traffic orchestrator unit 64 preferably includes a road infrastructure rule generator 86 and a vehicle rule generator 94, wherein both can receive evaluations of the predicted traffic state evaluator. Both the road infrastructure rule generator 86 and the vehicle rule generator 94 can have a series of processing components (e.g., application servers / applications), wherein each processing component can have a certain triggering condition. If an output of the predicted traffic state evaluator 84 suggests certain foreseeable traffic inefficiencies, then the corresponding processing components shall initiate.

[0134] The role of the processing components of the road infrastructure rule generator 86 and the vehicle rule generator 94 is to adjust the current traffic setup (either the infrastructure or the vehicles, respectively) to eliminate the predicted traffic inefficiency. In other words, the road infrastructure rule generator 86 and the vehicle rule generator 94 can create new traffic rules and commands for the infrastructure or the vehicles, respectively.

[0135] The road infrastructure rule generator 86 is preferably connected to an infrastructure response module 88, and the vehicle rule generator 94 is preferably connected to a vehicle response module 96. The output of the road infrastructure rule generator 86 and the vehicle rule generator 94 is then passed to the respective infrastructure response module 88 or vehicle response module 96, which then prepare a response message to the appropriate units, i.e. the static units 20 and / or the dynamic units 22. The response message can be a command requests to a set of vehicles (or dynamic units 22) or an 12V (Infrastructure-to-Vehicle) message to any units managing a physical infrastructure (e.g., RSU) so that the physical infrastructure can start sharing new traffic regulations.

[0136] Exemplary processes performed by the vehicle rule generator 94 and / or the vehicle response module 96 can be a lane-change penalization, for example, to impact manoeuvres. For a penalized area, a custom weighting scheme (for instance for cost calculation purposes) can be created between lanes A and B. The calculated cost / penalty could then be shared with vehicles, so that they include it in their cooperated manoeuvres (e.g., a V2X-based MCS - Manoeuvre Coordination Service utilizes such schemes in their calculations). In such a case, the vehicle rule generator 94 calculates an additive penalty cost, which then is shared with the vehicle response module 96 that communicates this information to the dynamic units 22.

[0137] A further example on the vehicles’ side can relate to a platoon formation. In this example, the vehicle rule generator 94 can be adapted to, upon predicted congestion information, initiating a platoon formulation. Such a request could be sent to the vehicle response module 96, which performs the required calculations and / or forwards this request to the involved vehicles.

[0138] A further example for vehicles is a speed recommendation. Upon predicted congestion, the vehicle rule generator 94 can calculate a suggested speed and communicate this information to the vehicles via the vehicle response module 96.

[0139] In case of the infrastructure, the infrastructure rule generator 86 and the infrastructure response module 88 can perform the following exemplary processes. Typically, the infrastructure rule generator 86 creates a new traffic rule based on the predicted traffic states, and the infrastructure response module 88 forwards this information to the static units 20.

[0140] - Priority lane tolling: the infrastructure rule generator 86 can assign a tolling fee to one or more specific lanes in a dynamic fashion (higher fees for more congested lanes), and the infrastructure response module 88 can forward this information to the static units 20.

[0141] - Dynamic speed limits: the infrastructure rule generator 86 can adjusts a virtual speed limit (to be sent via V2X) of specific road segments, and again, the infrastructure response module 88 can forward this information to the static units 20, e.g. to display it to the vehicles.

[0142] - Dynamic lane configuration: the infrastructure rule generator 86 can change the topology of an intersection, road, or highway segment to offset an upcoming traffic condition. For example, a normal lane can be changed into a priority lane (tolled lane); a bike lane can be temporarily accessible for cars; at an intersection an additional left-turning lane can be allocated.

[0143] - Dynamic parking fees: the infrastructure rule generator 86 can adjust parking fees for congested areas.

[0144] - Traffic light optimization: the infrastructure rule generator 86 can optimize traffic signal phase scheduling based on predicted traffic.

[0145] The infrastructure response module 88 can be connected to a distributed map 92 for receiving further topological information.

[0146] Fig. 6 illustrates a trajectory association method for a vehicle. Currently, non-V2X- based traffic collection approaches are tied to a sampling infrastructure (e.g., detectors, cameras) that may collect either vehicle trajectories, speeds, or volumes. This limits where and how to predict traffic. V2X is a rich data source, and it is a new technology, so it has not been widely used for this problem. Unlike loop detectors or smart cameras, V2X messages may contain several useful fields regarding the traffic state besides vehicle speed. For example, it may be feasible to reconstruct / estimate traffic flow via V2X messages as an input, even when the coverage of V2X-equipped vehicles is very low (e.g., 2-5% only), just from position and speed information. This invention goes further and suggests that information about the use of harsh braking and lane changes can collected between measurement points and it can further increase its expressiveness when it comes to efficient traffic modelling. The extraction of these data, however, requires an adequate V2X stack.

[0147] The system according to the invention has the following primary functional units:

[0148] - map matching,

[0149] - V2X data ingestion and extraction, and

[0150] - traffic prediction via neural network-based calculations with lane-level topology, as it was discussed in more detail in connection with Fig. 4. Trajectory association can be part of V2X data ingestion and extraction. The V2X messages 102 are typically received with a fixed frequency. The trajectory association module 32 labels specific (virtual) points of the underlying road network as measurement points 100 (similarly as would happen with fixed roadside sampling infrastructure deployments, such as loop detectors or cameras). The trajectory association module 32 preferably collect messages from its proximity, then it shall group the received messages from C-ITS-S (RSUs, OBUs, etc.) that are part of the network according to a relative distance 104 from these pre-defined measurement points 100. Each of the V2X messages 102 shall be assigned to a closest measurement point 100. After the assignment, the associated V2X messages 102 can be aggregated based on their data type. For instance, speed or volume is to be averaged, while frequency items (such as lane change) shall be accumulated (counted). This way an input can be created in discrete nodes 14 for the inference module to make predictions on future traffic states and parameters.

[0151] Fig. 7 illustrates a preferred implementation of a prediction process performed by the inference module 40. The inference module 40 provides, for a given input, estimated future lane-level traffic states.

[0152] The inference module 40 preferably receives real-time or very recent data as input, i.e., lane-matched speed / volume 120 and traffic characteristics info 118 (i.e., V2X data fields from V2X messages). The traffic characteristics info 118 may comprise, for instance, a harsh braking frequency 110, a lane change request frequency 112, or a turn signal activation frequency 114. The inference module 40 may also receive as additional input other external data 116, such as POI or weather data. Then, these are aggregated by a real-time data aggregator module 122 and appended to a matrix data format, whose columns are associated with the measurement points of the network, whereas rows represent measurement reference timestamps. The inference module 40 may use an arbitrary time horizon (number of rows) that fits its hardware computational and memory capacity. The inference module 40 preferably maintains a fixed-sized matrix by, upon update, appending a newest while removing an oldest one (FIFO - first in first out). The inference module 40 can also contains a static data aggregator module 126 that can receive inputs from a spatial graph module 128 and a temporal module 132. The spatial graph module 128 receives as input a road graph 130 (i.e. , a road graph topology), preferably in form of an adjacency matrix, from which the spatial graph module 128 creates a compact embedded representation via a method called node2vec (any other similar method may be used). On the other hand, the temporal module 132 extracts a day-of-week and time-of-day information from a timestamp (in short: a time 134), which is then embedded into an intermediate space. Furthermore, the temporal module 132 can extract further temporal information relevant for predictions, e.g., measurements from a predefined time (e.g., a day or week) before at a same or similar time of the day. The static data aggregator module 126 preferably merges the temporal and spatial information received, thereby forming a spatio-temporal static representation. This technique can produce predictions for multiple time horizons simultaneously, unlike most other prior art.

[0153] The inference module 40 can further contain a data aggregator module 124 that can collect both real-time and static inputs (from the real-time data aggregator module 122 and the static data aggregator module 126, respectively). The aggregated realtime and static data generated by the data aggregator module 124 is then forwarded to a lane-wise prediction module 136 that is adapted to calculate a prediction for any arbitrary time horizon 138. Preferably, time horizons 138 are calculated in 5-m inute time steps, preferably up until 3 hours (180 minutes) ahead.

[0154] From the predicted time horizons 138, traffic flow related data can be derived, such as speed 140, occupancy 142 an / or volume 146, etc.

[0155] In order to calculate lane-level predictions, a data format needs to be prepared for lane-level accuracy. Fig. 8 shows two ways to implement lane-level predictions compared to just side-level predictions, wherein for side-level prediction traffic flow is only predicted for each side of a road (i.e., for traffic going into opposite directions, regardless the number of available lanes on each side). After having a suitable lanelevel data format, a weighting mechanism should be applied. The middle image of Fig. 8 depicts a side-level configuration (original representation, without lanes), while the left and right images of Fig. 8 indicate how a lane-level granularity can be implemented starting from the original representation.

[0156] The right-hand side image of Fig. 8 illustrates a separated representation of the lanes, wherein measurement points 150 (nodes 14 of the graph) are be separated into multiple nodes 150’, wherein the number of multiplication corresponds to the amount of lanes present at a measurement point 150. For example, if two lanes are on one side of the road, instead of one node 150, two nodes 150’ are defined. For each node 150’ a complete bipartite graph may be formed for every edge 152 contained in the original graph. By doing so, essentially, a new road-graph topology will be created where the nodes 150’ are representing lanes rather than sides (collection of lanes). Then, for example, a GNN-based model can be applied to extract relevancy between lanes.

[0157] The left-hand side image of Fig. 8 illustrates a stacked representation of the lanes, wherein an input data can be split into a stack 154, yielding a multidimensional array, whose rows represent the individual lane-level inputs. In the shown illustrative example, the stacks 154 comprise speed as input data, corresponding to an average speed in a respective lane. Besides speed, other traffic related parameter or a combination such parameters can also be used. It is preferred to use attention layers (known in the prior art) between lanes of consecutive measurement points 150.

[0158] To further enhance the “communication” between lanes of consecutive measurement points 150, the invention uses of a new weighting mechanism. The weights are based on information extracted from V2X or CV-based messages that can be interpreted as a relationship between two lanes. For example, the weight of an edge between lane A of point N and lane B of point N+1 may be assigned based on a number of turn signal activations from lane A towards lane B between point N and N+1 , normalized by a distance between the two points. By giving suggestions to the lanes on which other lanes they shall attend, may not only improve the accuracy, but also make these lane attention / correlation coefficients serve as an extremely useful input for further traffic analytics (e.g., in analysing the relationship between lanes). Another example could be a relative ratio between an average time spent in lane A compared to lane B between measurement points by a vehicle. These sorts of inputs are new and may not be acquired (with sufficient precision) by any other technology than V2X. This weighting mechanism may then be utilized alongside the suggestions made before for lane-level predictions, i.e., these weights may be assigned as an initial state for a GNN-based approach or multiplied by the attention scores of lanes.

[0159] The method and system according to the invention can serve as an input for several smart city related use cases, such as traffic signal optimization, crash likelihood prediction, or road surface condition detection and prediction.

[0160] As traffic signal optimization is usually framed very similarly, this information may in fact be one of the most important inputs for that. For example, one can use a well- trained lane-wise traffic flow prediction model to calculate an estimated lane-level traffic volume for intersections, and thus use that information as an internal part (intrinsic, hidden) of a larger traffic signal optimization method that aims to control the traffic lights in an E2E (End-to-End) fashion.

[0161] The invention, furthermore, relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out an embodiment of the method according to the invention.

[0162] The computer program product may be executable by one or more computers.

[0163] The invention also relates to a computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out an embodiment of the method according to the invention.

[0164] The computer readable medium may be a single one or comprise more separate pieces.

[0165] The invention is, of course, not limited to the preferred embodiments described in detail above, but further variants, modifications and developments are possible within the scope of protection determined by the claims. Furthermore, all embodiments that can be defined by any arbitrary dependent claim combination belong to the invention.

[0166] List of reference signs

[0167] 10 ego vehicle

[0168] 12 prediction area

[0169] 14 node

[0170] 16 edge

[0171] 20 static unit

[0172] 22 dynamic unit

[0173] 23 V2X message

[0174] 24 CV (collected vehicle) data ingestion module

[0175] 26 sequence generator module

[0176] 28 trajectory-based map matching module

[0177] 30 point-based map matching module

[0178] 32 trajectory association module

[0179] 34 data signal association module

[0180] 36 point-wise association module

[0181] 38 traffic characteristics aggregator

[0182] 39 lane-matched traffic info

[0183] 40 inference module

[0184] 42 high-definition (HD) map

[0185] 44 static unit location

[0186] 46 graph definition generator

[0187] 48 graph associativity generator

[0188] 50 graph topology generator module

[0189] 52 transfer learning module

[0190] 54 road graph generator

[0191] 56 feedback module

[0192] 58 prediction sharing module

[0193] 60 prediction unit 62 V2X message

[0194] 64 connected traffic orchestrator unit

[0195] 66 traffic management centre (TMC)

[0196] 82 predicted traffic information

[0197] 84 predicted traffic state evaluator

[0198] 86 road infrastructure rule generator

[0199] 88 infrastructure response module

[0200] 92 distributed map

[0201] 94 vehicle rule generator

[0202] 96 vehicle response module

[0203] 100 measurement point

[0204] 102 V2X message

[0205] 104 relative distance

[0206] 110 harsh braking frequency

[0207] 112 lane change request frequency

[0208] 114 turn signal activation frequency

[0209] 116 other external data

[0210] 118 traffic characteristics info

[0211] 120 lane-matched speed / volume

[0212] 122 real-time data aggregator module

[0213] 124 data aggregator module

[0214] 126 static data aggregator module

[0215] 128 spatial graph module

[0216] 130 road graph

[0217] 132 temporal module

[0218] 134 time

[0219] 136 lane-wise prediction module

[0220] 138 time horizon

[0221] 140 speed

[0222] 142 occupancy

[0223] 146 volume

[0224] 150, 150’ node

[0225] 152 edge 154 stack

[0226] 5

Claims

CLAIMS1 . A method for traffic flow prediction, comprising the steps of- receiving traffic-related information from a static unit (20) and / or a dynamic unit (22),- performing traffic flow prediction by a prediction unit (60), wherein at least one future state of traffic is estimated based on the received traffic-related information, and- sharing the estimated at least one future state, by a prediction sharing module (58), characterized by- creating a road graph representation comprising nodes (14, 150, 150’) and edges (16) between the nodes (14, 150, 150’), wherein the road graph representation has a lane-level resolution,- assigning the received traffic-related information to a corresponding lane before performing the traffic flow prediction step to generate a lane-matched traffic-related information, if the received traffic-related information is not yet a lane-matched traffic-related information,- performing the traffic flow prediction step by deep learning based on the road graph representation and on the lane-matched traffic-related information.

2. The method according to claim 1 , wherein the traffic-related information is extracted from a message received from the static unit (20) and / or the dynamic unit (22).

3. The method according to claim 2, wherein the traffic-related information received through a message is aggregated and associated with a node (14, 150, 150’) or an edge (16) of the road graph representation.

4. The method according to any of claims 1 - 3, wherein the traffic-related information is one or more of the following: a speed of traffic, an occupancy, a volume of traffic, a density of traffic, a point-of-interest, a location of an event, weather data, or road work.

5. The method according to claim 4, wherein the traffic-related information further includes at least one of the following: a turn signal status of a vehicle, a lane change request of a vehicle, an emergency brake activation of a vehicle.

6. The method according to any of claims 1 - 5, wherein the estimated at least one future state is shared with a connected traffic orchestrator unit (64), and / or with a traffic management centre (TMC), and / or as part of a V2X message (62).

7. The method according to any of claims 1 - 6, wherein the step of creating the road graph representation having a lane-level resolution is performed by- defining the nodes (14, 150, 150’) of the road graph representation based on locations of static units (20), and / or based on a predefined map, and / or based on predefined locations, and- defining the edges (16) between the nodes (14, 150, 150’) based on physical connectivity of the nodes (14, 150, 150’) or based on a logical connectivity of the nodes (14, 150, 150’).

8. The method according to claim 7, wherein physical connectivity is determined between the nodes (14, 150, 150’) based on a physical distance between the nodes (14, 150, 150’) or based on map-based information.

9. The method according to any of claims 1 - 8, further comprising a step of updating the road graph representation via transfer learning to a new road graph representation.

10. The method according to claim 9, wherein transfer learning is performed by the following steps:- applying a trained graph neural network having a first layer, a last layer and one or more intermediate layers between the first layer and the last layer, wherein the trained graph neural network is trained for a road graph,- removing the first layer and the last layer of the trained graph neural network,- freezing all the parameters of the one or more intermediate layers, and- re-training the graph neural network in an E2E fashion on the new road graph representation to create a new first layer and a new last layer.11 . The method according to any of claims 1 - 10, wherein the step of assigning the received traffic-related information to a corresponding lane is performed by point-based map-matching and / or by trajectory-based map-matching.

12. A system for traffic flow prediction, adapted to implement the method according to claim 1 , the system comprising a prediction unit (60) adapted to- receive traffic-related information from a static unit (20) and / or a dynamic unit (22), and- perform traffic flow prediction and estimate at least one future state of traffic on the received traffic-related information and comprising, characterized in that the prediction unit (60) comprises- a road graph generator (54) generating a road graph representation of a road network, wherein the road graph representation has nodes (14, 150, 150’) and edges (16), and the road graph representation has a lane-level resolution, an inference module (40) adapted to receive lane-matched traffic- related information, and the inference module (40) is connected to theroad graph generator (54) to receive the road graph representation, wherein the inference module (40) is further adapted to perform the traffic flow prediction based on the traffic-related information and the road graph representation, and generate an estimation of at least one future state of traffic as an output, and- a prediction sharing module (58) connected to the inference module (40), wherein the prediction sharing module (58) is adapted to share the output of the inference module (40).

13. The system according to claim 12, wherein the prediction module (60) further comprises a point-based map-matching module (30) and / or a trajectorybased map-matching module (28) for lane-matching the received traffic- related information, if the received traffic-related information is not already lane-matched.

14. The system according to claim 12 or claim 13, further comprising a connected traffic orchestrator unit (64) connected to the prediction unit (60), wherein the connected traffic orchestrator unit (64) is adapted to receive the output of the inference module (40).

15. The system according to claim 14, wherein the connected traffic orchestrator unit (64) comprises- a traffic state evaluator (84) adapted to evaluate the output of the inference module (40) and generate an evaluated traffic state,- a road infrastructure rule generator (86) for generating a rule based on the evaluated traffic state in connection with an infrastructure element, an infrastructure response module (88) to forward the rule generated by the road infrastructure rule generator (86) to the infrastructure element,- a vehicle rule generator (94) for generating a rule based on the evaluated traffic state in connection with a vehicle, and- vehicle response module (96) to forward the rule generated by the vehicle rule generator (94) to the vehicle.

16. The system according to any of claims 12 - 15, wherein the prediction unit(60) is implemented in a road-side unit; a MEC application on a MEC server; or a central instance, such as a virtual cloud environment.

17. The system according to any of claims 12 - 16, wherein the static unit (20) is a road-side unit, a loop detector or any other static infrastructure sensor adapted to record a passing-by traffic on a lane-level, and the dynamic unit (22) is a vehicle, a smartphone, or an on-board unit of a vehicle.

18. A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 1-11.

19. A non-transitory computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 1-11.

Images