Automated generation of v2x intersection map message

Abstract

There is provided a technique of generating MAP message for V2X intersection. The technique comprises: obtaining road-informative data related to the vicinity of the intersection and collected by one or more sensor-based (SB) source-modalities and one or more V2X source-modalities monitoring the vicinity of the intersection and operatively connected to the control unit, wherein the obtained road-informative data comprises vehicle-related data informative of vehicles and of parameters thereof detected by the source-modalities in the vicinity of the intersection; separately for each of the detected vehicles aggregating the obtained road-informative data; processing the resulted data to obtain lane-level geometry of the intersection; and encoding the generated geometry to generate the MAP message. The MAP message is automatically regenerated responsive to detecting an anomaly event (e.g. anomaly event can be detected by continuously analyzing the collected and stored timestamped road-informative data and TLC-informative data to detect changes indicative of the anomaly event).

Description

AUTOMATED GENERATION OF V2X INTERSECTION MAP MESSAGECROSS-REFERENCES TO RELATED APPLICATIONS

[0001] The present application claims benefit from US Provisional Application No. 63 / 695,418 filed on September 17, 2024, and incorporated hereby by reference in its entirety.TECHNICAL FIELD

[0002] The presently disclosed subject matter relates to techniques of management of V2X traffic intersection and, more particularly, to methods and systems for generation of V2X intersection MAP messages.BACKGROUND

[0003] Intelligent Transportation Systems (ITS) is becoming increasingly spread worldwide. ITS integrates advanced information, communication, and sensing technologies into transportation infrastructure and vehicles. ITS allows to enhance road safety, improve traffic efficiency, reduce environmental impact, and provide better mobility services to users.

[0004] ITS includes V2X intersection management that can enable advanced intersection control by integrating real-time data, adaptive signal systems, vehicle-to-infrastructure (V2I) communication, and connected vehicle messages such as SPaT (Signal Phase and Timing) and MAP (Intersection Geometry). These capabilities support applications such as, for example, adaptive traffic signal control, transit signal priority, collision avoidance, and automated vehicle coordination, thereby reducing delays, mitigating congestion, and improving overall traffic safety.

[0005] ITS communication involves the exchange of various types of messages (e.g. Vehicle-to- Everything (V2X) and Infrastructure-to-Everything (I2X) messages) both among road users (vehicles, pedestrians, cyclists, etc.) and between road users and entities of transportation infrastructure. Types and content of V2X / I2X messages can be defined by different standard bodies (e.g. SAE J2735 (US), ETSI ITS-G5 (EU), ISO 19091 (Global), China GB / T). V2X / I2Xmessages include MAP messages (and alike) and SPaT messages which are defined across the standards in a similar way, mostly differing in naming, encoding and local rules adaptation.

[0006] Unless specifically stated otherwise, throughout the specification the term "intersection" refers to V2X intersection, i.e. an intersection augmented with V2X capabilities.

[0007] Unless specifically stated otherwise, throughout the specification the term “SPaT (Signal Phase and Timing) message” refers to an ITS message informative of dynamic signal control states and defined by any of suitable standards. SPaT message can be informative of current traffic light phases and status thereof (red / yellow / green), protected / permitted turns, time-to-change estimates, etc.

[0008] Unless specifically stated otherwise, throughout the specification the term “MAP message” refers to an ITS message informative of static road geometry (e.g. an intersection layout including lanes-to Signal Groups mapping) and defined by any of suitable standards. MAP message can be informative of lane configurations, permitted maneuvers, connection points, attributes of intersections / road segments, etc.

[0009] Generating a MAP message currently entails performing a site survey and then manually mapping the intersection using dedicated tools. However, survey data (e.g. GIS, CAD, OpenStreetMap) may lack sufficient accuracy and become outdated due to road changes, which could lead to misinterpretations and unsafe maneuvers of the vehicles.GENERAL DESCRIPTION

[0010] The inventors have appreciated that there is a need for a solution that enables continuous automated generation of MAP messages, thereby providing improved accuracy, coordination with signal controllers and efficiency in ITS deployments.

[0011] In accordance with certain aspects of the presently disclosed subject matter, there is provided a method of generating a MAP message informative of geometry of a vicinity of V2X intersection. The method comprises, by a control unit: obtaining road-informative data related to the vicinity of the intersection and collected by one or more sensor-based (SB) source-modalities and one or more V2X source-modalities monitoring the vicinity of the intersection and operativelyconnected to the control unit, wherein the obtained road-informative data comprises vehicle- related data informative of vehicles and of parameters thereof detected by the source-modalities in the vicinity of the intersection; aggregating the obtained road-informative data, wherein aggregating is provided separately for each of the detected vehicles, thus giving rise to fused data; processing the fused data to obtain lane-level geometry of the intersection; and encoding the generated geometry of the intersection to generate a MAP message.

[0012] In accordance with further aspects of the presently disclosed subject matter, the road- informative data can be collected during a collection time frame, wherein the length of the collection time frame is dynamically adjusted in accordance with traffic density. The length of the collection time frame can be adjusted separately for each of approaches.

[0013] In accordance with further aspects of the presently disclosed subject matter, aggregating can comprise synchronization of the collected road-informative data with the help of creating a single domain for data collected by all source-modalities.

[0014] In accordance with further aspects of the presently disclosed subject matter, obtaining the geometry of the intersection comprises: processing the fused data to calculate trajectories of the detected vehicles in the vicinity of the intersection; using a high-definition (HD) map to match the calculated trajectories to lanes presented by the HD map; processing the fused data using the trajectories-to-lanes match to define lane attributes informative of lanes types; processing the fused data to provide semantic segmentation and identify the widths and lengths of the lanes; and detecting the intersection topology.

[0015] Obtaining the geometry of the intersection can further comprise mapping the lanes to traffic light signal groups.

[0016] Mapping the lanes to traffic light signal groups can comprise: for each lane, collecting, during a sampling interval, per-vehicle data informative of movements of the respective vehicles and obtained from SB and V2X source-modalities, thereby yielding vehicle-related data; detecting traffic light controller (TLC) in the vicinity of the intersection and collecting, during the sampling interval, data informative of status state thereof thereby yielding TLC-related data; processing the collected vehicle-related data and TLC-related data to match TLC status states to correspondinglanes; using the matching results to map the TLC phases to corresponding lanes and further providing lanes to traffic light signal groups mapping.

[0017] In accordance with further aspects of the presently disclosed subject matter, defining attributes informative of lane types can comprise: applying to the fused data a IstMLM (Machine Learning Model) to segment the vicinity of the intersection, thereby yielding segment-indicative data; applying to the segment-indicative data a 2nd MLM to detect objects and classify the types thereof; applying to the detected objects a 3rd MLM to provide objects tracking; converting imagespace coordinates of tracking objects into real- world coordinates and matching to coordinates of the lanes' coordinates; and setting the types of the lanes in accordance with respective types of detected objects.

[0018] In accordance with further aspects of the presently disclosed subject matter, providing semantic segmentation can comprise projecting data collected by all source-modalities into respective camera images and using the results to assign semantic labels to pixels and / or groups thereof.

[0019] In accordance with further aspects of the presently disclosed subject matter, identifying the widths and lengths of a given lane can comprise: a) processing the fused data to obtain initial coordinates of start and end of the given lane and a fusion-based lane width thereof; b) processing, separately for each type of SB source modalities, road- informative data to calculate maximal width of vehicles located at the given lane; c) using the calculated trajectories to define the average position of a center of the given lane; and d) setting the lane width in accordance with results of steps b) and c).

[0020] In accordance with further aspects of the presently disclosed subject matter, the MAP message can be automatically regenerated. The MAP message can be automatically regenerated with a predefined periodicity or responsive to scheduled events influencing the geometry of the intersection. Alternatively or additionally, the MAP message can be automatically regenerated responsive to detecting an anomaly event.

[0021] The anomaly event can be detection of a change in the geometry of the vicinity of the intersection; of a change in traffic lights assignments and / or traffic patterns in the vicinity of the intersection, detection of special conditions related to the lanes, etc.

[0022] The anomaly event can be detected with the help of a 3rd party alert received by the control unit and / or with the help of information continuously requested by the control unit from a 3rd party. Alternatively or additionally, the anomaly event can be detected by continuously analyzing the collected and stored timestamped road-informative data and TLC-informative data to detect changes indicative of the anomaly event. The detected changes can be further used during regenerating the MAP message for generating the updated geometry of the intersection.

[0023] The anomaly event can be detected by analyzing statistics of vehicle-related data with respect to the lanes.

[0024] By way of non-limiting example, when the anomaly event is appearance of one or more reversible lanes, detecting said anomaly event can comprise: aggregating statistical data informative of trajectories in the lanes within last 24 hours; detecting trajectories with opposite driving directions in different hours thereby identifying a reversible lane; and using the respective trajectories to detect time-depending characteristics of the reversible lanes.

[0025] By way of another non-limiting example, when the anomaly event is appearance of permissive and / or protected lanes, detecting said anomaly event can comprise: aggregating statistical data informative of vehicles' velocity, stop and turn; aggregating statistical data informative of vehicles aiming to the egress lane and pedestrian; and running classification algorithm to detect yield for pedestrian / other vehicles in the egress lane of the turn.

[0026] By way of further non-limiting example, when an anomaly event is reconfiguration of the intersection, detecting said anomaly event can comprise detecting changes in traffic light status patterns.

[0027] In accordance with further aspects of the presently disclosed subject matter, regenerating the MAP messages can be provided responsive to detection of an identified anomaly event. Alternatively or additionally, regenerating the MAP messages can be provided responsive toreceiving indication of a potential anomaly event without determination of the specific nature of the event.

[0028] In accordance with further aspects of the presently disclosed subject matter, the stored data can be used to validate the regenerated MAP message. The method can further comprise broadcasting the validated regenerated MAP message.

[0029] In accordance with further aspects of the presently disclosed subject matter, the anomaly event can be detected with the help of a digital twin representing the geometry of the intersection and continuously updated by feeding road-informative and TLC-informative data.

[0030] In accordance with other aspects of the presently disclosed subject matter, there are provided one or more computing devices comprising processors and memory, the one or more computing devices configured, via computer-executable instructions, to perform operations for operating, in a cloud computing environment, a system capable of generating a MAP message and perform operations of the method above.

[0031] In accordance with other aspects of the presently disclosed subject matter, there is provided a system capable of generating a MAP message, the system comprising a computer configured to perform the operations of the method above.BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to understand the presently disclosed subj ect matter and to see how it can be carried out in practice, embodiments will be described, by way of non-limiting examples, with reference to the accompanying drawings, in which:Fig- 1 illustrates a generalized block diagram of a control unit in accordance with certain embodiments of the presently disclosed subject matter;Fig- 2 illustrates a generalized flow-chart of operating the control unit in accordance with certain embodiments of the presently disclosed subject matter;Figs. 3a and 3b are exemplified environment diagrams illustrating obtaining road-informative data from sensor-based and V2X based source-modalities;Fig- 4 illustrates a simplified geometry of an exemplified intersection;Fig. 4a illustrates an example of a visual diagram of a MAP message structure;Fig. 5 illustrates a generalized flow-chart of generating the geometry of the intersection in accordance with certain embodiments of the presently disclosed subject matter;Fig. 6 illustrates a generalized flow-chart of setting the lane types in accordance with certain embodiments of the presently disclosed subject matter;Fig. 7 illustrates a generalized flow-chart of using semantic segmentation for high-accuracy determination of lane width and length;Fig. 8 illustrates a generalized flow-chart of Lanes -to-Signal Groups mapping in accordance with certain embodiments of the presently disclosed subject matter;Fig. 8a illustrates a non-limiting example of TLC Phase Classifier;Fig. 8b illustrates a non-limiting example of look-up table representing Lane - to - TLC Phase matching;Fig. 8c illustrates a non-limiting example of look-up table representing Lane - to- Signal Group matching;Fig. 9 illustrates a generalized flow-chart of continuous automated generation of MAP messages in accordance with certain embodiments of the presently disclosed subject matter; andFig. 10 illustrates a generalized flow-chart of using a digital twin for continuous automated generation of MAP messages in accordance with certain embodiments of the presently disclosed subject matter.DETAILED DESCRIPTION

[0033] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently disclosed subject matter. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the presently disclosed subject matter.

[0034] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as "processing", "computing", "representing", "fusing", "applying", “detecting”, “generating” or the like, refer to the action(s) and / or process(es) of a computer that manipulate and / or transform data into other data, said data represented as physical, such as electronic, quantities and / or said data representing the physical objects. The term “computer” should be expansively construed to cover any kind of hardware-based electronic device with data processing capabilities including, by way of nonlimiting example, Control Unit and processing and memory (PMC) circuitry(s) therein disclosed in the present application.

[0035] The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a non-transitory computer- readable storage medium.

[0036] Bearing this in mind, attention is drawn to Fig. 1 illustrating a generalized block diagram of a control unit in accordance with certain embodiments of the presently disclosed subject matter.

[0037] The illustrated control unit 100 is operatively connected (wired and / or wirelessly) to a plurality of source- modalities 110, each source-modality configured to collect road-informative data within a sensing zone covering, at least, a part of intersection. The sensing zones of sourcemodalities in plurality 110 are configured so that their combined coverage substantially overlaps with or entirely encompasses the vicinity of the intersection.

[0038] Unless specifically stated otherwise, throughout the specification the term vicinity of the intersection refers to the intersection itself plus approaching and departing road segments (with lanes, crosswalks, merging / diverging areas, etc.).

[0039] Unless specifically stated otherwise, throughout the specification the term “road- informative data” refers to any data, metadata and derivatives thereof informative of road users, a road (including roadways, intersections, road structures, sidewalks, bike lanes, etc.) and traffic therein.

[0040] The road-informative data can be captured, gathered, received or otherwise acquired by a given source-modality and / or can be derived from the acquired data resulting from pre-processing provided by the given source-modality.

[0041] The term "road user" refers to any entity using the road, for example, pedestrians, cyclists, motorcycles, private cars, trucks, buses, emergency vehicles, etc.

[0042] The plurality 110 of source modalities includes one or more sensor- based sourcemodalities 110-1 (referred to hereinafter also as SB source-modalities) configured to collect road- informative data acquired by stationary sensors (e.g. mounted on elements of road structures) and / or mobile sensors (e.g. mounted on vehicles, mobile devices or otherwise connected and / or integrated with road users, mounted on unmanned aerial vehicles (UAVs) of different types, etc.). Sensors in source-modalities 110-1 can be of different types such as, for example, cameras, LIDARs, long-range radars, short-range radars, etc.

[0043] At least part of SB source-modalities 110-1 can include one or more processing and memory circuitries (not shown) configured to provide an initial processing of the sensor-acquired data (e.g. to recognize the road objects and road users, define at least part of such road users’ parameters as location, speed, acceleration, bearing, past and predicted future trajectory and track at least part of the road users, etc.). PMCs can be integrated into one or more sensors and / or can be operatively connected thereto. SB source-modalities 110-1 further forward the captured and / or pre-processed data to control unit 100.

[0044] The plurality of source-modalities 110 further includes one or more V2X communicationbased source-modalities 110-2 (referred to hereinafter also as V2X source-modalities) that cancollect road-informative data from respective connected vehicles and / or other suitable entities with the help of V2X messages. For example, a V2X message can be informative of ID of a respective vehicle, its location, bearing, speed, acceleration, past trajectory and predicted future trajectory.

[0045] It is noted that, unless specifically stated otherwise, in the following description the terms V2X communications, V2X source-modalities and V2X messages are also referred to cellular C- V2X communication when a connected vehicle uses cellular technologies or to MEC (Multi-access Edge Computing) technologies for V2X communications (rather than or in addition to DSRC / ITS- G5).

[0046] At least part of V2X source-modalities 110-2 can include one or more processing and memory circuitries (not shown) configured to pre-process received V2X messages prior to forwarding to Control Unit 100.

[0047] At least part of source-modalities 110 can be configured to record and save locally the road-informative data collected during a predetermined limited time.

[0048] Further to source-modalities 110, control unit 100 can be operatively connected to road infrastructure (e.g. one or more traffic controllers 150) and can be configured to gather therefrom data informative of real-time traffic lights status and duration, configuration information, the timing and sequence of the lights, the presence of any relevant signals or signs (e.g. dynamic message signs, dynamic lane indicators, etc.) and alike (e.g. with the help of SPaT messages).

[0049] Control unit 100 can be further operatively connected to one or more external databases 140 (e.g. open-source maps, HD (High-definition) maps based on results of manual surveys, etc.).

[0050] Control unit 100 comprises Input / Output Interface 120 operatively connected to a processing and memory circuitry (PMC) 130 and enabling communication between the control unit and road entities operatively connected thereto.

[0051] PMC 130 comprises a processor and a memory (not shown separately within the PMC). PMC 130 is configured to execute several program components in accordance with computer- readable instructions implemented on a non-transitory computer-readable storage medium therein. Such executable program components are referred to hereinafter as functional modules comprisedin the PMC. The functional modules can be implemented in any appropriate combination of hardware with software and / or firmware.

[0052] The functional modules in PMC 130 can comprise operatively connected data fusion engine 131, trajectory generator 132, geometry extraction unit 133 and encoding& MAP message generation unit 134 configured to enable operations further detailed with reference to Figs. 2 - 10 below.

[0053] It is noted that the teachings of the presently disclosed subject matter are not bound by the control unit described with reference to Fig. 1. Equivalent and / or modified functionality can be consolidated or divided in another manner and can be implemented in any appropriate combination of hardware with firmware and / or software and executed on suitable device(s). The sourcemodalities can be consolidated or divided in other manner.

[0054] At least part of source modalities can be integrated with the control unit. For example, control unit 100 can be configured to directly receive I2V / V2I messages from at least part of connected vehicles.

[0055] Control unit 100 can be a standalone entity or integrated, fully or partly, with other entities. In certain embodiments, control unit 100 can be located within a traffic cabinet. In certain embodiments, a single control unit 100 can be operatively connected to one or more pluralities of source-modalities collecting road-informative data related to two or more intersections.

[0056] Optionally, control unit 100 can be implemented, at least partly, in a distributed and / or cloud and / or virtualized computing environment. The functional modules (and / or parts thereof) shown in Fig. 1 can be distributed over several local and / or remote computers (including computers located in a cloud environment) and can be linked through a communication network.

[0057] Referring to Fig. 2, there is illustrated a generalized flow chart of operating the control unit in accordance with certain embodiments of the presently disclosed subject matter.

[0058] Control unit 100 uses SB source-modalities to obtain (201) sensor-based (SB) road- informative data and uses V2X source-modalities to obtain (202) V2X road-informative data based on V2X communications, road-informative data related to the vicinity of the interception.

[0059] Obtaining SB road-informative data is schematically illustrated by an exemplified environment diagram presented in Fig. 3a.

[0060] SB source-modalities 110-1 are configured to monitor the vicinity of the illustrated intersection. At least one of SB source-modalities 110-1 comprises camera image sensors with sensing zone 300 and is configured to detect vehicles and locations thereof using a fine-grained object detection neural network model. The real-time locations of a given vehicle are used to create an updated trajectory using <location, time> pairs.

[0061] Further, at least one of SB source-modalities 110-1 comprises a radar camera (sensing zone is not shown) and is configured to detect vehicles and locations thereof (e.g. with the help of neural network processing) to create a high- resolution point cloud. The data of the radar is updated using <point_cloud, time> pairs.

[0062] The sensing zones of SB source-modalities are configured to enable the maximal combined coverage of the vicinity of the intersection as well as the maximal coverage by each type of sensors.

[0063] Upon detection of vehicles (denoted as 301 and 302) approaching the intersection, SB source-modalities 110-1 use the obtained camera and radar data to generate SB vehicle- informative data for each of the detected vehicles. SB vehicle-informative data can include respective vehicle ID, location, speed, acceleration, heading and vehicle’s size. The SB vehicle- informative data is further transmitted to control unit 100 as a part of SB road-informative data.

[0064] At least one of source-modalities 110-1 further uses camera and map information to calculate coordinates of a center point (303) of the intersection and transmits the calculated coordinates to control unit 100.

[0065] Likewise, at least one of source-modalities 110-1 further uses GPS data of one of its sensors (or GPS source of an edge device placed in the vicinity of the intersection to support V2X services) to calculate coordinates of a reference point (304) that defines the geodetic anchor point for the intersection, and to transmit the calculated coordinates to control unit 100.

[0066] Obtaining V2X road-informative data is schematically illustrated by an exemplified environment diagram presented in Fig. 3b.

[0067] V2X source-modality 110-2 is configured to receive V2X BSM (Basic Safe Message) messages from a plurality of connected vehicles in the vicinity of the illustrated intersection. BSM messages are configured to provide real-time vehicle state information including vehicle position (GPS-based latitude, longitude, elevation), speed, heading, acceleration, brake status, and vehicle size. Optionally, BSM messages can be further informative of path history / predicted path, special vehicle status (e.g., emergency vehicle, transit vehicle), safety extensions (lights, wipers, ABS events, etc.).

[0068] Likewise, V2X source-modality 110-2 is configured to receive cellular-based BSM messages from connected vehicles (denoted 302, 305 and 306) in the coverage area of cellular base station 307. The BSM messages can be also received using MEC (Mobile Edge Computing).

[0069] For illustrative purposes, the following description refers to BSM messages (i.e. messages defined by U.S. standard SAE J2735). It is noted that the teachings of the presently disclosed subject matter are likewise applicable to equivalent BSM-like messages defined by other standards (e.g. Cooperative Awareness Message (CAM) defined by European Standard ETSI EN 302 637- 2, etc.).

[0070] Likewise, the teachings of the presently disclosed subject matter are likewise applicable to PSM (Personal Safety Message), i.e. V2X broadcast message generated by smart devices of pedestrians, cyclers and alike road users to share their location, motion, and intent with nearby vehicles and infrastructure.

[0071] Upon receiving a BSM message (308), V2X source-modality 110-2 extracts data of vehicle size, location, speed, acceleration and other data informative of vehicle state (also referred to hereinafter as V2X vehicle-informative data). Optionally, accuracy of V2X vehicle-informative data can be enhanced by matching C-V2X -based data with data extracted from DSRC / ITS-G5 V2X communications. The V2X vehicle-informative data is further transmitted to control unit 100 as a part of V2X road-informative data.

[0072] Referring back to Fig. 2, control unit 100 receives, synchronizes and per-vehicle aggregates road-informative data collected by the plurality of source-modalities and, thereby, provides (203)data fusion (during or after the data collections). Road-informative data are collected during a collection time frame with a sufficient length.

[0073] In certain embodiments, the sufficient length can be predefined by selecting from a set of values (e.g. between 5 minutes and 30 minutes) corresponding to predefined levels of traffic density in the vicinity of the intersection. Depending on traffic density, the sufficient length can be different for different approaches of the intersection or can be selected in accordance with approach with the lowest traffic density.

[0074] In other embodiments, the sufficient length can be dynamically adjusted in accordance with traffic density and results of further verification. For example, the length of the collection time frame can be adjusted by the following algorithm:(1) setting the default length of collecting time frame as 5 minutes (or any other suitable predefined duration) for each approach and setting the minimal traffic density is defined as Densmin;(2) using the respectively obtained road-informative data to calculate the actual traffic density Densactuai per approach;(3) when the ratio of Densactuai to Densmin below a predefined threshold, increasing the collecting time in 5 minutes and repeating step (2).

[0075] Optionally, control unit 100 can further generate a test geometry of intersection (in a manner further detailed with reference to Figs. 2 - 10) and compare with an available ground truth. If discrepancies are detected (e.g., in the number of lanes, signal group phases, etc.), control unit 100 extends the collection time frame with the next 5 minutes.

[0076] Synchronization includes synchronizing GPS-clock based data received in V2X communications, radar- captured data based on a radar clock and camera-captured data based on a camera clock.

[0077] Data received from different source-modalities can have different frame rates. Synchronization includes creating a single domain for all sources.

[0078] For example, creating the single domain can include finding nearest timestamps of radar and camera followed by calibration phase and using an extrinsic matrix to transform radar data from the radar’s coordinate system into the pixel coordinates of the camera frame.

[0079] Further, data from all sources can be synchronized all together using a transformation matrix to the single domain.

[0080] Synchronization process can include:1) running external PTP Grandmaster and setting camera(s) to PTP slave mode;2) using ISR (Interrupt Service Routine) driver and system clock to timestamp radar data for each frame;3) verifying that PTP offset is less than tolerance required to ensure reliable temporal alignment (e.g. 100 microsecond);4) capturing simultaneous radar and camera data;5) comparing timestamps and measuring clock drift to ensure it remains below a threshold ensuring reliable multi sensor fusion for fast moving objects (e.g. 1 millisecond)

[0081] Data fusion and synchronization for each detected vehicle can be provided in accordance with the following order: 1) vehicle-informative data received from camera sensor(s); 2) vehicle- informative data received from radar(s); and 3) vehicle-informative data received from V2X / C- V2X communications.

[0082] Optionally, the obtained data can be filtered prior to fusing, thereby increasing further accuracy of processing thereof.

[0083] As will be further detailed with reference to Figs. 4 - 8, control unit 100 processes the fused data to obtain (204) a geometry (lane-level layout and lane-to signal group mapping) of the intersection.

[0084] Fusing data from different source-modalities prior to calculating the intersection geometry enables temporal consistency (as inputs from source-modalities are aligned along the same timeline), reduces errors and errors propagation and enhances the accuracy.

[0085] Control unit 100 further encodes the generated geometry in accordance with a relevant standard and generates (205) a respective MAP message. The generated MAP message is further validated (206) and broadcasted (207) over a respective RSU (Roadside Unit) or directly to the connected (Direct V2X / Sidelink) vehicles and infrastructure. The generated MAP message isbroadcasted with a frequency requested by implementation guidelines or respective standards (e.g. MAP message may be transmitted every 1 second and every 100 milliseconds for SPaT messages).

[0086] By way of non-limiting example, validating the generated MAP can include its visualization and further comparing its graphical representation to the real-world intersection map (e.g. OpenStreetMap, GIS, CAD drawing, etc.). Validating can further comprise verifying consistency with the signal group mapping. As will be further detailed with reference to Fig. 10, validation can be also provided with the help of a digital twin generated for the vicinity of the interception.

[0087] Referring to Fig. 4, there is illustrated a simplified geometry of an exemplified intersection. The intersection geometry is characterized by lane-level layout and lane-to signal group mapping.

[0088] The illustrated intersection comprises four incoming / outgoing roads (referred to hereinafter as “approaches”) leading to / from the intersection.

[0089] It is noted that the teachings of the presently disclosed subject matter are likewise applicable to other 2D and 3D intersections, including the intersections with 5 approaches (Star), 6 approaches (Diamond) and 7+ approaches (multi-Leg).

[0090] The illustrated intersection comprises 12 lanes (enumerated Lane 1 - Lane 12) associated with respective approaches and characterized by lane attributes:Lane Type Attributes (e.g. dedicated vehicle lane, crosswalk, bike lane, pedestrian path, etc.). In the illustrated intersection Lane 1 is dedicated for buses, Lane 3 is dedicated for taxis and Lane 7 is available for any transport;Directional AttributesManeuver Attributes (e.g. straight, left-turn, right-turn, U-turn, merge, diverge, etc.) Regulatory / Control Attributes (e.g. stop lines and other control points, restrictions (e.g., bus-only, HOV, truck restrictions), etc.)Geometric Attributes (lane width and length, length of approach segment, lane's connections to other lanes (e.g. e.g., left-turn lane — outbound lane), reference node points (e.g. polyline points with latitude / longitude / elevation in WGS84), etc.).

[0091] The intersection geometry is further characterized by lane-level intersection topology including lane-to-lane connections specifying how ingress lanes connect to egress lanes and allowed maneuvers (e.g. turn left from Lane 10 to Lane 2, etc.).

[0092] The intersection geometry can also include signal group mapping indicative of the links between specific lanes or movements and traffic signal phases (used with SPaT) that are characterized by associated Signal Groups.

[0093] Fig. 4a illustrates a non-limiting example of a MAP message structure and shows the hierarchy of the MAP message elements and how they relate to each other.

[0094] A MAP message can be informative of one or more intersections, each intersection characterized by intersection ID and a reference point (usually GPS coordinates of the intersection center). For each intersection, MAP message specifies the existing lanes (including each lane geometry, attributes and connections) and linking to SPaT signal groups.

[0095] Lanes within each approach are characterized by Lane ID (unique within the intersection) and lane attributes (e.g. as specified with reference to Fig. 4).

[0096] Fig. 5 illustrates a generalized flow-chart of obtaining the geometry of the intersection in accordance with certain embodiments of the presently disclosed subject matter.

[0097] As will be further detailed with reference to Figs. 6 - 8, to obtain the geometry of the intersection, control unit 100 processes the fused data to calculate (501) trajectories of detected vehicles in the vicinity of intersection; uses the calculated trajectories to obtain (502) an enhanced HD map of the intersection and processes the fused data to define (503) the lane type attributes.

[0098] Upon processing the fused data to calculate trajectories of plurality of vehicles, control unit 100 uses HD (high definition) maps (e.g. Google maps, Openstreetmaps, etc.) to provide trajectories-to-lanes matching. HD maps can include lane-level geometry, road edges, curbs, crosswalks, traffic signs, signals, elevation, etc.

[0099] Typically, HD maps are built from survey-grade data and need to be updated with field changes. Control unit 100 uses the available HD maps as a base layer and corrects and / or updates the lane-level layout using the respective trajectories. As a result, control unit 100 generatesenhances HD map (referred to hereinafter as EHD map) further suitable for encoding into MAP messages.

[0100] Matching aggregated per-lane trajectories that are indicative of actual vehicle behavior to survey-based data enables capturing dynamic changes and increasing accuracy of representing geometry of intersection.

[0101] For example, control unit 100 can set lanes directional and maneuver attributes according to the heading of the vehicles, can set approaches as respective ingresses or egresses, can mark the correct position of regulatory / control attributes on the map of the intersection; can set lanes connections according to vehicles heading trajectories, etc.

[0102] In certain embodiments, lane type attributes can be defined (503) by processing the fused data as illustrated in Fig. 6.

[0103] Control unit 100 can apply to the fused data a 1stMLM (Machine Learning Model) to segment (601) the intersection vicinity; and apply to the segmented data a 2ndMLM to detect (602) the objects and classify the types thereof (e.g. car, truck, pedestrian, bike, crosswalk, sidewalk, parking, road signs, etc.). Each detected object is associated with a bounding box {[pl_x, pi_ y], [p2_x, p2_y] } for each object.

[0104] Control unit 100 further applies to the detected objects a 3rdMLM to provide objects tracking (603).

[0105] Control unit 100 further converts image-space coordinates (bounding boxes coordinates) of tracking objects into real- world coordinates and matched (604) to lanes coordinates. The conversion is provided using verified GNSS reference point and the radar / camera intrinsic matrix. Thereby, control unit 100 sets (605) the lane types in accordance with respective types of detected objects (e.g. bus only for Lane 1 in Fig. 4, taxi only for Lane 3 and mixed traffic for Lane 7, etc.).

[0106] Optionally, the process can include also applying a 4thMLM to detect anomalies and / or irregular behavior of the detected objects. It can allow to filter the objects that can cause a wrong lane type classification (e.g. moving in directions that are irrelevant to lane trajectory, do notdescribe the relevant road area and attributes thereof as, for example, trash on the road, stuck vehicle, etc.).

[0107] As will be further detailed with reference to Fig. 9, the detected anomalies can be used to trigger an update of the generated MAP message. Referring back to Fig. 5, control unit 100 can use semantic segmentation to high-accuracy definition of the widths and lengths of the lanes (504).

[0108] Typically, the accuracy of vehicle positioning (e.g. from GNSS and BSM messages) has a certain error (achieving, in certain embodiments, ±1-3 m). Thus, if the lane width is incorrect, even a small error in vehicle positioning can lead to misclassification of which lane a vehicle is in. Such misclassification can propagate to wrong maneuver predictions, wrong SPaT mapping, and potentially unsafe decisions.

[0109] In accordance with certain embodiments of the presently disclosed subject matter, control unit 100 uses road-informative data from source-modalities to provide semantic segmentation of the vicinity of intersection, to define (504) the widths and lengths of the lanes.

[0110] Semantic segmentation is a computer vision technique (e.g. algorithms of DeepLab Family, Segformer, etc.) where each pixel in an image is classified into a category (class label).

[0111] Using semantic segmentation for high-accuracy determination of lane width and length is illustrated in Fig. 7.

[0112] Control unit 100 can use fused road-informative data to provide semantic segmentation of the vicinity of intersection (701). Available data from all source-modalities are projected into camera image and further used to assign semantic labels to pixels and / or groups thereof.

[0113] It is noted that although the sensing zones of SB source modalities are configured to maximize coverage for each sensor type, there may still be areas within the vicinity of intersection where the fused data lacks (e.g. temporarily) input from certain sensor types. Likewise, there may still be areas within the vicinity of intersection where the fused data lacks V2X inputs (e.g. due to low density of connected vehicles).

[0114] Semantic segmentation for such areas can be performed using the available types of source data. The highest accuracy can be achieved by fusing camera, radar, and V2X data, whichoutperforms combinations of camera-radar or camera-V2X. These combinations, in turn, provide higher accuracy than relying on any single modality alone.

[0115] In certain embodiments, the geometry of the intersection vicinity can be obtained across multiple time frames. Control unit 100 can derive the geometry for each frame from the source data available at that moment and store the respective road-informative data in a database. When deriving the geometry in the next time frame, control unit 100 can increase the accuracy by using road-informative data from the previous time frames.

[0116] Non-limiting examples of semantic segmentation based on the limited types of source data are detailed below.

[0117] In certain embodiments semantic segmentation can be provided based on camera-captured data in combination with radar-captured data as follows: a) Projecting 3D point cloud obtained from radar-based data on a camera-based image using intrinsic camera matrix. b) Separately processing camera-based and radar-based data for real-time separate detection of objects and identifying their characteristics.Detecting objects using the camera-captured image data can be provided using YOLO (You Look Only Once) or alike algorithm. YOLO-like algorithms treat detection as a single regression problem and takes an entire image in one pass through a deep neural network and directly outputs bounding boxes indicative of locations of objects and class probabilities indicative of what the objects are. Thus, YOLO-like algorithm sets for each detected object 3D bounding box with the parameters [x center, y center, z center, width, length, height] label indicative of the object’s type (e.g. vehicle, pedestrian, animal, etc.).Likewise, detecting objects using the radar-based point cloud data can be provided using YOLORadar or alike algorithm that sets for each detected object 3D bounding box with the parameters [x_center, y_center, z_center, width, length, height] and object’s velocity, heading, elevation, width and heigh.c) Fusing the results of camera-based and radar-based detection to filter invalid detection and provide accurate 3D detection. d) Applying to the fused data a tracking algorithm to assign a unique identifier (IDx) to each detected object; and e) Applying a mapping algorithm and using coordinates of GNSS verified reference point to convert, for each object with IDx, image- based coordinates thereof to latitude, longitude and elevation coordinates (LatX, LatY, ElavationZ).

[0118] In certain embodiments semantic segmentation can be provided based on camera-captured data as follows: a) Lane detection can comprise:- applying to the image Lane2Seq transformer architecture neural network thereby detecting the lane therein and setting lane coordinates:[ {laneX start base Jatl , laneX start base Jongl }, {laneX start base Jat2, laneX start b ase long2}, laneX base width] which represents the detected lane structure; and using Lane2Seq transformer architecture neural network to detect all moving vehicles with their bounding boxes usable for later extracting the lane width. Apply free space fill algorithm to mark the lane areas.Extracting lane accurate coordinates based on[ {laneX start vision Jatl, laneX start vision longl}, {laneX end vision lat2, laneX end vision long2{ lane width ] b) Stop line detection can comprise: using previously generated trajectories to detect vehicles stopping point when its velocity decreases to a value close to zero; saving for each approach all stop points, [{stop _vN Jatl, stop _vN longl},{stoppvN Jat2 ,stoppvN Jong2}] where N is the number of the detected vehicle; averaging all above stop points to two points for start and end of stop bar line[ {stop bar Jatl , stop bar longl}, {stop J>arJat2, stop J>arJong2} ] c) Crosswalk detection can comprise:1 detecting crosswalk using classical vision algorithms by detecting the zebra line. With the following properties: } cw width, cw height, cw number of lines, cw line step / ; detecting all pedestrians in the intersection with [pedN lat,pedN long, pedN speed, ped -heading] ; building dynamical pedestrian trajectories (referred to as pedNTrajectoty); for each pedNTrajectoty checking the nearest stop bar and the ones in parallel to it; applying anomaly detection algorithm to detect and remove trajectories corresponding to crossing on red or jaywalking pedestrians and alike; averaging the rest of trajectories resulting in [ {cw latl ,cw longl}, {cw laid, cw long!} , cw width, cw line step] and setting the crosswalk location accordingly.

[0119] In certain embodiments semantic segmentation can be provided based on radar-obtained data as following: detecting all moving vehicles and project the detected movements on the image (using the intrinsic matrix); using a free space filling algorithm to mark the areas where movements are detected (in a manner similar to vision based-detection detailed above): detecting traffic density and set dens quant (0 - no traffic , 1 dense traffic); apply 30min sampling time for value below 0.7 and apply 5 min sampling time for value above 0.7. defining the fill area with the following points [ {lane start Jail, laneX start longl}, {lane end latl , laneX end longl}, lane! width ].For lanes which include more than 2 points the generic case will be [{{lane start latl, laneX start longl}, {lane end latl, laneX end longl}, lane width},.., { lane latN, laneX longN}, {lane end JatN, laneX end longN}, lane width} / mainly when lanes are not straight line.

[0120] In certain embodiments semantic segmentation can be provided based on V2X communications as follows: extracting location from the BSM / PSM V2X messages or any other GNSS location data shared by the connected vehicle;applying filling free space algorithm (in a manner similar to described above for radar) to mark the lanes with different colors according to the trajectory / matched to the vehicle movement.

[0121] In certain embodiments semantic segmentation can be provided upon fusion of data captured by cameras, radars and data received via V2X communications as follows:- fusing camera and radar data and complementing with V2X data when available;- Calculating differences in real-world GPS coordinates lat diff, long diff, lane_width_diff between camera and radar data (cr diff); between camera and V2X communications data (cc diff) and between radar and V2X communications data (rc di fff,- For each lane calculating : [{fusion JaneX start Jail ,fu ion lan X start longl}, {fusion laneX end latl fusion laneX end long! [ fusion laneX width /

[0122] Upon providing the semantic segmentation of the intersection vicinity, control unit 100 fuses data received from SD source-modalities and V2X source-modalities to obtain (702) initial coordinates (base value) of start and end of a given lane and a base value of width thereof.

[0123] Control unit 100 further uses data received from SD and V2X source-modalities to define(703), for each type of sensors, maximal width of vehicles located at the given lane and to calculate(704) the average position of lane center according to multiple trajectories of respective connected and non-connected vehicles and variance thereof.

[0124] Further, control unit 100 uses data obtained in steps (703) - (704) to calculate (705) width of the given lane for a given trajectory.

[0125] By way of non-limiting example, the width of the given lane (lane X) can be calculated(705) as following: i. Use fusion laneX width as a base value for the lane width. ii. Extract multiple car sizes from the V2X messages on the corresponding lane and extract lane width carw com which is the max detected width. iii. Calculate carw vision which is the max width of car based on camera and radar fusioniv. Aggregate lidar, radar, cameras data from approaching semi-autonomous to fully autonomous (L3-L5) connected vehicles (using v2x communications) and calculate as below:1. Match location of the vehicle to the specific trajectory I and then to specific lane.2. For each trajectory / and lanes from previous step aggregate the sensors data from the driving / stopped vehicles3. calculate the laneXw cooperative perception which is extracted from the aggregated data per lane in previous step v. Calculate lane center avg as the average position of lane according to multiple trajectories of connected and non-connected vehicles. vi. Calculate lane center var which is the lane center variance. vii. Calculate vision com lane width =(carw com + carw vision) / 2 (in case carw_com is zero use only vision) viii. Calculate trajectory l lwd base = lane center avg + 2 *lane center var ix. Calculate final lane width for trajectory / as: tracjectory l lane width = (vision com Jane width + trajectory Iwd base +lanew cooperative perception) 3

[0126] When the width of the given lane is calculated, control unit 100 compares values of lane width obtained in steps (702) and (705), and, when the difference exceed a predefined threshold, sets (706) the lane width as equal to the width obtained in step (705).

[0127] It’s noted that the size of the lane can change dynamically (e.g. lane boundaries can be altered due to construction or temporary changes, road marking can be inconsistent or misaligned, sensor data related to lane edges can be misinterpreted in view of environmental conditions, etc.).

[0128] In certain embodiments, control unit 100 can be configured to periodically repeat steps (701) - (706) for dynamic correction (707) of the lane width and length. Alternatively oradditionally, control unit 100 can be configured to dynamically correct the calculated size of the lane responsive to detected anomalies (as further detailed with reference to Figs. 9 - 10).

[0129] Referring back to Fig. 5, control unit 100 further identifies the reference point of the intersection and detects (505) intersection topology and control attributes (e.g. lane connections, allowed maneuvers, stop-lines, approaches, length, etc.).

[0130] By way of non-limiting example, intersection reference point and GNSS verified point can be defined using multiple cameras from all intersection directions. The verified GNSS point may be defined as the most accurate position derived from the GNSS location available via V2X communications and the GNSS location (if available) of the control unit 100. The elevation data can be obtained from Google Maps.

[0131] Crosswalk and stop line locations are set using accurate coordinates calculated as detailed above (e.g. with reference to camera-based semantic segmentation).

[0132] Ingress and Egress approaches are set with size {ingressN width, ingressN_height} egressN idth, egressN Jieight} . The width of each approach will match the lane width, and the height will be specified as 6 feet.

[0133] Lane enumeration is performed incrementally, beginning with the first enumerated lane, based on traffic observed across all lanes.

[0134] Connections between lanes are set with the help of trajectories matching algorithm. Data for each lane are aggregated in the {from Jane JdX, to Jane JdY, Allowed manuouver Z, Singal group JD}. The allowed maneuvers are further validated based on traffic directions (upon filtering traffic anomalies, if any).

[0135] Upon identifying the lane-level intersection topology, control unit 100 further maps traffic light behavior to specific vehicle or pedestrian paths through the intersection thereby providing lanes -to-Signal Groups mapping (506).

[0136] A Signal Group corresponds to a set of one or more lanes (including cycling and pedestrian paths) at an intersection that share the same traffic signal indication (e.g., green, yellow, red). It maps traffic light behavior to specific vehicle or pedestrian paths through the intersection.

[0137] A Traffic Light Controller (TLC) Phase is a time interval within a traffic signal controller’s cycle during which one or more signal groups are simultaneously granted the right-of-way.

[0138] Lanes -to-Signal Groups mapping is further illustrated in Fig. 8.

[0139] For each lane, control unit 100 collects (801), during a sampling interval, per-vehicle data informative of movements of the respective vehicles and obtained from SB and V2X sourcemodalities. The length of sampling interval is based on traffic density (the denser traffic - the shorter sampling interval) and is sufficient for collecting reliable statistical data. Optionally, the sampling interval can be defined similarly to the way of setting the "collection time frame" detailed with reference to step 203 in Fig. 2,

[0140] For each vehicle detected in the lane during the sampling interval, the collected vehicle- related data include, if available: vehicle stop time, vehicle stop location, vehicle intersection crossing time, vehicle turn right / left indication, vehicle yield indication, vehicle U-turn indication, etc.

[0141] Thus, the data for each vehicle will be collected as an array

[0142] [ [vN stop time tl ,vN_stop_location_tl , vN_i ntersection cross time t 1 ,vN_turn_left_in d_t 1 ,vN_turn_right_ind_tl ,vN_yield_indc,vN_uturn_tl },...].

[0143] Separately, control unit 100 detects traffic light controller (TLC) in the vicinity of intersection. For the detected TLC, control unit 100 collects (802), during the sampling interval, data informative of status state thereof. The collected TLC-related data can include time of red vehicle / pedestrian (time red yv and time red ped , time of yellow vehicle / pedestrian (time yellow y and time yellow ped} and time of green vehicle / pedestrian (time green yv and time green ped},

[0144] Optionally, TLC-related data can be obtained by processing data collected by camera sensors.Control unit 100 analyzes the intersection geometry together with the collected vehicle-related data to generate a phase classifier. The phase classifier is configured to label phases by movement type / functional role, and its generating can comprise:Classifying the movement types for each lane or lane connection (e.g. left-turn, through, right-turn, pedestrian crossing, etc.).Detecting and filtering anomalies (e.g. flashing yellow in all traffic lights, etc.).Clustering the movements into maximal conflict-free sets (two movements conflict if their paths cross as, for example, left-turn vs oncoming through). Clustering the movement can comprise; detecting trajectory patterns (e.g., straight vs left-turn vs right-turn); clustering movements by lane position and approach direction; providing temporal correlation: when multiple vehicles consistently move in the same direction simultaneously, infer the active phase; providing pedestrian / bike detection in correspondence with movement across crosswalks or bike lanes; providing preemption detection in correspondence with sudden changes in flow priority (e.g., all vehicles stop except an emergency vehicle path); providing flashing / dark detection: irregular or unsynchronized movement patterns.Assigning for each conflict-free movement set a phase ID; andAssigning semantic labels based on dominant movement type, for example: through movement phase — straight-ahead movements. left-turn phase — protected or permissive lefts. right-turn phase (less common as separate, often overlaps). pedestrian phase — walk / don’t-walk signals. transit / bicycle phase — priority signals for buses or bikes. overlap phases — movements served simultaneously with another phase (e.g., right turn served during through green).

[0145] Non-limiting example of phase classifier is illustrated in Fig. 8a.

[0146] Control unit 100 further processes the collected vehicle-related data and TLC-related data to match (803) status states to corresponding lanes and to match (804) the TLC phases to the lanes.

[0147] In certain embodiments the matching between lanes and phases can comprise:1) For each phase of each given TLC, finding the most likely movement timing from all lanes in the intersection. By way of non-limiting example, such most likely movement timing can be found by applying k-nearest neighbor algorithm (or other instance-based learning algorithms) to the collected data.2) Using the result of step (1) to set a lane-to-TLS phases look-up table (LUT) representing matching between the lanes and TLC phases controlling thereof.

[0148] Non-limiting example of such look-up table for an exemplified TLC is illustrated in Fig. 8b. The example illustrates an intersection with 4 approaches: North, South, East, West. Each approach has 2 lanes: Left-turn (L) and Straight / Right (S / R). The traffic light controller has phases illustrated in Fig. 8a.

[0149] It is noted that while the TLC phase classifier covers every TLC phase defined in the traffic light controller, lanes-to-TLC phases LUT can comprise only phases that are relevant to the lanes in the MAP (e.g. the LUT can miss pedestrian phases, flashing yellow, emergency preemption, etc.). Following TLC phase-to-lanes matching, control unit 100 further provides mapping (805) of lanes to signal groups. Non-limiting example of lanes-to-signal groups mapping is illustrated in Fig. 8c (for the TLC illustrated in Figs. 8a - 8b). The provided mapping is further usable also in SPaT messages.

[0150] In certain embodiments, movement similarity can be defined, for example, with the help of Siamese / Triplet Networks:Given pairs / triplets of detected trajectories, the network learns embeddings so that “same phase” movements cluster together while different phases stay apart.Trajectories (paths over time) can be turned into respective features, like sequence of coordinates (x,y,t), speed, direction and start-end zone.Preparing training Pairs / Triplets comprises:Siamese: feed two trajectories (A, B)Triplet: feed three trajectories — Anchor (A), Positive (P), Negative (N). A & P = same movement / phase A & N = different movement / phaseA shared neural network encodes each trajectory into an embedding vector. Similar movements — embeddings close together. Different movements — embeddings far apart. Now, when new trajectories come in, compare their embeddings. Cluster / group them — infer which phase type (through, left-turn, pedestrian, etc.) they belong to.

[0151] Temporal sequence modeling can be handled with the help of LSTM (Long Short-Term Memory ) / GRU (Gated Recurrent Unit) networks. The inputs for network are:Phase duration: e.g., left-turn protected lasts ~10s, then switches.Transitions: after through movement — left-turn — pedestrian.Irregular events: preemption, flashing, or dark phases.For example: xt= [#left, #through, #right, #ped, #bike] raw data for example:X = [[3, 10, 1, 0, 0], # time Is [5, 12, 2, 0, 0], # time 2s[0, 0, 0, 4, 1] # time 30s]

[0152] When feeding the above to LSTM network, the exemplified output can be:If through vehicles are moving consistently — "Through phase". If left-turn vehicles dominate + others stopped — "Left protected".If peds move — "Pedestrian phase".

[0153] Thus, lane - to-signal group mapping includes: providing lane to movement mapping followed by movement-to-phase mapping (e.g using Siamese / LSTM inference). Further, defining signal groups and mapping phases to signal groups and, finally, obtaining lanes - to- signal groups mapping.

[0154] As was detailed with reference to Fig. 2, MAP message is generated based on road- informative data collected during a time frame with a certain length and then the same MAPmessage is broadcasted with a frequency requested by implementation guidelines or respective standards.

[0155] Optionally, the MAP message can be regenerated with a predefined periodicity or responsive to scheduled events influencing the geometry of intersection.

[0156] In accordance with certain embodiments of the presently disclosed subject matter, MAP message can be automatically regenerated responsive to events satisfying updating criteria.

[0157] Among the events satisfying updating criteria (referred to hereinafter also as "anomaly events") can be detection of various changes in the geometry (e.g. widths, connections, added / removed lanes, etc.), traffic lights assignments and / or traffic patterns in the vicinity of intersection, detection of special conditions (e.g. reversable lanes, closed roads, blocked lanes, etc.), etc.

[0158] Anomaly events can be detected by control unit 100 with the help of 3rdparty alerts (e.g. from Waze or Google Map users, etc.) received by control unit 100. Likewise, anomaly events can be detected by control unit 100 with the help of information continuously requested / received by control unit 100 from a 3rdparty (e.g. data about closed lanes, construction works, changes in lane directions, etc. received from a city authority, etc.).

[0159] Alternatively or additionally, control unit 100 can detect anomaly events by analyzing road-informative data.

[0160] Automated continuous generation of MAP messages is illustrated in Fig. 9.

[0161] In certain embodiments, control unit 100 continuously collects and stores (901) time- stamped road-informative data and / or derivatives thereof. Likewise, control unit 100 continuously collects and stores time-stamped data informative of light status and / or derivatives thereof (902). Further, control unit 100 continuously analyzes road-informative data and TLC to detect (903) anomaly events in the vicinity of intersection. Some anomaly events can be detected using data collected during a recent time frame, while other anomaly events can be detected by combining and / or comparing with respective historical data.

[0162] Unless specifically stated otherwise, throughout the specification the term "continuously collecting" refers to an ongoing data acquisition process that persists throughout the operation of control unit 100.

[0163] Unless specifically stated otherwise, throughout the specification the term "continuously analyzing" refers to real time or near real time processing the obtained road-informative data, such that the analysis output is persistently updated in correspondence with newly collected data.

[0164] Following are several examples of detecting anomaly events in accordance with certain embodiments of the presently disclosed subject matter:Appearance of blocked lanes can be identified by processing camera data received in a recent time frame (e.g. defined similarly to the way of setting the "collection time frame" detailed with reference to step 203 in Fig. 2). An anomaly detection algorithm in combination with a YOLO-like neural network for object detection and classification enable detecting blocking objects (e.g. cones, work zone signs, workers with yellow vests, work vehicles, stuck vehicles, trees, etc.) that can be indicative of the blocked lanes. Appearance of reversible lanes can be identified by aggregating statistics for movements in the lanes within last 24h; detecting “wrong” way driving in different hours thereby identifying a reversible lane; and using the respective trajectories to detect time-depending characteristics of the reversible lanes.Appearance of permissive / protected lanes can be identified based on detecting in the vicinity of intersections signs indicative of permissive / protected lanes. Alternatively or additionally, it can be identified with the help of movement classification.Identifying permissive lanes can comprise: aggregating statistics of vehicle velocity, stop and turn; aggregating statistics for vehicle aiming to the egress lane and pedestrian; and running classification algorithm to detect yield for pedestrian / other vehicles in the egress lane of the turn.Identifying protected lanes can comprise: aggregating statistics of vehicle velocity, stop and turn; aggregating statistics on oncoming traffic to the lane; and running classification algorithm to detect that oncoming traffic always stops at this lane, hence turn protected.Changes in movement patterns can be detected, for example, by comparing the current and historical trajectories and can be indicative of anomaly events;Changes in traffic light status patterns (e.g. comparing with historical data) can be indicative of temporary reconfiguration of the intersection, e.g. due to road works, lane closures, incident management, etc.

[0165] As will be further detailed with reference to Fig. 10, detecting anomaly events can be provided with the help of a digital twin of the vicinity of intersection.

[0166] Responsive to detecting one or more anomaly events, control unit 100 regenerates (904) the MAP message (e.g. in a manner detailed with reference to Figs. 2 - 8).

[0167] It is noted that in certain embodiments regenerating MAP messages is provided responsive to identified (meaningful) anomaly event (e.g. appearance of a blocked lanes). In other embodiments, MAP message regeneration can be triggered by receiving indication (e.g. changes of moving patterns and / or light status patterns, etc.) of potential anomaly event(s) without determination of the specific nature of the event. Optionally, regeneration of MAP messages can be provided only when the detected anomalies meet predefined criteria (e.g. exceed a statistical threshold).

[0168] Control unit 100 further validates the regenerated MAP message. Further to validation detailed with reference to Fig. 2, MAP message can be validated (905) using historical data and, optionally, the detected anomalies. By way of non-limiting example, the regenerated MAP message can be compared with the previous MAP message in considerations with the detected anomalies.

[0169] Control unit 100 broadcasts (906) the regenerated MAP message over a respective RSU directly to the connected vehicles and infrastructure.

[0170] Automated continuous generation of MAP messages enables integration of MAP messages with SPaT messages (signal phase and timing), thereby allowing connected vehicles to make better route and maneuvering decisions.

[0171] Referring to Fig. 10, there is illustrated a generalized flow-chart of using a digital twin for continuous automated generation of MAP messages, in accordance with certain embodiments of the presently disclosed subject matter.

[0172] A digital twin of an intersection is a dynamic 3D digital model informative of the geometry (lanes, approaches, crosswalks, signal groups), the operational attributes (signal timings, SPaT, traffic flows) and the real-time conditions (vehicle positions, pedestrian presence, anomalies, blocked lanes, etc.).

[0173] In accordance with certain embodiments of the presently disclosed subject matter, the generated MAP message can be used to generate (1001) a baseline 3D model of the intersection. By way of non-limiting example, converting lane geometry into 3D model can be provided with the help of NVIDIA DRIVE Sim / Omniverse tool.

[0174] Control Unit 100 further enriches the baseline 3D model using Cooperative Perception L3- L5 (sensor sharing), thereby yielding (1002) the digital twin.

[0175] Cooperative perception for L3-L5 (SAE Automation Levels) vehicles means that highly automated vehicles exchange sensor data and / or perception information with each other and with infrastructure via V2X. When applied to L3-L5 automated vehicles, sensor sharing includes Sensor Data Sharing when raw or processed data (camera images, LiDAR point clouds, radar detections) are transmitted between vehicles or between vehicles and RSUs) and Perception Information Sharing when, instead of raw data, vehicles share detected objects, trajectories, or semantic segmentation maps.

[0176] L3-L5 sensor sharing can enrich the baseline 3D model as following:Lidar can provide 3d point cloud of the environment objects (e.g. Buildings, non-moving objects near the road, moving vehicles, etc.;Camera and Radar can provide accurate lanes reconstruction, etc,;V2X communications can provide data informative of allowed speed, alerts and warnings for traffic incidents, vehicle types and size, emergency vehicle preemption, TSP (Traffic Signal Priority), etc.IMU (Inertial Measurement Unit) data are usable for accurate lanes reconstruction, accurate trajectories, etc.

[0177] Control Unit 100 continuously feeds (1003) in real-time sensor-based data and SPaT messages into the 3D model, thereby updating the digital twin.

[0178] Feeding sensor-based data into 3D model can be based on radar point cloud and include:Aggregating radar point cloud and camera video from multiple sensors, where each data point consists of the following:1. [radar pc north, timestamp jiorth], [camera jiorth, timestamp jiorth]2. [radar _pc south, timestamp south] , [camera south, timestamp south]3. [radar _pc east, timestamp east], [camera east, timestamp east]4. [radar _pc west, timestamp west], [camera vest, timestamp west]Applying time synchronization for radar and camera samples using the steps below:5. Each radar point cloud and camera frame has timestamp tN.6. Frames in camera are saved in order array and also the radar point cloud: a. radar data: {[pcl,tl],[pc2,t2] ... ,[pcN,tN]} b. camera data: {[Framel,tl],[Frame2,t2] ... ,[FrameN,tN]} c. As the camera has specific FPS, while the radar is sampled in different rate the time delta between the samples is calculated d. Minium time delta is used to match the radar point cloud to the corresponding camera frame e. Synced data of the radar and camera array as below:{[F ram el,pc5,tl],[F ram e2,pc9, t2] ... ,[Fram eN, pcN, tN] }7. Camera and radar data array will be synced on each approach of the intersection: north, south, east, west.Fusing camera and radar data to create 3D model of the intersection:8. Apply intrinsic matrix for each sensor image to project the point cloud on the image9. Use multiple radar and camera data from all four directions to create a 3d model of each moving object using a deep learning 3D reconstruction model.10. Create detailed description for below intersection elements from the data: a. Lanes - [width, length, elevation, type] b. Intersection center - [width, height, center Jat, center long] c. Traffic lights - [lat, long, elevation, type] d. Traffic signs - [lat, long, type] e. Crosswalks - [width, height, center lat, center long / f. Sidewalks - [width, length, start Jat, start Jong, end Jat, end Jong] g. Work zones - [width, height, center Jat, center Jong, type]

[0179] It is noted that as each lane in the digital twin is associated with a respective signal group (from MAP message), feeding SPaT messages further provides real-time phase info for each signal group and, thereby updates the twin’s lane state according to the current signal phase.

[0180] Control Unit 100 continuously compares (1004) updated version of the digital twin with the previous one and, responsive to a detected discrepancy, triggers (1005) generating a new MAP message.

[0181] It is noted that each version of digital twin has an assigned ID (e.g. timestamp of the provided update, sequential number of the version, etc.).

[0182] In certain embodiments, a discrepancy between versions is deemed to be detected upon identification of one or more meaningful changes. Among such changes can be changes in lanelevel geometry (e.g. widths, connections, added / removed lanes, etc.), lane-to-signal group association, special conditions (blocked lanes, constructions, accidents, etc.).

[0183] Alternatively or additionally, control unit 100 can calculate an integrity value (checksum, hash, etc.) of each version. A discrepancy between versions is considered as detected when the difference between the integrity values exceeds a predefined threshold.

[0184] Upon triggering, control unit 100 generates (1006) the new MAP message in accordance with the updated digital twin.

[0185] The generated new MAP message can be validated by with the help of the digital twin running in a simulation mode.

[0186] Simulation and respective validation can include several scenarios as, for example: replaying recorded traffic with emulated newly generated MAP message and verifying the correct mapping of vehicles to intersection geometry; running a set of predefined scenarios (e.g. straight through, protected left, right turn, etc.) with emulated newly generated MAP message and verifying, for each scenario, that the vehicles are correctly associated with the corresponding elements of the intersection geometry; simulating the next few minutes of traffic based on current inflows, queues, and predicted phase sequences; emulating the newly generated MAP message, broadcasting thereof to virtual vehicles and verifying accurate vehicle-to-geometry mapping at the intersection.1.

[0187] It is to be understood that the presently disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the presently disclosed subject matter.

[0188] It will also be understood that the system according to the presently disclosed subject matter may be, at least partly, implemented on a suitably programmed computer. Likewise, the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the method of the presently disclosed subject matter. The presently disclosed subject matter further contemplates a non-transitory computer-readable memory tangibly embodying a program of instructions executable by the computer for executing the method of the presently disclosed subject matter.

[0189] Those skilled in the art will readily appreciate that various modifications and changes can be applied to the embodiments of the presently disclosed subject matter as hereinbefore described without departing from its scope, defined in and by the appended claims.

Claims

1. CLAIMS1. A method of generating a MAP message informative of geometry of a vicinity of a V2X intersection, the method comprising, by a control unit: obtaining road-informative data related to the vicinity of the intersection and collected by one or more sensor-based (SB) source-modalities and one or more V2X source-modalities monitoring the vicinity of the intersection and operatively connected to the control unit, wherein the obtained road-informative data comprises vehicle-related data informative of vehicles and of parameters thereof detected by the source-modalities in the vicinity of the intersection; aggregating the obtained road-informative data, wherein aggregating is provided separately for each of the detected vehicles, thus giving rise to fused data; processing the fused data to obtain lane-level geometry of the intersection; and encoding the generated geometry of the intersection to generate a MAP message.

2. The method of Claim 1 , wherein the road- informative data is collected during a collection time frame, and wherein the length of the collection time frame is dynamically adjusted in accordance with traffic density.

3. The method of Claim 2, wherein the length of the collection time frame is adjusted separately for each of approaches.

4. The method of any one of Claims 1 - 3, wherein aggregating comprises synchronization of the collected road-informative data with the help of creating a single domain for data collected by all source-modalities.

5. The method of any one of Claims 1 - 4, wherein obtaining the geometry of the intersection comprises: processing the fused data to calculate trajectories of the detected vehicles in the vicinity of the intersection;using a high-definition (HD) map to match the calculated trajectories to lanes presented by the HD map; processing the fused data using the trajectories-to-lanes match to define lane attributes informative of lanes types; processing the fused data to provide semantic segmentation and identify the widths and lengths of the lanes; and detecting the intersection topology.

6. The method of Claim 5, wherein obtaining the geometry of the intersection further comprises mapping the lanes to traffic light signal groups.

7. The method of Claim 6, wherein the mapping the lanes to traffic light signal groups comprises: for each lane, collecting, during a sampling interval, per-vehicle data informative of movements of the respective vehicles and obtained from SB and V2X source-modalities, thereby yielding vehicle-related data; detecting traffic light controller (TLC) in the vicinity of the intersection and collecting, during the sampling interval, data informative of status state thereof, thereby yielding TLC- related data; processing the collected vehicle-related data and TLC-related data to match TLC status states to corresponding lanes; and using the matching results to map the TLC phases of to corresponding lanes and further providing lanes to traffic light signal groups mapping.

8. The method of any one of Claims 5 - 7, wherein defining attributes informative of lane types comprises: applying to the fused data a 1stMLM (Machine Learning Model) to segment the vicinity of the intersection, thereby yielding segment-indicative data;applying to the segment-indicative data a 2ndMLM to detect objects and classify the types thereof; applying to the detected objects a 3rdMLM to provide objects tracking; converting image-space coordinates of tracking objects into real- world coordinates and matching to coordinates of the lanes' coordinates; and setting the types of the lanes in accordance with respective types of detected objects.

9. The method of any one of Claims 5 - 8, wherein providing semantic segmentation comprises projecting data collected by all source-modalities into respective camera images and using the results to assign semantic labels to pixels and / or groups thereof.

10. The method of any one of Claims 5 - 9, wherein identifying the widths and lengths of a given lane comprises: a) processing the fused data to obtain initial coordinates of start and end of the given lane and a fusion-based lane width thereof; b) processing, separately for each type of SB source modalities, road-informative data to calculate maximal width of vehicles located at the given lane; c) using the calculated trajectories to define the average position of a center of the given lane; and d) setting the lane width in accordance with results of steps b) and c).

11. The method of any one of Claims 1 - 10, wherein the MAP message is automatically regenerated with a predefined periodicity or responsive to scheduled events influencing the geometry of the intersection.

12. The method of any one of Claims 1 - 11, wherein the MAP message is automatically regenerated responsive to detecting an anomaly event.

13. The method of Claim 12, wherein the anomaly event is detection of at least one of: a change in the geometry of the vicinity of the intersection; a change in traffic lights assignments and / or traffic patterns in the vicinity of the intersection, detection of special conditions related to the lanes.

14. The method of Claims 12 or 13, wherein the anomaly event is detected with the help of a 3rd party alert received by the control unit.

15. The method of any one of Claims 12 -14, wherein the anomaly event is detected with the help of information continuously requested by the control unit from a 3rd party.

16. The method of any one of Claims 12 -15, wherein the anomaly event is detected by continuously analyzing the collected and stored timestamped road-informative data and TLC- informative data to detect changes indicative of the anomaly event.

17. The method of Claim 16, wherein the detected changes are used during regenerating the MAP message for generating the updated geometry of the intersection.

18. The method of Claims 16 or 17, wherein the anomaly event is detected by analyzing statistics of vehicle-related data with respect to the lanes.

19. The method of any one of Claims 16 - 18, wherein the anomaly event is appearance of one or more reversible lanes, and wherein detecting said anomaly event comprises: aggregating statistical data informative of trajectories in the lanes within last 24 hours; detecting trajectories with opposite driving directions in different hours thereby identifying a reversible lane; and using the respective trajectories to detect time-depending characteristics of the reversible lanes.

20. The method of any one of Claims 16 - 19, wherein the anomaly event is appearance of permissive and / or protected lanes, and wherein detecting said anomaly event comprises: aggregating statistical data informative of vehicles' velocity, stop and turn; aggregating statistical data informative of vehicles aiming to the egress lane and pedestrian; and running classification algorithm to detect yield for pedestrian / other vehicles in the egress lane of the turn.

21. The method of any one of Claims 16 - 20, wherein an anomaly event is reconfiguration of the intersection and wherein detecting said anomaly event comprises detecting changes in traffic light status patterns.

22. The method of any one of Claims 16 -21, wherein regenerating the MAP messages is provided responsive to detecting identified anomaly event.

23. The method of any one of Claims 16 -22, wherein regenerating the MAP messages is provided responsive to receiving indication of a potential anomaly event without determination of the specific nature of the event.

24. The method of any one of Claims 16 - 23, further comprising using the stored data to validate the regenerated MAP message.

25. The method of Claim 24, further comprising broadcasting the validated regenerated MAP message.

26. The method of any one of Claims 12 - 25, wherein the anomaly event is detected with the help of a digital twin representing the geometry of the intersection and continuously updated by feeding road-informative and TLC-informative data.

27. One or more computing devices comprising processors and memory, the one or more computing devices configured, via computer-executable instructions, to perform operations for operating, in a cloud computing environment, a system capable of generating a MAP message, the operations comprising: obtaining road-informative data related to the vicinity of the intersection and collected by one or more sensor-based (SB) source-modalities and one or more V2X source-modalities monitoring the vicinity of the intersection and operatively connected to the control unit, wherein the obtained road-informative data comprises vehicle-related data informative of vehicles and of parameters thereof detected by the source-modalities in the vicinity of the intersection; aggregating the obtained road-informative data, wherein aggregating is provided separately for each of the detected vehicles, thus giving rise to fused data;processing the fused data to obtain lane-level geometry of the intersection; and encoding the generated geometry of the intersection to generate a MAP message.

28. The one or more computing devices of Claim 15 further configured to perform operations of any one of Claims 2 - 26.

29. A system capable of generating a MAP message, the system comprising a computer configured to perform the operations of any one of Claims 1 - 26.

30. The system of Claim 29, wherein at least part of the operations is provided in a cloud environment.

31. A non-transitory computer-readable medium comprising instructions that, when executed by a computing system comprising a memory storing a plurality of program components executable by the computing system, cause the computing system to operate in accordance with any one of Claims 1- 30.

Images