Map-based annotation for training autonomous mobility models

Abstract

A method, apparatus, and computer program product for augmenting the training of an autonomous travel model based on map-based annotations. The method includes augmenting an original training set with map-based annotated modification scenarios, at least one of which affects the route and at least one of which does not affect the route. The method includes generating modification scenarios that introduce synthetic route-changing objects into a feature map representing the original travel or travel scenario, which induce a travel path different from the original route, and other modification scenarios that introduce non-route-changing objects into the feature map, within which the original route is applicable. An autonomous travel model for predicting a travel path within a road segment based on the feature map representation is trained using the modification scenarios.

Description

[Technical Field]

[0001] This application claims the benefit of U.S. Patent Application No. 18 / 052,922, filed November 6, 2022, entitled "MAP BASED ANNOTATION FOR AUTONOMOUS MOVEMENT MODELS TRAINING," which is incorporated herein by reference in its entirety without creating a disclaimer.

[0002] The present disclosure relates generally to generating autonomous movement models, and more particularly to augmenting the training of autonomous movement models based on map-based annotations. [Background technology]

[0003] Control systems for autonomous vehicles are configured to identify appropriate navigation paths, obstacles, and relevant signs, helping the vehicle control its autonomous movement, e.g., for self-driving. At the core of an autonomous vehicle's control system is a stack of perception modules whose purpose is to perceive, reconstruct, and understand the system's environment. Modern approaches to environmental perception utilize specialized lightweight neural networks, which are capable of generating precise situational awareness models that can be used by other modules in the system to create motion plans, interact with, and ultimately navigate within the environment. Summary of the Invention

[0004] One exemplary embodiment of the subject matter of this disclosure is a method including: obtaining a first driving scenario including at least a first functional map including a representation of a first non-composite static topology of a first road segment, thereby defining a first original driving route for a vehicle traveling within the first road segment. The method further includes receiving a first modified driving scenario and a first driving route, the first driving route being applicable within the first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first functional map, the route change object being a composite object, and the route change object inducing the first driving route different from the first original driving route. The method further includes obtaining a second driving scenario, the second driving scenario including at least a second feature map including a representation of a second non-composite static topology of the second road segment, thereby defining a second original driving route for the vehicle traveling within the second road segment. The method further includes receiving a second modified driving scenario, the second modified driving scenario introducing non-path-changing objects into the second feature map, the non-path-changing objects being composite objects, thereby the second original driving route being applicable to the second modified driving scenario. The method further includes training an autonomous driving model, the autonomous driving model being configured to provide a predicted driving path within a road segment based on a functional map representation of the road segment, the training including instructing the autonomous driving model that a first driving path is applicable to a first modified driving scenario and instructing the autonomous driving model that a second, original driving path is applicable to a second modified driving scenario, whereby an original training set is augmented with map-based annotated modified scenarios, at least one of which affects the driving path and at least one of which does not affect the driving path.

[0005] Optionally, the method further comprises utilizing the autonomous driving model by the autonomous vehicle to determine a driving route in real time.

[0006] Optionally, the first non-composite static topology includes at least one of a road layer, a stop line layer, a lane marking layer, a sign layer, and a speed bump layer.

[0007] Optionally, the first feature map includes at least one of a static object layer representing static objects that may physically block the vehicle's path, a dynamic object layer representing dynamic objects with movement vectors that may collide with the vehicle, and an occlusion layer representing one or more occlusion sections in the first road section.

[0008] Optionally, the re-route object is a dynamic object having a syntactically defined movement vector, whereby the syntactically defined movement vector interferes with the first original travel path.

[0009] Optionally, the route change object is a static object having a syntactically defined location within the first road segment, whereby the static object creates a physical interference and reduces the advisability of using the first original driving route.

[0010] Optionally, the route change object is a modification to the first non-composite static topology that reduces the desirability of using the first original traveled route.

[0011] Optionally, the re-route object and the non-re-route object are the same.

[0012] Optionally, the method further includes receiving a third modified driving scenario and a third driving route, wherein the third driving route is applicable within the third modified driving scenario, the third modified driving scenario being defined based on the third driving scenario, the third modified driving scenario introducing a third route change object into the first function map, the third route change object being a synthetic object, and the third route change object inducing a third driving route different from the first original driving route and the first driving route, and the training further includes instructing the autonomous driving model that the third driving route is applicable to the third modified driving scenario.

[0013] Optionally, the method further includes receiving a third modified driving scenario, the third modified driving scenario introducing a third path non-changing object into the first capability map, the third path non-changing object being a synthetic object, and a type of the third path non-changing object being similar to a type of the path changing object, whereby the first original driving path is applicable to the third modified driving scenario, and the training further includes instructing the autonomous driving model that the first original driving path is applicable to the third modified driving scenario.

[0014] Optionally, the third non-path-changing object is identical to the path-changing object, and a location of the third non-path-changing object in the first functional map is different from a location of the path-changing object in the first functional map.

[0015] Optionally, the first feature map further comprises a layer representing driving information of the vehicle within a recent time frame.

[0016] Optionally, the functional map representation of the road segment includes an aggregate decision layer representing possible directions of applicable driving routes, the aggregate decision layer including at least one of a straight ahead label, a left label, and an all-route label.

[0017] Optionally, the feature map representation represents an aggregation of multiple driving scenarios associated with a road segment.

[0018] Another exemplary embodiment of the subject matter of the present disclosure is a computerized apparatus having a processor, the processor including: obtaining a first driving scenario, the first driving scenario including at least a first functional map including a representation of a first non-composite static topology of a first road segment, thereby defining a first original driving route of a vehicle traveling within the first road segment; receiving a first modified driving scenario and a first driving route, the first driving route being applicable within the first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first functional map, the route change object being a composite object, which induces a first driving route different from the first original driving route; and obtaining a second driving scenario, the second driving scenario including at least a second functional map including a representation of a second non-composite static topology of a second road segment, thereby defining a first original driving route of a vehicle traveling within the second road segment. a step of receiving a second modified driving scenario, the second modified driving scenario introducing non-path-changing objects into a second function map, the non-path-changing objects being synthetic objects, such that the second original driving route is applicable to the second modified driving scenario; and a step of training an autonomous driving model, the autonomous driving model configured to provide a predicted driving route within the road segment based on the function map representation of the road segment, the training including instructing the autonomous driving model that the first driving route is applicable to the first modified driving scenario and instructing the autonomous driving model that the second original driving route is applicable to the second modified driving scenario, such that the original training set is augmented with map-based annotated modified scenarios, at least one of which affects the driving route and at least one of which does not affect the driving route.

[0019] Yet another illustrative embodiment of the subject matter of the present disclosure is a computer program product comprising a non-transitory computer-readable storage medium bearing program instructions that, when read by a processor, cause the processor to perform a method, the method including obtaining a first driving scenario, the first driving scenario including at least a first feature map including a representation of a first non-synthetic static topology of a first road segment, thereby defining a first original driving path of a vehicle traveling within the first road segment; and obtaining a first modified driving scenario. receiving a driving scenario and a first driving route, the first driving route being applicable within a first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first function map, the route change object being a composite object, and the route change object inducing the first driving route different from the first original driving route; and obtaining a second driving scenario, the second driving scenario including at least a second non-conformity of a second road segment. a second feature map including a representation of a synthetic static topology, thereby defining a second original driving path for a vehicle traveling within the second road segment; receiving a second modified driving scenario, the second modified driving scenario introducing non-path-changing objects into the second feature map, the non-path-changing objects being synthetic objects, thereby defining the second original driving path for the vehicle traveling within the second road segment; and training an autonomous driving model configured to provide a predicted driving path within the road segment based on the functional map representation of the road segment, the training including instructing the autonomous driving model that the first driving path is applicable to the first modified driving scenario and instructing the autonomous driving model that the second original driving path is applicable to the second modified driving scenario, wherein an original training set is augmented with map-based annotated modified scenarios, at least one of which affects the driving path and at least one of which does not affect the driving path.

[0020] The disclosed subject matter will be more fully understood and appreciated from the following detailed description taken in conjunction with the drawings, in which corresponding or like numbers or characters indicate corresponding or like components. Unless otherwise specified, the drawings provide illustrative examples or aspects of the disclosure and do not limit the scope of the disclosure. [Brief explanation of the drawings]

[0021] [Figure 1A] 1 is a schematic diagram of an example functional map representation of a road segment, according to some example embodiments of the subject matter of this disclosure. [Figure 1B] 1 is a schematic diagram of an example functional map representation of a road segment, according to some example embodiments of the subject matter of this disclosure. [Figure 2A] 1 is a flow diagram of a method according to some example embodiments of the disclosed subject matter. [Figure 2B] 1 is a flow diagram of a method according to some example embodiments of the disclosed subject matter. [Figure 3A] 1 is a schematic diagram of an example functional map with objects synthetically added, according to some example embodiments of the subject matter of this disclosure. [Figure 3B] 1 is a schematic diagram of an example functional map with objects synthetically added, according to some example embodiments of the subject matter of this disclosure. [Figure 4] 1 is a block diagram of an apparatus according to some example embodiments of the disclosed subject matter. DETAILED DESCRIPTION OF THE INVENTION

[0022] In this disclosure, the term "autonomous vehicle," unless otherwise specified, shall be broadly construed to cover any machine or vehicle that may have the capability to move, navigate, etc. anonymously without the guidance of a human operator, driver, pilot, etc. The term "autonomous vehicle" is not limited to any particular form of vehicle, and should not be construed to be limited solely to vehicles utilized as a means of transportation. The term "autonomous vehicle" shall be construed to cover wheeled and non-wheeled vehicles, air vehicles, maritime vessels, submarines, etc. For the avoidance of doubt, the term also covers autonomous aircraft, autonomous aerial vehicles, unmanned aerial vehicles (UAVs), drones, unmanned surface vehicles (USVs), unmanned ground vehicles (UGVs), self-driving cars, autonomous underwater vehicles (AUVs), autonomous cargo ships, and legged robots.

[0023] The term "travel" in this disclosure, unless otherwise specified, shall be interpreted broadly to cover any form of autonomous vehicle movement. This term is not limited to wheeled vehicle movement alone, but also encompasses flying UAVs, piloting aircraft, and piloting maritime vessels. On the other hand, the term "road" in this disclosure is not limited to paths (e.g., paved roads) for carrying vehicular traffic having a surface, but also encompasses any area in which autonomous vehicle movement may occur. For example, without limiting the scope of this disclosure, the term "road" should be interpreted to cover sections of the ocean that unmanned surface vehicles can navigate, surfaces on which legged robots can move, 3D aerial volumes in which drones can fly, 3D underwater volumes in which autonomous submersible vehicles can submerge, etc.

[0024] The term "feature map" in this disclosure, unless otherwise specified, shall be interpreted broadly to cover any type of map or device that represents a space through which an autonomous vehicle travels and various obstacles or objects within that space that the autonomous vehicle may encounter or avoid. The term "feature map" is not limited to a particular type of map and may cover a top-down feature map representation of a road segment, a feature map provided from a predefined viewpoint, a three-dimensional (3D) map representing a three-dimensional space or environment, a rapid 3D map, etc. As one example, a top-down feature map representing a road segment may be utilized for an autonomous vehicle. As another example, a 3D feature map representing a flight domain may be utilized for a drone. As yet another example, a vertical map may be utilized for a wall-crawling robot, etc.

[0025] One technical challenge addressed by the subject matter of this disclosure is providing an accurate decision-making system for self-navigation of an autonomous vehicle. In some demonstrative examples, the decision-making system for self-navigation of an autonomous vehicle may be based on machine learning techniques, such as regression algorithms, decision matrix algorithms, neural networks, etc. The decision-making system may be configured to continuously render the surrounding environment, perceive the world, and learn and plan the best moves for the autonomous vehicle in real time, such as through a neural network, to provide optimized navigation for the autonomous vehicle. The neural network may be configured to learn how to make complex data-driven decisions in real time through observation and imitation. However, the understanding of the world perceived through a neural network may be gaps and incomplete, and the perception may often lack reliability. This is because it may be susceptible to noise, artifacts, and errors, may include false positives, and may be difficult to analyze.

[0026] Another technical challenge addressed by the subject matter of this disclosure is enhancing the training phase of a motion planning decision system for self-navigation of an autonomous vehicle to enable the decision system to independently navigate new locations. In some demonstrative examples, the largest portion of the maturation effort in an autonomous driving system may be spent on collecting and understanding training data. The training data sets used to train the system may be very large because they are sprawling across infinite space, but may not be useful because they do not cover all possible situations and locations, are not generated by an extremely careful driver, or are not generated based on strict rules. An autonomous driving system may need to handle complex data-driven decisions in real time based on the observed environment. However, the system may be unable to handle unknown scenarios and may misbehave when inputs deviate from their expected domain.

[0027] One technical solution is to augment an existing training set with synthetically modified scenarios (e.g., data cases). Each scenario may include an aggregated, comprehensive representation of the environment as generated by various perception modules of the autonomous vehicle. In some demonstrative examples, an existing driving scenario may be modified by inserting one or more synthetic objects, some of which may change the driving path associated with the driving scenario (e.g., rerouting objects) and others of which may not. For each existing training set, one or more enhanced training sets may be generated. In some demonstrative examples, the enhanced training set may include, in addition to the original driving scenario, one or more modified driving scenarios that are applicable to the associated original driving path because the synthetic objects inserted therein are non-modifying objects, e.g., objects that do not change the driving path. Additionally or alternatively, the enhanced training set may further include one or more modified driving scenarios having a driving path that is modified compared to the original driving path because the synthetic objects inserted therein are rerouting objects. Additionally or alternatively, each original training set may be augmented with modification scenarios, at least one of which affects the traveled path and at least one of which does not affect the traveled path.

[0028] In some demonstrative examples, synthetic modified scenarios may be generated manually, automatically, semi-automatically, or a combination thereof. As one example, synthetic modified scenarios may be created manually, such as to simulate rare use cases that introduce unusual scenarios. In another example, synthetic modified scenarios may be created automatically by performing rare substitutions on objects or events within an existing driving scenario. It may be noted that such substitutions may be changes to the objects or events represented in the scenario, rather than simple changes in parameter values. As yet another example, synthetic modified scenarios may be created automatically by blending synthetic and non-synthetic driving data. As yet another example, synthetic modified scenarios may be generated by modifying a computer representation of the original driving scenario.

[0029] Additionally or alternatively, annotation of the synthetic modified scenario, e.g., synthetic additional objects that may or may not be route-changing objects and how they change the route, may be performed manually, automatically, semi-automatically, or a combination thereof. As one example, a spatial reasoning deep neural network may be utilized to automatically annotate the synthetic modified scenario according to how a human driver would react. The network may be configured to infer generalizations by leveraging learned behavior to independently navigate through the modified driving scenario and determine a driving path. As another example, annotation may be performed automatically according to safe driving policies, such as a physically permissible maximum speed or rules defined based on the area range of physical limitations. It may be noted that annotation may be configured to mimic the behavior of a real or skilled driver. As one example, a driving scenario may be in a school area where the speed limit is 40 km / h, but if there are many children, the recommended maximum speed may be 30 km / h instead of 40 km / h, depending on how the driver drives. As another example, the annotation or corrected driving path may include "less typical" behavior but safer decisions, such as ignoring a traffic light to allow an emergency vehicle to pass or avoiding a collision with another vehicle.

[0030] It may be noted that synthetic addition objects, both when they are reroute objects and when they are non-reroute objects, may be dynamic objects (e.g., pedestrians, moving vehicles) or static objects (e.g., road signs, road elements, parked cars, traffic cones, bollards, fences, etc.). Dynamic reroute objects may have syntactically defined movement vectors that interfere with the original travel path. Static reroute objects may have syntactically defined locations within a road segment that create physical interference and reduce the recommendation to use the first original travel path.

[0031] It may be further noted that the same object, or object from the same type, may be a route-changing object when synthetically added to one driving scenario and a non-route-changing object when synthetically added to another driving scenario. Additionally or alternatively, the same object may be a route-changing object when synthetically added to one driving scenario at a particular location or at a particular time, and a non-route-changing object when synthetically added to the same driving scenario at a different location or at a different time.

[0032] In some demonstrative examples, an autonomous driving model may be configured to provide a predicted driving path within a road segment based on a functional map representation of the road segment that allows for representing a driving scenario, such as, but not limited to, a top-view functional map representation of the road segment, a functional map provided from a predefined viewpoint, a 3D map representing a 3D space or environment, a rapid 3D map, etc. As an example, a top-view functional map representing a road segment may be used for an autonomous vehicle. As another example, a 3D functional map representing a flight domain may be used for a drone. As yet another example, a vertical map may be used for a wall-crawling robot, etc. Additionally or alternatively, the functional map may represent a space through which the autonomous vehicle will navigate and various obstacles the autonomous vehicle may encounter within that space. In some demonstrative examples, an enhanced training set may be generated by modifying the functional map. The autonomous driving model may be trained using the enhanced training set. In some demonstrative examples, training the autonomous driving model may include instructing the autonomous driving model that a new driving path is applicable to a particular modified driving scenario while instructing that the original driving path is applicable to another modified driving scenario. The autonomous driving model can be used by autonomous vehicles to determine their driving routes in real time.

[0033] In some demonstrative examples, a driving scenario may be represented using functional maps that describe road segments in which the driving scenario occurs and possible next actions or movements that may be performed by the autonomous vehicle. Each map may represent at least a non-composite static topology of a road segment and define a driving path for an autonomous vehicle traveling within the road segment. The non-composite static topology may include at least one of a road layer representing the road along which the autonomous vehicle travels, a layer representing elements associated with the road, such as a stop line layer, a lane marking layer, a speed bump layer, etc., and a layer representing various traffic signs, such as a stop sign layer, a traffic light, etc. Additionally or alternatively, the map may further represent dynamic elements, such as pedestrians, other vehicles, etc.

[0034] In some exemplary embodiments, the functional road map may be generated on the fly using real-time visual input based on sensor data captured by sensors mounted on the vehicle, such as a front-mounted camera, LiDAR, odometry, inertial measurement unit, etc. In some exemplary embodiments, the functional road map may include precise real-time information and true absolute ground accuracy. Additionally or alternatively, the functional road map may be a high-definition (HD) map generated offline and dynamically updated based on real-time sensor data. In some exemplary embodiments, the functional road map may be of higher resolution than maps found in current traditional resources, such as cloud-based road maps. The functional road map may be utilized by the autonomous vehicle's navigation system. In some exemplary embodiments, the functional road map generated on the fly may be utilized in place of a pre-prepared HD map that may be correlated with the vehicle's current environment based on location information, such as that obtained using a GPS connection.

[0035] In some exemplary embodiments, functional road maps may be vehicle-centric. The vehicle may be determined to be at a fixed location and orientation within the map. Additionally or alternatively, functional road map formats may be functional. Each functional road map may include only the information necessary for the vehicle to safely navigate its environment. As an example, information may include information necessary for lane determination, identifying locations to stop the vehicle, etc. Additional information, such as that related to structures detected by sensors, may be missing from the functional road map.

[0036] In some demonstrative embodiments, the functional map may include a static layer and a dynamic layer. Each static layer may represent one or more types of static objects that may physically block the vehicle's path. Each dynamic layer may represent one or more types of dynamic objects with movement vectors that may collide with the vehicle. Additionally or alternatively, the functional map may include one or more occlusion layers that represent one or more occlusion sections within a road segment. In some demonstrative embodiments, the functional map may include a layer representing the vehicle's driving information in the most recent time frame, such as the vehicle's recent movement patterns, recent driving behavior, recent relative locations, the vehicle's speed history, etc. Additionally or alternatively, the layer representing the vehicle's driving information in the most recent time frame may include environment-based parameters, such as traffic light color history, navigation instructions provided in the most recent time frame, environmental understanding based on modules external to the map, etc. The time frame may be approximately 5 seconds, 10 seconds, 20 seconds, etc. Additionally or alternatively, the functional map may include one or more layers representing driving and environmental information from a series of recent time frames, which may be implemented as a raster. In some demonstrative examples, the modified driving scenario may be generated by modifying a representation of the original driving scenario, such as modifying a map representing the original driving scenario. Each functional map representing the modified driving scenario may represent a non-composite static topology of the road segment as represented in the functional map representing the original driving scenario, and a representation of the composite additional objects. Each functional map representing the modified driving scenario may define a driving path for the autonomous vehicle traveling within the road segment according to the composite additional objects. In some demonstrative examples, the route change object may be a modification to the non-composite static topology that reduces the recommendation to use the first original driving path.

[0037] Additionally or alternatively, the feature map representation of a road segment may include an aggregate decision layer that represents possible directions for applicable driving routes. The aggregate decision layer may include at least one path that defines one potential driving decision, such as "stay in lane," "turn right," or "change lane to left." Additionally or alternatively, the feature map may represent an aggregation of multiple driving scenarios associated with the road segment. It may be noted that the feature map may be a dynamic map that is updated in real time. Some layers of the feature map may remain unchanged, while other layers may change. The aggregate decision layer may change when an added object is a route change object, or it may not change when the object does not affect the original driving route.

[0038] In some demonstrative embodiments, the aggregate decision layer may be annotated manually or semi-automatically, such as by adding points, dots, labels, pixels, etc., representing possible decisions, movements, directions, etc. An automatic trajectory or path may be generated based on the manually added points. Additionally or alternatively, the aggregate decision layer may be automatically annotated using AI learning models, neural networks, existing or dynamically trained autonomous driving models, etc.

[0039] One technical effect of utilizing the subject matter of the present disclosure is to enhance the training of autonomous driving models. Training based on map-based annotations allows for variations in the natural environment to test decision-making. Such training allows for the transmission of subtle changes that may alter decisions as if they were made by an experienced driver. In addition, the training of the present disclosure is resistant to problems of habitual learning, such as by training the model (e.g., the respective neural network) to ignore noise and artifacts, thereby overcoming repetitive and unhelpful cases.

[0040] Additionally or alternatively, the subject matter of this disclosure provides a contextual understanding of dynamic environments, unlike classical motion planning algorithms that may be required to perceive the dynamic environment either by the motion planning model itself or by an intermediate understanding layer that is conceptually in the domain of perception but separate from it. Training based on map-based annotations makes it possible to obviate the requirement to manually develop algorithms for classifying the behavior of dynamic objects in the environment, classifying the "status" (if you will) of the ego-vehicle, and so on.

[0041] Another technical effect of utilizing the presently disclosed subject matter is that it provides a solution to the infinitely complex problem of generating training data sets that cover all possible scenarios by transforming the problem from the algorithmic domain into the world of data. While previous solutions provide unmaintainable code bases that can become irrelevant in new environments, the presently disclosed subject matter allows the natural environment to adapt to any possible changes that may occur. Due to the way classical algorithms operate, layering hundreds of rules one on top of the other can require extreme care for backward compatibility. Due to the explosion of case combinations, handling one type of behavior can unintentionally degrade handling of another type of behavior. However, the presently disclosed subject matter solves the decision-making aspect of motion planning using neural networks, thereby effectively transforming driving scenarios into data problems as opposed to conceptual algorithmic problems.

[0042] Yet another technical effect of utilizing the subject matter of the present disclosure is to provide an automatic annotation tool that does not involve a programmer. The burden of implicitly (or explicitly) classifying the behavior of various actors in the environment and understanding the situation of the vehicle can be transferred from the program to the annotator, which can be a human driver. This makes the problem tractable, and all annotators know how to "drive" given a particular situation. Therefore, instead of developing algorithms that always have blind spots and always cause degradation in existing behavior, the subject matter of the present disclosure can add diverse data to the training set and transfer the responsibility for understanding contextual classification to the network.

[0043] The subject matter of the present disclosure may provide one or more technical improvements over any previous techniques and over any techniques previously routine or conventional in the art. Additional technical problems, solutions, and advantages may become apparent to those skilled in the art in light of the present disclosure.

[0044] Reference is now made to FIG. 1A, which illustrates a schematic diagram of an example functional map representation of a road segment, in accordance with some example embodiments of the subject matter of this disclosure.

[0045] In some demonstrative embodiments, map 100a may be a functional road map generated by an artificial neural network (ANN) based on visual input 110a. In some demonstrative embodiments, map 100a may be a top-view map. Visual input 110a may include one or more images 111a representing road segment 120a in front of autonomous vehicle 130a, behind vehicle 130a, around vehicle 130a, etc. Additionally or alternatively, visual input 110a may represent vehicle 130a capturing a field of view, its surroundings, etc. In some demonstrative embodiments, the ANN may be trained to detect road features and transform input image space into a single-shot top-view space. The ANN may be configured to process image 111a and generate map 100a.

[0046] In some demonstrative embodiments, map 100a may be configured to be provided to an autonomous navigation system of vehicle 130a. The autonomous navigation system may be utilized by vehicle 130a to perform autonomous navigation, such as to find a directionally relative position on a road, adjust a route, etc. The autonomous navigation system may enable vehicle 130a to navigate autonomously according to the functional features represented by map 100a.

[0047] In some demonstrative embodiments, map 100a may be a multi-layered map including multiple cumulative layers, each representing a different functional feature. In some demonstrative embodiments, a layer may represent functional features associated with road geometry, such as asphalt, lane markings, crosswalks, parking lots, stop lines, speed bumps, etc. Additionally or alternatively, a layer may represent functional features associated with static objects, such as cones, fences, general static objects, traffic signs, pavement markings, traffic lights, etc. Additionally or alternatively, a layer may represent functional features associated with objects defining traffic rules, such as traffic lights, "yield" signs (inverted triangles), "no U-turns" signs, "no left turn arrows" signs, etc., which may be further categorized based on their status (e.g., green light) and their relevance to the ego-vehicle (e.g., stop signs facing other lanes of travel, no right turn on red signs, whose relevance depends on the ego-vehicle's driving decisions and intentions). Additionally or alternatively, a layer may represent functional features associated with dynamic objects (e.g., other vehicles, pedestrians), such as other vehicles' brake lights, side indicators, reverse lights, etc., pedestrian intent (e.g., is this person looking / gestures / moving / sitting? What are they trying to communicate?), etc. Additionally or alternatively, a layer may represent the trajectories of dynamic objects, such as vehicles, pedestrians, motorbikes, bicycles, animals, etc.

[0048] In some demonstrative embodiments, the map 100a may include, for each pixel, one or more features related to assisting the autonomous navigation of the vehicle 130a within the road segment 120a. Such elements and features may include the road itself (101a), road elements (103a), road geometry, signs, vehicles or portions thereof (104a), obstacles, moving elements (105a), trajectories of dynamic objects, etc. In some demonstrative embodiments, each pixel in the map 100a may be associated with a predetermined relative position of the vehicle 130a. The contents of each pixel in the map 100a may be assigned a set of values. Each value may be configured to represent a functional feature associated with the location at the pixel's predetermined relative position. By way of example, the functional feature may be a drivable road indication (101a), an available driving route indication (135b), a stop line indication (106a), a speed bump indication, a lane marking indication, etc. The set of values ​​assigned to each pixel in the map 100a may include at least two different functional features. Additionally or alternatively, map 100a may be configured to demonstrate navigation decisions. As an example, map 100a may be configured to demonstrate a decision to stop at stop sign 125a using short route 145a.

[0049] Reference is now made to FIG. 1B, which illustrates a schematic diagram of an example functional map representation of a road segment, in accordance with some example embodiments of the subject matter of this disclosure.

[0050] In some demonstrative examples, map 100b may be a functional road map similar to map 100a. Map 100b may be an overhead view map. Map 100b may be generated by an ANN based on visual input 110b including one or more images 111b depicting road segment 120b surrounding autonomous vehicle 130b. Vehicles 130a and 130b may be the same vehicle. Road segment 120b may be similar to or adjacent to road segment 120a, such as a segment consecutive to road segment 120a, a road segment that vehicle 130a (e.g., 130b) travels after road segment 120a, etc.

[0051] In some demonstrative embodiments, map 100b may be configured to represent the driving environment within road segment 120b on a frame-by-frame basis while maintaining historical information over time. The historical information may be associated with trajectories or movement vectors of dynamic objects, movement patterns of autonomous vehicles, etc. Additionally or alternatively, map 100b may be configured to represent recent frame profiles, such as a vehicle speed profile (e.g., vehicle travel speed within a time window of the last N seconds, where N may be 10, 20, 60 seconds, etc.), a traffic light classification profile (e.g., the status of all existing traffic lights within a time window of the last N seconds, where N may be 10, 20, 60 seconds), etc.

[0052] Additionally or alternatively, map 100b may be configured to represent one or more decision layers that indicate driving options, navigation commands, driving instructions, driving intent, etc. In some exemplary embodiments, different decision layers may be associated with different levels of specification that may be accumulated in a motion plan, applied sequentially, etc. As an example, one layer may be a navigation layer (e.g., where am I and where do I want to go?). Another layer may be a navigation command layer (given that I know where I want to go in the current environment, what maneuver should I perform? (i.e., stay in lane, change lanes, etc.)). As an example, map 100b may be configured to represent a vehicle coming to a full stop at a stop sign as shown in map 100a and to demonstrate a decision to continue driving after the full stop, showing three distinctly different paths: right 150b, forward 160b, and left 170b.

[0053] Reference is now made to FIG. 2A, which illustrates a flow diagram of a method according to some example embodiments of the disclosed subject matter.

[0054] In step 210, a driving scenario can be obtained that defines an original driving path of a vehicle traveling within the road segment. In some demonstrative embodiments, the driving scenario may include a feature map, such as a top-view feature map representation that includes a representation of a non-composite static topology of the road segment. Each non-composite static topology may be represented using one or more layers, such as a road layer, a stop line layer, a lane marking layer, a sign layer, a speed bump layer, etc. Additionally or alternatively, the feature map may include a static object layer representing static objects that may physically occlude the vehicle's path, a dynamic object layer representing dynamic objects with movement vectors that may collide with the vehicle, an occlusion layer representing one or more occlusion segments within the first road segment, etc.

[0055] In some demonstrative examples, the driving scenario may be obtained from a training set used to train the autonomous driving model. Additionally or alternatively, the driving scenario may be a real-world driving scenario observed by a system of the autonomous vehicle.

[0056] In step 220, a modified driving scenario may be received. The modified driving scenario may introduce synthetic objects into the capability map. In some demonstrative embodiments, the synthetic objects may be dynamic objects with syntactically defined movement vectors, such as moving vehicles, pedestrians, animals, etc. Additionally or alternatively, the synthetic objects may be static objects with syntactically defined locations within the road segment, such as signs, road additions, static elements, etc.

[0057] A determination of whether the composite object is a re-route object may be made in step 230. In some example embodiments, a composite object may be a re-route object if it modifies the original travel path or necessitates a modification of the original travel path.

[0058] In some demonstrative examples, a reroute object may be a modification to a non-composite static topology that may reduce the advisability of using the original driving route. As an example, given that a composite object is a dynamic object, its syntactically defined movement vector may interfere with the original driving route, thus necessitating a modification of the driving route to prevent an accident or other undesirable situation. As another example, given that a composite object is a static object, the original driving route may be modified when the static object creates a physical interference that reduces the advisability of using the original driving route. It may be noted that the same composite object, or an object from the same type, may be a reroute object when introduced into a first driving scenario and a non-reroute object when introduced into another driving scenario, even when inserted at a similar location, similar timing, etc.

[0059] Additionally or alternatively, the same composite object may be a path-changing object when introduced into a driving scenario at a particular location or at a particular time, but may be a non-path-changing object when inserted at a different location or at a different time.

[0060] In response to determining that the composite object is a route change object, an applicable modified driving path triggered by the composite object may be determined for the modified driving scenario in step 240. In some demonstrative embodiments, the modified driving scenario may be annotated based on a map representation thereof, e.g., based on the map, an applicable driving path is determined.

[0061] In step 245, in response to determining that the composite object is not a route change object, an indication is made that the original driving route is applicable to the modified driving scenario.

[0062] The training set can be enriched with modified driving scenarios in step 250. Map-based annotated versions of the modified driving scenarios and applicable modified driving paths can be added to the training set.

[0063] It may be noted that steps 210-250 may be repeated periodically to generate additional map-based annotated training data and enhance the training data set.

[0064] In step 260, the reinforced training set can be used to train the autonomous driving model.

[0065] In step 270, the autonomous vehicle may utilize the autonomous driving model to determine a driving route in real time.

[0066] Reference is now made to FIG. 2B, which illustrates a flow diagram of a method according to some example embodiments of the disclosed subject matter.

[0067] In step 210b, a first driving scenario and a second driving scenario may be obtained. The first driving scenario may define a first original driving path of the vehicle traveling within the first road segment. The second driving scenario may define a second original driving path of the vehicle traveling within the second road segment. Each of the first driving scenario and the second driving scenario may include a feature map including a representation of a non-composite static topology of the first road segment and the second road segment, respectively.

[0068] In step 220b, a first modified driving scenario and a second modified driving scenario may be received. The first modified driving scenario and the second modified driving scenario may introduce a composite object into the function maps of the first driving scenario and the second driving scenario, respectively. In some demonstrative embodiments, the composite object may be a route-changing object for the first driving scenario and a route-non-changing object for the second driving scenario.

[0069] In step 240b, an applicable modified driving path induced by the synthetic object can be determined for the first modified driving scenario.

[0070] Step 245b indicates that the second original driving route is applicable to the second modified driving scenario.

[0071] In step 250b, the training set may be augmented with the first modified driving scenario and the second modified driving scenario.

[0072] It may be noted that steps 210b through 250b may be repeated periodically to generate additional map-based annotated training data and enhance the training data set.

[0073] In step 260b, the reinforced training set can be used to train the autonomous driving model.

[0074] In step 270b, the autonomous vehicle route can be determined in real time using the autonomous driving model.

[0075] Reference is now made to FIG. 3A, which illustrates a schematic diagram of an example functional map with composite additive objects, in accordance with some example embodiments of the subject matter of this disclosure.

[0076] In some demonstrative embodiments, map 300 may be a functional map representing a driving scenario of a vehicle within a road segment. Map 300 may be a top-down functional map. Map 300 may include a representation of a non-composite static topology of the road segment in which the driving scenario occurs, such as a representation of road 301, a representation of crossing pedestrians or crosswalks 302 and 303, a representation of stop lines 304, a representation of lane markings or different lanes (305), a representation of signs, speed bumps, etc.

[0077] In some demonstrative examples, a driving scenario may define a driving path for a vehicle traveling within a road segment, such as route 310, other options such as route 315, or optional driving directions. As an example, a driving scenario may represent decision-making at an empty roundabout.

[0078] In some demonstrative embodiments, map 300 may include representations of static objects that may physically occlude the vehicle's path, such as object 316 occluding path 315. Additionally or alternatively, map 300 may include representations of dynamic objects with movement vectors that may collide with the vehicle, such as vehicle 320 traveling in lane 305, other vehicles 330, etc.

[0079] It may be noted that map 300 may represent a genuine driving scenario or may be a map-based annotated scenario that may be generated based on a series of one or more modifications to other driving scenarios. As one example, object 316 may be a composite object inserted into a previous version of a driving scenario that modifies driving path 315. Object 316 may also be a static object with a syntactically defined location within a road segment that creates a physical interference that reduces the advisability of using the original driving path (e.g., continuing driving path 315). As another example, a route change object may be a modification to the non-composite static topology of a road segment (e.g., 345) that reduces the advisability of using the original driving path or creates a blocked segment within the road segment, such as segment 340, where a vehicle cannot travel.

[0080] Reference is now made to FIG. 3B, which illustrates a schematic diagram of an example functional map with composite additive objects, in accordance with some example embodiments of the subject matter of this disclosure.

[0081] In some demonstrative embodiments, map 350 may be a modified functional map representing a modified driving scenario generated as a modification to the driving scenario represented by map 300. Map 350 may be a top-down map. Map 350 may be generated by introducing synthetic objects into map 300, such as pedestrians 360, moving vehicles 370, etc.

[0082] A synthetic object, such as pedestrian 360, may be a route-changing object that triggers a different travel path 380, requiring the vehicle to come to a complete stop before crosswalk 302. Pedestrian 360 may be a dynamic object with a syntactically defined movement vector that interferes with travel path 310 and therefore needs to be changed. Travel path 380 may differ from original travel path 310a and may be annotated based on modifications to map 300 as represented in map 350. Additionally or alternatively, a synthetic object, such as vehicle 370, may be a non-route-changing object for paths 310 and 315 because its movement vector may not collide with the vehicle when at that particular location within travel path 310. However, if vehicle 370 is inserted into map 350 at a different location that collides with the vehicle, with a different movement vector that interferes with travel path 310, or at a different time (e.g., when the vehicle is still located near the location of vehicle 370), vehicle 370 becomes a route-changing object. Additionally or alternatively, vehicle 370 may be a rerouting object with respect to other driving scenarios of other vehicles, such as vehicle 330, within the road segment.

[0083] It may be noted that while map 350 exhibits multiple different optional modifications to the driving scenario represented in map 300, a single composite object may be added and annotated in each case to generate the modified scenario.

[0084] Reference is now made to Figure 4, which illustrates a block diagram of an apparatus 400, in accordance with some example embodiments of the presently disclosed subject matter. The apparatus 400 can be configured to support the generation of a model for generating a functional road map for an autonomous vehicle, in accordance with the presently disclosed subject matter.

[0085] In some demonstrative embodiments, device 400 may include one or more processors 402. Processor 402 may be a central processing unit (CPU), a microprocessor, an electronic circuit, an integrated circuit (IC), etc. Processor 402 may be utilized to perform calculations required by device 400 or any of its subcomponents.

[0086] In some example embodiments of the subject matter of this disclosure, the device 400 can include an input / output (I / O) module 405. The I / O module 405 can be utilized to provide output to and receive input from a user, device, sensor, etc., such as, for example, receiving input from one or more sensors of the connected car 450, providing output to one or more systems of the connected car 450, etc.

[0087] In some demonstrative embodiments, device 400 may include memory 407. Memory 407 may be a hard disk drive, a flash disk, random access memory (RAM), a memory chip, etc. In some demonstrative embodiments, memory 407 may hold program code operable to cause processor 402 to perform operations associated with any of the subcomponents of device 400.

[0088] In some demonstrative examples, map generator 410 may be configured to generate a functional map of a road in front of an autonomous vehicle, such as one or more of autonomous vehicles 480. The map generated by map generator 410 may be an overhead view map. The functional map may be functionally represented using functional features related to assisting the autonomous vehicle in autonomous navigation within the map. Each pixel in the generated functional map may be associated with a predetermined relative location with respect to autonomous vehicle 480. The content of each pixel in the generated functional map may be assigned a set of values, each representing functional features related to a location at the pixel's corresponding predetermined relative location.

[0089] In some demonstrative embodiments, map generator 410 may be configured to process real-time inputs provided by sensors of autonomous vehicle 480 to generate a functional map without relying on predetermined precise mapping obtained using GPS or other information. In some demonstrative embodiments, the functional map may be configured to provide functional information useful to navigation system 485 for implementing autonomous navigation of autonomous vehicle 480. Navigation system 485 may cause autonomous vehicle 480 to navigate autonomously according to the functional features represented by the generated functional map in accordance with the autonomous navigation model generated by model generator 430.

[0090] In some demonstrative embodiments, the functional map generated by the map generator 410 may be a multi-layered map. Each layer may represent a different component of a driving scenario, a different functional feature, etc. The functional map may include multiple layers representing a non-composite static topology of a road segment, such as roads, lanes, stop lines, lane markings, signs, speed bumps, etc. Additionally or alternatively, the functional map may include one or more layers representing static objects that may physically occlude the vehicle's path, such as barriers, obstacles, poles, traffic cones, parked vehicles, etc. Additionally or alternatively, the functional map may include one or more layers representing dynamic objects with movement vectors that may collide with the vehicle, such as other vehicles, motorcycles, trucks, buses, pedestrians, animals, etc. Additionally or alternatively, the functional map may include one or more occlusion layers representing one or more occlusion segments or regions within a road segment or region. Blockages can be caused by physical elements such as irregularly parked vehicles, rocks, etc., physical conditions of the road such as damaged, deteriorated, destroyed sections, etc. Additionally or alternatively, blockages can be caused by implicit conditions created by elements that cause visual obstructions such as traffic signs, directions, structures, large vehicles (trucks and buses), weather and lighting conditions (e.g., fog), automatic gates and barriers, large groups of pedestrians on crosswalks, etc.

[0091] In some demonstrative embodiments, the feature map generated by the map generator 410 may include a possible direction layer representing applicable driving paths, such as drive left / right labels, keep straight, full route applicable instructions, etc. Additionally or alternatively, the feature map generated by the map generator 410 may include an aggregate decision layer representing possible directions of applicable driving paths, such as lane keeping, turn right / left, change left / right lane, park straight labels, left labels, right labels, full route labels, etc. Additionally or alternatively, the feature map generated by the map generator 410 may include an instruction layer, such as instructing the vehicle to make a full stop, a sudden stop, etc. It may be noted that the feature map representation may represent an aggregation of multiple driving scenarios associated with a road segment.

[0092] In some demonstrative embodiments, the model generator 430 can be configured to utilize data from the training data set 420 to generate and train the autonomous driving model 450. The model generator 430 can be configured to utilize the map-based annotation module 440 to syntactically generate and annotate the training data cases.

[0093] In some demonstrative embodiments, the map-based annotation module 440 can obtain a driving scenario and an associated feature map representing the driving scenario, such as an overhead feature map. The feature map can include a representation of a non-synthetic static topology of a road segment. The driving scenario can define an original driving path of a vehicle traveling within the road segment.

[0094] In some exemplary embodiments, the map-based annotation module 440 can be configured to generate a modified driving scenario that defines a driving path based on an existing driving scenario. In some exemplary embodiments, the map-based annotation module 440 can be configured to generate the modified driving scenario by introducing a composite object into a functional map associated with the original driving scenario. In some exemplary embodiments, the composite object can be a route-changing object that induces a driving path different from the original driving path. Additionally or alternatively, the composite object can be a non-route-changing object that does not modify the original driving path (e.g., leaving the original driving path applicable to the modified driving scenario). In some exemplary embodiments, the map-based annotation module 440 can introduce a composite object that has already been classified as a route-changing object and automatically determine whether the composite object can affect the driving scenario when introduced at a particular location at a particular time. In some exemplary embodiments, the annotation module 440 can be configured to implicitly select a route-changing object based on previously annotated data (e.g., a pedestrian ahead of the host vehicle becomes a route-changing object if the pedestrian shortens the host vehicle's path before reaching the pedestrian).

[0095] In some demonstrative examples, the reroute objects added by the map-based annotation module 440 may be dynamic objects with syntactically defined movement vectors that may interfere with the original traveled route. In some demonstrative examples, layers representing the reroute objects may be added to the original functional map. As an example, a layer may be added to the map simulating a distant pedestrian running toward the original traveled route, forcing a slowdown. As another example, an added layer may simulate an emergency situation that forces an unusual decision (such as swerving away from a fast-moving oncoming vehicle that accidentally enters the traveled lane). Additionally or alternatively, the reroute objects added by the map-based annotation module 440 may be static objects with syntactically defined locations within a road segment that may create a physical interference that reduces the advisability of using the original traveled route.

[0096] Additionally or alternatively, the route change objects added by the map-based annotation module 440 may be modifications to the non-composite static topology that make using the original traveled route less advisable, such as road occlusions, the addition of traffic signs (e.g., no entry signs), etc. As an example, a route change object may be a large vehicle occluding the current lane forcing a lane change, the placement of a "no entry" sign at an intersection exit that prevents the ego vehicle from traveling there, etc.

[0097] In some demonstrative embodiments, the map-based annotation module 440 may be configured to annotate the training data set with binary labels indicating, for example, whether the original driving path is applicable within the modified driving scenario. Additionally or alternatively, the map-based annotation module 440 may be configured to determine a modified driving path when the synthetically added object is determined to be a reroute object, such as by using a neural network, a navigation model, or the like. The modified driving path may be represented in a modified feature map representation of the road segment according to the modified driving scenario. The modified feature map representation may include a new aggregate decision layer representing possible directions of the modified driving path, modifications to a recent aggregate decision layer, etc.

[0098] In some demonstrative embodiments, training module 460 may be configured to train autonomous driving model 450. Autonomous driving model 450 may be configured to provide a predicted driving path within a road segment based on a functional map representation of the road segment. For each modified driving scenario, training module 445 may be configured to instruct model 450 whether the original driving path is applicable or to define a new driving path.

[0099] In some demonstrative embodiments, autonomous driving model 450 may be trained using training data set 420. Training data set 420 may include original driving scenarios, manually annotated scenarios, etc., that define real-world driving scenarios as obtained from an autonomous vehicle system. Additionally or alternatively, training data set 420 may include map-based annotated modified scenarios generated by modification module 435. It may be noted that training data set 420 may be continuously augmented with map-based annotated modified scenarios generated by map-based annotation module 440, some of which affect the original driving path and some of which do not affect the original driving path.

[0100] In some demonstrative embodiments, autonomous driving model 450 may be provided to one or more connected autonomous vehicles 480 via I / O module 405 for use by navigation systems 485 of the autonomous vehicles 480 to determine driving routes in real time.

[0101] The present invention may be a system, method, and / or computer program product, which may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0102] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves having instructions recorded thereon, and any suitable combination of the above. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating in waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0103] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or may be downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmission cables, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0104] The computer-readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and the like, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to implement aspects of the present invention.

[0105] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executing on the processor of the computer or other programmable data processing apparatus create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0107] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to create a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0108] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts, or a combination of dedicated hardware and computer instructions.

[0109] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless clearly indicated otherwise. Furthermore, it will be understood that the terms "comprises" and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0110] The corresponding structure, material, acts, and equivalents of all means-plus-function or step-plus-function elements in the appended claims are intended to include any structure, material, or acts for performing the function that is specifically claimed in combination with other claimed elements. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The examples were chosen and described in order to best explain the principles and practical applications of the invention and to enable others skilled in the art to understand the invention in various embodiments with various modifications as may be suited to the particular uses contemplated.

Claims

1. 1. A method comprising: obtaining a first driving scenario, the first driving scenario including at least a first feature map including a representation of a first non-composite static topology of a first road segment, thereby defining a first original driving path of a vehicle traveling within the first road segment; receiving a first modified driving scenario and a first driving route, the first driving route being applicable within the first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first function map, the route change object being a composite object, and the route change object inducing the first driving route different from the first original driving route; obtaining a second driving scenario, the second driving scenario including at least a second feature map including a representation of a second non-composite static topology of a second road segment, thereby defining a second original driving path for a vehicle traveling within the second road segment; receiving a second modified driving scenario, the second modified driving scenario introducing a non-route-changing object into the second capability map, the non-route-changing object being a composite object, such that the second original driving path is applicable to the second modified driving scenario; training an autonomous driving model, the autonomous driving model configured to provide a predicted driving path within a road segment based on a functional map representation of the road segment, the training including: Instructing the autonomous driving model that the first driving path is applicable to the first modified driving scenario; and instructing the autonomous driving model that the second original driving path is applicable to the second modified driving scenario; Thus, the original training set is augmented with map-based annotated modification scenarios, at least one of which affects the driving path and at least one of which does not affect the driving path. A method comprising:

2. The method of claim 1 , further comprising utilizing the autonomous driving model by an autonomous vehicle to determine a driving route in real time.

3. The first non-composite static topology comprises: road layer, stop line layer, Lane division means layer, a label layer, and Speed ​​Bump Layer The method of claim 1 , comprising at least one of:

4. The first functional map comprises: a static object layer representing static objects that may physically occlude the path of the vehicle; a dynamic object layer representing dynamic objects having movement vectors that may collide with the vehicle; and a closure layer representing one or more closure sections within the first road segment; The method of claim 1 , comprising at least one of:

5. The method of claim 1 , wherein the re-route object is a dynamic object having a syntactically defined movement vector, whereby the syntactically defined movement vector interferes with the first original travel path.

6. 2. The method of claim 1, wherein the reroute object is a static object having a syntactically defined location within the first road segment, whereby the static object creates a physical interference and reduces the desirability of using the first original driving route.

7. The method of claim 1 , wherein the route change object is a modification to the first non-composite static topology that reduces the desirability of using the first original traveled route.

8. The method of claim 1 , wherein the re-route object and the non-re-route object are the same.

9. The method comprises: receiving a third modified driving scenario and a third driving route, the third driving route being applicable within the third modified driving scenario, the third modified driving scenario being defined based on the third driving scenario, the third modified driving scenario introducing a third route change object into the first function map, the third route change object being a composite object, and the third route change object inducing the third driving route different from the first original driving route and the first driving route; The training includes: The method of claim 1 , further comprising instructing the autonomous driving model that the third driving path is applicable to the third modified driving scenario.

10. The method comprises: receiving a third modified driving scenario, the third modified driving scenario introducing a third route non-changing object into the first function map, the third route non-changing object being a composite object, and a type of the third route non-changing object being similar to a type of the route changing object, such that the first original driving route is applicable to the third modified driving scenario; The training includes: The method of claim 1 , further comprising instructing the autonomous driving model that the first original driving path is applicable to the third modified driving scenario.

11. 11. The method of claim 10, wherein the third non-re-path object is identical to the re-path object, and a location of the third non-re-path object in the first functional map is different from a location of the re-path object in the first functional map.

12. The method of claim 1 , wherein the first feature map further comprises a layer representing driving information of the vehicle within a recent time frame.

13. 2. The method of claim 1, wherein the feature map representation of the road segment includes an aggregate decision layer representing possible directions of applicable driving routes, the aggregate decision layer including at least one of a straight ahead label, a left label, and an entire route label.

14. The method of claim 1 , wherein the feature map representation represents an aggregation of multiple driving scenarios associated with the road segment.

15. 1. A computerized device having a processor, the processor comprising: obtaining a first driving scenario, the first driving scenario including at least a first feature map including a representation of a first non-composite static topology of a first road segment, thereby defining a first original driving path of a vehicle traveling within the first road segment; receiving a first modified driving scenario and a first driving route, the first driving route being applicable within the first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first function map, the route change object being a composite object, and the route change object inducing the first driving route different from the first original driving route; obtaining a second driving scenario, the second driving scenario including at least a second feature map including a representation of a second non-composite static topology of a second road segment, thereby defining a second original driving path of a vehicle traveling within the second road segment; receiving a second modified driving scenario, the second modified driving scenario introducing a non-route-changing object into the second capability map, the non-route-changing object being a composite object, such that the second original driving path is applicable to the second modified driving scenario; training an autonomous driving model, the autonomous driving model configured to provide a predicted driving path within the road segment based on the functional map representation of the road segment, the training including: Instructing the autonomous driving model that the first driving path is applicable to the first modified driving scenario; and instructing the autonomous driving model that the second original driving path is applicable to the second modified driving scenario; training, whereby the original training set is augmented with map-based annotated modification scenarios, at least one of which affects the driving path and at least one of which does not affect the driving path; 1. A computerized apparatus adapted to perform the steps of:

16. 1. A computer program product comprising a non-transitory computer-readable storage medium bearing program instructions that, when read by a processor, cause the processor to perform a method, the method comprising: obtaining a first driving scenario, the first driving scenario including at least a first feature map including a representation of a first non-composite static topology of a first road segment, thereby defining a first original driving path of a vehicle traveling within the first road segment; receiving a first modified driving scenario and a first driving route, the first driving route being applicable within the first modified driving scenario, the first modified driving scenario being defined based on the first driving scenario, the first modified driving scenario introducing a route change object into the first function map, the route change object being a composite object, and the route change object inducing the first driving route different from the first original driving route; obtaining a second driving scenario, the second driving scenario including at least a second feature map including a representation of a second non-composite static topology of a second road segment, thereby defining a second original driving path for a vehicle traveling within the second road segment; receiving a second modified driving scenario, the second modified driving scenario introducing a non-route-changing object into the second capability map, the non-route-changing object being a composite object, such that the second original driving path is applicable to the second modified driving scenario; training an autonomous driving model, the autonomous driving model configured to provide a predicted driving path within a road segment based on a functional map representation of the road segment, the training including: Instructing the autonomous driving model that the first driving path is applicable to the first modified driving scenario; and instructing the autonomous driving model that the second original driving path is applicable to the second modified driving scenario; Thus, the original training set is augmented with map-based annotated modification scenarios, at least one of which affects the driving path and at least one of which does not affect the driving path.

2. A computer program product adapted to perform the steps of:

Images