Tool for performance testing and / or training of an autonomous vehicle planner

Abstract

A computer-implemented method of evaluating a goal planner of an autonomous robot in a real or simulated scenario, the method comprising: receiving evaluation data for evaluating performance of the goal planner in the scenario, the evaluation data being generated by applying the goal planner at incremental planning steps to compute a series of autonomous plans in response to changes in the scenario, the series of autonomous plans being implemented in the scenario to cause changes in autonomous states; the evaluation data comprising: an autonomous plan computed by the goal planner at one of the planning steps, and a scenario state at a time instant of the scenario; wherein the evaluation data is used to evaluate the goal planner by: computing a reference plan for the time instant based on the scenario state, the scenario state comprising autonomous states at time instants caused by one or more previous autonomous plans in the series of autonomous plans computed by the goal planner, and computing at least one evaluation score for comparing the autonomous plan to the reference plan.

Description

Technical Field

[0001] The present disclosure relates to tools and techniques for testing the performance of autonomous vehicle planners, as well as methods, systems, and computer programs for implementing the tools and techniques. The techniques can also be applied in a training context to generate training data for training planners. Background Art

[0002] There has been major and rapid development in the field of autonomous vehicles. An autonomous vehicle is a vehicle equipped with sensors and an autonomous driving system that enables the vehicle to operate without human control of its behavior. In this article, the term autonomous driving covers both semi-autonomous and fully autonomous driving behaviors. Sensors enable the vehicle to perceive its physical environment and may include, for example, cameras, radars, and lidars. An autonomous vehicle is equipped with a suitably programmed computer that is able to process the data received from these sensors and make safe and predictable decisions based on the context that these sensors have perceived. There are different aspects of testing the behavior of sensors and autonomous driving systems on a specific autonomous vehicle or a class of autonomous vehicles. AV testing can be conducted in the real world or based on simulated driving scenarios. An autonomous vehicle in a test (real or simulated) may be referred to as an autonomous vehicle.

[0003] One testing approach in the industry relies on “shadow mode” operation. Such tests attempt to use human driving as a baseline for evaluating automated driving decisions. An automated driving system (ADS) operates in shadow mode on inputs captured from a sensor-equipped but human-driven vehicle. The ADS processes the sensor inputs of the human-driven vehicle and makes driving decisions as if it were theoretically controlling that vehicle. However, those automated driving decisions are not actually implemented, but are simply recorded with the goal of comparing them to the actual driving behavior of a human. “Shadow miles” are accumulated in this way, with the goal typically of demonstrating that the ADS can perform more safely or more efficiently than a human. Summary of the Invention

[0004] Existing shadow mode testing has significant shortcomings. Shadow mode testing can flag scenarios where available test data indicates an ADS would perform differently from a human driver. However, this approach suffers from two fundamental flaws: First, shadow mode operations do not provide a reliable indication of how the ADS will actually perform in scenarios where it is controlling the vehicle; and second, to the extent that shadow mode operations can meaningfully demonstrate some differences between human and autonomous driving behavior, shadow mode provides little insight into the causes of those differences.

[0005] Expanding on the first point, at a given planning step where the ADS needs to make some immediate driving decisions, it will have access to some approximation of the autonomous vehicle's own state, as well as various observations of the vehicle's surroundings derived from the vehicle's onboard sensors. All the ADS can do is use that data to decide some immediate action it will take accordingly. However, in the next planning step, the ADS does not start from the state it would have reached if the human had done something different—it starts from a different state (in all likelihood) caused by the human's actions. In other words, as the planning scenario progresses, the ADS does not have to address any consequences of its decisions, as those autonomous driving decisions have no consequences in shadow mode. This is in contrast to actual autonomous driving scenarios where the ADS controls the vehicle—in such scenarios, at any given planning step, the ADS will evaluate the situation based on the currently available observations and decide on some optimal course of action, thereby utilizing some form of predictive logic to plan for the future. However, as the scenario evolves and new observations become available, the optimal course of action may deviate significantly over the course of multiple planning steps—not only through deviations from earlier predictions (which can never be perfect), but also through the ADS's own actuation systems (as they are implemented). In summary, existing shadow mode systems can at best provide some insight into the instantaneous reasoning of the ADS at a specific planning step in a scenario, but not into how it actually executes over the duration of the scenario.

[0006] For human drivers, on the other hand, the situation is exactly the opposite. The only insight into human driver behavior is the actual, final trajectory that the human driver decides to take during some driving scenario; but there is no structured insight into the reasons for those decisions, or into the long-term plans in the expert human driver's mind. For example, it is impossible to say with any certainty whether a specific event during a scenario caused the human driver to change his mind about some earlier plan; an experienced human driver for whom driving is "second nature" may not even be able to express such an event in a qualitative way.

[0007] Expanding on the second point above, human drivers and ADSs do not operate from the same perspective of the world. Human drivers observe the world from the driver's seat with their own senses, while ADSs observe the world via the vehicle's sensors. At best, this might indicate that a particular AV sensor, combined with a particular ADS, performs better than a human in similar situations. However, such techniques cannot provide more targeted insights.

[0008] A first aspect herein provides a computer-implemented method for evaluating the performance of a goal planner of an autonomous robot in a real or simulated scenario, the method comprising:

[0009] receiving evaluation data for evaluating performance of a goal planner in a scenario, the evaluation data generated by applying the goal planner at incremental planning steps to compute a series of autonomous plans responsive to a change in the scenario, the series of autonomous plans being implemented in the scenario to cause a change in the autonomous state;

[0010] The evaluation data includes: the autonomous plan computed by the goal planner at one of the planning steps, and the state of the scene at the instant of the scene;

[0011] The evaluation data is used to evaluate the goal planner in the following ways:

[0012] calculating a reference plan at the time instant based on a scene state, the scene state comprising: an autonomous state at the time instant resulting from implementing one or more previous autonomous plans in a series of autonomous plans calculated by the target planner; and

[0013] At least one evaluation score is calculated for comparing the autonomous plan with a reference plan.

[0014] One application of this method is to provide a systematic comparison between a target planner and a reference planner. The reference planner provides a target benchmark for evaluating the capabilities of the target planner. An important benefit is the ability to compare apples to apples. Both planners produce comparable plans, and the reference planner provides a more meaningful benchmark than human behavior. Another benefit is the ability to implement the method on simulated scenarios, which makes the method more scalable (although the possibility of implementing the target planner in the real world, either alternatively or in addition, is not excluded).

[0015] In embodiments, the autonomous plan may take the form of an instantaneous autonomous trajectory, which at the planning step may extend further in time than the next planning step and which may be only partially implemented in the scene until the next planning step before the instantaneous autonomous trajectory for the next planning step is calculated (i.e., only an initial portion of the instantaneous autonomous trajectory may be implemented until the next planning step).

[0016] In this context, the goal planner considers further into the future than the next planning step in order to make decisions about what action to take. The instantaneous autonomous trajectory is a form of "internal state" that provides insight into the goal planner's reasoning at a specific point in time (above and beyond the actual trajectory taken during the scenario).

[0017] The reference plan may take the form of an instantaneous reference trajectory, and the at least one evaluation score is used to compare the instantaneous autonomous trajectory with the instantaneous reference trajectory.

[0018] Such embodiments address the performance testing of an autonomous vehicle planner (goal planner) that takes the future into account in order to effectively plan trajectories for an autonomous robot (autonomous trajectories), for example, in the presence of static and / or dynamic obstacles. That is, through a series of incremental planning steps, the planner plans an "instantaneous" autonomous trajectory at each planning step that extends further into the future than the next planning step. This means that it is impossible to fully achieve any given instantaneous autonomous trajectory, as the scenario will most likely have evolved in a way that does not exactly match the planner's early expectations. In this sense, the instantaneous autonomous trajectory is a form of "internal state" of the planner. Over the course of a scenario, the instantaneous trajectories under different planning scenarios provide valuable insight into the reasons for the autonomous robot's behavior (whether in the real world or in simulation).

[0019] Unlike existing shadow mode systems, the autonomous robot (real or simulated) is actually under the control of the target planner being tested. That is, planning decisions are actually implemented as changes in the state of the autonomous robot (autonomy state) via the actuator system of the real-world robot, or in simulation (possibly using robot dynamics modeling for additional realism). For the reasons explained in the previous paragraph, it is impossible to fully implement any given instantaneous trajectory in such a way that only the initial part of the instantaneous trajectory will actually be implemented as a change in the autonomy state before the next plan is reached and the target planner replans according to the new autonomy state (e.g., taking into account any changes in the scene, i.e., any new obstacle data that has become available in the medium term).

[0020] The performance of the target planner is evaluated not relative to a human driver, but rather relative to a reference planner that plans instantaneous "reference" trajectories in a comparable manner. The reference planner can be applied at any given planning step and, importantly, applied with the same autonomous state as the target planner, the same level of information about obstacles, and the goal of achieving the same objectives. This, in turn, facilitates meaningful comparisons between instantaneous autonomous trajectories from the target planner and the instantaneous reference planner—an apples-to-apples comparison in a manner that provides more rigorous, quantitative insight not only into the differences in the decisions made by the two planners, but also into the reasons for those differences.

[0021] The actual trajectory of the autonomous robot (which can be real or simulated) taken over the duration of the scenario is determined not by any individual one of the instantaneous autonomous trajectories, but rather by repeated planning of the instantaneous autonomous trajectories at incremental planning steps. Because the goal planner plans further into the future than the next planning step, typically no instantaneous autonomous trajectory is fully realized—changes in the scenario (specifically, changes in available obstacle data) can cause instantaneous autonomous trajectories to deviate from each other at different planning steps.

[0022] The planning step may occur at a moment in the scene (ie, the scene state may be the scene state at the time of the above-mentioned planning step), and both the autonomous plan and the reference plan may be calculated based on the scene state at that moment.

[0023] In the case where the plan is an instantaneous trajectory, both the instantaneous autonomous trajectory and the instantaneous reference trajectory may extend beyond the next planning step.

[0024] Alternatively, the instant may be later than the planning step, so that the reference plan takes into account changes in the scene since the planning step was carried out in the scene state.

[0025] This technique can also be applied to training, where the goal is to make a planner learn to imitate a reference planner via imitation learning. In this context, the technique can be applied to generate high-level training data that can be used, for example, in dataset aggregation (DAgger) training. In this context, the reference planner acts as a queryable expert that can be used to augment the training set with the problem (scene state) actually encountered by the target planner. In this imitation learning context, it is important to follow the autonomous plan from the target planner to travel the scene.

[0026] In the presence of obstacles, the planner will typically reason about the best or acceptable trajectory based on some level of predicted obstacle behavior. In a more "modular" autonomous driving architecture, the predicted obstacle trajectory can be provided by different prediction systems.

[0027] In other architectures, planning and prediction can be more tightly integrated. At one extreme are end-to-end driving models, where the entire AV stack is implemented as an end-to-end trainable function approximator that receives raw sensor inputs and computes a trajectory based on them.

[0028] In general, the term planner accommodates any level of modularity or integration—in the context of this aspect, a planner is any component or system that can plan an instantaneous trajectory (i.e., a trajectory that extends further into the future than the next planning step, without expecting such instantaneous trajectory to be fully realized); or, in other words, any component or system that autonomously decides what actions to take between the current planning step and the next planning step by reasoning further into the future than the next planning step.

[0029] Where the above planning step occurs at the above time instant, the state of the scene at that time instant of the planning step may comprise the current set of observations used by the target planner to compute the autonomous plan, or the state of the scene at that time instant of the planning step may be derived from the current set of observations used by the target planner to compute the autonomous plan, and the reference planner may use the same set of observations to compute the reference plan.

[0030] That is, the scene state may include the available obstacle dataset (observed and / or predicted).

[0031] The available obstacle dataset constitutes the shared input common to both the target planner and the reference planner. These are the inputs upon which the decision-making / reasoning processes of both planners are based, and they can take different forms depending on the context. For example, with a modular architecture that separates planning from prediction, both the target planner and the reference planner typically prefer (but are not required to) plan based on a common set of predicted obstacle trajectories (in which case, they form part of the set of available obstacle data shared between the planners). By essentially freezing the prediction "dimension" between the planners (i.e., removing it as a degree of freedom), it becomes easier to attribute differences between the target and reference trajectories to high-level decisions made within the planners, as opposed to differences in the predictions. With a planner that integrates predictions, the shared inputs can, in extreme cases, be only "low-level" inputs, such as (real or simulated) perceptual outputs or even (real or simulated) raw sensor data, with each planner relying on its own form of prediction logic—the reference trajectory still provides a useful baseline in this case.

[0032] For example, the observations can be noisy observations subject to real or modeled observation errors, and in this case both the target planner and the reference planner are subject to the same observation errors. Alternatively, the scenario can be simulated, and the observations can be simulated observations without any observation errors, so that neither the target nor the reference planner are subject to observation errors.

[0033] The scene state may include a current set of observations and / or one or more current obstacle predictions derived from the current set of observations, in which case the target planner and the reference planner may use the same observations and / or the same obstacle predictions to compute the autonomous plan and the reference plan, respectively.

[0034] As mentioned above, the evaluation data can be captured by applying the target planner to a real or simulated scenario. Therefore, the available obstacle data to which the reference planner is also applied can be real or simulated.

[0035] The ability to implement this approach in simulation eliminates or reduces the need for actual driving test miles. One estimate is that in order to match a human driver in terms of safety, an ADS should be able to make and implement a prediction with an error rate of no more than 10 7 At this level, validating performance based solely on actual driving test miles would require many times this value. The ability to provide meaningful benchmarking through a test regime implemented at least partially in simulation is a significant benefit.

[0036] The aforementioned planning step can occur at the time of the aforementioned scenario (i.e., the scenario state can be the scenario state at the time of the aforementioned planning step), and both the autonomous plan and the reference plan can be calculated based on the scenario state at that time. In other words, the reference planner can be applied at the same time of the scenario as the target planner.

[0037] In the case where the plan is an instantaneous trajectory, both the instantaneous autonomous trajectory and the instantaneous reference trajectory may extend beyond the next planning step.

[0038] Alternatively, the instant may be later than the planning step, so that the reference plan takes into account changes in the scene since the planning step was carried out in the scene state.

[0039] There are various cases where the reference plan outperforms the autonomous plan from the goal planner. This difference can be a useful way to evaluate the performance of the goal planner in a meaningful way.

[0040] For example, the goal planner may be constrained to operate in real time, and the reference plan may not be computed in real time.

[0041] Alternatively or additionally, more computing resources and / or memory resources are allocated to computing the reference plan than are granted to the target planner for computing the autonomous plan.

[0042] As noted, the reference plan may be computed by applying the reference planner to the scene state at that time. Alternatively, however, that time may be later than the planning step, such that the reference plan takes into account changes to the scene since the planning step was performed in the scene state.

[0043] A reference plan is computed by applying the reference planner to the state of the scene at that time instant.

[0044] For example, as in the embodiments described below, the reference planner may be a planner based on constraint optimization.

[0045] Alternatively, a goal planner can be used to compute a reference plan, but:

[0046] have a larger allocation of computational or memory resources than those used to compute the autonomous plan,

[0047] In the case of real-time calculation and autonomous planning, it is not constrained to real-time operation.

[0048] The time is later than the planning step, and / or

[0049] In cases where autonomous planning is affected by modeled perceptual errors, there are no modeled perceptual errors.

[0050] The evaluation data may include instantaneous autonomous plans calculated at a plurality of planning steps in the incremental planning steps, and a plurality of scene states each associated with one of the plurality of planning steps. A reference plan may be calculated for each of the plurality of planning steps based on the associated scene state at the planning step, and at least one evaluation score may be calculated for each of the plurality of planning steps for comparing the autonomous plan for the planning step to the reference plan.

[0051] For each of the plurality of planning steps, the associated scene state may be the scene state at the planning step, or the scene state a set amount of time after the planning step.

[0052] The at least one evaluation score may include: a separate evaluation score calculated for the target plan and a separate evaluation score calculated for the reference plan; the separate evaluation scores being calculated with respect to the common metric.

[0053] That is, different individual metrics may be calculated for the autonomous plan and the reference plan, but they are comparable with respect to the same metric.

[0054] Additionally or alternatively, the at least one evaluation score may include a comparison score that quantifies a difference between the target plan and a reference plan relative to a comparison metric.

[0055] Common or comparative metrics can quantify safety, comfort, or progress relative to defined goals.

[0056] Target plans and reference plans can be calculated with a common goal of achieving defined objectives.

[0057] For example, the target may be defined as a static or moving target location to be reached.

[0058] The evaluation score may be a numerical evaluation score for quantifying the difference between the autonomous plan and the reference plan.

[0059] The scenario can be a real or simulated driving scenario, in which case the autonomous state is the autonomous vehicle state.

[0060] The evaluation score may be calculated with respect to a metric that quantifies the different levels of compliance with predetermined driving rules between the autonomous plan and the reference plan.

[0061] Multiple evaluation metrics may be used to calculate multiple evaluation scores, each evaluation metric encoding a different predetermined driving rule.

[0062] The method may include the step of outputting the evaluation score on a user interface.

[0063] The evaluation score can be output along with a visual representation of the scene, and visual representations of the autonomous plan and the reference plan.

[0064] The at least one assessment score may be used to compare:

[0065] The autonomous trajectory and the reference trajectory progress towards the defined goal,

[0066] Deviation of autonomous trajectory and reference trajectory from the reference path,

[0067] Acceleration along the autonomous trajectory and the reference trajectory,

[0068] The jerk, i.e., the rate of change of acceleration, along the autonomous and reference trajectories.

[0069] Different evaluation scores may be calculated for the autonomous trajectory and the reference trajectory respectively for the comparison.

[0070] This technique can also be applied to training, where the goal is to make a planner learn to imitate a reference planner via imitation learning. In this context, the technique can be applied to generate high-level training data that can be used, for example, in dataset aggregation (DAgger) training. In this context, the reference planner acts as a queryable expert that can be used to augment the training set with the problem (scene state) actually encountered by the target planner. In this imitation learning context, it is important to follow the autonomous plan from the target planner to travel the scene.

[0071] The target planner may include one or more trainable models, and the evaluation data may provide one or more training examples for training the target planner to mimic a reference planner. The evaluation score may be a component of a cost function (or objective / error function) that measures the deviation between the autonomous plan computed by the target planner and a reference plan computed by the reference planner, with the goal of performing training with the goal of minimizing those deviations.

[0072] The goal planner may have been pre-trained on earlier training examples, and the training examples provided by the evaluation data are new training examples used to retrain the goal planner.

[0073] Another aspect herein provides a computer-implemented method of training a goal planner to emulate an expert planner for an autonomous robot, the method comprising:

[0074] applying the goal planner to the simulated scenario at incremental planning steps to compute a series of autonomous plans responsive to changes to the scenario, the series of autonomous plans being implemented in the scenario to cause changes in the autonomous state;

[0075] computing an expert plan for at least one of the planning steps by applying an expert planner to the scene state at the planning step, and

[0076] One or more models of a target planner are trained to mimic the expert planner using scene states and expert plans as training examples, the training being performed with the objective of minimizing a deviation between an autonomous plan computed by the target planner when applied to the scene state and an expert plan computed by the expert planner when applied to the scene state, the scene state comprising: an autonomous state at a planning step resulting from implementing one or more previous autonomous plans in a series of autonomous plans computed by the target planner.

[0077] In an embodiment, the autonomous plan may take the form of an instantaneous autonomous trajectory calculated by the target planner, wherein the instantaneous autonomous trajectory calculated at at least one planning step extends further in time than a next planning step, wherein the expert plan may also take the form of an instantaneous expert trajectory that extends further in time than a next planning step.

[0078] The expert planner can be a planner based on constrained optimization.

[0079] In the context of training, all of the above descriptions related to the reference planner also apply to the expert planner.

[0080] In the above aspects and embodiments, the reference planner may be said to be implemented in a shadow mode, in that it does not make decisions in the scene—instead the scene states to which it is applied are (partially) determined by the decisions of the target planner.

[0081] Further aspects herein allow for the comparison of similar structures, but with a reference planner implemented "in-the-loop", making its own decisions in a second instance of a simulated scenario.

[0082] Another aspect herein provides a computer-implemented method for evaluating the performance of a goal planner of an autonomous robot in a simulated scenario, the method comprising:

[0083] receiving evaluation data for evaluating performance of a goal planner, the evaluation data generated by applying the goal planner in a first instance of a scenario according to an initial scene state at a first incremental planning step to compute a series of autonomous plans responsive to changes in the first instance of the scenario, implementing the first series of autonomous plans in the first instance of the scenario to cause changes in the first autonomous state, wherein an actual autonomous trajectory is defined by the changes in the first autonomous state over a duration of the first instance of the scenario;

[0084] The evaluation data includes the initial scene state and actual autonomous trajectory;

[0085] The evaluation data is used to evaluate the goal planner in the following ways:

[0086] applying the goal planner in a second instance of the scenario based on the same initial scene state at a second incremental planning step to compute a series of reference plans responsive to changes in the second instance of the scenario, the series of reference plans being implemented in the second instance of the scenario to cause changes in the second autonomy state, wherein the actual reference trajectory is defined by the changes in the second autonomy state over the duration of the second instance of the scenario; and

[0087] At least one evaluation score is calculated for comparing the actual autonomous trajectory to the actual reference trajectory.

[0088] The autonomous plan may take the form of an instantaneous autonomous trajectory, wherein the instantaneous autonomous trajectory of at least one planning step in the first series of planning steps may extend further in time than the next planning step, and before the instantaneous autonomous trajectory of the next planning step is planned, the instantaneous autonomous trajectory of at least one planning step in the first series of planning steps may be only partially implemented in a first instance of the scenario until the next planning step, and the actual autonomous trajectory is not defined by any individual one of the instantaneous self-trajectories, but rather by changes in the first autonomous state over the duration of the scenario. The reference plan may take the form of an instantaneous reference trajectory, wherein the instantaneous reference trajectory of at least one planning step in the second series of planning steps may extend further in time than the next planning step, and before the instantaneous reference trajectory of the next planning step is calculated, the instantaneous reference trajectory of at least one planning step in the second series of planning steps may be only partially implemented until the next planning step, and the actual reference trajectory is not defined by any individual one of the instantaneous reference trajectories, but rather by changes in the second autonomous state over the duration of the scenario.

[0089] The method may include the steps of receiving an indication of a starting state at a user interface, and initiating at least a second instance of the scenario in response to the indication.

[0090] The actual trajectories can be scored / compared in exactly the same way as the planned / instantaneous trajectories in the previously described aspects / embodiments. All the above descriptions apply.

[0091] Another aspect herein provides a computer-implemented method for evaluating the performance of a goal planner of an autonomous robot in a real or simulated scenario having one or more obstacles, the method comprising:

[0092] receiving evaluation data for evaluating performance of a goal planner in a scenario, the evaluation data generated by applying the goal planner at incremental planning steps of the scenario;

[0093] Wherein, at each planning step, a goal planner plans an instantaneous autonomous trajectory with the goal of achieving a defined goal based on the current autonomous state and the currently available obstacle dataset;

[0094] wherein the available obstacle dataset is updated between consecutive planning steps, and an instantaneous autonomous trajectory planned at an early one of those planning steps extends further in time than a late one of those planning steps such that only an initial portion of the early instantaneous autonomous trajectory is substantially implemented as a change in the autonomous state before planning the late instantaneous autonomous trajectory based on the changed autonomous state and the updated available obstacle dataset;

[0095] The evaluation data includes: an instantaneous autonomous trajectory of at least one of the planning steps, and an autonomous state and an available obstacle dataset of the planning step on which the instantaneous autonomous trajectory is based, the method further comprising:

[0096] applying a reference planner to plan a corresponding instantaneous reference trajectory for that planning step based on the same autonomy state and the same available obstacle dataset with the goal of achieving the same goal; and

[0097] At least one trajectory evaluation score is calculated for comparing the instantaneous autonomous trajectory to the corresponding instantaneous reference trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Example embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which:

[0099] Figure 1 An example autonomous vehicle (AV) runtime stack is shown;

[0100] Figure 1A An example of a planned trajectory with spatial and motion components is shown;

[0101] Figure 2 A schematic block diagram of a computer system for running a simulation scenario and evaluating a trajectory planned in the simulation scenario is shown;

[0102] Figure 2A An embodiment of a simulator using a perceptual error model is shown;

[0103] Figure 3 A schematic block diagram of a goal planner operating in-the-loop is shown;

[0104] Figure 3A is a schematic block diagram of a reference planner operating in shadow mode;

[0105] Figure 3Bshows how trajectories can be scored for a reference planner operating in shadow mode;

[0106] Figure 4 shows a schematic block diagram of a reference planner operating in a loop test;

[0107] Figure 5 schematically illustrates instantaneous reference trajectories planned by the reference planner in shadow mode at different moments;

[0108] Figure 6A and Figure 6B An example driving scenario is schematically depicted;

[0109] Figure 7 showing a time plot of metric-based scores assigned to instantaneous autonomous trajectories and reference trajectories over time intervals;

[0110] Figure 8 Schematic showing how actual trajectories are generated by both the target planner and the reference planner operating in the loop test;

[0111] Figure 9 An example of a user interface for comparing metric-based scores for different trajectories is shown;

[0112] Figure 10 A reference planner and a neural network trained to imitate the reference planner through imitation learning are shown;

[0113] Figure 11 A schematic block diagram of an imitation learning training scheme is shown;

[0114] Figure 12 An example network architecture that can be trained to mimic a reference planner is shown;

[0115] Figure 13A shows the mathematical transformations to and from a reference frame based on a reference path; and

[0116] Figure 13B The representation from the global reference frame to the reference path based reference frame is depicted geometrically. DETAILED DESCRIPTION

[0117] Sample AV Stack

[0118] Figure 1 A highly schematic block diagram of a runtime stack 100 for an autonomous vehicle (AV), also referred to herein as an ego vehicle (EV), is shown. The runtime stack 100 is shown to include a perception system 102, a prediction system 104, a planner 106, and a controller 108.

[0119] In the real-world context, the perception system 102 will receive sensor inputs from the AV's onboard sensor system 110 and use those sensor inputs to detect external bodies and measure their physical states, such as their position, velocity, acceleration, etc. The onboard sensor system 110 can take different forms, but typically includes a variety of sensors (such as image capture devices (cameras / optical sensors), lidar and / or radar units, satellite positioning sensors (GPS, etc.), motion sensors (accelerometers, gyroscopes, etc.), etc.), which collectively provide rich sensor data from which detailed information about the surrounding environment and the state of the AV and any external actors (vehicles, pedestrians, cyclists, etc.) within that environment can be extracted. The sensor inputs typically include sensor data from multiple sensor modalities, such as stereo images from one or more stereo optical sensors, lidar, radar, etc.

[0120] The perception system 102 includes multiple perception components that cooperate to interpret sensor input and thereby provide perception output to the prediction system 104. External agents may be probabilistically detected and represented in a manner that reflects the level of uncertainty in their perception within the perception system 102.

[0121] The perception output from perception system 102 is used by prediction system 104 to predict the future behavior of external actors (agents), such as other vehicles near the AV. Other agents are dynamic obstacles perceived by the EV. The output of prediction system 104 can, for example, take the form of a set of predicted obstacle trajectories.

[0122] The predictions calculated by the prediction system 104 are provided to the planner 106, which uses the predictions to make automated driving decisions to be performed by the AV in a given driving scenario. A scenario is represented as a set of scene description parameters used by the planner 106. A typical scenario will define a drivable area and will also capture any static obstacles within the drivable area as well as the predicted movements of any external bodies.

[0123] The core functionality of the planner 106 is the planning of the AV trajectory (autonomous trajectory) taking into account any static and / or dynamic obstacles (including any predicted motion of the latter). This can be referred to as trajectory planning. The trajectory is planned so as to execute the desired goal within the scenario. The goal can be, for example, to enter a roundabout and leave the roundabout at the desired exit; to overtake the vehicle in front; or to stay in the current lane at a target speed (lane tracking). The goal can be determined, for example, by an autonomous driving route planner (not shown). In the following examples, the goal is defined by a fixed or moving target position, and the planner 106 plans a trajectory from the current state of the EV (autonomous state) to the target position. For example, this can be a fixed target position associated with a specific intersection or roundabout exit, or a moving target position that remains in front of the vehicle in front in an overtaking situation. The trajectory in this article has both a spatial component and a motion component, defining not only the spatial path planned for the autonomous vehicle, but also the planned motion profile along this path.

[0124] The planner 106 is required to navigate safely in the presence of any static or dynamic obstacles (such as other vehicles, bicycles, pedestrians, animals, etc.).

[0125] Figure 1A The operation of the planner 106 in an example scenario is schematically illustrated. It should be understood that Figure 1A is highly schematic, with certain aspects highlighted for illustrative purposes, meaning that the depicted scenarios do not necessarily correspond to real driving scenarios.

[0126] The planner 106 will typically be employed in a series of planning steps over the course of a scenario. At each planning step, the planner 106 may obtain a certain level of information about the vehicle's surroundings from the perception and prediction systems 102, 104. This includes, among other things, information about the predicted behavior of any obstacles near the vehicle. At each planning step, the planner 106 plans an "instantaneous" trajectory (the autonomous trajectory) for the autonomous vehicle based on the current autonomous state, with the goal of achieving some defined goal, such as reaching the (current) target location. This trajectory is instantaneous in the sense that it is the trajectory of the autonomous vehicle that the planner considers to be substantially optimal (or at least feasible) at a given planning step, given the level of information available to the planner from perception and prediction at that instant. However, a typical planner 106 will plan further ahead in time than the next planning step, as reasoning further into the future will generally improve the quality of the planner's decisions. The information available at that planning step is a "snapshot," which takes into account the observations up to that point and the prediction system's "best guess" about how the scenario will develop based on those observations. Therefore, in all likelihood, only the initial portion of the instantaneous autonomous trajectory will actually be achieved in any given planning step, particularly since the predictions it is based on will never be perfect in practice. Once an instantaneous autonomous trajectory has been planned, the autonomous vehicle will typically follow that trajectory until the next planning step, at which point the process will be repeated to replan the autonomous trajectory based on the new autonomous state, taking into account any relevant changes in the outputs of perception and prediction systems 104, 106. By that time, it is likely that observations since the last planning step have substantially changed the predictions from prediction system 106, potentially leading to substantial changes in the output of planner 106.

[0127] Figure 1A Two consecutive planning steps for an example scenario are shown, with time indices t and t+1, respectively. At time t, the autonomous vehicle is shown in a current autonomous state 120. Up until time t, an external agent has been observed as a time series 122A of perceived agent states. Prediction system 106 uses those observations 122A to predict the agent's future trajectory 124A from time t, which planner 106 in turn uses to plan an autonomous trajectory 128A from current autonomous state 120 to some defined target location 130. Figure 1A The trajectories in are represented as sequences of states (black circles) at evenly spaced intervals, such that relatively close intervals between states represent relatively low speeds along the trajectory in question. According to the agent's predicted trajectory 124A at time t+1, the agent will obstruct the autonomous vehicle's progress toward the target location 130. To ensure safe operation of the autonomous vehicle, the planner responds by reducing the autonomous vehicle's speed along the planned trajectory 128A to allow the agent to pass. However, those events are predicted to occur sometime after the next planning step at t+1.

[0128] By the next planning step at time t+1, a new observed state 122B of the agent has become available, and this deviates significantly from the prediction at the previous planning step. This, in turn, results in a very different predicted trajectory 124B for the agent, such that the agent is no longer predicted to be obstructing the autonomous vehicle from traveling toward target position 130 (which is stationary in this example). Since the last planning step at time t, the autonomous vehicle has already traveled the initial portion of the most recently planned autonomous trajectory 128A (from time t to time t+1), and thus now arrives at new state 126, at least approximately, along the trajectory 128A planned at time t. Now, at time t+1, presented with a very different predicted agent trajectory 124B, planner 106 plans a new trajectory 128B at time t+1 based on the new autonomous state 126 at time t+1, which travels faster toward target position 130.

[0129] In the above sense the planning trajectories 128A, 128B are instantaneous trajectories, representing the planner's current optimal plan given the level of information available to the planner at times t and t+1, respectively.

[0130] return Figure 1 Within the stack 100, the controller 108 implements the decisions made by the planner 106. The controller 108 does this by providing appropriate control signals to the AV's onboard participant systems 112. At any given planning step (having planned an instantaneous autonomous trajectory), the planner 106 will provide the controller 108 with enough data about the planned trajectory to allow the controller 108 to implement the initial portion of the planned trajectory until the next planning step. For example, the planner 106 may plan the instantaneous autonomous trajectory as a series of discrete autonomous states at incrementally future times, but actually provide only the first (or first few) of the planned autonomous states to the controller 108 for implementation.

[0131] In a physical AV, participant systems 112 include the engine, brakes, etc., which can be controlled to affect the vehicle's movement and other physical changes in the real-world autonomous state.

[0132] The control signals from the controller 108 are typically low-level instructions that may be frequently updated for the participant systems 112. For example, the control signals may specify changes in speed, acceleration, jerk, etc., and / or changes in steering angle.

[0133] Practice Test - Overview

[0134] The embodiments herein have useful applications in simulation-based testing. Referring to stack 100 as an example, to test the performance of all or part of stack 100 through simulation, the stack is exposed to simulated driving scenarios. The following examples consider testing planner 106 alone, but also in conjunction with one or more other subsystems or components of stack 100.

[0135] In the simulated driving scenario, the autonomous agent implements decisions made by the planner 106 based on simulated inputs derived from the simulated scenario as it progresses. Typically, the autonomous agent needs to navigate within a static drivable area (e.g., a specific static road layout) in the presence of one or more simulated obstacles of the kind that a real vehicle would need to interact safely. Dynamic obstacles (e.g., other vehicles, pedestrians, cyclists, animals, etc.) can be represented as dynamic agents in the simulation.

[0136] Simulated inputs are processed in exactly the same way as corresponding physical inputs, ultimately forming the basis for autonomous driving decisions made by the planner during the simulation. These decisions, in turn, cause the autonomous agent to make these decisions, simulating the behavior of a physical autonomous vehicle in those circumstances. In the simulation, these decisions ultimately materialize as changes in the simulated autonomous state. This two-way interaction exists between planner 106 and the simulator, where decisions made by planner 106 influence the simulation, and changes in the simulation influence subsequent planning decisions. The results can be recorded and analyzed with respect to safety and / or other performance criteria.

[0137] A "slice" in this document means a stack or subset of components of a stack (substack) under test. The way a stack is used as a "slice" for testing dictates the form of simulated perceptual input that needs to be provided to the slice (stack / substack).

[0138] Referring to stack 100 as an example, if the full stack (including the entire perception system 102) is to be tested, then simulated inputs would take the form of simulated sensor inputs provided to the lowest-level components of perception system 102. Perception system 102 would then interpret the simulated sensor inputs as if they were real sensor data in order to provide perception outputs (simulated in the sense that the perception outputs are derived from the interpretation of the simulated sensor data). This can be referred to as "full" simulation and typically involves generating sufficiently realistic simulated sensor inputs (such as realistic image data and / or equally realistic simulated lidar / radar data, etc.) that can in turn be fed to perception system 102 and processed in exactly the same manner as real sensor data. The outputs produced by the perception system would, in turn, feed higher-level prediction and planning systems, thereby testing the responses of those components to the simulated sensor inputs.

[0139] Alternatively, in what may be referred to herein as a “headless” simulation, some or all of the perception system 102 is bypassed, and simulated perception outputs are computed directly from the simulation. In a real-world context, equivalent perception outputs would be derived by one or more perception components of the perception system 102 interpreting lower-level sensor inputs from sensors. In a headless simulation, those perception components are not applied—instead, their perception outputs are computed directly from the simulated ground truth without having to simulate the inputs to those perception components. For example, for a bounding box detector, rather than generating simulated sensor data and applying a bounding box detector to the simulated sensor data, simulated bounding box detection outputs are computed directly from the simulation.

[0140] As a general rule, the further you slice the stack "down", the more complex the simulated perceptual inputs required (ranging from full sensor modeling at one extreme to simple simulated fused position / orientation measurements, etc., at the other extreme, which can be directly derived using efficient techniques like ray tracing).

[0141] In between those two extremes, there is a range of many different levels of input slicing, for example, testing only a subset of the perception system, such as "late" perception components, i.e., components (such as filters or fusion components) that operate on the outputs from lower-level perception components (such as object detectors, bounding box detectors, motion detectors, etc.).

[0142] Turning to the output of stack 100, there are different ways to implement the decisions of planner 106 in testing. In a "planning-level" simulation, the autonomous agent can be assumed to follow the portion of the most recently planned trajectory exactly from the current planning step to the next. This is a simpler form of simulation that does not require any implementation of controller 108 during simulation. More complex simulations recognize that, in reality, any number of physical conditions (e.g., due to wheel slip, delayed or imperfect responses of participant systems, or inaccuracies in measurements of the vehicle's own state 112) may cause a real autonomous vehicle to deviate slightly from the planned trajectory. These factors can be accommodated by appropriately modeling the autonomous vehicle's dynamics. In that case, controller 108 is used in simulation, just as it would in real life, and a suitable autonomous dynamics model (instead of participant systems 112) is used to convert control signals into changes in autonomous state in order to more realistically simulate the autonomous vehicle's response to the control signals. In this case, as in real life, the portion of the planned trajectory from the current planning step to the next planning step can only be roughly implemented as a change in autonomous state.

[0143] Example test pipeline

[0144] Figure 2A schematic block diagram of a test pipeline 200 is shown. The test pipeline is highly flexible and can adapt to many forms of AV stacks, operating at any level of autonomous driving. As indicated, the term autonomous driving herein encompasses any level of fully or partially autonomous driving, from Level 1 (driver assistance) to Level 5 (fully autonomous driving).

[0145] The test pipeline 200 is shown as including a simulator 202 and a test oracle 252. The simulator 202 runs a simulation for the purpose of testing all or part of the AV runtime stack.

[0146] For example only, the description of the test pipeline 200 is shown in FIG. Figure 1 As discussed, it is possible to test only a sub-stack of the runtime stack, but for simplicity, the following description always refers to the AV stack 100; note that the actual test content may only be Figure 1 A subset of the AV stack 100, depending on how the AV stack 100 is sliced ​​for testing. Figure 2 In FIG, reference numeral 100 may therefore denote the full AV stack or just a sub-stack, depending on the context.

[0147] Figure 2 The prediction system, planning system, and control system 104, 106, and 108 within the AV stack 100 under test are shown, with simulated sensory inputs 203 fed into the stack 100 from the simulator 202. However, this does not necessarily mean that the prediction system 104 directly operates on those simulated sensory inputs 203 (although this is a feasible slice, in which case the simulated sensory inputs 203 would formally correspond to the final output of the sensory system 102). For example, in Figure 2A In the example depicted in and described in detail below, the AV stack 100 is sliced ​​such that (only) a subset of the late perception components 102B are included in the sub-stack 100S under test, along with the prediction, planning, and control components 104, 106, and 108 (in which case the simulated perception inputs 203 would correspond to the outputs of the remaining early perception components that were not implemented with the stack 100 during testing). In cases where the full perception system 102 is implemented in the stack under test (or at least includes one or more lower-level perception components that operate on raw sensor data), the simulated perception inputs 203 would include simulated sensor data.

[0148] Simulated sensory input 203 serves as the basis for predictions and ultimately decisions made by planner 108. Controller 108, in turn, implements the planner's decisions by outputting control signals 109. In a real-world setting, these control signals would drive the AV's physical actor systems 112. The format and content of the control signals generated during testing are identical to their real-world counterparts. However, within test pipeline 200, these control signals 109 instead drive autonomous dynamics model 204 to simulate the motion of the autonomous agent within simulator 202.

[0149] To the extent that an external agent exhibits autonomous driving behavior / decision-making within simulator 202, some form of agent decision logic 210 is implemented to execute those decisions and, accordingly, drive the external agent's dynamics within simulator 202. Agent decision logic 210 can be comparable in complexity to the autonomy stack 100 itself, or it can have more limited decision-making capabilities. The goal is to provide sufficiently realistic external agent behavior within simulator 202 to effectively test the decision-making capabilities of the autonomy stack 100. In some cases, this may not require any agent decision-making logic 210 at all (open-loop simulation), while in other cases, relatively limited agent logic 210 (such as basic adaptive cruise control (ACC)) can provide useful testing. Similar to the autonomy stack 100, any agent decision logic 210 is driven by output from simulator 202, which is in turn used to derive input to agent dynamics model 206, which serves as the basis for the simulation of the agent's behavior.

[0150] A simulation of a driving scenario is run according to a scenario description 201 having both a static layer 201 a and a dynamic layer 201 b .

[0151] The static layer 201a defines the static elements of the scene, which will typically include a static road layout.

[0152] The dynamic layer 201b defines dynamic information about external entities within the scene (e.g., other vehicles, pedestrians, bicycles, etc.). The scope of the dynamic information provided can vary. For example, for each external entity, the dynamic layer 201b may include a spatial path for the entity to follow and either or both motion data and behavioral data associated with the path.

[0153] In a simple open-loop simulation, external actors simply follow spatial paths and motion data defined in a non-reactive (i.e., not reacting to the autonomous agents within the simulation) dynamic layer. Such an open-loop simulation can be implemented without any agent decision logic 210.

[0154] However, in a "closed-loop" simulation, the dynamic layer 201b instead defines at least one behavior (e.g., an ACC behavior) to be followed along a static path. In this case, the agent decision logic 210 implements the behavior within the simulation in a reactive manner (i.e., reacting to the autonomous agent and / or other external agents). Motion data may still be associated with the static path, but in this case is less standardized and may, for example, be used as a target along the path. For example, for an ACC behavior, a target speed may be set along the path that the agent will seek to match, but the agent decision logic 110 may be allowed to reduce the speed of the external agent below the target at any point along the path in order to maintain the target vehicle distance from the vehicle ahead.

[0155] For a given simulation, the output of simulator 202 includes an autonomous trace 212a for the autonomous agent and one or more agent traces 212b for one or more external agents (traces 212).

[0156] A trajectory is a complete history of an agent's behavior within a simulation that has both spatial and motion components. For example, a trajectory can take the form of a spatial path with motion data associated with points along the path, such as velocity, acceleration, jerk (rate of change of acceleration), snap (rate of change of jerk), etc.

[0157] Additional information is also provided to supplement and provide context to the traces 212. This additional information is referred to as "environmental" data 214, which may have both a static component (such as the road layout) and a dynamic component (such as the extent to which weather conditions change during the simulation).

[0158] To some extent, environment data 214 may be "pass-through" in that it is directly defined by scenario description 201 and is not affected by the results of the simulation. For example, environment data 214 may include a static road layout directly from scenario description 201. Typically, however, environment data 214 will include at least some elements derived within simulator 202. This may include, for example, simulated weather data, where simulator 202 is free to change weather conditions as the simulation proceeds. In this case, the weather data may be time-dependent, and this time dependence will be reflected in environment data 214.

[0159] The test oracle 252 receives the trace 212 and the environmental data 214 and scores these outputs against a predefined numerical performance metric 254. The performance metric 254 encodes what is referred to herein as a "Digital Highway Code" (DHC) or digital driving rules. Some examples of suitable performance metrics are given below.

[0160] The scoring is time-based: for each performance metric, the test oracle 252 tracks how the value of that metric (the score) changes over time as the simulation progresses. The test oracle 252 provides output 256 that includes a score-versus-time curve for each performance metric.

[0161] Metrics 254 provide useful information to experts, and these scores can be used to identify and mitigate performance issues within the test stack 100 .

[0162] Perceptual Error Model

[0163] Figure 2A A specific form of slice is shown and reference numerals 100 and 100S are used to denote the full stack and the sub-stack, respectively. The sub-stack 100S will be subjected to Figure 2 Tests within the test pipeline 200.

[0164] A plurality of "late" perceptual components 102B form part of the sub-stack 100S to be tested and are applied to simulated perceptual inputs 203 during testing. The late perceptual components 102B may, for example, comprise filtering or other fusion components that fuse perceptual inputs from a plurality of early perceptual components.

[0165] In the full stack 100, the late perception component 102B will receive actual perception input 213 from the early perception component 102A. For example, the early perception component 102A may include one or more 2D or 3D bounding box detectors, in which case the simulated perception input provided to the late perception component may include simulated 2D or 3D bounding box detections derived in simulation via ray tracing. The early perception component 102A will typically include components that operate directly on sensor data.

[0166] With this slicing, the simulated perceptual input 203 will formally correspond to the actual perceptual input 213 typically provided by the early perceptual component 102A. However, the early perceptual component 102A is not used as part of the test, but rather is used to train one or more perceptual error models 208 that can be used to introduce realistic errors into the simulated perceptual input 203 fed to the late perceptual component 102B of the sub-stack 100 under test in a statistically rigorous manner.

[0167] Such perceptual error models may be referred to as Perception Statistical Performance Models (PSPMs) or by a synonym “PRISMs”. Further details of the principles of PSPMs and suitable techniques for building and training them may be found in European patent application No. 20168311.7, the entire contents of which are incorporated herein by reference. The idea behind PSPMs is to efficiently introduce realistic errors into the simulated perceptual inputs provided to the sub-stack 102B (i.e., reflecting the type of errors that would be expected when applying the early perceptual components 102A in the real world). In the simulation context, “perfect” ground truth perceptual inputs 203G are provided by the simulator, but these ground truth perceptual inputs 203G are used to derive more realistic perceptual inputs 203 with realistic errors introduced by the perceptual error model 208.

[0168] As described in the aforementioned references, the PSPM can depend on one or more variables representing physical conditions (“confounding factors”), allowing for the introduction of different levels of error reflecting different possible real-world conditions. Thus, the simulator 202 can simulate different physical conditions (e.g., different weather conditions) by simply changing the value of the weather confounding factor, which will in turn change how the perception error is introduced.

[0169] The late perception component 102b within the sub-stack 100S processes the simulated perception input 203 in exactly the same way as it would process the real-world perception input 213 within the full stack 100, and the output of the late perception component 102b in turn drives prediction, planning, and control.

[0170] according to Figure 1A The general principle shown in Figure 3 A block diagram showing how the planner 106 is applied iteratively during testing is shown. In this example, simulated perception output and autonomous dynamic modeling are used. However, as noted, the present technology can be applied using simpler forms of simulation.

[0171] At a given planning step, (directly using Figure 2ASimulated perception outputs are derived from the simulator (either by modeling perception errors in the ego vehicle or by applying perception system 102 to simulated sensor data). These perception outputs encode the current state of perception of the ego vehicle and the current states of any other agents in the scene. Prediction system 104 uses the available perception outputs to generate predictions of the behavior of external agents in the scene in the form of one or more predicted agent trajectories. Additionally, predictions may utilize earlier perception outputs. Planner 106 uses the outputs of perception system 102 to determine a new instantaneous trajectory for the ego vehicle in its current state, taking into account the states and predictions of other agents in the scene. Control system 108 determines the control signals to be passed to simulator 202. As described above, simulator 202 includes an ego vehicle dynamics model 204, which simulates the motion of the ego vehicle based on the received control signals. Agent decision logic 210 uses the outputs of the simulator to derive inputs to agent dynamics model 206, which determines the motion of the agents. This, in turn, allows new perception outputs to be simulated for the next planning step, and these steps are iteratively executed with new instantaneous trajectories planned for each updated motion of the ego vehicle and the agents in the scene. This continues until some termination condition is reached, such as the autonomous agent reaching a defined target position, a collision occurring, or a time limit being reached.

[0172] Test Metrics

[0173] The performance metrics 254 may be based on different factors, such as distance, speed, etc. In the described system, these factors may reflect an applicable set of road rules, such as the Highway Code applicable to road users in the United Kingdom. The terms "Digital Highway Code" (DHC) and "Digital Driving Rules" may be used synonymously with respect to a collection of performance metrics 254. The DHC term is a convenient shorthand and does not imply any particular driving jurisdiction. The DHC may consist of a collection of any performance metrics 254 that are capable of numerically assessing driving performance. As described above, each metric is numerical and time-dependent. The value of a given metric at a portion of time is referred to as a score for that metric at that time.

[0174] Relatively simple metrics include those based on vehicle speed or acceleration, jerk, etc., distance to another agent (e.g., distance to the nearest cyclist, distance to the nearest oncoming vehicle, distance to the curb, distance to the centerline, etc.). Comfort metrics can score a path based on acceleration or first or higher order time derivatives of acceleration (jerk, jerk, etc.). Another form of metric measures progress toward a defined goal, such as reaching a specific roundabout exit. Simple progress metrics can simply consider the time it takes to reach the goal. More complex metrics quantify concepts such as "missed opportunities," for example, in the context of a roundabout, the extent to which the autonomous vehicle missed an opportunity to enter the roundabout.

[0175] For each metric, an associated “failure threshold” is defined. If the autonomous agent’s score for that metric drops below the threshold, the autonomous agent is said to have failed that metric.

[0176] Not all metrics 252 will necessarily apply to a given scenario. For example, a subset of metrics 254 may be selected as being applicable to a given scenario. The selection of the applicable metric subset may be performed by the test oracle 252 based on one or both of the environmental data 214 associated with the scenario under consideration and the scenario description 201 used to simulate the scenario. For example, certain metrics may only apply to roundabouts, intersections, etc., or to certain weather or lighting conditions.

[0177] One or both of the metrics 254 and their associated fault thresholds may be adapted to a given scenario. For example, a speed-based metric and / or its associated fault metric may be adapted based on applicable speed limits and weather / lighting conditions, etc.

[0178] The planner continuously updates the planned trajectory based on the latest perception and prediction outputs from the perception system 102 and the prediction system 104 .

[0179] Planner Benchmarks

[0180] Figure 3A A block diagram illustrating how the target planner 106T can be compared to the reference planner 106R running in "shadow mode". Figure 3 As depicted, the target planner determines a new instantaneous trajectory at each time step based on the current perceptual output received from the simulator and the predicted outputs of the other agents determined by the prediction component 104. The reference planner 106R receives the same perceptual and prediction inputs as the target planner 106T at each time step and outputs its own instantaneous planned trajectory.

[0181] The goal planner 106T drives the simulation because the output of the goal planner 106T is passed to the control component 108, which provides input to the simulator to update the scene. In contrast, the instantaneous trajectory predicted by the reference planner is not input to the simulator and does not affect the scene, but each instantaneous trajectory is collected and can be compared with the instantaneous planned trajectory of the goal planner 106T at the corresponding time step. This means that the actual trajectory taken by the autonomous vehicle is driven only by the goal planner, and the reference planner follows the vehicle along this trajectory, "shadowing" the goal planner and outputting the vehicle's instantaneous trajectory at each time step.

[0182] Note that shadow mode is only used to describe the mode of the reference planner; the target planner always runs in "in-the-loop testing," i.e., it always runs as part of the autonomous vehicle's driving process, determining the autonomous vehicle's next steps as it progresses through the simulation.

[0183] Figure 3B The target planner is tested against a reference planner running in shadow mode for one or more test metrics at a given time step. For each planner, the internal state of the planner at a given time step can be defined as its instantaneous planned trajectory at that time step. The test oracle 252 receives the instantaneous planned trajectory of each target planner and the reference planner and compares the trajectories (i.e., the common sense and prediction inputs) for the corresponding time step. The trajectory includes the position and motion parameters of the autonomous vehicle, from which metrics of the vehicle's respective performance along each planned trajectory can be derived and compared.

[0184] Figure 4 FIG2 is an example block diagram illustrating the evaluation of a target planner 106T relative to a reference planner 106R, where the reference planner 106T is operating in a "test-in-the-loop" mode. This shows autonomous trajectories generated by the target planner 106T and associated simulator 202T, as well as the reference planner 106R and simulator 202R. The trajectories have a common starting state but are generated independently by the respective target and reference systems.

[0185] The goal planner iteratively updates its predictions and guides the autonomous vehicle forward along the path based on the output of its own simulator 202T. Over time, the actual trajectory taken by the autonomous vehicle is output to the test oracle. Figure 3 In the described process, reference planner 106R utilizes simulator 202R to drive its own autonomous vehicle simulation, iteratively updating its planned trajectory and moving the autonomous vehicle along an independent path based on its own planner and simulation outputs. Test oracle 252 receives autonomous vehicle travel trajectories generated by respective planners 106T and 106R and their associated simulators 202T and 202R. These trajectories can be measured against one or more performance metrics 254, examples of which are described above.

[0186] In contrast to shadow mode, only the trajectories actually driven by the ego vehicle are evaluated, and the instantaneous trajectory of each planner at each planning iteration is not used for evaluation. This is because, when using two different planners and following two different paths, the instantaneous trajectories are planned based on different sensory and prediction inputs, as the agent behaves differently for the two simulated scenarios. Therefore, measuring the instantaneous planned trajectories given different starting points does not provide a useful measure of comparison. However, both planners start from the same overall ego vehicle state, so measuring the actual performance of the ego vehicle along the trajectories generated by each planning system is a fair measure of the planner's performance.

[0187] In some cases, the reference planner 106R may be able to produce a superior trajectory because it will not be subject to the same constraints as the target planner. Specifically, the target planner 106T is typically required to operate in real time and may operate on a resource-constrained platform (such as an onboard computer system of an autonomous vehicle) (with limited computational and / or memory resources). The reference planner 106R may be granted a larger amount of computational and / or memory resources, so the reference planner 106R does not have to be subject to the same constraints and does not necessarily need to operate in real time.

[0188] For example, reference is made to UK patent applications No. 2001200.1, No. 2001202.7 and No. 2001277.9 and F. Eiras, M. Hawasly, SV Albrecht and S. Ramamoorthy, “A two-stage optimization approach to safe-by-design planning for autonomous driving”, arXiv preprint arXiv:2002.02215, 2020, the entire contents of which are incorporated herein by reference. The above discloses a multi-stage constrained optimization planner that is capable of robustly planning high-quality trajectories, but is not necessarily feasible to implement in real time using the state of the hardware and solver (at least without compromising performance). Such a planner can be used as a reference planner to evaluate the performance of a real-time target planner. As will be understood, this is only one example of a suitable reference planner.

[0189] There are other ways that the reference planner 106R can be used to provide a good benchmark. As mentioned above, simulations can be performed using simulated perceptual (or, more generally, observational) errors. One way is to Figure 2AAnother approach is to use simulated sensor data to which perception system 102 is applied. One possibility is to implement a target planner with such perception errors and a reference planner without such perception errors. A metric-based comparison can then indicate whether the difference between the plans is the result of observation errors.

[0190] While the following examples consider different target planners and reference planners (i.e., implementing different planning logic), the technique can also be implemented using the target planner 106T itself instead of the reference planner 106R (i.e., using the same planning logic to implement both the autonomous trajectory and the reference trajectory), but with the target planner 102T operating under different conditions when computing the reference plan (e.g., no perceptual error or with more resources, etc.). This provides a way to assess whether those different conditions are substantial via a structured, metric-based comparison of the different trajectories generated by the target planner under different circumstances. Note that all descriptions herein relating to the reference planner 106R also apply to the target planner 102T in this embodiment.

[0191] Figure 5 is a simple illustration of the planned autonomous vehicle trajectory A by the target planner 106T (planner A) and the second planned trajectory B by the reference planner 106R (planner B) operating in shadow mode. The instantaneous planned trajectory at the current time step t is Figure 5 The path of the ego vehicle and the external agent up to the current time t is shown as a solid line, with time intervals represented by dots along the trajectory, such that the space between two dots represents the distance covered in a single time unit. Therefore, dots that are more widely spaced indicate that the vehicle is moving at a higher speed.

[0192] In this example, the autonomous vehicle moves toward a predetermined target location, and the subject in the scene moves in the same direction. The perception system determines the state of the subject's position variables and motion variables at regular intervals along the path. The prediction system uses the series of observed states of the autonomous vehicle and the subject to generate a predicted trajectory for the subject. The predicted trajectory of the subject at the current time step t is shown as Figure 5 The dashed line continues the current trajectory of the agent in . In this example, the predicted trajectory of the agent shows the agent moving to the right toward the path of the autonomous vehicle.

[0193] The two planners 106T, 106R use the predicted trajectory of the agent and the current state of the autonomous vehicle determined by the perception system 102 to plan a trajectory for the autonomous vehicle, which includes the position and motion of the autonomous vehicle at a series of future time steps until a predetermined target position 500. Figure 5As shown in the left figure of FIG, at time step t, the reference planner 106R plans trajectory B in which the ego vehicle turns right to avoid a collision with the subject vehicle after the subject has already passed it according to its predicted trajectory, and then later turns toward the target. The target planner determines trajectory A in which the ego vehicle slows down to allow the subject to pass according to its predicted trajectory before continuing toward the target location 500. Note that, as previously described, the planned trajectory does not determine the final path taken by the ego vehicle at any given time because the planner updates the trajectory in response to new perception and prediction outputs received for each update of the autonomous state.

[0194] As described above, the test oracle 252 uses one or more metrics to compare the instantaneous trajectories predicted by each planner for the same sensory and prediction inputs. The metrics can assess the success of the planned trajectory in achieving some predetermined goal. For example, in Figure 6A In a passing scenario, a possible metric could simply be a measurement of the vehicle's travel in a given time frame. Planner A could determine a trajectory that intersects the other agent's predicted trajectory at the current time step. However, planner A could instruct the ego vehicle to slow down as the agent passes. Planner B could plan a different trajectory, where the ego vehicle avoids completely intersecting the agent's predicted path by turning right.

[0195] Controller 108 can take actions during the current time step to move the autonomous vehicle along the trajectory planned by planner A. The agent simultaneously moves along its corresponding trajectory, which may differ from the planned trajectory output by prediction system 104. At the next time step t+1, the perception system calculates the new states of the agent and the autonomous vehicle, and the prediction system calculates the updated trajectory prediction for the agent.

[0196] Figure 5 The right side of shows the updates to the planned trajectories of the two planners at the subsequent time step t+1, shown as dashed lines from the circled point identifying the actual position of the ego vehicle at the next time step, showing that the simulator had already moved the ego vehicle along the trajectory planned by the target planner 102T in the previous time step. The planned trajectory of each planner in the previous time step is also shown by the lighter dashed line. The agent's actual motion in the latest update differs from its predicted trajectory (also shown in the lighter dashed line). The prediction stack 104 determines an updated predicted trajectory for the agent, which now veers away from the path of the ego vehicle. Each planner 106 uses sensory input from the ego vehicle's current state and the agent's predicted motion to determine an updated instantaneous trajectory based on the ego vehicle's new position. Both trajectories differ from the corresponding planned trajectories at the previous time step, having determined that the ego vehicle can take a more direct path to the target location 500 based on the agent's predicted path.

[0197] It should be noted that this example shows an exaggerated scenario to illustrate trajectory updates, which is not typical of everyday driving situations. Typically, the vehicle's trajectory is constrained by the road layout, and the variability between planners is mostly due to differences in speed, acceleration, etc. along the road. In this example, the target position remains fixed, but in other driving scenarios, such as driving straight within a lane on a road, the target position may move (for example, to maintain a fixed distance from the ego vehicle or a fixed distance in front of the preceding vehicle in a passing situation).

[0198] Figure 6A An example of the instantaneous planned trajectories of the target planner 106T and the reference planner 106R is shown for a scenario in which the ego vehicle overtakes an agent in the same lane.

[0199] Figure 6A The left side of FIG shows two instantaneous planned trajectories, A and B, at the current time step t. At this time step, the ego vehicle is behind the subject, and the subject's predicted trajectory, as given by prediction stack 104, is a straight trajectory along its current lane. Target planner 106T determines instantaneous trajectory A toward target position 500, in which the ego vehicle closely passes the subject without fully entering the opposing driving lane. Reference planner B determines a slightly different planned trajectory for the ego vehicle, which leaves a wider gap between the ego vehicle and the subject vehicle during the passing maneuver.

[0200] Figure 6A The right side of shows the instantaneous trajectory determined by each planner at a later time t+1. The simulator 202 updates the positions of both the autonomous vehicle and the subject using the subject and autonomous vehicle dynamics, respectively. The autonomous vehicle has begun an overtaking maneuver based on the previously planned trajectory A determined by the target planner 106T. The target planner determines a new instantaneous trajectory toward an updated target position 500 that maintains a fixed distance in front of the autonomous vehicle. The new trajectory plan passes through the subject while remaining close to the center of the road and not completely entering the opposing lane. The reference planner determines an updated planned trajectory in which the autonomous vehicle moves further into the opposing lane and overtakes the subject vehicle while maintaining a larger distance. These trajectories can be compared with multiple metrics in the test oracle 252. Examples of metrics that can be applied to this driving scenario are the autonomous vehicle's movement over time and the autonomous vehicle's distance from the subject during the overtaking maneuver.

[0201] Figure 6B An example of two instantaneous planned trajectories A and B planned by the target planner 106T and the reference planner 106R at the current time step t and the subsequent time step t+1, respectively, for the target right turn determined for the autonomous vehicle is shown. Figure 6BOn the left side of the figure, the two planners plan similar paths because the ego vehicle's position is nearly identical along both planned trajectories. However, as shown by the distribution of ego vehicle states, represented by the black dots, the vehicle's speed differs between the two planners. Planner A generates a slower predicted trajectory in which the ego vehicle decelerates significantly before turning right, while the planned reference trajectory approaches the turn at a higher speed and the ego vehicle is planned to decelerate to a lesser extent before entering the roadway.

[0202] Figure 6B The right side of shows the updates of the two planned trajectories at a subsequent time step t+1. The subject vehicle is now moving along the road in the opposite lane, and the predicted trajectory of the subject vehicle is to continue in a straight line along that lane. The target planner 106T now plans trajectory A in which the autonomous vehicle slows to a stop, allowing the subject to pass through the opposite lane before turning right. However, the reference planner 106R is still approaching the turn at a higher speed and only slows down slightly to turn before the subject vehicle passes through the turn in the opposite lane. This may be because the reference planner has determined that with the current motion of the autonomous vehicle and the subject, there is enough time to turn before the subject vehicle passes, where the target planner may have determined that this should not be done. Different planners may prioritize different driving requirements (such as safety and progress) differently and determine different planning behaviors based on their respective combinations of driving goals.

[0203] Based on one or more metrics of the performance of the test autonomous vehicle, each trajectory may be passed to the test oracle 252 to be evaluated. Figure 7 The diagram shows the process for executing the target planner 106T and the target planner 106T in shadow mode (ie, Figure 3A 6T and reference planner 106R are calculated at each planning step, i.e., the progress of the entire instantaneous trajectory is calculated for each planner at a given planner step size.

[0204] A failure threshold can be selected that defines the minimum allowable score for a given metric. For example, the autonomous vehicle's travel may need to be above a given distance within a predetermined time interval. The normalization of the planner score can be determined such that the failure threshold is set to 0.

[0205] like Figure 7As shown, the target planner falls below the failure threshold during the time interval, which means that the instantaneous planned trajectory planned during the time interval did not make enough progress. Figure 6A In the overtaking example, the target planner might, for example, plan a trajectory in which the ego vehicle waits behind the subject vehicle for a period of time until all subjects have passed in the opposing lane, potentially causing the planned trajectory to fail to meet driving momentum. Meanwhile, the reference planner might determine that it is safe to overtake before the approaching subject vehicle passes in the opposing lane, thus assigning a higher driving score to that planned trajectory. Failure thresholds can encode driving rules and be defined in the Digital Highway Code. As described above, planners can be evaluated on a range of test metrics, of which driving is just one example.

[0206] Figure 8 The figure shows the updates to the planned trajectory for each planner at each time step in "test-in-the-loop" mode. That is, the reference planner is also used for in-the-loop testing. On the left, given the current state of the ego vehicle and the agent's predicted path, the two planners determine the planned trajectory for the current time step. At the second time step, the planners each make a new prediction based on the ego vehicle state updated according to each planner's planned trajectory. In this case, after the first time step, the ego vehicle state is different for each planner.

[0207] In in-the-loop testing mode, planners are compared only against the ego vehicle's actual trajectory, which includes its position and motion at each time step until it reaches the target position. The instantaneous trajectory output by the planner along the path is not used for evaluation. Thus, in-the-loop testing mode compares planners based on the ego vehicle's true motion, which is determined by the accumulation of multiple trajectory updates determined for the instantaneous planner decisions. However, the planner's intermediate decisions at any given point along the path are not used for evaluation in this mode.

[0208] When evaluating the target planner 106T against the reference planner 106R in shadow mode, the reference planner 106R must be provided with the same sensory inputs and predictions as the planner 106T. This provides both planners with the same starting point, making it possible to directly compare the two planners' trajectories. For example, a metric measuring the time it takes to complete the roundabout from the starting point allows for a direct comparison of the time taken for each trajectory. A common initial state is required to compare the two planners' trajectories because any changes in the sensory outputs affect the predicted behavior of other agents in the scene. It is not possible to determine whether any differences between the trajectory outputs in this case are due to different initial states of the ego vehicle or differences between the planners.

[0209] The planning trajectories of the respective target planner and reference planner, as well as metrics used to evaluate these planning trajectories, may be presented to the user in a user interface 1000 that is connected to the test oracle 252 . Figure 9 An example scenario that can be presented in user interface 1000 is shown, in which the goal of an ego vehicle is to overtake an agent moving in the same direction along a road. The predicted trajectory of the agent in the scenario is shown by a thin green line. The planned trajectories of target planner 106T and reference planner 106R are shown, where the reference planner operates in shadow mode, i.e., the ego vehicle state is updated based on the target reference planner 106T, and at a given time step, the instantaneous trajectory planned by reference planner 106R starts from the same ego vehicle state as the planned trajectory of target planner 106T.

[0210] exist Figure 9 In the example shown, a simulation of the autonomous vehicle's path has been performed for this scenario, and the trajectory that the autonomous vehicle actually took in the simulation is shown in blue as the "ground truth" autonomous vehicle trajectory. For example, the user can select the initial state of the autonomous vehicle that generates the instantaneous planned trajectory for the target planner and the reference planner by clicking with the cursor at a point along the autonomous vehicle's ground truth trajectory. Figure 9 In FIG, the instantaneous planning trajectory of the target planner is shown as a thick green line, and the instantaneous planning trajectory of the reference planner 106R is shown as a thick yellow line.

[0211] By sending a request to the API as described above, a reference planner trajectory can be obtained for a given autonomous vehicle state, wherein the resulting trajectory is depicted in the UI based on the information received from the API. The API request can be sent once the user selects an initial autonomous vehicle state from which to generate the planned trajectory.

[0212] The metrics associated with each planned trajectory are displayed to the user in results window 1002. User interface 1000 is not limited to displaying metrics for individual trajectories of a single target planner and a single reference planner starting from the same autonomous vehicle state. For example, metrics can be displayed for trajectories of the same planner starting from different autonomous vehicle states along the ground truth path traveled by the autonomous vehicle.

[0213] The user can display metrics for the associated trajectory by selecting a point along the autonomous "ground truth" trajectory as the initial state for the planned trajectory and by selecting the planner for which the metrics should be displayed. For example, in Figure 9 In the results window 1002, the parameter sets and calculated metrics of the instantaneous metrics (shown in yellow and green) of two different onboard planners are displayed for the same initial state of the autonomous vehicle and the predicted agent behavior. If any metric falls below the defined failure threshold, as described above in reference Figure 7 As described, the user interface 1000 may display a failure signal for the selected trace in the results window 1002. A success signal may be displayed in the results window 1002 if the trace does not fall below the failure threshold for any defined metric.

[0214] Metrics can be selected by the user from any available trajectory and compared. Metrics may include, but are not limited to, travel along a path, average or maximum deviation from some predefined path, average or maximum acceleration or deceleration, and average or maximum positive or negative jerk (rate of change of acceleration).

[0215] In the specific case above, the target planner trajectory is compared to the reference planner trajectory on the route of the actual autonomous "ground truth" path, and the user can select from the presented metrics which metrics should be used to compare and evaluate the planned trajectories. Figure 7 As described above, the user can use the selected metric to generate a time plot of the target planner and the reference planner over the autonomous vehicle's ground truth trajectory.

[0216] For a reference planner running in in-the-loop test mode, user interface 1000 can display two different scenarios, one for each planner. Metrics of the actual driven trajectory can be displayed in results window 1002 for each scenario, since the agent's behavior and the autonomous vehicle's path are different for each planner when the reference planner is running in this mode. However, in this case, given the same initial autonomous vehicle state, the user can still view metrics associated with the autonomous vehicle's path for each planner and compare these metrics for the two trajectories. As described above for shadow mode, metrics can include travel, deviation, and positive and negative acceleration, as well as jerk.

[0217] exist Figure 9 In the example of [ 0 ] , each trajectory is defined relative to a predetermined reference path (e.g., defined to follow the shape of a road). Although the planner aims to follow the reference path when possible, the planner is allowed to deviate from the reference path.

[0218] Figure 9 An example is shown where scores are assigned to trajectories individually, but with respect to the same metric to facilitate meaningful comparisons for the user, namely:

[0219] - travel along a reference path,

[0220] - Deviation of the trajectory from the reference path (maximum and average),

[0221] - maximum acceleration along the trajectory (positive and negative),

[0222] - Maximum and average jerk along the trajectory, ie, the rate of change of acceleration (positive and negative).

[0223] This metric can be applied to both instantaneous traces in shadow mode and actual traces in in-the-loop test mode.

[0224] Although Figure 9 Individual trajectories are depicted, but similar comparison metrics can be defined to provide a score that encodes the comparison results (e.g., relative travel, deviation, jerk, acceleration, etc. between trajectories).

[0225] Although the above description of shadow mode assumes that the target planner 106T and the reference planner 106R are applied at the same time in the scenario, Figure 9 The UI is more flexible. It allows reference planners to be applied at different times (e.g., some set amount of time from the time the target planner 106T is applied (the set amount of time may be a user-configurable parameter)). For example, the UI allows a user to request a metric evaluation of an instantaneous trajectory produced by the target planner 106T at some time, and a metric evaluation of an instantaneous trajectory produced by the same target planner 106T or some other reference planner 106R at some later time step.

[0226] Training Application

[0227] The above techniques can also be used for training. Reference is made to International Patent Publication No. WO 2021 / 152047 and UK Patent Application No. GB2017252.4, the contents of which are incorporated herein by reference in their entirety. Therein, a form of imitation learning is described, in which a function approximator (in this context, a goal planner) is trained using dataset aggregation (DAgger) training to mimic a planner based on constrained optimization.

[0228] In this context, a reference planner operating in shadow mode can be queried to provide additional training examples to augment the training set with the problems (scene states) encountered by the target planner, thereby allowing retraining of the target planner.

[0229] Typical supervised learning problems assume that the data generating process is iid (independent and identically distributed). In the context of autonomous driving, this assumption is flawed. A key aspect for effective driving is the accumulation of errors over time. The distribution of "problems" (scene states) seen by the agent while driving depends on the agent's previous actions. If autonomous actions lead to problems outside the range of the learner's observed representation, the learner's planning relies on generalization.

[0230] The scene state in this context is a snapshot of the scene at a given moment in time, where the autonomy (simulated agent or real vehicle) is to be planned and the trajectories of all other agents have been predicted.

[0231] The following solution (called “PILOT”) improves the efficiency of expensive planners (the reference planner that has the role of “expert” in the current imitation learning context), such as optimization-based planners. Assume that the input to the planning problem is given by Given, and the planning goal is to obtain the state sequence Make it optimized:

[0232]

[0233] Where g=(g1,…,g L ) and h=(h1,…,h M ) may be nonlinear, nonconvex inequality and equality constraints on the planning state, is the cost function defined in planning. While solving this optimization problem globally is known to be NP-hard [D-23], [D-24], efficient solvers exist that actually compute local solutions in acceptable time, assuming a reasonable initial guess is provided [D-25], [D-26]. Here, let ν be defined as an efficient optimizer that solves Equation D-1 (e.g., the optimizer in [D-25]), and Ω be defined as an expert-running, expensive optimizer that attempts to improve the local optimum of Equation D-1 found by ν. Examples of Ω can include performing a recursive decomposition of the problem, taking the minimum cost [D-27], or applying other warm-start procedures [D-4], [D-28].

[0234] The goal of PILOT is to achieve the To this end, PILOT adopts the imitation learning paradigm to train deep neural networks. (900, Figure 9 ), the deep neural network The output of Ω is mimicked and then used to initialize ν in order to output a feasible and smooth trajectory.

[0235] To achieve this, we use a dataset of questions labeled by experts. The neural network is pre-trained on 900. The trained network is then used as a planner in the simulator, employing a DAgger-type training loop [D-17] to adapt to the covariate shift between the training dataset and the learner's experience in the simulator. See Algorithm 1. In principle, a well-trained neural network can be used as Ω at deployment if it produces feasible trajectories that resemble the output of an expert on problems close to those in the training set. However, the raw output of the neural network is not guaranteed to satisfy solution optimality and constraints without substantial investment in robust training [D-29], [D-30] or post-hoc analysis [D-31]. Instead, the neural network Initialize ν to maintain safety and smoothness guarantees. See Algorithm 2.

[0236]

[0237]

[0238] A. Motion planner based on two-stage optimization

[0239] Figure 10 (Top) shows the Figure 3 The system inputs are: 1) a bird's-eye view of the planning scenario, which includes relevant features of the ego vehicle, other road users, and the static layout; 2) a route plan provided by an external route planner as a reference path; and 3) predicted trajectories for all road users provided by the prediction module.

[0240] In 2s-OPT, projecting the world state and road user predictions into a reference-path-based coordinate system produces the optimized inputs. A first optimization phase 302 uses a mixed-integer linear programming (MILP) solver to solve a linear version of the planning problem. This minimizes a cost function that encodes desired planning characteristics, such as passenger comfort and travel along the reference path, while satisfying hard constraints that encode safety requirements. The output of the MILP solver is fed into a constrained, nonlinear optimizer 304 as a warm-start initialization. This second optimization phase ensures that the output trajectory is smooth and feasible while maintaining safety guarantees.

[0241] As mentioned above, although the framework produces superior output compared to alternatives with respect to solution quality (measured by convergence guarantee and output cost values), it is limited by pure optimization methods in terms of solution time, as the method effectively trades off efficiency for better solution quality.

[0242] B. Implementation Details

[0243] The following example uses the in-the-loop Dagger [D-17] method for imitation learning to train a deep neural network to mimic the output of an expert planner. Online augmentation using DAgger enriches the learner's dataset with relevant problem settings that may be lacking in the expert planner's dataset. This benefits from the fact that (unlike human experts) Figure 3 The expert planner is always available to update the training dataset and label new instances experienced by the learner with the expert output, which greatly reduces the training cost compared to using human expert data. Furthermore, this objective augmentation is in contrast to other dataset augmentation techniques that rely on random perturbations to the expert problem (e.g., [D-19], [D-21]).

[0244] See also Figure 10 (Bottom) A convolutional neural network 900 is configured to take as input a graphical representation 902 of a reference path projection planning scenario (including predictions of other road users), in addition to other scalar parameters of the problem (e.g., the velocity of the ego vehicle), and output a smoothed trajectory that mimics the output of the optimizer when presented with the same problem. The expert planner (2S-OPT) runs on the problem dataset to initiate training and is used to label new planning problems generated by the learner in simulation. A post-optimizer 712 implements a nonlinear constrained optimization phase, similar to the second phase 304 in 2S-OPT, to maintain safety and smoothness guarantees.

[0245] In this example, the expert planner takes the form of a two-stage optimization based planner (2S-OPT), as taught in WO2021 / 152047, which includes a linear optimization stage 302 that seeds a nonlinear optimization 712. However, the present technique can be used to train a neural network (or other function approximator) to mimic any form of planner.

[0246] Further details of reference path based representations are described below.

[0247] 1) System Architecture: Figure 10 (Bottom) shows an example system architecture incorporating a trainable neural network 900 that mimics the expert planner 2S-OPT. ) After transforming the planning problem from the global coordinate system to the reference path based coordinate system, a sequence of images of the scene 902 is then generated to encode the predicted future of the dynamic road user. These images 902 are part of the input to the neural network 900. The output of the network 900 is a trajectory in the reference path coordinate system, which is used as a warm start initialization for the constrained nonlinear programming (NLP) optimization stage 712 in order to enforce certain desired qualities of the output trajectory. Finally, the output of the optimizer is then inversely transformed Transform back to the global coordinate system.

[0248] 2) Network Input Representation: The planning problem input consists of a static road layout, road users with their predicted trajectories, and a reference path to follow. When the problem is transformed into the reference path coordinate system, the transformed scene automatically aligns with the region of interest (the road along the reference path). This simplifies the representation of neural network 900.

[0249] In order to encode the predicted trajectory of dynamic road users, The positions of road users along their predicted trajectories are sampled uniformly in time for the planning horizon h, producing C grayscale top-down images of the scene of size W × H. These images create an input tensor of size C × W × H, allowing the use of convolutional layers to extract semantic features of the scene and its temporal evolution. Static layout information is presented on all channels.

[0250] Additional information of the planning problem that is not visible in the top-down image, such as the initial velocity of the ego vehicle, is appended to the first dense layer of the network along with the flattened convolutional neural network (CNN) output as a scalar input.

[0251] Figure 12 Further details of an example convolutional architecture of neural network 900 are shown.

[0252] 3) Network output representation:

[0253] The output of the network 900 is a trajectory in the reference path coordinate system. One possibility is to output a tensor of size 2×(h·f) for the planning time domain h and the planning frequency f, encoding the time stamp space point τ = {(x j ,y j )} j＝1,...,N To enforce output smoothness, an alternative is to train the network to produce parameters of a family of smooth functions, e.g., time-varying polynomials and B-splines, i.e., f x (t) and f y (t).

[0254] The post-hoc NLP optimization stage 712 expects as input a sequence of timestamped states, each of which consists of: (x, y) position, velocity, orientation, and control inputs (steering and acceleration), all in the reference path coordinate system. Velocity and orientation are calculated from the sequence of points generated by the network (or sampled from the smoothing function output). Control inputs are derived from the inverse dynamics model. The NLP optimization stage is an optional addition to ensure that certain constraints are satisfied in the final trajectory; for further details, see WO 2021 / 152047.

[0255] 4) Neural Network Training:

[0256] a) Pre-training: The neural network 900 is trained to mimic the output of the expert planner 2s-OPT. In a supervised learning approach, expert data is generated by running the expert planner 2s-OPT on problems generated by various configurations of simulated driving examples to create a training dataset. The training loss is defined as the L2 norm distance between the expert trajectory and the network output:

[0257]

[0258] where θ refers to the neural network parameter vector, is a dataset of training examples, and the identifier exp indicates the expert trajectory from this dataset. The ADAM optimizer [D-32] is used to determine the update step size.

[0259] b) DAgger Training: Dataset Aggregation (DAgger) [D-17] is a training regime that reduces the difference between the distribution of problems in an expert dataset and the distribution of problems seen by a learner when interacting with its environment. This is accomplished by online augmenting the training dataset with additional problems generated by a pre-trained learner network as it interacts with the environment in simulation. DAgger requires an interactive expert who can query solutions to new problems in a scalable manner. This is satisfied in our case by 2s-OPT, which labels new problems with high-quality solutions. As described in Algorithm 1, the DAgger process alternates between problem generation in simulation and training the network.

[0260] Extending the benefits of DAgger training, typical supervised learning problems assume that the data generating process is iid (independent and identically distributed). In the case of autonomous driving, this assumption is flawed. A key aspect for effective driving is the accumulation of errors over time. The distribution of "problems" (scene states) seen by the agent while driving depends on the agent's previous actions. If autonomous actions lead to problems outside the range of the display seen by the learner, the learner's planning relies on generalization.

[0261] The scene state in this context is a snapshot of the scene at a given moment in time, where the autonomy (simulated agent or real vehicle) is to be planned and the trajectories of all other agents have been predicted.

[0262] Figure 11 A schematic block diagram of a training scheme (referred to herein as the “PILOT” scheme) is shown. PILOT uses an expert-in-the-loop testing imitation learning paradigm to train deep neural networks. (900), the deep neural network (900) imitates the output of the optimization planner Ω(top) based on the high running cost. At inference time, use The output of is used to initialize an efficient optimizer ν (712) to compute feasible and low-cost trajectories (bottom).

[0263] According to Algorithm 2, the scheme alternates between training steps and augmentation steps. For a large dataset of examples obtained using the reference planner Ω The first training set is performed. In each subsequent training step, the parameters θ are adjusted by the augmented training set (as augmented in the previous augmentation step) In each augmentation step, the (partially trained) network 900 is applied to simulated scenarios in the simulator 904 by using the partially trained network 900 to plan trajectories for simulated autonomous agents (typically in response to other agents in the scenario). The expert planner Ω is then trained in shadow mode (i.e., a given snapshot of the scenario encountered in the latest set of simulations). The reference planner Ω is used to generate expert trajectories at those moments, which are then used to augment the training set to proceed to the next training step. Note that the evolution of each simulated scenario is at least partially determined by the actions of the partially trained network 900 (e.g., at some time into the scenario, x0 typically depends on earlier actions taken by the partially trained network 900, e.g., in the case of other agents reacting to the autonomous agent, other parameters The reference planner Ω then provides expert trajectories for those scenarios (but, unlike network 900, does not control the autonomous agent or influence the simulated scenarios, as the purpose is to expose the reference planner Ω to scenarios in which the errors of the partially trained network 900 may have accumulated over time).

[0264] 5) Post-optimization phase: In the post-optimizer 712, a nonlinear optimization phase following the constraints of 2s-OPT is designed to smooth and correct the trajectory generated by the imitation learning network. More specifically, a discrete bicycle model for the autonomous vehicle is used:

[0265]

[0266] Among them, z k =[x k ,y k ,φ k ,v k ] T Describes the state of the autonomous system at time k, where (x k ,y k ) is the reference path projection position, φ k is the vehicle's deflection angle, v k is the speed of the vehicle. (a k ,δ k ) represents the acceleration and steering control applied to the vehicle at time k, and L is the distance between the axles.

[0267] Constraints are enforced to maintain acceleration and steering values ​​within allowed ranges, limit maximum jerk and angular jerk, and maintain speed within the allowed speed limit for the road. Furthermore, constraints are added to ensure that the corners of the autonomous vehicle's footprint are always constrained to the road surface and to prevent collisions with other road users.

[0268] The post-optimizer 712 optimizes the cost function defined as follows:

[0269]

[0270] in is a scalar weight, θ i (z k ,u k ) are soft constraints that measure deviations from the desired velocity, reference path, and final target position and govern the criteria for acceleration and steering control inputs. A grid search is used in the parameter space to fine-tune the optimized parameters. A more detailed formulation of the optimization problem is provided in Appendix B.

[0271] The output of the imitation learning neural network is used as the initialization for this optimization problem. As shown in Appendix A, since the NLP solver converges faster when initialized with a feasible solution, the initialization trajectory is processed to cover outliers in the calculated velocity, acceleration, and steering that may violate some hard constraints. This process is detailed in Appendix C.

[0272] The described approach places no restrictions on the design of the optimization objective that is expected to adhere to the planner output, whereas in settings such as [D-20] the optimization problem should be carefully designed to ensure that the optimization constraints are differentiable in order to make them useful for training the planner network.

[0273] Typically, the function approximator takes the form of one or more trainable models, rather than ones that can be trained to mimic an expert. Those models can take any form, including but not limited to neural networks. As in the example above, the expert can be a computer-implemented planner (not necessarily operating in real time). In that case, one or more models are chosen so that they can be applied with fewer computational or memory resources than the expert planner (preferably in real time). This typically means trading efficiency (time and / or memory efficiency) for accuracy (i.e., the accuracy with which the function approximator can mimic the expert) - in the above embodiment, the trade-off is that the trained function approximator may no longer be guaranteed to meet the hard constraints of the NLP stage. However, this problem is addressed by implementing NLP logic at runtime to refine the initial trajectory produced by the function approximator, and in this context, the trained function approximator does not have to perfectly mimic the expert.

[0274] As will be appreciated, while the above treats a specific form of multi-level optimization-based planner as the expert to be emulated, function approximators can be trained to implement other types of planners. This is particularly beneficial when the expert planner has the ability to produce high-quality trajectories but requires significant computational and / or memory resources to do so.

[0275] Reference path based representation

[0276] In order to simplify the process of defining planning objectives, the global coordinate system is reversibly transformed The path is transformed into a reference path-based representation. This representation significantly simplifies the path tracking problem.

[0277] The goal of the autonomous vehicle is defined as following a differentiable and bounded two-dimensional reference path in the global coordinate system By the distance from the starting point of the path Parameterized. Along the path At any point λ, the tangent vector and normal vector of the reference path in the global coordinate system can be obtained as follows:

[0278]

[0279] Reference Path The reference path is the path that the autonomous vehicle typically aims to follow at a set target speed. However, in the event of an obstacle, deviations from the reference path and target speed are permitted as long as no hard constraints (such as collision avoidance constraints) are violated. The reference path can be determined using knowledge of the road layout, which can be determined using predetermined map data (such as an HD map of the driving area), information from the perception stack 104, or a combination of both. Specifically, for complex layouts (such as complex intersections or roundabouts), the reference path can be learned by monitoring the behavior of other drivers in the area over time.

[0280] Figure 13A Shows the input Proceed to the desired output X 0:N Given input (uppercase), reversible transformation is applied to produce in a reference path based representation (lowercase), the planner 106 then uses The output of the planner 106 is the expected trajectory x in a representation based on the reference path 0:N , the expected trajectory x 0:N Then by applying the inverse transform Transform back to the global coordinate system to obtain X in the global coordinate system 0:N Further details of the transformation are described below.

[0281] Reversible transformation It operates on three types of input: (1) pose, (2) velocity, and (3) covariance matrix. Each of these operations is described next.

[0282] 1) Posture change: The global coordinate system The pose (X, Y, Φ) in the image is mapped to the reference path system The pose (x,y,φ) in , such as Figure 13B As shown in .

[0283] · is [XY] to The projected distance from the starting point of the path is defined as the solution to the following optimization problem:

[0284]

[0285] Due to the nature of the optimization, no closed-form solution can be obtained for x.

[0286] · Among them, n x is the normal vector of the reference path at λ = x as in (1), and

[0287] ·φ=∠t x -Φ, where

[0288] Figure 13B Shows the transformation between the world reference frame and the reference path based frame and inverse transform Visual representation of .

[0289] As will be understood, the inverse transform It can be derived directly by applying the same geometric principles.

[0290] 2) Speed ​​change: because Defined in space, so velocity is invariant to it:

[0291] 3) Covariance transformation: Consider a traffic participant with pose O and covariance Γ, so that The transformed covariance matrix in the reference path coordinate system is given by:

[0292]

[0293] Among them, t x is evaluated at λ = x , R∈SO(2) is the rotation matrix.

[0294] References

[0295] Reference is also made above to the following documents, each of which is incorporated herein by reference in its entirety:

[0296] [D-4] F. Eiras, M. Hawasly, S. V. Albrecht, and S. Ramamoorthy, “Twostage optimization-based motion planner for safe urban driving” arXiv preprint arXiv:2002.02215, 2020.

[0297] [D-17] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning” in Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011, pp. 627–635.

[0298] [D-19] J. Chen, B. Yuan, and M. Tomizuka, “Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety” in 2019 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, pp. 2884–2890.

[0299] [D-20] W. Zhan, J. Li, Y. Hu, and M. Tomizuka, “Safe and feasible motion generation for autonomous driving via constrained policy net,” in IECON 2017-43rd Annual Conference of the IEEE Industrial Electronics Society, 2017, pp. 4588–4593.

[0300] [D-21] T. Tosun, E. Mitchell, B. Eisner, J. Huh, B. Lee, D. Lee, V. Isler, H. S. Seung, and D. Lee, “Pixels to plans: Learning non-prehensile manipulation by imitating a planner” arXiv preprint arXiv:1904.03260, 2019.

[0301] [D-23] C.A. Floudas and P.M. Pardalos, State of the art in global optimization: computational methods and applications. Springer Science & Business Media, 2013, Vol. 7.

[0302] [D-24] J. Nocedal and S. Wright, Numerical optimization. Springer Science & Business Media, 2006.

[0303] [D-25] A. Wachter and L.T. Biegler, “On the implementation of an interior-¨point filter line-search algorithm for large-scale nonlinear programming,” Mathematical Programming, vol. 106, no. 1, pp. 25–57, 2006.

[0304] [D-26] A. Zanelli, A. Domahidi, J. Jerez, and M. Morari, “Forces nlp: an efficient implementation of interior-point methods for multistage nonlinear nonconvex programs,” International Journal of Control, pp. 1–17, 2017.

[0305] [D-27] A.L. Friesen and P. Domingos, “Recursive decomposition for nonconvex optimization” arXiv preprint arXiv:1611.02755, 2016.

[0306] [D-28] T. S. Lembono, A. Paolillo, E. Pignat, and S. Calinon, “Memory of motion for warm-starting trajectory optimization,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2594–2601, 2020.

[0307] [D-29] M. Mirman, T. Gehr, and M. Vechev, “Differentiable abstract interpretation for provably robust neural networks” in International Conference on Machine Learning, 2018, pp. 3578–3586.

[0308] [D-30] E. W. Ayers, F. Eiras, M. Hawasly, and I. Whiteside, “PaRoT: A practical framework for robust deep neural network training” in NASA Formal Methods. Springer International Publishing, 2020, pp. 63–84.

[0309] [D-31] C. Liu, T. Arnon, C. Lazarus, C. Barrett, and M. J. Kochenderfer, “Algorithms for verifying deep neural networks” arXiv preprint arXiv:1903.06758, 2019.

[0310] [D-32] D.P. Kingma and J.Ba, “Adam: A method for stochastic optimization” arXiv preprint arXiv:1412.6980, 2014.

Claims

1. A computer-implemented method for evaluating the performance of a goal planner of an autonomous robot in a real or simulated scenario, the method comprising: receiving evaluation data for evaluating the performance of the goal planner in the scenario, the evaluation data generated by applying the goal planner at incremental planning steps to compute a series of autonomous plans responsive to changes in the scenario, the series of autonomous plans being implemented in the scenario to cause changes in autonomous states; The evaluation data includes: the autonomous plan calculated by the target planner at one of the planning steps, and the state of the scene at the time instant of the scene; The evaluation data is used to evaluate the target planner in the following manner: calculating a reference plan at the time instant based on the scenario state, the scenario state comprising: the autonomous state at the time instant resulting from implementing one or more previous autonomous plans in the series of autonomous plans calculated by the target planner; and At least one evaluation score is calculated for comparing the autonomous plan with the reference plan.

2. The method according to claim 1, wherein The autonomous plan is in the form of an instantaneous autonomous trajectory; wherein the instantaneous autonomous trajectory at the planning step extends further in time than a next planning step, and before calculating the instantaneous autonomous trajectory for the next planning step, the instantaneous autonomous trajectory is only partially implemented in the scene until the next planning step; The reference plan is in the form of an instantaneous reference trajectory, and the at least one evaluation score is used to compare the instantaneous autonomous trajectory with the instantaneous reference trajectory.

3. The method according to claim 2, wherein: The planning step occurs at the moment in time of the scene, wherein both the autonomous plan and the reference plan are calculated based on the state of the scene at the moment in time.

4. The method according to claim 3, wherein: Both the instantaneous autonomous trajectory and the instantaneous reference trajectory extend beyond the next planning step.

5. The method according to claim 1 or 2, wherein: The time instant is later than the planning step so that the reference plan takes into account changes in the scene since the planning step was performed in the scene state.

6. The method according to claim 4, wherein: The target planner is constrained to operate in real time, and the reference plan is not computed in real time.

7. The method according to claim 6, wherein: More computing resources and / or memory resources are allocated to computing the reference plan than the computing resources and / or memory resources granted to the target planner for computing the autonomous plan.

8. The method according to claim 7, wherein: The reference plan is computed by applying a reference planner to the scene state at the time instant.

9. The method according to claim 8, wherein The reference planner is a planner based on constraint optimization.

10. The method according to claim 8 or 9, wherein: The state of the scene at the time of the planning step comprises, or is derived from, a current set of observations used by the target planner to compute the autonomous plan; wherein the reference planner uses the same set of observations to compute the reference plan.

11. The method according to claim 10, wherein: The observations are noisy observations subject to real observation errors or modeled observation errors, wherein both the target planner and the reference planner are subject to the same observation errors.

12. The method according to claim 10, wherein: The scenario is simulated, and the observation result is a simulated observation result without any observation error, so that neither the target planner nor the reference planner is subject to observation error.

13. The method according to claim 12, wherein: The scene state comprises: the current set of observations and / or one or more current obstacle predictions derived from the current set of observations, wherein the target planner and the reference planner use the same observations and / or the same obstacle predictions to compute the autonomous plan and the reference plan, respectively.

14. The method according to claim 9, wherein The autonomous plan is computed by the target planner based on a set of observations subject to modeled perceptual errors, and the reference plan is computed based on the same observations but without the modeled perceptual errors.

15. The method according to claim 14, wherein Use the goal planner to compute the reference plan, but: Having greater computational or memory resource allocation; Not constrained to real-time operations; The time is later than the planning step; and / or There is no perceptual error modeled.

16. The method according to claim 15, wherein The evaluation data comprises: the series of autonomous plans calculated at a plurality of planning steps in the incremental planning steps, and a plurality of scene states each associated with one of the plurality of planning steps; Wherein, based on the associated scene state at the planning step, a reference plan is calculated for each of the multiple planning steps, and at least one evaluation score is calculated for each of the multiple planning steps for comparing the autonomous plan and the reference plan of the planning step.

17. The method according to claim 16, wherein For each of the plurality of planning steps, the associated scene state is the scene state at the planning step, or the scene state a set amount of time after the planning step.

18. The method according to claim 17, wherein: The at least one evaluation score includes: a separate evaluation score calculated for the target plan and a separate evaluation score calculated for the reference plan; the separate evaluation scores are calculated with respect to a common metric.

19. The method according to claim 18, wherein The at least one evaluation score includes a comparison score that quantifies a difference between the target plan and the reference plan relative to a comparison metric.

20. The method according to claim 18 or 19, wherein The common metric or comparative metric quantifies safety, comfort or progress in relation to a defined goal.

21. The method according to claim 20, wherein The target plan and the reference plan are calculated with a common goal of achieving a defined goal.

22. The method according to claim 21, wherein The target is defined as a static or moving target location to be reached.

23. The method according to claim 22, wherein The evaluation score is a numerical evaluation score for quantifying the difference between the autonomous plan and the reference plan.

24. The method according to claim 23, wherein The scenario is a real driving scenario or a simulated driving scenario, and the autonomous state is a state of an autonomous vehicle.

25. The method according to claim 23 or 24, wherein The evaluation score is calculated with respect to a metric that quantifies a different level of compliance with predetermined driving rules between the autonomous plan and the reference plan.

26. The method according to claim 25, wherein A plurality of evaluation metrics are used to calculate a plurality of evaluation scores, each evaluation metric encoding a different predetermined driving rule.

27. The method according to claim 26, comprising the step of outputting the assessment score on a user interface.

28. The method according to claim 27, wherein The evaluation score is output along with a visual representation of the scene and visual representations of the autonomous plan and the reference plan.

29. The method according to claim 2, wherein The at least one assessment score is used to compare: The autonomous trajectory and the reference trajectory progress towards the defined target, The deviation of the autonomous trajectory and the reference trajectory from the reference path, The acceleration along the autonomous trajectory and the reference trajectory, The jerk, i.e., the rate of change of acceleration, along the autonomous and reference trajectories.

30. The method according to claim 29, wherein Different evaluation scores are calculated for the autonomous trajectory and the reference trajectory respectively for use in the comparison.

31. The method according to claim 8 or 9, wherein The target planner includes one or more trainable models, and the evaluation data provides one or more training examples for training the target planner to mimic the reference planner; the evaluation score is a component of a cost function that measures the deviation between an autonomous plan calculated by the target planner and a reference plan calculated by the reference planner; The training is performed with the goal of minimizing the deviation.

32. The method according to claim 31, wherein The goal planner has been pre-trained on earlier training examples; the training examples provided by the evaluation data are new training examples for retraining the goal planner.

33. A computer system comprising one or more computers programmed to perform the method of any one of claims 1 to 32.

Images