Technology for planning actions of an at least partially automated vehicle

Abstract

A technique for planning actions of an at least partially automated vehicle in a planning phase comprises a computer-implemented method. Sensor data representing an actual state of a traffic scene around the vehicle in a first time step are received. Actions of other road users in the traffic scene are predicted in a second time step immediately following the first time step based on the received sensor data. Actions of the vehicle are reactively planned for the second time step. The reactive planning is conditioned on the predicted actions of the other road users in the traffic scene in the second time step. The prediction and reactive planning are repeated for a further time step, in particular an immediately following one, and specifically a maximum of a predeterminable number of N iteratively per planning phase.At least after the maximum N repetitions, a value of a quality function is determined for each state of the traffic scene reached by the repeated prediction and reactive planning.

Description

[0001] The patent application relates to a technology for planning actions of an at least partially automated vehicle, particularly during a planning phase. A method, a computing device, and a system for planning the actions are provided. Furthermore, a method for training, a computer program, and a computer-readable medium are provided. State of the art

[0002] In modular approaches for autonomous driving, the software components are usually divided into perception, prediction, and planning components. The planning methods used in practice within the planning component can be divided into three classes: learned methods, which use traditional control and optimization techniques, and hybrid methods, which represent a combination of the above methods. In recent years, the use of machine learning (ML), particularly deep learning (DL), has become the de facto standard in learned methods, as it allows for the inclusion of diverse contextual information.

[0003] To plan safe and predictable maneuvers, autonomous vehicles must anticipate how the situation they are in will develop. To do this, they often predict future trajectories of other road users (vehicles, cyclists, pedestrians) and pass them on to the planning components.

[0004] In a large class of learned methods, a planning component is learned that, based on the current state of the scene (the state of the autonomous vehicle, as well as all other available data), provides a probability distribution of possible future actions (steering angles and accelerations) or complete planned trajectories as output. For this class of methods, a further algorithm must then be used to determine which action or trajectory is selected and passed on to the control component. Algorithms based on tree search here use the combination of planning and prediction components to predict future states of the scene, based on the quality of which the decision for the action or trajectory is then made.

[0005] This means that the predicted quality of future states is retroactively used to make a choice in the present. These tree search approaches for finding the next action also find applications outside of autonomous driving [1]. A popular algorithm is the Monte Carlo Tree Search (MCTS) [1, 2], which strikes a balance between exploring new states and exploring previously visited states (a tradeoff between exploration and exploitation) to choose the best current action or trajectory.

[0006] Conventionally, the planning of the autonomous vehicle's action or trajectory in a time step is independent of the prediction of the other road users in the same time step, which limits the predicted quality of the future states and thus the overall planning in its accuracy. Disclosure of the invention

[0007] In the following, the inventive solution is described with reference to the claimed methods for planning and training, as well as with reference to the claimed computing device. Features, advantages, or alternative embodiments herein can be assigned to the other claimed subject matter (e.g., the system, the computer program, or a computer program product), and vice versa. In other words, the claims for the computing device and / or the system comprising the computing device can be improved by features described or claimed in connection with the methods. In this case, the functional features of the method are embodied by structural units of the computing device and / or the system, and vice versa.

[0008] According to a first method aspect, a computer-implemented method for planning actions of an at least partially automated vehicle in a planning phase is provided. The method comprises a step of receiving sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step. The method further comprises a step of predicting actions of other road users in the traffic scene in a second time step immediately following the first time step based on the received sensor data (e.g., by means of a prediction component). The method further comprises a step of reactively planning actions of the at least partially automated vehicle for the second time step (e.g., by means of a planning component).The reactive planning is conditioned on the predicted actions of the other road users in the traffic scene in the second time step. The method further comprises a step of repeating the prediction (e.g., using the prediction component) and the reactive planning (e.g., using the planning component) for a further (in particular, immediately following) time step. The prediction and reactive planning are repeated iteratively a maximum of N times per planning phase. At least after the maximum of N repetitions, a value of a quality function is determined for each state of the traffic scene reached by the repeated prediction and reactive planning.

[0009] Using this technology, a safe and comprehensible maneuver can be planned for a vehicle (especially one that is at least partially automated). By predicting the actions of other road users in the traffic scene (also: the developments in the traffic scene) in the same timeframe in which planning of the at least partially automated vehicle is carried out (planning step), an improved quality assessment of future conditions and / or an improved selection of the vehicle's planned next actions can be enabled.

[0010] By conditioning the predicted actions of other road users during the reactive planning step, the search space for planning can be restricted. This can improve planning reliability in real time (and / or while the vehicle is moving).

[0011] The traffic scene can include at least one other road user in addition to the at least partially automated vehicle (also: ego vehicle), in particular another vehicle, a pedestrian, a motorcyclist and / or a cyclist. The (e.g. at least partially automated and / or other) vehicle can in particular be a road-based vehicle (e.g. a car or truck) with auto-automatic or partially automated driving functionality in (in particular a high, e.g. L4 or L5, level below) levels L1 to L5. Automated operation can include full automation (also referred to as autonomous driving, in particular L5), at least partial automation (e.g. high, L4, conditional, L3, or partial, L2) and / or a driver assistance system (e.g. L1, comprising adaptive cruise control). For example, partially automated driving functionality can include a traffic jam pilot, which, for example,is used only in limited areas and / or for limited periods of time. Alternatively or additionally, the at least partially automated vehicle may comprise a robot that can move, in particular, on a factory site.

[0012] The traffic scene can represent and / or include an environment of the at least partially automated vehicle, in particular an environment detectable by sensors (e.g., video cameras, radar sensors, and / or LiDAR sensors on the vehicle). Alternatively or additionally, the traffic scene can be represented by an environment model. The environment model can include one agent per road user (e.g., one agent for the at least partially automated vehicle and one agent for each additional road user).

[0013] The traffic scene evolves over time. This is recorded (by sensors). A traffic scene state refers to the state of the traffic scene at a specific time step.

[0014] The traffic scene and / or the surroundings of the at least partially automated vehicle can be spatially limited by a maximum range of one or more sensors from which the sensor data is received and / or by a maximum distance from the at least partially automated vehicle. For example, the maximum range of a LiDAR sensor can be up to 200 m.

[0015] The traffic scene may be asymmetrical around the at least partially automated vehicle. For example, more sensors (and / or sensors with a longer range) may be directed forward (and / or rearward) than to the sides of the at least partially automated vehicle.

[0016] The sensor data can be received by at least one sensor system comprising one or more sensors, in particular a (e.g., video) camera, a radar device, a LiDAR device, an ultrasound device, a motion sensor, and / or a thermal imaging sensor. Alternatively or additionally, the sensor data can include data from a positioning system and / or a navigation system (e.g., GPS or Galileo) and / or traffic reports or other messages, which are received, for example, wirelessly from a transmitting station. Furthermore, alternatively or additionally, the sensor data can include vehicle-to-everything (V2X, also: Car2x) data.

[0017] The sensor system can be arranged internally within the vehicle and / or on the at least partially automated vehicle (e.g., a camera mounted on the vehicle). Alternatively or additionally, the sensor system can be arranged externally on the vehicle (e.g., a camera mounted on a road for traffic monitoring, a positioning system and / or navigation system, a weather determination system, and / or internally on another road user). The sensor data received by the external sensor system can be transmitted (and / or received) wirelessly, for example, via V2X.

[0018] The sensor data can be received currently and / or buffered. Alternatively or additionally, the sensor data can include historical data, particularly relating to one or more previous states of the traffic scene.

[0019] Map data (e.g., from a navigation system) of the surrounding area can be stored in the at least partially automated vehicle. Alternatively or additionally, the received sensor data can (in particular additionally) include map data of the surrounding area.

[0020] Receiving the sensor data (also known in technical terms as perception) with regard to the traffic scene and / or other road users may include pre-processing, in particular object classification and / or object detection (e.g. of another vehicle or pedestrian).

[0021] The method may further comprise a step of aggregating the received sensor data. In particular, sensor data may be received from various sensors and / or sources. The aggregation may comprise combining the sensor data from various sources and / or sensors. Alternatively or additionally, the aggregation may comprise combining sensor data associated with different time steps (e.g., the current and / or first, as well as, in particular, directly preceding time steps).

[0022] The at least partially automated vehicle and / or all or selected other road users can (in particular each) be represented by an agent.

[0023] An action can relate to an automatically controlled or regulated function and / or behavior (e.g., of the at least partially automated vehicle). The action can be dynamic. For example, the action can include acceleration (and / or deceleration), a change in the steering angle, a jerk, and / or a waypoint. The action can include the determination of a trajectory for movement (e.g., of the at least partially automated vehicle). The trajectory can include a position, a speed, an acceleration, an orientation, a drift, a lane change, and / or a path (e.g., a path curvature and / or curve). Alternatively or additionally, the trajectory can include a state of the associated (in particular, another) road user or of the at least partially automated vehicle.

[0024] Alternatively or additionally, the action can be static. For example, the action can include flashing, honking, and / or not moving (e.g., the at least partially automated vehicle).

[0025] Receiving the sensor data, predicting, and reactive planning can be performed by a perception component, a prediction component, or a planning component, respectively. At least the prediction component and / or the planning component can be represented by artificial intelligence (AI) and / or include deep learning (DL).

[0026] The prediction (also: prediction and / or temporal development or forecasting of the actions of other road users in the traffic scene over time) of the actions of a (particularly other) road user and / or the planning of the actions of the at least partially automated vehicle can be individual for the road user and / or the at least partially automated vehicle, in particular based on object classification and / or object recognition. For example, the prediction and / or planning can be carried out depending on one or more parameters of the road user and / or the at least partially automated vehicle.The parameter(s) may include, in particular, technical specifications and / or dimensions, for example, a distance between the wheel axles, a vehicle model, a maximum steering angle, and / or a maximum acceleration of the at least partially automated vehicle and / or the (in particular, other) road user. Alternatively or additionally, the prediction and / or planning may use a kinematic model of the respective road user and / or the at least partially automated vehicle.

[0027] Using a state prediction function (e.g., as a combination of a prediction component and a planning component), a (particularly encoded) future state of the entire traffic scene (including all or selected road users, in particular not only the at least partially automated vehicle) can be determined (and / or estimated) and / or its temporal progression can be described. The (e.g., encoded) future state can also include objects and / or actions (e.g., rolling balls and / or the course of a road after a curve) in the traffic scene that are not represented by agents.

[0028] Prediction may include a deterministic method and / or a stochastic method.

[0029] Planning is preferably reactive. Planning can be reactive in that it depends on predicting the actions of other road users in the traffic scene. Planning can serve to provide a probability distribution over possible future actions (e.g., including trajectories) and / or possible future traffic scenes. The quality function can provide an approximate (and / or predicted) quality value (in short: value) for a possible future traffic scene.

[0030] The planning component can use a conventional control and / or optimization method, a planning method using deep learning (DL), and / or a hybrid planning method. The planning component can be designed to transmit its results to a control component and / or a control component. The control and / or control component can interact with actuators of an action layer to move and / or control the vehicle (in particular, one that is operated at least partially automatically) in an at least partially automated manner.

[0031] The quality function can evaluate an (achieved) state of the traffic scene in a time step, in particular including the planned actions of the at least partially automated vehicle and preferably all or selected other road users. Based on the best value of the quality function, an action (in particular a sequence of) of the at least partially automated vehicle can be selected. Determining the value of the quality function for the state can lead to a more global traffic scene optimum being achieved (namely, taking into account the other road users and their behavior / actions) and, in particular, not only ensuring that the at least partially automated vehicle reaches its desired destination. For example, the consistency of the achieved traffic scene can be evaluated (e.g.that there are no future overlapping road users or agents and / or the road users or agents have not left the lane).

[0032] The planning component can be used for stationary applications, such as in a particularly permanently installed traffic control system, and / or can be used for mobile applications, such as on a component in an at least partially automated vehicle with assistance function or a fully autonomous or partially autonomous vehicle.

[0033] Conditioning on the actions of other road users in the planning step, in particular for the same (e.g. second) time step as the prediction of the actions of the at least partially automated vehicle, can enable predictive driving (and / or behavior, in particular adapted to the traffic scene and / or environment).

[0034] The traffic scene (and / or the environment of the at least partially automated vehicle and / or the environment model) can be represented by a rasterized representation (also: rasterized map). The actions can be represented by vectors in the rasterized representation.

[0035] In a further development, additional sensor data relating to dynamic traffic control in the traffic scene (and / or the surroundings of the at least partially automated vehicle) can be received, in particular relating to a traffic light with switching states (e.g., red phase, green phase). Predicting the dynamic traffic control (e.g., traffic light switching) can be incorporated as further conditioning in the planning of the actions of the at least partially automated vehicle. For example, a speed of the at least partially automated vehicle can be planned depending on a predicted dynamic traffic control (e.g., traffic light phase).

[0036] Repeating prediction and reactive planning can also be called iterative prediction and reactive planning.

[0037] A time step can also be referred to as a (especially discrete) point in time.

[0038] The current state can be assigned to a time t. The first prediction and reactive planning can be assigned to a time t+1. Alternatively or additionally, an mth iteration of the prediction and planning can be assigned to a time t+m.

[0039] The sensor data representing the actual state (in particular in the first time step) may have been recorded (and / or acquired) at least partially in the first time step. Alternatively or additionally, the sensor data may have been recorded (and / or acquired) at least partially in one or more earlier time steps (also: preceding, previous, and / or past the first time step). Alternatively or additionally, the actual state of the traffic scene may be understandable based on its past. For example, a speed (and / or acceleration) in the actual state of the at least partially automated vehicle and / or of another road user may be determined using (in particular position) sensor data, preferably from several earlier time steps of the same vehicle.

[0040] In one embodiment, the received sensor data for a predetermined prior time period (and / or a predetermined number of prior time steps) can be aggregated to represent the current state. Alternatively or additionally, a compressed state can be stored, which is updated (e.g., in each time step) based on newly received sensor data.

[0041] The repeated (and / or iterative) prediction and reactive planning cannot exceed a maximum number of N iterations. For example, the repeated (or iterative) prediction and reactive planning can be aborted if a predetermined termination criterion is met. In one embodiment, the predetermined termination criterion can only be applied to individual branches of the tree search; for example, other branches of the tree search can not be further developed (e.g., no further prediction and reactive planning from a termination time step onwards) if a sufficiently high value of the quality function (and / or exceeding a predetermined threshold) is reached in a branch (in particular in the termination time step). Alternatively or additionally, the predetermined termination criterion can represent the achievement of a planning goal. For example, a parking space can be found within a number of iterations less than N (e.g.,within 1s).

[0042] The repeated (and / or iterative) prediction and reactive planning in a planning phase can comprise up to N time steps. The maximum number of N time steps can also be referred to as the (e.g., maximum) depth of the tree (and / or tree search). The maximum number of N time steps can, for example, correspond to a maximum time horizon of 20 s. Alternatively or additionally, in one embodiment, the planning phase can comprise up to N=10, preferably up to N=5 time steps, and particularly preferably N=3 time steps.

[0043] Each time step can last between 1 and 3 seconds, preferably between 1 and 2.5 seconds. Alternatively or additionally, a planning phase can last up to 20 seconds, for example, 10 seconds, preferably up to 8 seconds.

[0044] The repeated (and / or iterative) prediction and reactive planning (preferably including the respective determination of the value of the quality function) based on the sensor data can also be referred to as tree search and / or "unrolling the tree." The starting point of the tree can be the current state. Alternatively or additionally, the "unrolling the tree" and / or tree search can take less time per planning phase than one time step, for example, 100 ms.

[0045] The planning phase may include a period of "unrolling the tree" and / or N-fold repeated (and / or iterative) prediction and planning. For example, at the end of the planning phase, a planned sequence of actions may be provided. The planned sequence of actions may be represented by a "branch of the tree search."

[0046] The method may further comprise a step of providing the (particularly iteratively) reactively planned actions of the at least partially automated vehicle (particularly with the best determined value of the quality function). Providing may comprise transmitting the action to a closed-loop control component and / or control component of the at least partially automated vehicle in order to execute one of the actions.

[0047] The received sensor data may include scene-specific data and / or be received from at least one sensor. The scene-specific data may include image data (in particular video data), radar data, LiDAR data, ultrasound data, motion data, thermal imaging data, navigation data, weather data, and / or traffic reports.

[0048] The at least one sensor can be arranged on the at least partially automated vehicle. Alternatively or additionally, the at least one sensor can be arranged outside the at least partially automated vehicle, in particular in the surroundings of the at least partially automated vehicle, for example at an intersection toward which the at least partially automated vehicle is moving. The sensor data, in particular for one or more sensors arranged outside the at least partially automated vehicle and / or sensor data comprising traffic reports, can be received wirelessly.

[0049] Furthermore, map data can be stored in the at least partially automated vehicle and / or (e.g. updates of) map data can be received as sensor data.

[0050] Using sensor data, the environment of the at least partially automated vehicle can be monitored as comprehensively and quasi-continuously as possible. This constantly updated knowledge of the environment allows the at least partially automated vehicle's journey to be reliably planned, thus increasing driving safety.

[0051] The method may further comprise a step of extracting the additional road users from the received sensor data. Alternatively or additionally, the method may further comprise a step of representing one or selected extracted additional road users by a respective road user agent. The prediction may be performed for each or selected road user agents.

[0052] By representing a road user as an agent, independent prediction can be realized. Alternatively or additionally, by representing two or more road users, mutual conditions can be predicted for their actions (e.g., for vehicles driving behind each other or an encounter at an intersection).

[0053] Each agent (e.g. per road user and / or for the at least partially automated vehicle) can include map data of the environment, an intelligent driver model (IDM) and / or a condition for compliance with traffic rules.

[0054] Predicting the development of the traffic scene may involve negotiation between the agents of other road users.

[0055] The predictive planning of the actions of the at least partially automated vehicle can be carried out by the agent of the at least partially automated vehicle, in particular based on negotiations with the agents of the other road users.

[0056] The IDM can include a continuous-time vehicle-following model for simulating road traffic. The IDM can describe the dynamics of the positions and velocities of individual vehicles. Accelerations can be subject to conditions for a clear road and / or for a road occupied by other road users. In particular, a (e.g., minimum) distance to other road users can be taken into account in the evolution of the movement.

[0057] The steps of prediction and / or reactive planning can involve machine learning (ML), particularly deep learning (DL). For example, a (particularly non-recurrent) multi-layer perceptron (MLP) can perform the prediction and / or planning (e.g., one MLP per agent).

[0058] The method may further comprise a step of extracting dynamic traffic regulations from the received sensor data (in particular with regard to one or more traffic light cycles). Alternatively or additionally, the method may further comprise a step of representing the extracted dynamic traffic regulations (in particular the one or more traffic light cycles) by one or more traffic regulation agents. The prediction may be performed for the one or more traffic regulation agents.

[0059] In addition to predicting other road users, dynamic traffic control (e.g., traffic light and / or barrier control at level crossings, tunnels, and / or bridges) can be incorporated into the planning of the actions of the at least partially automated vehicle. This allows, in particular, speeds and / or acceleration to be optimized, and / or alternative routes to be selected for the at least partially automated vehicle (e.g., if a long closing time of a barrier-controlled level crossing has been predicted).

[0060] Prediction can involve negotiation between agents per road user and / or per dynamic traffic control system. This can, for example, improve prediction depending on traffic light phases.

[0061] Planning can involve negotiations between the agent of the at least partially automated vehicle and the agents of the road users and / or the agents of the dynamic traffic control system. This can, for example, improve planning based on future traffic light phases.

[0062] Predicting the actions of road users in the environment of the at least partially automated vehicle and optionally predicting dynamic traffic regulation can be performed using scene consistency and / or stochastically sampled methods. In particular, scene consistency can be based on negotiations between the road user agents and, optionally, the traffic regulation agents.

[0063] Scene consistency can include compliance with traffic rules (e.g., the IDM) by one or every road user. Alternatively or additionally, scene consistency can include behavior coordinated between road users (e.g., negotiated between the respective agents) (e.g., actions where no accident or local conflict is expected). This can limit the data set conditioning for planning. For example, traffic scene developments in which two other road users are simultaneously in the same location can be excluded from the conditioning data set.

[0064] The stochastic sampling may comprise multiple (and / or n-fold) execution (and / or repetition) of future time steps of the prediction, in particular between five and fifty and further in particular between ten and thirty times.

[0065] Alternatively or additionally, a particular stochastic distribution of a prediction component may not be sampled.

[0066] Alternatively or additionally, the prediction, in particular stochastic sampling, can comprise a predetermined number of a maximum of N future time steps. For example, the number of time steps and / or predictions per (in particular combination of) road users, dynamic traffic control, and / or per (in particular combination of) agent(s) can be three.

[0067] Using stochastic sampling, the behavior of other road users (and / or dynamic traffic control) can be predicted according to a probability distribution.

[0068] Prediction may involve encoding and / or compressing data. For example, prediction of a road user (and / or agent) may be limited to 1 gigabyte (GB).

[0069] By means of the condition on scene consistency and / or the restriction on stochastic sampling, it can be ensured that the method is limited to collision-free predictions of the traffic scene and planning of the actions of the at least partially automated vehicle.

[0070] The planning component can be conditioned on the (in particular the entire) determined stochastic distribution of the prediction component and / or react (e.g. reactively) to the (in particular the entire) determined stochastic distribution.

[0071] Alternatively or additionally, the planning component (especially in a deterministic process) can plan reactively for a set of possible futures (e.g., the traffic scene and / or, in particular, the other road users). This can enable contingency planning. Alternatively or additionally, an action of the at least partially automated vehicle can be valid for a set and / or distribution of possible futures.

[0072] The prediction of the actions of other road users can be limited to a set of a maximum of K developments of the traffic scene in a time step, starting from the previous time step. Optionally, K can be a maximum of ten, preferably a maximum of five, and even more preferably a maximum of three.

[0073] The developments of the traffic scene can include combinations of the actions of all other road users. Alternatively or additionally, the developments of the traffic scene can include inactivity (e.g., of the agents) of the road users (e.g., of the other road users and / or of the at least partially automated vehicle). For example, a development of a traffic scene can include a ball rolling onto a road.

[0074] The restriction to a small set of developments of the traffic scene (and / or predicted combinations of the actions of all other road users) enables a fast (especially for real-time applications while driving) rollout of the tree.

[0075] The reactive planning of actions of the at least partially automated vehicle can be limited to a set of a maximum of L actions in a time step, starting from the previous time step. Alternatively or additionally, a stochastic distribution of a maximum of L actions can be sampled. Optionally, L can be a maximum of ten, preferably a maximum of five, and even more preferably a maximum of three.

[0076] The restriction to a small set of actions of the at least partially automated vehicle enables a fast (especially for real-time applications during driving) rollout of the tree.

[0077] The predeterminable maximum number N of repetitions of prediction and reactive planning may be at most ten, preferably five, and particularly preferably three.

[0078] Due to the small predeterminable number N of repetitions, the tree can be rolled out so quickly that it meets safety standards for time-critical planning of automated driving.

[0079] In the prediction step and / or the reactive planning step, a value of the quality function can be assigned to a possible future state of the traffic scene. Optionally, the reactive planning can include optimizing the quality function of the possible future state of the traffic scene.

[0080] The quality function (also known as a quality measure and / or quality metric) can assign a value to an achieved state of the traffic scene (e.g., in tree development). Alternatively or additionally, the quality function can estimate a value for one or more not yet predicted and reactively planned time steps (and / or a not yet deployed future of the tree).

[0081] Alternatively or additionally, the quality function may include a value function and / or an actor-critic algorithm.

[0082] The quality function can consider several attributes, in particular driving safety and / or driving comfort. The quality function can, for example, assign a high value to a safe action (and / or a safe planned trajectory). Furthermore, the quality function can, for example, evaluate a distance between road users (in particular between the at least partially automated vehicle and other road users), a distance to boundaries (e.g., a road edge and / or a center line), compliance with a minimum distance and / or a safety distance, freedom from collisions, and / or compliance with traffic regulations.

[0083] The quality function can be implemented as a classical algorithm or by means of a data-driven method (in particular as a machine learning method).

[0084] The quality function is executed at least after the last iteration of the prediction and reactive planning steps, i.e., after the last hierarchy level of the tree. Alternatively, the quality function can also be executed after each tree level to determine where the tree should be further explored. The tree level and / or hierarchy level can each represent a time step in the tree's rollout.

[0085] The quality function can also be referred to as a cost function. Optimizing the quality function can involve maximizing a value of the quality function. Alternatively or additionally, optimizing the cost function can involve minimizing costs.

[0086] The quality function of the prediction can be determined in relation to the actions of other road users (e.g. individually for each road user and / or as an assessment of an overall situation).

[0087] The quality function can be used to assess the consistency of the predicted and / or planned traffic scene. Overlapping agent footprints (and / or overlapping bounding boxes) can be assigned a lower quality function value. Alternatively or additionally, a lower quality function value can be assigned to future states in which agents are predicted to be at least partially outside or adjacent to a lane.

[0088] Using the value of the quality function, planned actions of the at least partially automated vehicle can be ordered (e.g., prioritized). Alternatively or additionally, a small number (and / or a single) optimized actions can be selected from a large number of planned actions. This ensures rapid execution of the prediction and / or planning, which is relevant to the safety of the at least partially automated operation. Alternatively or additionally, the tree search can be restricted to the optimal branch with progressive time steps. This can save computing time, memory space, and / or processor resources.

[0089] The quality function (and / or the value of the quality function) can consider at least one operating parameter regarding the travel of the at least partially automated vehicle. In particular, the at least one operating parameter can include energy consumption, a number and / or magnitude of accelerations, a number and / or magnitude of jolts, a time period until reaching a planned destination, a number of lane changes, driving outside of designated lanes, a probability of violating traffic regulations, and / or an accident probability. By considering the at least one operating parameter in the quality function, an energy-saving and / or vehicle component-friendly driving style can be favored.

[0090] Optimizing (especially maximizing) the quality function can involve minimizing the probability of an accident and / or the probability of violating traffic regulations. Alternatively or additionally, maneuvers (especially dangerous) such as frequent lane changes, lane changes across multiple lanes (e.g., on a multi-lane highway), and / or driving off-road or beyond designated lanes can be avoided by optimizing the quality function.

[0091] Alternatively or additionally, the optimized (in particular maximized) quality function can be used to ensure that the at least partially automated vehicle is operated in an energy-efficient (also: energy-saving) manner and / or in a way that protects vehicle components. For example, low-jerk and / or low-acceleration journeys can extend the service life (e.g. of parts) of the braking system of the at least partially automated vehicle. Alternatively or additionally, driving comfort for vehicle occupants can be increased by minimizing the number and / or intensity of jerks and / or accelerations. Alternatively or additionally, a travel time (and / or a period of time until reaching a destination) can be optimized. For this purpose, a longer route can be selected, for example (based on traffic reports received in the sensor data).

[0092] The method may further comprise a step of selecting the reactively planned actions for the at least partially automated vehicle with the highest determined value of the quality function within a set of the states of the traffic scene achieved by the repeated prediction and reactive planning.

[0093] The selected reactively planned actions can be forwarded to the control and / or control component of the at least partially automated vehicle for execution.

[0094] The selection of actions with the highest value of the quality function can enable a particularly safe, energy-efficient and / or vehicle component-friendly automated operation of the at least partially automated vehicle.

[0095] The tree search in the reactive planning (and / or predicting) step may comprise selecting an action of the planning component of the at least partially automated vehicle (and / or selecting actions of the prediction component of the other road users). The selected action may comprise a new action and / or a deep exploration of an already known and / or selected action. Furthermore, the tree search in the planning (and / or predicting) step may comprise expanding states of the at least partially automated vehicle based on the selected action of the planning component of the at least partially automated vehicle (and / or expanding states of the other road users based on the selected actions of the prediction component).

[0096] An expansion (technically also: expansion and / or time evolution) of the states can include an assignment of states to future time steps.

[0097] The planning component of the at least partially automated vehicle can be assigned to an actuator (e.g. steering and / or speed).

[0098] A tree search (TS) can encompass nodes and connections (also known as branches) between nodes in a hierarchical arrangement. The tree search typically starts from the tree root, a single node at one (e.g., highest) hierarchy level, with branches to nodes of the adjacent (e.g., next lowest) hierarchy level. The connections and / or branches can represent content-related assignments between the data structures assigned to the respective nodes. Alternatively or additionally, the connections and / or branches can represent temporal orderings.

[0099] The nodes of the tree search for providing the planned actions for the at least partially automated vehicle can represent future states in which the actions of the at least partially automated vehicle are conditioned by future developments in the traffic scene (in particular, the future actions of other road users and / or developments in dynamic traffic regulations). In particular, each hierarchy level of the tree search nodes can be assigned to a time step (e.g., the only node in the first, e.g., highest, hierarchy level to time step t, and the nodes in the next, e.g., next-lowest, hierarchy level to time step t+1).

[0100] According to the method for planning actions for an at least partially automated vehicle, the prediction of the development of the traffic scene (especially before, e.g., complete construction of the tree) can be incorporated into the (especially reactive) planning of the actions for the at least partially automated vehicle. This can improve and / or accelerate the planning process (especially compared to conventional tree searches, in which the prediction of the development of the traffic scene takes place with a delay).

[0101] A combination of the prediction of the development of the traffic scene (in particular comprehensively predicted actions of other road users), for example assigned to a node of the tree search, may deviate from a Cartesian product (e.g. due to scene consistency).

[0102] A conventional tree search can be extended by simulating and / or backpropagating values ​​of the quality function.

[0103] The tree search (TS) can be implemented as a Monte Carlo tree search (MCTS). The fundamentals of MCTS are described, for example, in [3], the content of which is hereby incorporated by reference. The MCTS can further comprise, in the planning step, simulating a value of a merit function for each predicted (and / or expanded) state. The MCTS can further comprise backpropagating and / or updating values ​​of the merit function of the previous states from the values ​​of the merit function of the expanded states.

[0104] In technical terms, backward propagation can also be referred to as backpropagation.

[0105] The MCTS can involve exploring new states and / or evaluating previously visited (and / or known) states. Typically, the MCTS is unbalanced. A statistical or heuristic function, particularly the quality function, can determine which tree regions are examined more closely.

[0106] The runtime of the tree search, especially the MCTS, can be less than or equal to 100 milliseconds (100 ms). This enables real-time planning of automated operations.

[0107] According to one aspect of the device, a computing device for planning actions of an at least partially automated vehicle is provided. The computing device comprises a perception component configured to receive sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step. The computing device further comprises a prediction component configured to predict actions of other road users in the traffic scene in a second time step immediately following the first time step based on the received sensor data. The computing device further comprises a planning component configured to reactively plan actions of the at least partially automated vehicle for the second time step.The reactive planning is conditioned on the predicted actions of the other road users in the traffic scene in the second time step. The prediction and the reactive planning for a further time step (in particular the immediately following one) are repeated iteratively a maximum of a predeterminable number of N times per planning phase. The computing device further comprises a quality determination component configured to determine (in particular at least after a maximum of N repetitions of the prediction and reactive planning) a value of a quality function for each state of the traffic scene achieved by the repeated prediction and reactive planning.

[0108] The computing device can be embodied in a planning module that exchanges data with a prediction module and a perception module, wherein the planning module is intended for planning actions of the at least partially automated vehicle.

[0109] Optionally, the computing device may further comprise an interface for providing the - in particular predictive - planned actions for the at least partially automated vehicle, in particular to a control component and / or control component.

[0110] Furthermore, the computing device may optionally comprise a selection component which is designed to select the planned action for the at least partially automated vehicle with the highest value of the quality measure within the set of planned (and / or provided) actions.

[0111] The computing device may further be configured to carry out the method according to the first method aspect and / or comprise features disclosed for the first method aspect.

[0112] According to a second method aspect, a method is provided for training a computing device (e.g., the computing device according to the device aspect) for iteratively and reactively planning actions of an at least partially automated vehicle. The method comprises a step of providing a training data set comprising sensor data of a traffic scene around the at least partially automated vehicle and at least one associated planning result. The method further comprises a step of training the computing device using the sensor data provided in the training data set and a loss function (or reward function) that compares the at least one associated planning result with an output of the planning component (and optionally an output of the prediction component) after the predeterminable number of maximum N times of iterative prediction and planning.

[0113] The training data set can further include the actions of the at least partially automated vehicle and other road users (e.g., each represented by an agent). Alternatively or additionally, the actions of the at least partially automated vehicle and other road users and / or the associated planning result (e.g.,) can be implicitly encoded and / or determined by the sensor data.

[0114] The training data set may alternatively or additionally comprise values ​​of a quality function, for example assigned to time steps of the traffic scene and / or to actions of the at least partially automated vehicle and / or the other road users.

[0115] The sensor data and the at least one associated planning result can comprise data from a real trip (e.g., performed by an expert) and / or a trip in a simulator. Alternatively or additionally, the at least one associated planning result can comprise ground truth.

[0116] The training can be supervised (technical term: supervised learning).

[0117] The loss function (or reward function) can compare the actions of the at least partially automated vehicle and other road users, optionally included in the training dataset, with the outputs of the planning component and the prediction component. Alternatively or additionally, an output of the quality determination component can be compared with the values ​​of the quality function, optionally included in the training dataset.

[0118] An L1 loss function, an L2 loss function (and / or a mean squared error, MSE) and / or a Huber loss function can be used as the loss function, especially for actions that correspond to trajectories.

[0119] Alternatively or additionally, for binary actions (e.g. honking or not honking, and / or blinking or not blinking), the loss function can be a classification loss function and / or a binary cross entropy loss function.

[0120] Furthermore, the MSE loss function can be used alternatively or additionally for the quality function, especially in supervised learning (also: supervised training).

[0121] Alternatively or additionally, the computing device can be trained by reinforcement learning (RL). The reward function can be a weighted or unweighted sum of several attributes. For the specification options of a reward function, reference is made to [4], the content of which is hereby incorporated by reference. The reward can correspond to the negative cost and / or the value of the quality of a time step, e.g., if one is in a state z t find, action a t (and / or ate,Ato ) and thus enters the state z t+1In RL, the goal can be to maximize the total reward when executing (e.g., rolling out the tree) the policy (e.g., the prediction component and / or planning component, and / or the quality function), which is equivalent to minimizing the total costs. In RL, the value function can be an estimate of the summed and, with time (e.g., in time steps), low-weighted (and / or discounted) rewards. This can be learned, e.g., based on the rewards obtained in a simulation (as in actor-critic algorithms in RL), or manually defined costs for trajectories (in particular, according to supervised learning), e.g., when in a state z t find, action a t (and / or ate,Ato ) and thus enters the state z t+1 comes.

[0122] Alternatively or additionally, the computing device can be trained using an end-to-end learning process.

[0123] The computing device may comprise a deep learning (DL) architecture, for example, a multi-layer perceptron (MLP) per component. Weights of the architecture can be determined through training.

[0124] Alternatively or additionally, each component (e.g. individually or jointly) can be pre-trained (e.g. using known data sets) and / or post-trained (e.g. using specially selected and / or simulated data sets).

[0125] According to one system aspect, a system for planning actions of an at least partially automated vehicle is provided. The system comprises at least one environmental sensor. The environmental sensor is configured to acquire sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step. The system further comprises a computing device according to the device aspect, which can be trained according to the second method aspect. The perception component of the computing device is configured to receive the sensor data from the at least one environmental sensor.

[0126] According to a further aspect, a computer program product is provided with program elements that cause a computing device to execute the steps of the method for planning actions of an at least partially automated vehicle, and / or the method for training a computing device for iteratively reactively planning actions of an at least partially automated vehicle, according to the first and / or second method aspect when the program elements are loaded into a memory of the computing device.

[0127] According to yet another aspect, a computer-readable medium is provided on which program elements are stored that can be read and executed by a computing device to carry out steps of the method for iteratively reactively planning actions of an at least partially automated vehicle, and / or the method for training a computing device for iteratively reactively planning actions of an at least partially automated vehicle, according to the first and / or the second method aspect when the program elements are executed by the computing device. Short description of the drawings Fig. 1 is a flowchart of a method for planning actions of an at least partially automated vehicle in a planning phase according to a preferred embodiment. Fig. 2 is an overview of the structure and design of a computing device for planning actions of an at least partially automated vehicle in a planning phase according to a preferred embodiment. Fig. 3 is a flowchart of a method for training the computing device for iteratively reactive planning of actions of an at least partially automated vehicle according to a preferred embodiment. Fig. Figure 4 shows schematically a difference in the probability distribution of planned actions according to a conventional tree search and with the reactive planning of the method of Fig. 1. Fig. Figure 5 schematically shows the four phases of a Monte Carlo tree search. Fig. 6A and Fig. 6B show schematically a comparison of a rolling out of a conventional tree and a rolling out by means of the method of Fig. 1. Fig. 7A, Fig. 7B, Fig. 7C and Fig. 7D show details of the training procedure of the Fig. 3. Detailed description

[0128] Fig. 1 schematically shows a flowchart of a computer-implemented method 100 for planning actions of an at least partially automated vehicle in a planning phase.

[0129] The method 100 comprises a step S102 of receiving sensor data that represents an actual state of a traffic scene around the at least partially automated vehicle in a first time step. The method 100 further comprises a step S104 of predicting actions of other road users in the traffic scene in a second time step immediately following the first time step on the basis of the received sensor data (e.g., by means of a prediction component). The method 100 further comprises a step S106 of reactively planning actions of the at least partially automated vehicle for the second time step (e.g., by means of a planning component). The reactive planning S106 is conditioned on the predicted S104 actions of the other road users in the traffic scene in the second time step. The prediction S104 (e.g., by means of the prediction component) and the reactive planning S106 (e.g.,The planning components (predicted by the planning component) are repeated for a further time step (in particular the immediately following one), namely a maximum of N predeterminable iterative times per planning phase. At least after the maximum of N repetitions, a value of a quality function is determined for each state of the traffic scene achieved by the repeated prediction S104 and reactive planning S106.

[0130] Optionally, the method 100 includes a step S103-A of extracting the additional road users from the received S102 sensor data. Alternatively or additionally, the method 100 may include a step S103-B of representing one or selected extracted S103-A additional road users by a road user agent. The prediction S104 may be performed for each or for selected road user agents.

[0131] Further optionally, the method 100 comprises a step S103-C of extracting dynamic traffic regulations (in particular with regard to one or more traffic light cycles) from the received S102 sensor data. Alternatively or additionally, the method 100 may comprise a step S103-D of representing the extracted S103-C dynamic traffic regulations (in particular the one or more traffic light cycles) by one or more traffic regulation agents. The prediction S104 may be performed for the one or more traffic regulation agents.

[0132] Even further optionally, the method 100 comprises a step S108 of selecting the reactively planned S106 actions for the at least partially automated vehicle with the highest determined value of the quality function within a set of the states of the traffic scene achieved by the repeated prediction S104 and reactive planning S106.

[0133] The selected S108 reactively planned S106 actions can be provided to a control and / or control component of the at least partially automated vehicle for execution.

[0134] Fig. 2 schematically shows a structure of a computing device 200 for planning actions of an at least partially automated vehicle. The computing device 200 comprises a perception component 202 configured to receive sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step. The computing device 200 further comprises a prediction component 204 configured to predict actions of other road users in the traffic scene in a second time step immediately following the first time step based on the received sensor data. The computing device 200 further comprises a planning component 206 configured to reactively plan actions of the at least partially automated vehicle for the second time step.The reactive planning is conditioned on the predicted actions of the other road users in the traffic scene in the second time step. The prediction and the reactive planning are repeated for a further time step, in particular the immediately following one, a maximum of a predeterminable number of N iterative times per planning phase. The computing device 200 further comprises a quality determination component 207, which is configured to determine, at least after a maximum of N repetitions of the prediction and reactive planning, a value of a quality function for each state of the traffic scene achieved by the repeated prediction and reactive planning.

[0135] Optionally, the computing device 200 comprises a road user extraction component 203-A configured to extract the additional road users from the received sensor data. Alternatively or additionally, the computing device may comprise a road user representation component 203-B configured to represent one or selected additional road users through a road user agent. The prediction may be performed for each or selected road user agents.

[0136] Further optionally, the computing device 200 comprises a traffic regulation extraction component 203-C, which is configured to extract dynamic traffic regulations (in particular with regard to one or more traffic light cycles) from the received sensor data. Alternatively or additionally, the computing device 200 may comprise a traffic regulation representation component 203-D, which is configured to represent the extracted dynamic traffic regulations (in particular of the one or more traffic light cycles) by one or more traffic regulation agents. The prediction may be performed for the one or more traffic regulation agents.

[0137] Even further optionally, the computing device 200 comprises a selection component 208 which is designed to select the reactively planned actions for the at least partially automated vehicle with the highest determined value of the quality function within a set of the states of the traffic scene achieved by the repeated prediction and reactive planning.

[0138] The perception component 202 can be implemented by an input-output interface 210 (e.g., in combination with a processor 212).

[0139] The prediction component 204, the planning component 206, the quality determination component 207 and / or the optional selection component 208 can be implemented by a processor 212.

[0140] The optional road user extraction component 203-A, the optional road user representation component 203-B, the optional traffic regulation extraction component 203-C and / or the optional traffic regulation representation component 203-D can also be implemented by the one (in particular the same) processor 212 and / or (not in Fig. 2) may be arranged in the perception component 202.

[0141] The computing device 200 may further include a memory 214.

[0142] The computing device 200 may be configured to carry out the method 100.

[0143] Fig. 3 schematically shows a flowchart of a method 300 for training a computing device (e.g., the computing device 200 of Fig. 2) for the iterative reactive planning of actions of an at least partially automated vehicle.

[0144] The training method 300 comprises a step S302 of providing a training data set comprising sensor data of a traffic scene around the at least partially automated vehicle and at least one associated planning result. The training method 300 further comprises a step S304 of training the computing device 200 based on the sensor data provided in the training data set and a loss function (or reward function) that compares the at least one associated planning result with an output of the planning component 206, and optionally an output of the prediction component 204, after the predeterminable number of maximum N times of iterative prediction and planning.

[0145] The technique (e.g., comprising the method 100, the computing device 200 and / or the method 300) for planning actions of an at least partially automated vehicle (in particular in a planning phase) can also be referred to as prediction-conditioned tree search for automated driving.

[0146] The technique can modify conventional tree search-based methods (e.g., MCTS) for selecting the next action or trajectory for an at least partially automated vehicle by incorporating predictions into the planning process (especially in the same time step) rather than during tree construction.

[0147] The technique can improve the quality determination and thus the selection of the planned next action or trajectory of the at least partially automated vehicle in tree-search-based planning components. This is achieved by the planning component considering predictions from the prediction component (especially in the same time step). By considering the predictions in the planning component, the future evolution of the traffic scene is taken into account in the planning, leading to an improved quality determination of future states.

[0148] According to the technique (e.g., comprising method 100, computing device 200, and / or method 300), the planning component of the at least partially automated vehicle takes into account the prediction of the prediction component (e.g., in the same time step). The nodes of the tree search represent future states in which the action or trajectory of the at least partially automated vehicle is conditioned on the future action of the other agents. This modifies the conventional tree search method such that, instead of a Cartesian product of the possible actions or trajectories of the at least partially automated vehicle and the other agents (and / or road users), the actions or trajectories of the other agents and / or road users are first predicted, upon which the planning of the at least partially automated vehicle is then conditioned.

[0149] The sequence of prediction and reactive planning results in an improved quality determination and thus an improved selection of the planned action or trajectory of the at least partially automated vehicle, since in each step of the tree search, the predictive behavior of the at least partially automated vehicle is evaluated by conditioning the planning component on the prediction of the prediction component.

[0150] According to one embodiment, a tree search in the planning application is used to select the best current action or trajectory based on the anticipated quality of determined future states. The technique is described using an MCTS as an example, in which the best action is selected for the at least partially automated vehicle. However, the technique is not limited to MCTS, but can be applied to any tree search.

[0151] The following components and / or notations can be used for an MCTS: State of the scene at time t : Z t ; • Action of the partially automated vehicle at the time t:ate (e.g. with “e” for “ego” and / or ego vehicle); • Set of actions of all M agents in the scene at time t : Ato={ato1,ato2,…,atoM} (e.g. with “o” for “other”) • Prediction component q, which predicts (e.g. K) possible future actions of all M agents of the traffic scene (short: scene): q(Zt,Ato,ate)={A1t+1o,A2t+1o,…,Akt+1o} ◯ OB d. A. it can be assumed that the prediction component predicts the future actions of all agents in the scene in a scene-consistent manner. ◯ OB d. A. the prediction component can also represent a stochastic function from which K possible actions are sampled. • Quality function V (and / or cost function), which assigns a scalar value k to each state: V(Z t ) = k; • State prediction function H (and / or determination function): H(Zt,At+1o,at+1e)=Zt+1; • Planning component p, which includes the current state of the scene, as well as previous actions, in order to plan L possible actions. According to the technique (e.g., comprising the planning method 100, the computing device 200 and / or the training method 300), additional predicted actions of the other agents At+1o included in order to realize reactive planning (also: forward-looking planning). ◯ p(Zt,Ato,ate)={a1t+1e,a2t+1e,…,aLt+1e} (especially conventional) ◯ p(Zt,Ato,At+1o,ate)={a1t+1e,a2t+1e,…,aLt+1e} (especially according to the technology) ◯ OB d. A. the planning component can also represent a stochastic function from which L possible actions are sampled.

[0152] Fig. 4 shows, by way of example, a distribution of possible actions of the at least partially automated vehicle without conditions on the predicted actions of the other agents at reference numeral 406 and with conditions on the predicted actions of the other agents at reference numeral 408. The horizontal axis at reference numeral 402 schematically shows a distribution of actions of the at least partially automated vehicle. The vertical axis at reference numeral 404 schematically shows a probability with which the action is planned.

[0153] Because the behavior of the at least partially automated vehicle receives information about the likely future actions of the other agents, it can adapt its behavior accordingly in order to avoid conflicting with the actions of the other agents.

[0154] As in Fig. 5, an MCTS comprises four phases (and / or steps) which are repeated several times (e.g. n times), as shown schematically at reference numeral 510: - Selection (and / or selection) at reference numeral 502: An action of the planning component is selected (e.g., as schematically shown by the arrows pointing in the tree development). During the tree search, a balance is struck between exploring new actions and deeper exploration of already selected actions and thus subsequent states. - Expansion at reference numeral 504: The following states of the selected action are expanded (e.g. as shown schematically by means of the additional lower node). - Simulation at reference number 506: For each visited subsequent state (also: child node), the quality of the expanded state is determined with the help of the quality function (e.g. as shown schematically by the wavy line of the lower node). - Backpropagation at reference numeral 508: The quality of the parent nodes and / or parent states (also: previous states) is calculated or updated from the quality of the child nodes (and / or child states) (e.g. as shown schematically by means of the arrows directed against the tree development direction).

[0155] Fig. 6A and Fig. 6B schematically shows an abstract visualization comparison, with exemplary two steps of a tree search. Fig. 6A shows a conventional tree search in which the selection (and / or planning) of the action of the at least partially automated vehicle at reference numeral 606 is not conditioned on the prediction of the other agents (and / or other road users) at reference numeral 604 (in particular in the same time step, e.g., t+1 or t+2). Fig. 6B shows a tree search according to the technique described here (e.g., comprising the planning method 100, the computing device 200, and / or the training method 300), in which the selection (and / or the planning) S106 is conditioned on the prediction S104 of the other agents (and / or other road users).

[0156] In Fig. 6A and Fig. 6B, three (and / or L=3) possible future actions of the at least partially automated vehicle are sampled by the planning component and two (and / or K=2) possible future scene-consistent (in particular combinations of) actions are sampled by the prediction component.

[0157] By conditioning the choice (and / or planning) of the action of the at least partially automated vehicle on the prediction of the other agents (and / or other road users), the determined qualities in the last step (e.g. step 2 and / or t+2 in Fig. 6B) Vt+2(Zt+2(at+2e(At+2o),…)) better, as they evaluate the predictive (and / or reactive) behavior of the at least partially automated vehicle. This improved quality determination also reaches the parent nodes (e.g., the nodes at t + 1) in the backpropagation and thus improves the choice of action in the previous steps (e.g., step 1 in Fig. 6B), since the choice (and / or planning) is made on the basis of the highest quality.

[0158] Fig. 7A, Fig. 7B, Fig. 7C and Fig. 7D schematically show details of the training method 300.

[0159] In Fig. 7A, reference numeral 710 schematically shows data of a traffic scene (in particular associated with several past time steps up to the current time step, t -h, ..., t0), which are summarized in a scene data set 702. The data set can comprise a real and / or simulated traffic scene. At reference numeral 704, the data set 702 is extended by a value of the quality function (also: quality value). At reference numeral 708, the thus obtained extended and / or annotated data set 708 is received and / or stored.

[0160] The data 720 (and / or the data set 702) may contain sensor data and / or environmental model data (e.g., trajectory data). Each action of the at least partially automated vehicle and of the other agents (and / or other road users) is and / or can be assigned a quality value. For each time step t -h , ... , t0 a quality value can and / or is assigned to the traffic scene.

[0161] In Fig. Figure 7B schematically shows an example of supervised training. The future qualities and actions (in particular of the at least partially automated vehicle and the other road users and / or other agents) 714 of the branch in the tree whose actions are closest to the ground truth (e.g., based on an MSE loss function) are compared with the ground truth in the loss function 716, and the result is used as a training signal for the computing device 200 (also: the network). The considered past horizon of the input data 710, which are received (and / or processed) by the perception component 202, as well as the planning horizon, are h-time steps long. o is the encoded initial traffic scene state.

[0162] In other words, the past data (e.g. at reference 710) on the left in Fig. 7B to the same training element (and / or sample) as in the future data (e.g. at reference numeral 714) on the right in Fig. 7B, for which the predicted and / or planned qualities and actions are compared.

[0163] In Fig. 7C schematically shows an exemplary sequence of training and inference within a time step (e.g., from t to t+1) in the iteratively spanned tree. The prediction component 204 predicts K scene-consistent (e.g., combinations of) actions of other agents, as schematically shown at reference numeral 720. As schematically shown at reference numeral 722, a quality value is output for each of the K predicted developments of the traffic scene (in particular, comprising the scene-consistent prediction of the actions of the other agents) (in particular for L=1 and / or no further branches in the reactive planning step S106). The entire output and / or the entire result after prediction and planning in time step t+1 further includes the planned actions of the at least partially automated vehicle and the quality values ​​of the planned actions (e.g.,with K×L planned actions of the at least partially automated vehicle, quality values ​​and states of the traffic scene for K branches in the prediction step S104 and / or K scene-consistent actions of the other agents and L subsequent branches in the reactive planning step S106 or L reactively planned actions of the at least partially automated vehicle).

[0164] In Fig.7D schematically shows an exemplary inner step i (in particular, a branch of the tree in time step t+1). The prediction of the scene-consistent actions of the other road users (and / or other agents) as well as the state of the traffic scene at time t are used as input data 722'. The prediction component 204 and the planning component 206 contribute to determining a value of the quality function, which is performed, for example, by the quality determination component 207. The exemplary output at reference numeral 728 includes a state of the traffic scene (in particular, comprising a scene-consistent prediction of the actions of the other agents) with a reactively planned action of the at least partially automated vehicle and a quality value.

[0165] The technology includes, in particular, a software solution, the proof of which can be provided by looking at the documentation (e.g. of partially automated driving). Zitierter Stand der Technik: [1] Schrittweiser et al., „Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model“, arXiv:1911.08265, 2019. [2] S. Mo, X. Pei and C. Wu, „Safe Reinforcement Learning for Autonomous Vehicle Using Monte Carlo Tree Search,“ in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6766-6773, July 2022, doi: 10.1109 / TITS.2021.3061627. [3] Lu, L.; Zhang, W.; Gu, X.; Ji, X.; Chen, J. HMCTS-OP: Hierarchical MCTS Based Online Planning in the Asymmetric Adversarial Environment. Symmetry 2020, 12, 719. https: / / doi.org / 10.3390 / sym12050719 [4] „Reward (Mis)Design for autonomous driving“, Knox W. et al, arXiv:2104.13906v2 [cs.LG] 11 Mar 2022 ZITATE ENTHALTEN IN DER BESCHREIBUNG

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Stepwise et al., “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model,” arXiv:1911.08265, 2019

[0165] S. Mo, X. Pei and C. Wu, "Safe Reinforcement Learning for Autonomous Vehicle Using Monte Carlo Tree Search," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6766-6773, July 2022, doi: 10.1109 / TITS.2021.3061627

[0165] Lu, L.; Zhang, W.; Gu, X.; Ji, X.; Chen, J. HMCTS-OP: Hierarchical MCTS Based Online Planning in the Asymmetric Adversarial Environment. Symmetry 2020, 12, 719. https: / / doi.org / 10.3390 / sym12050719

[0165] Reward (Mis)Design for autonomous driving“, Knox W. et al, arXiv:2104.13906v2 [cs.LG] 11 Mar 2022

[0165]

Claims

[1] Computer-implemented method (100) for planning actions of an at least partially automated vehicle in a planning phase, comprising the following method steps: - receiving (S102) sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step; - predicting (S104) actions of other road users in the traffic scene in a second time step immediately following the first time step on the basis of the received sensor data; - Reactive planning (S106) of actions of the at least partially automated vehicle for the second time step, wherein the reactive planning (S106) is conditioned on the predicted (S104) actions of the other road users in the traffic scene in the second time step; and - Repeating the prediction (S104) and the reactive planning (S106) for a further time step, in particular an immediately following one, wherein the prediction (S104) and the reactive planning (S106) are repeated iteratively a maximum of a predeterminable number of N times per planning phase; and wherein at least after the maximum of N repetitions, a value of a quality function is determined for each state of the traffic scene reached by the repeated prediction (S104) and reactive planning (S106). [2] The method (100) of claim 1, further comprising the steps of: - Extracting (S103-A) the other road users from the received (S102) sensor data; and - Representing (S103-B) one or selected extracted (S103-A) further road users by a road user agent; wherein the prediction (S104) is carried out for each or for selected road user agents. [3] Method (100) according to one of the preceding claims, further comprising at least one of the steps: - Extracting (S103-C) dynamic traffic regulations, in particular with regard to one or more traffic light circuits, from the received (S102) sensor data; and - Representing (S103-D) the extracted (S102-C) dynamic traffic regulations, in particular the one or more traffic light circuits, by one or more traffic regulation agents; wherein the prediction (S104) is carried out for the one or more traffic regulation agents. [4] Method (100) according to one of the preceding claims, wherein the prediction (S104) for actions of the road users in the environment of the at least partially automated vehicle and optionally the prediction (S104) of dynamic traffic regulations is carried out in a scene-consistent and / or stochastically sampled manner, in particular wherein the scene consistency is based on negotiations of the road user agents, and optionally the traffic regulation agents. [5] Method (100) according to one of the preceding claims, wherein the prediction (S104) of the actions of the further road users in a time step starting from the previous time step is limited to a set of a maximum of K developments of the traffic scene, optionally where K is at most ten, preferably at most five, and even more preferably at most three. [6] Method (100) according to one of the preceding claims, wherein the reactive planning (S106) of actions of the at least partially automated vehicle in a time step starting from the previous time step is limited to a set of a maximum of L actions, and / or wherein a stochastic distribution of a maximum of L actions is sampled, optionally wherein L is at most ten, preferably at most five, and even more preferably at most three. [7] Method (100) according to one of the preceding claims, wherein the predeterminable maximum number N of repetitions of the prediction and the reactive planning is at most ten, preferably five, and particularly preferably three. [8] Method (100) according to one of the preceding claims, wherein in the step of predicting (S104) and / or in the step of reactive planning (S016) a value of the quality function is assigned to a possible future state of the traffic scene, optionally wherein the reactive planning (S106) comprises optimizing the quality function of the possible future state of the traffic scene. [9] Method (100) according to one of the preceding claims, wherein the value of the quality function comprises at least one operating parameter with regard to the travel of the at least partially automated vehicle, in particular wherein the at least one operating parameter comprises an energy consumption, a number and / or magnitude of accelerations, a number and / or magnitude of jerks, a period of time until reaching a planned destination, a number of lane changes, driving outside of designated lanes, a probability of violating traffic regulations and / or an accident probability. [10] Method (100) according to one of the preceding claims, further comprising the step: - Selecting (S108) the reactively planned (S106) actions for the at least partially automated vehicle with the highest determined value of the quality function within a set of the states of the traffic scene achieved by the repeated prediction (S104) and reactive planning (S106). [11] Computing device (200) for planning actions of an at least partially automated vehicle, comprising: - a perception component (202) configured to receive sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step; - a prediction component (204) which is designed to predict actions of other road users in the traffic scene in a second time step immediately following the first time step on the basis of the received sensor data; - a planning component (206) designed to reactively plan actions of the at least partially automated vehicle for the second time step, wherein the reactive planning is conditioned on the predicted actions of the other road users in the traffic scene in the second time step; wherein the prediction and the reactive planning are repeated for a further, in particular immediately following, time step, wherein the prediction and the reactive planning are repeated iteratively a maximum of a predeterminable number of N times per planning phase; and - A quality determination component (207) which is designed to determine, at least after a maximum of N repetitions of the prediction and reactive planning, a value of a quality function for each state of the traffic scene reached by the repeated prediction and reactive planning. [12] Computing device (200) according to the directly preceding claim, wherein the computing device (200) is further configured to carry out the method according to any one of claims 2 to 10 and / or wherein the computing device (200) comprises features according to any one of claims 2 to 10. [13] Method (300) for training a computing device (200) according to claim 11 or 12 for iteratively reactively planning actions of an at least partially automated vehicle, comprising the method steps: - Providing (S302) a training data set comprising sensor data of a traffic scene around the at least partially automated vehicle and at least one associated planning result; and - Training (S304) the computing device (200) based on the sensor data provided in the training data set and a loss or reward function that compares the at least one associated planning result with an output of the planning component (206), and optionally an output of the prediction component (204), after the predeterminable number of maximum N times of iterative prediction and planning. [14] System for planning actions of an at least partially automated vehicle, comprising: - at least one environmental sensor designed to acquire sensor data representing an actual state of a traffic scene around the at least partially automated vehicle in a first time step; and - a computing device (200) according to claim 11 or 12, wherein the perception component (202) is configured to receive the sensor data from the at least one environmental sensor. [15] Computer program product with program elements that cause a computing device (200) to carry out the steps of the method (100) for planning actions of an at least partially automated vehicle, and / or the method (300) for training the computing device (200) for iteratively reactively planning actions of an at least partially automated vehicle, according to one of the preceding method claims, when the program elements are loaded into a memory of the computing device (200).

Images