Method for training the prediction of a trajectory of a motor vehicle along a roadway which has a specified course using a neural network

Abstract

The invention relates to a method for training the prediction of a trajectory (T*) of a motor vehicle (1) along a roadway (2) which has a specified course (3) using a neural network, according to which a loss function (L2) is calculated in order to characterize a deviation (Δ_T) of the trajectory (T*) predicted by the neural network from the actual trajectory (T). The loss function (L2) is calculated for at least one trajectory point (P1, P2, P3) of the actual trajectory (T) in a local coordinate system (K1, K2, K3), said trajectory point being based on the course (3) of the roadway (2).

Description

[0001]Method for training the prediction of a trajectory of a motor vehicle along a road having a predetermined course using a neural network. The present invention relates to a method for training the prediction of a trajectory of a motor vehicle along a road having a predetermined course using a neural network. The invention further relates to a deep learning system having such a neural network, which is set up / programmed to carry out this method. In addition, the invention relates to a computer program product and a data carrier for carrying out the method. Modern vehicle assistance systems for motor vehicles enable partially automated driving on a roadway. Corresponding driving strategies, which can be defined with the aid of said assistance systems, include the calculation of a trajectory along which the motor vehicle is to follow the course of the roadway to be traveled.In order to obtain such driving strategies – both for simulation and real-world driving – and to continuously improve them, it is known to use so-called deep learning systems with neural networks that can train said driving strategies. For training, a trajectory of the vehicle along the road predicted by the neural network is compared with a simulated trajectory or with a trajectory actually traveled by the vehicle, and the resulting deviation is determined in the form of a so-called loss function. Conventional methods use the so-called L2 loss to calculate the loss function. In this case, the deviation of the predicted trajectory from the simulated / real trajectory is determined by separately determining the X-deviation and the Y-deviation in a uniform, stationary, and typically Cartesian coordinate system with specified X- and Y-axes.This allows for different weighting of the X and Y deviations. The X and Y deviations are determined for all trajectory points of the predicted trajectory and the simulated / real trajectory in a uniform coordinate system, the so-called "world coordinate system." A disadvantage of this approach is that the course of the roadway is not taken into account when determining the loss function. Thus, it is not possible to determine, in particular, the magnitude of the determined deviation along the longitudinal direction of the roadway, typically in the direction of travel of the vehicle, and transverse to the longitudinal direction of the roadway, which is of considerable importance for automated driving on a roadway.It is therefore an object of the present invention, against the above background, to provide an improved method for training a neural network, in which the above-mentioned disadvantage is at least partially, preferably largely, particularly preferably completely, eliminated. This object is achieved by the subject matter of the independent patent claims. Preferred embodiments are the subject matter of the dependent patent claims. The basic idea of ​​the invention is therefore to calculate a loss function, which characterizes the deviation of a trajectory of a motor vehicle predicted by a neural network when driving along a roadway from an actual trajectory, not in a uniform world coordinate system, but in local coordinate systems dependent on the course of the roadway.This makes it possible to take into account the course of the roadway, in particular the directional course of the roadway, which usually does not extend exclusively in a straight line, when calculating the loss function. In particular, the use of local coordinate systems makes it easy to calculate the loss function component by component with an X component in the - local - longitudinal direction of the roadway and a Y component in the - local - transverse direction of the roadway. The individual local coordinate systems can be aligned differently and, in particular, rotated relative to one another, depending on the course of the roadway. In particular, the individual local coordinate systems can be positioned and aligned such that at each trajectory point the loss function can be split into a loss component in the longitudinal direction of the roadway and a loss component in the transverse direction of the roadway perpendicular to the longitudinal direction of the roadway.This allows a different weight to be assigned to the loss component in the longitudinal direction of the road for each trajectory point than to the loss component in the transverse direction. This proves to be particularly important and advantageous because knowledge of the deviation in the transverse direction is of great importance for vehicle assistance systems when driving on roads with two or more lanes, especially if the vehicle is to follow a selected lane of the road or if the vehicle assistance system is to assist or even automate a lane change. Therefore, when calculating the loss function, it is a decisive advantage if the Y component of the loss function in the transverse direction can be assigned a higher weight than the X component in the longitudinal direction of the road.Analogously, in the case of a planned stop of the motor vehicle, for example, at a traffic light, the X component can be assigned a higher weight than the Y component. The term "actual trajectory" in the present sense preferably refers to a trajectory according to "ground truth." This can be a trajectory actually traveled by a motor vehicle or a trajectory simulated by simulation. In the course of the method according to the invention, the trajectory point of the actual trajectory can be selected as the origin for the respective local coordinate system. According to this variant, it is also proposed to define an X coordinate axis of this local coordinate system such that it runs along a longitudinal direction of the roadway at the trajectory point of the actual trajectory.Accordingly, a Y-coordinate axis of this coordinate system should be selected such that it runs along a transverse direction of the road at the trajectory point, i.e., perpendicular to the longitudinal direction of the road. This allows the loss function to be calculated at each trajectory point with an X-component along the road course and a Y-component orthogonal to the road course. The method according to the invention serves to train the prediction of the trajectory of a motor vehicle along a road with a predetermined course using a neural network. According to the method, a loss function is calculated to characterize a deviation of the trajectory predicted by the neural network from the actual trajectory.According to the invention, the loss function is not calculated in a single, uniform world coordinate system, but for at least one trajectory point, preferably for several trajectory points, particularly preferably for all trajectory points, of the actual trajectory in local coordinate systems that depend on the course of the roadway. According to a preferred embodiment of the method according to the invention, it is therefore proposed to individually define a coordinate system for calculating the deviation or loss function for each trajectory point of the actual trajectory for which the deviation from the corresponding trajectory point is to be calculated. Particularly preferably, the origin of this coordinate system can be placed at the trajectory point of the actual trajectory.Furthermore, an X-coordinate of this coordinate system can be selected along the longitudinal direction of the roadway, i.e., along the direction along which the roadway runs at the trajectory point. Accordingly, a Y-coordinate of this coordinate system can be selected orthogonal to the longitudinal direction of the roadway, i.e., along a transverse direction of the roadway that extends orthogonal to the course of the longitudinal direction of the roadway at the trajectory point. Therefore, the coordinate system to be used to determine the deviation or the loss function is preferably individually defined for at least one trajectory point, preferably for several of the trajectory points, particularly preferably for all of the trajectory points. To calculate the deviation or loss function, a deviation of the at least one actual trajectory point from the associated predicted trajectory point is particularly expediently determined in the individual coordinate system.It is understood that both the actual and the predicted trajectory are each defined as a function of time. Thus, in the method presented here, or in the calculation of the deviation or loss function, trajectory points of the predicted trajectory are compared with trajectory points of the actual trajectory points at the same time. In a preferred embodiment, the actual trajectory comprises a plurality of trajectory points defined in a, preferably Cartesian, world coordinate system. In this embodiment, the desired loss function is calculated for at least two, preferably several, trajectory points of the actual trajectory. For this purpose, the loss function is calculated for each of these trajectory points in a local coordinate system assigned to the respective trajectory point of the actual trajectory.The local coordinate system is determined by the course of the roadway at the trajectory point. According to an advantageous development of the method according to the invention, the coordinate origin of the respective local coordinate system is selected as the trajectory point to which the relevant local coordinate system is assigned. In this development, an X-coordinate axis of the respective local coordinate system is selected as running away from the assigned trajectory point in a roadway longitudinal direction, along which the roadway extends at the trajectory point. Particularly preferably, a Y-coordinate axis of the local coordinate system can be selected as running away from the respective trajectory point in a roadway transverse direction, which extends orthogonally to the roadway longitudinal direction at this trajectory point.In another preferred embodiment, a local X-function component and a local Y-function component are calculated to calculate the loss function at the respective trajectory point. For this purpose, the X-function component is calculated as the deviation of the predicted trajectory from the actual trajectory along the longitudinal direction of the roadway, i.e., along the X-coordinate axis of the local coordinate system at this trajectory point, or at least as a function of this deviation.In addition, the Y-function component is calculated as the deviation of the predicted trajectory from the actual trajectory along the transverse direction of the roadway, i.e., along the Y-coordinate axis of the local coordinate system, at this trajectory point, or at least calculated as a function of this deviation. In a further preferred embodiment, to calculate the X-function component and the Y-function component, the deviation of the predicted trajectory from the actual trajectory along the X-axis and the Y-axis of the world coordinate system is first calculated. In this embodiment, the deviations calculated in the world coordinate system are transferred to the local coordinate system of the respective trajectory point of the actual trajectory by means of a rotation transformation.Particularly preferably, the rotational transformation can be carried out by a rotation about a rotation axis that extends perpendicular to both the X-axis and the Y-axis of the world coordinate system. The rotational transformation then takes place by a rotation angle that is an intermediate angle between the X-axis of the world coordinate system and the X-coordinate axis of the local coordinate system at the respective trajectory point of the actual trajectory. Analogously, the rotation angle can be defined as the intermediate angle between the Y-axis of the world coordinate system and the Y-coordinate axis of the local coordinate system at the respective trajectory point. In another preferred embodiment, the actual trajectory can be a trajectory actually traveled by the motor vehicle or a trajectory simulated by means of simulation.The invention further relates to a deep learning system comprising at least one neural network, which in turn is configured / programmed to carry out the above-described method according to the invention. The above-explained advantages of the method according to the invention are therefore transferred to the deep learning system according to the invention. Furthermore, the invention relates to a computer program product designed to carry out the method, in particular by means of the deep learning system. The computer program product contains instructions which, when the computer program product is executed by a computer system and / or by the deep learning system, cause the computer system and / or the deep learning system to carry out the method. The computer program product is preferably stored / stored on a memory comprising at least one non-volatile memory. The invention likewise comprises a computer-readable data carrier for carrying out the method.The data carrier comprises instructions which, when executed, cause a computer system and / or the deep learning system to carry out the method according to the invention explained above. Further important features and advantages of the invention emerge from the subclaims, from the drawings and from the associated description of the figures with reference to the drawings. It is understood that the features mentioned above and those to be explained below can be used not only in the respectively specified combination, but also in other combinations or on their own, without departing from the scope of the present invention. Preferred embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description, wherein like reference numerals refer to like or similar or functionally identical components. They show, each schematically: Fig.1 shows a roadway traveled by a motor vehicle, with the predicted and actual trajectory drawn therein. Fig. 2 shows a detailed illustration of Figure 1 in the area of ​​a specific trajectory point of the actual trajectory of the motor vehicle. Figure 1 shows a plan view of a roadway 2 having a predetermined course 3. The roadway 2 is traveled by a motor vehicle 1, which is to follow the course 3 of the roadway. The method according to the invention is used here. The method according to the invention is explained below by way of example with reference to Figure 1. The method according to the invention serves to train the prediction of a trajectory T* of the motor vehicle 1 along the roadway 2 using a neural network. According to the method, for the characterization orTo determine a deviation Δ_T of a trajectory T* of motor vehicle 1 predicted by the neural network from the actually traveled trajectory T, a loss function L2 is calculated for various trajectory points P1, P2, P3 of the actual trajectory T. However, the calculation is not performed – as in conventional methods – in a uniform Cartesian world coordinate system K0 with a uniform X-axis and Y-axis, but rather in local coordinate systems K1, K2, K3, which depend on the course 3 of roadway 2. The trajectory referred to as "actual trajectory T" in the context of the present invention can be a trajectory actually traveled by motor vehicle 1 or a trajectory simulated by means of simulation.As Figure 1 shows, the actual trajectory T of the motor vehicle 1 can therefore have a plurality of trajectory points, of which three trajectory points P1, P2, P3 are shown as examples in the example of Figure 1, at which the loss function L2 is to be calculated. The trajectory points P1, P2, P3 can be defined with respect to a uniform Cartesian world coordinate system K0 in the example scenario. In the method according to the invention, the desired loss function L2 is calculated individually for each of the trajectory points P1, P2, P3 of the actual trajectory T. For the calculation, for each of these trajectory points P1 to P3, the deviation of the trajectory T* predicted by the neural network from the actual trajectory T is calculated by determining the distance a1, a2, a3 of the respective trajectory point P1, P2, p3 of the actual trajectory T from the associated trajectory point P1*, P2*, P3* of the predicted trajectory T*.Here, both the actual trajectory T and the predicted trajectory T* are defined as a function of time, i.e., T=T(t) and T*=T*(t), respectively. Thus, when calculating the deviation or loss function L2, trajectory points P1*, P2*, P3* of the predicted trajectory T* are compared with trajectory points P1, P2, P3 of the actual trajectory points T at the same time, i.e., P1 = T(t1), P1* = T*(t1), P2 = T(t2), P2* = T*(t2), P3 = T(t3), P3* = T*(t3). In the method according to the invention, the loss function L2 is calculated using a local coordinate system K1, K2, K3 assigned to the respective trajectory point P1 to P3. The local coordinate system K1, K2, K3 is determined by the course 3 of the roadway 2 at the respective trajectory point P1, P2, P3. Thus, the individual local coordinate systems K1, K2, K3 can be defined as shown in Figure 1.their position and orientation differ from one another and may also be different from the world coordinate system K0. To define the individual local coordinate systems K1, K2, K3, the respective trajectory point P1, P2, P3 of the actual trajectory T is selected as the respective coordinate origin U1, U2, U3 of the relevant local coordinate system K1, K2, K3. An X-coordinate axis X1, X2, X3 of the respective local coordinate system K1, K2, K3 is selected such that it extends from the respective trajectory point P1, P2, P3 along the roadway longitudinal direction FLR1, FLR2, FLR3. In other words, the X-coordinate axis X1 of the local coordinate system K1 extends away from the trajectory point P1 along the roadway longitudinal direction FLR1 of roadway 2 at the trajectory point P1. The X-coordinate axis X2 of the local coordinate system K2 extends from the trajectory point P2 along the roadway longitudinal direction FLR2 of roadway 2 at the trajectory point P2.The X-coordinate axis X3 of the local coordinate system K3 extends from the trajectory point P3 along the longitudinal roadway direction FLR3 of roadway 2 at the trajectory point P3. Accordingly, the Y-coordinate axis Y1, Y2, Y3 of the respective local coordinate system K1, K2, K3 is selected such that it extends from the respective trajectory point P1, P2, P3 along a transverse roadway direction FQR1, FQR2, FQR3, which is orthogonal to the longitudinal roadway direction FLR1, FLR2, FLR3 at this trajectory point P1, P2, P3. Thus, the respective Y-coordinate axis Y1, Y2, Y3 also extends perpendicular to the associated X-coordinate axis X1, X2, X3. In other words, the Y-coordinate axis Y1 of the local coordinate system K1 extends from the trajectory point P1 along a road transverse direction FQR1, which extends orthogonally to the road longitudinal direction FLR1 at the trajectory point P1.The Y-coordinate axis Y2 of the local coordinate system K2 extends from the trajectory point P2 along a roadway transverse direction FQR2, which at the trajectory point P2 extends orthogonally to the roadway longitudinal direction FLR2. The Y-coordinate axis Y3 of the local coordinate system K3 extends from the trajectory point P3 along a roadway transverse direction FQR3, which at the trajectory point P3 extends orthogonally to the roadway longitudinal direction FLR3. To calculate the loss function L2 at the respective trajectory point P1, P2, P3, a local X-function component L2-X and a local Y-function component L2-Y of the loss function L2 are determined. This is explained below using Figure 2 for the trajectory point P3 as an example. Figure 2 is a detailed representation of Figure 1 in the area of ​​the trajectory point P3.The following explanations regarding trajectory point P3 also apply mutatis mutandis to the two other trajectory points P1, P2 shown in Figure 1, as well as to all trajectory points for which the method according to the invention is to be carried out as described here. According to Figure 2, the X-function component L2-X is determined as the deviation ΔT_FLR of the predicted trajectory T* from the actual trajectory T along the roadway longitudinal direction FLR, i.e., along the X-coordinate axis X3 of the local coordinate system K3 at trajectory point P3. Correspondingly, the Y-function component L2-Y is determined as the deviation ΔT_FQR of the predicted trajectory T* from the actual trajectory T along the roadway transverse direction FQR3, i.e., along the Y-coordinate axis Y3 of the local coordinate system K3 at trajectory point P3.For the concrete calculation of the X-function component L2_X and the Y-function component L2_Y, the deviation ΔT_X0 and ΔT_Y0 of the predicted T* from the actual trajectory T along the X-axis X0 and the Y-axis Y0 of the world coordinate system K0 can first be calculated at the trajectory point P3. The deviations ΔT_X0 and ΔT_Y0 calculated in the world coordinate system K0 are transferred to the local coordinate system K1, K2, K3 of the respective trajectory point P1, P2, P3 using a rotation transformation DT. The rotation transformation DT can be performed by rotating around a rotation axis that extends perpendicular to both the X-coordinate axis X0 and the Y-coordinate axis Y0 of the world coordinate system K0.The rotational transformation DT then occurs by a rotation angle, which is an intermediate angle θ between the X-coordinate axis X0 of the world coordinate system K0 and the X-coordinate axis X1, X2, X3 of the local coordinate system K1, K2, K3 at the respective trajectory point P1, P2, P3. The rotational transformation DT is described by the following rotation matrix: This results in the deviations ΔT_FLR, ΔT_QLR or the X-function components L2_X, L2_Y in the coordinate system K3: The distance a3 of the trajectory point P3 of the actual trajectory T to the assigned trajectory point P3* of the predicted trajectory T* is therefore calculated as ^3 = ^(ΔT_FLR ^ + ΔT_FQR ^ ) The loss function L2 can thus be calculated from the two functional components L2_X, L2_Y, i.e. L2 = f (L2_X, L2_Y). In particular, an individual weighting G_X, G_Y of the two functional components L2_X, L2_Y is conceivable, for example in the form ^2 = (^ ^^2 ^ + ^ ^ ^2 ^ ) or, alternatively, in the form ^2 = ( (^ ^ ^2 ^ ) ^ + (^ ^ ^2 ^ ) ^)^ / ^ conceivable. In both examples, the individual weighting of the loss function L2 at the trajectory point P3 in the longitudinal direction of the road (FLR) or perpendicular to it in the transverse direction (FQR), which is essential to the invention, is possible and also implemented. *******

Claims

Patent claims 1. Method for training the prediction of a trajectory (T*) of a motor vehicle (1) along a roadway (2) having a predetermined course (3) by means of a neural network, - according to which a loss function (L2) is determined to characterize a deviation (Δ_T) of the trajectory (T*) predicted by the neural network from the actual trajectory (T), - wherein the loss function (L2) is calculated for at least one trajectory point (P1, P2, P3), preferably for several trajectory points (P1, P2, P3), particularly preferably for all trajectory points (P1, P2, P3), of the actual trajectory (T) in a local coordinate system (K1, K2, K3) which depends on the course (3) of the roadway (2).

2. Method according to claim 1, characterized in that for calculating the deviation (Δ_T) orA (local) coordinate system (K1, K2, K3) is individually defined for a loss function (L2) for at least one, preferably for each, trajectory point (P1, P2, P3) of the actual trajectory (T).

3. The method according to claim 1 or 2, characterized in that, to calculate the deviation (Δ_T) or loss function (L2) in the individual coordinate system (K1, K2, K3), a deviation of the at least one actual trajectory point (P1, P2, P3) from the associated trajectory point (P1*, P2*, P3*) of the predicted trajectory (T*) is determined.

4. The method according to claim 3, characterized in that - the trajectory point (P1, P2, P3) of the actual trajectory (T) is selected as the origin (U1, U2, U3) for this coordinate system (K1, K2, K3), - an X-coordinate (X1, X2, X3) of this coordinate system is selected along a roadway longitudinal direction (FLR) of the roadway (2) at the trajectory point (P1, P2, P3), - a Y-coordinate (Y1, Y2, Y3) of this coordinate system (K1, K2, K3) is selected along a roadway transverse direction (FQR) orthogonal to the roadway longitudinal direction (FLR) of the roadway (2) at the trajectory point (P1, P2, P3).Method according to one of claims 2 to 4, characterized in that - the actual trajectory (T) comprises a plurality of trajectory points (P1, P2, P3) which are defined in a, preferably Cartesian, world coordinate system (K0), - the loss function (L2) is calculated for at least two, preferably several, trajectory points (P1, P2, P3) of the actual trajectory (T), - for each of these trajectory points ((P1, P2, P3)), the loss function (L2) is calculated in a local coordinate system (K1, K2, K3) assigned to the respective trajectory point ((P1, P2, P3)), wherein the local coordinate system (K1, K2, K3) depends on the course (3) of the roadway (2) at the trajectory point (P1, P2, P3).Method according to one of claims 2 to 5, characterized in that - a coordinate origin (U1, U2, U3) of the respective local coordinate system (K1, K2, K3) is that trajectory point (P1, P2, P3) to which this local coordinate system (K1, K2, K3) is assigned;. - an X-coordinate axis (X1, X2, X3) of the respective local coordinate system (K1, K2, K3) extends away from the assigned trajectory point (P1, P2, P3) in a roadway longitudinal direction (FLR1, FLR2, FLR3), along which the roadway (2) extends at the trajectory point (P1, P2, P3).

7. The method according to one of claims 2 to 6, characterized in that a Y-coordinate axis (Y1, Y2, Y3) of the local coordinate system (K1, K2, K3) extends away from the respective trajectory point (P1, P2, P3) in a roadway transverse direction (FQR1, FQR2, FQR3), which extends orthogonally to the roadway longitudinal direction (FLR1, FLR2, FLR3) at this trajectory point (P1, P2, P3).

8. Method according to one of claims 2 to 8, characterized in that - for calculating the loss function (L2) in the respective trajectory point (P1, P2., P3) a local X-function component (L2_X) and a local Y-function component (L2_Y) are calculated, - the X-function component (L2-X) is calculated as the deviation (ΔT_FLR) of the predicted trajectory (T*) from the actual trajectory (T) along the longitudinal direction of the road (FLR), i.e. along the X-coordinate axis (X1, X2, X3) of the local coordinate system (K1, K2, K3) in this trajectory point (P1, P2, P3) or is calculated as a function of this deviation (ΔT_FLR), - the Y-function component (L2-Y) is calculated as the deviation (ΔT_FQR) of the predicted trajectory (T*) from the actual trajectory (T) along the transverse direction of the road (FQR), i.e. along the Y-coordinate axis (Y1, Y2, Y3) of the local coordinate system (K1, K2, K3) in this trajectory point is calculated or is calculated depending on this deviation (ΔT_FQR).

9. Method according to one of claims 2 to 8, characterized in that - to calculate the X-function component (L2_X) and the Y-function component (L2_Y), the deviation (ΔT_X0, ΔT_Y0) of the predicted trajectory (T*) from the actual trajectory (T) along the X-axis (X0) and the Y-axis (Y0) of the world coordinate system (K0) is first calculated, - the deviations (ΔT_X0, ΔT_Y0) calculated in the world coordinate system (K0) are transferred into the local coordinate system (K1, K2, K3) of the respective trajectory point (P1, P2, P3) by means of a rotation transformation (DT).Method according to claim 9, characterized in that the rotational transformation (DT) takes place by a rotation about a rotation axis that extends perpendicular to both the X-coordinate axis (X0) and the Y-coordinate axis (Y0) of the world coordinate system (K0), and takes place about a rotation angle that is an intermediate angle (θ) between the X-coordinate axis (X0) of the world coordinate system (K0) and the X-coordinate axis (X1, X2, X3) of the local coordinate system (K1, K2, K3) at the respective trajectory point (P1, P2, P3).

11. Method according to one of the preceding claims, characterized in that the actual trajectory (T) is a trajectory actually traveled by the motor vehicle (1) or a trajectory simulated by means of simulation.

12. Deep learning system, comprising at least one neural network that is set up / programmed to carry out the method according to one of the preceding claims.

13. A computer program product containing instructions which, when executed by a computer system and / or by the deep learning system according to claim 12, cause the computer program product to execute the method according to any one of claims 1 to 11.

14. A data carrier containing instructions which, when executed by a computer system and / or by the deep learning system according to claim 12, cause the computer system to execute the method according to any one of claims 1 to 11. *******

Images