Methods for predicting road user movements

Abstract

The present invention relates to a method for predicting the movement of road users (V1 to Vm) around a vehicle, where the surroundings of the road users (V1 to Vm) are displayed in the form of a raster map (RK) having a set number of raster cells, and this raster map (RK) is supplied as input information to an artificial neural network (2), and from this input information, the trajectories (T1 to Tn) of the road users (V1 to Vm) are predicted using the neural network (2). According to the present invention, in order to enlarge or reduce the display range (DB) of the raster map (RK), the scale of the raster cells is dynamically enlarged or reduced according to the speed of the road users (V1 to Vm), and when the speed of the road users (V1 to Vm) is high, a wider range around the road users (V1 to Vm) is displayed by the raster map (RK) than when the speed is low. Furthermore, the present invention also relates to the use of the movement of road users (V1 to Vm) around the vehicle predicted in such a way to activate automated vehicle functions.

Description

[Technical Field]

[0001] The present invention relates to a method for predicting the movement of road users around a vehicle according to the preamble of claim 1 .

[0002] Furthermore, the invention relates to the use of road user movements predicted in such a way. [Background technology]

[0003] Making reliable predictions about the behavior of road users around a vehicle is essential for automated, and especially highly automated or autonomous, operation of vehicles. Only if an automated vehicle system can reliably predict the behavior of surrounding road users can it safely plan the future behavior of the automated vehicle.

[0004] Various approaches for such prediction are known from the prior art. These approaches include, in particular, raster map-based approaches, in which the vehicle's surroundings or parts of the vehicle's surroundings, such as the road geometry and the movement history of surrounding road users, also called agents, are mapped onto a raster map. The raster map here displays the vehicle's surroundings in a bird's-eye view. The predicted road user is, for example, positioned in the center of the raster map. Subsequently, an artificial neural network, such as a so-called convolutional neural network (CNN for short), processes the information from the raster map and uses it to estimate the future trajectory of the predicted road user.

[0005] Such a method for predicting road user behavior is known from "Henggang Cui et al.: Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks; In:arXiv:1809.10732v2[cs.RO]1 Mar 2019." There, multiple possible trajectories are predicted for road users around a vehicle or robot, and their probabilities are estimated. Furthermore, the surrounding context of each road user is encoded into a raster image, and the road user's trajectory is automatically derived by a deep artificial neural network. Here, the raster map has a set number of raster cells and three layers corresponding to the RGB image. These layers include both static infrastructure, such as roads, and dynamic information, such as the predicted road user and the past behavior of surrounding road users. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention is based on the object of providing a new method for predicting road user movements and a use of road user movements predicted in such a way. [Means for solving the problem]

[0007] This object is achieved according to the present invention by a method having the features of claim 1 and a use having the features of claim 6.

[0008] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0009] In a method for predicting the movement of road users around a vehicle, the surroundings of the road user are represented in the form of a raster map having a set number of raster cells, and this raster map is provided as input information to an artificial neural network, from which the trajectory of the road user is predicted using the neural network.

[0010] According to the present invention, the scale of the raster cells is dynamically increased or decreased depending on the speed of the road user in order to increase or decrease the display range of the raster map: if the road user is moving at a high speed, the raster map will show a larger area around the road user than if the road user is moving at a low speed.

[0011] Using this method and dynamic scaling of the viewing area, also called field of view, particularly advantageously avoids the need for a raster map-based approach to define, prior to the training process of the artificial neural network, how much viewing area (e.g., 50 meters by 50 meters) this raster map will cover in the real world for a given resolution of the raster map, e.g., 256 pixels by 256 pixels. Thus, the viewing area covered by each raster cell of the raster map does not need to be fixed.

[0012] This avoids the need to define a clear and constant (consistent) display range and the compromises that would be made if a fixed display range for a raster map were selected. This is due to the fact that the dynamic setting of the display range for high speeds of predicted road users, e.g., vehicles on a highway, ensures that the widest possible display range is covered, and that road shapes and surrounding road users that are far away from the predicted road users are also included in the prediction. The dynamic setting of the display range allows the method to select a lower (lower altitude) display range for low speeds of predicted road users, e.g., vehicles driving in urban areas, in order to obtain as much detailed information as possible about all nearby surroundings.

[0013] Particularly advantageously, existing prediction approaches can be suitably extended to implement the method.

[0014] The method eliminates the main weaknesses of raster map-based prediction approaches, particularly the fixed display range of the raster map and the problems that result from it. In this case, the method eliminates the display range limitations and the need to make compromises between low and high speed travel. An additional advantage is that the resolution of the raster map does not need to be automatically increased when road users are moving at high speeds.

[0015] In a possible embodiment of the method, the display range is calculated according to the following formula:

number

[0016] In a further possible embodiment of the method, the input information is scaled according to the scaled display range, which makes the method feasible by extending existing prediction approaches.

[0017] In a further possible embodiment of the method, the scaling factor of the raster cell scale is fed to a neural network and / or a network for further processing of the results calculated using this network, thereby making it possible to convert the results calculated from the scaled input information as needed, and thus also to implement the method by extending existing prediction approaches.

[0018] In a further possible embodiment of the method, after the trajectory has been predicted, the display range is scaled back to a set value, so that the display range is, for example, returned to its initial state again.

[0019] The inventive use of the movements of road users around the vehicle predicted by the method according to any one of the preceding claims for activating automated vehicle functions allows for particularly accurate and reliable activation of automated vehicle functions, such as automated longitudinal and / or lateral control of the vehicle, due to the advantageous prediction of road users around the vehicle.

[0020] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a plan view showing a schematic traffic situation including multiple road users; [Figure 2] FIG. 10 is a plan view schematically illustrating a further traffic situation including a plurality of road users and a display area of a raster map. [Figure 3] FIG. 10 is a plan view schematically illustrating a further traffic situation including a plurality of road users and a display area of a raster map. [Figure 4] 1 is a block diagram that shows a schematic diagram of a device for predicting the movement of road users around a vehicle according to the prior art; [Figure 5] 1 is a block diagram that shows a schematic diagram of a device for predicting the movement of road users around a vehicle; DETAILED DESCRIPTION OF THE INVENTION

[0022] In all the figures, corresponding parts are given the same reference numerals.

[0023] FIG. 1 shows a plan view of a traffic situation including a number of road users V1 to V5, for example cars.

[0024] In automated, in particular highly automated or autonomous, driving of a vehicle not shown in detail, it is necessary to predict the future behavior and thus the future movements of road users V1 to V5 present in the vehicle's surroundings, where the possible movements of road user V1 are represented by different possible trajectories T1 to Tn.

[0025] For such prediction, a raster map-based approach is used, for example. The surroundings or parts of the surroundings, such as the road geometry and the movement histories of surrounding road users V1-V5, also called agents, are shown in a bird's-eye view on a raster map RK, which is shown in detail in Figures 2, 3 and 5. The predicted road users V1-V5, such as vehicles, pedestrians, cyclists or other road users V1-V5, are positioned, for example, in the center of the raster map RK. Subsequently, an artificial neural network 2, such as a so-called convolutional neural network (CNN for short), shown in detail in Figures 4 and 5, is used to process the information on the raster map RK and to estimate future trajectories T1-Tn of the predicted road users V1-V5.

[0026] 2 shows a plan view of a further traffic situation and display range DB of a raster map RK including a number of road users V1 to V3, which are in particular vehicles.

[0027] When using a raster map-based approach to predict the movements of the corresponding road users V1-V3 as described above, the display range DB is dynamically scaled to eliminate any compromises in the selection of the display range DB of the raster map RK, thereby eliminating the need to define a fixed (consistent) display range DB and to find compromises for different traffic scenarios, e.g., highway or urban environments.

[0028] This scaling is performed by dynamically scaling the scale of the raster cells of the raster map RK according to the predicted speeds of the road users V1 to V3.

[0029] The resolution of the raster map RK (e.g. 256 pixels x 256 pixels) remains fixed in particular. However, the display area DB covered by the raster map RK is determined dynamically during operation. This dynamic determination is performed depending on the predicted instantaneous speeds of the road users V1 to V3.

[0030] If the speed of the corresponding road users V1 to V3 is high, the display range DB will cover a wider range than if the speed is low. Therefore, the display range DB covered by each raster cell will also change.

[0031] To determine the dimensions of the display range DB, the following formula is considered:

number

[0032] At this time, the size of the display range DB is set to the predicted braking distance of road users V1 to V3, that is, v 2 The acceleration a of road users V1 to V3 is typically 3 m / s for vehicles. 2 ~5m / s 2 The "max" operator defines a lower limit of 50 meters.

[0033] In the example shown in Figure 2, road users V1 to V3 are located in an urban environment, road user V1 is the predicted vehicle, and road users V2 and V3 are vehicles located around road user V1. Image B1 showing the traffic situation is rotated so that the predicted direction of movement of road user V1 faces upwards.

[0034] Due to the urban environment and the relatively slow speed of the associated road users V1 to V3, particularly the predicted speed of road user V1, a relatively small range is selected as the display range DB of the raster map RK.

[0035] 3 shows a plan view of a further traffic situation and the display range DB of the raster map RK including multiple road users V1-Vm, which are located on a highway. In this case, road user V1 is the predicted vehicle, and road users V2-Vm are vehicles located around road user V1. Image B2 showing the traffic situation is rotated so that the predicted direction of movement of road user V1 faces upward.

[0036] Compared to the embodiment shown in FIG. 2, the speed of road users V1 to Vm on the expressway is faster, so a wider range is selected for the display range DB of the raster map RK.

[0037] FIG. 4 shows a block diagram of an apparatus 1′ for predicting the movement of road users V1 to Vm around a vehicle according to a raster map-based approach according to the prior art.

[0038] The device 1' comprises an artificial neural network 2, for example a convolutional neural network, to which data of a predefined, fixedly sized raster map RK' is supplied as input.

[0039] Network 2 forms raster features RM from the input information and sends them to network 3, which combines the raster features RM with state information sEI, e.g., regarding speed, position, angle, etc., resulting in prediction of trajectories T1 to Tn of road users V1 to Vm.

[0040] The result is stored in memory 4 and output by output unit 5.

[0041] FIG. 5 shows a block diagram of a device 1 for predicting the movements of road users V1 to Vm around a vehicle.

[0042] In addition to the device 1' shown in FIG. 4, the device 1 comprises a scaling module 6 used to calculate (determine) a scaling factor SF for scaling the raster map RK described in FIGS.

[0043] Here, first, the predicted speeds of the road users V1 to Vm are calculated from the state information sEI.

[0044] Next, a scaling coefficient SF for scaling the display range DB of the raster map RK is calculated by the scaling module 6, and the raster map RK is scaled accordingly. The data of the raster map RK scaled in this manner is then supplied to the network 3.

[0045] Furthermore, the state information sEI is scaled according to the calculated display range DB, and a scaling coefficient S is supplied as additional input information to the network 3. Next, the corresponding trajectories T1 to Tn are predicted, and after this prediction, the display range DB is scaled back to the set value (the scale is returned). [Prior art documents] [Non-patent literature]

[0046] [Non-Patent Document 1] Henggang Cui et al.:Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks;In:arXiv:1809.10732v2[cs.RO]1 Mar 2019

Claims

1. A method for predicting the movement of road users (V1 to Vm) around a vehicle, comprising: - the surroundings of said road users (V1 to Vm) are displayed in the form of a raster map (RK) with a set number of raster cells, - said raster map (RK) is fed as input information to an artificial neural network (2), - a method in which trajectories (T1 to Tn) of said road users (V1 to Vm) are predicted using said artificial neural network (2) from said input information, - the scale of the raster cells is dynamically scaled depending on the speed of the road users (V1 to Vm) in order to scale the display range (DB) of the raster map (RK); When the speed of the road user (V1 to Vm) is high, the area around the road user (V1 to Vm) is displayed by the raster map (RK) in a larger area than when the speed is low. A method characterized by:

2. The display range (DB) is as follows: [Equation 1] Here, B = width of the display range (DB), L = length of the display range (DB), v = the speed of the road user (V1 to Vm), a = acceleration of the road users (V1 to Vm) is calculated 2. The method of claim 1.

3. The input information is scaled according to the scaled display range (DB).

3. The method according to claim 1 or 2.

4. The scaling factor (SF) of the scale of the raster cells is fed to the artificial neural network (2) and / or a network for further processing the results calculated using the artificial neural network (2).

2. The method of claim 1.

5. After the trajectory (T1 to Tn) is predicted, the display range (DB) is scaled back to the set value.

2. The method of claim 1.

6. Use of the movements of road users (V1 to Vm) around the vehicle predicted with the method of claim 1 for activating automated vehicle functions.

Images