METHOD FOR PREDICTING THE MOVEMENT OF A ROAD USER

Description

[0001] The invention relates to a method for predicting the movement of a road user in a vehicle environment according to the preamble of claim 1.

[0002] The invention further relates to a use of a movement of a road user predicted in such a method.

[0003] Reliable prediction of the behavior of road users in a vehicle's environment is essential for automated, and especially highly automated or autonomous, driving. Only if an automated vehicle system is able to reliably predict the behavior of surrounding road users can the future behavior of the automated vehicle be planned with certainty.

[0004] Several approaches for such prediction are known from the prior art. These approaches include raster map-based methods, in which the vehicle's surroundings, or parts thereof—for example, road geometry and the movement history of surrounding road users, also referred to as agents—are represented in a raster map. The raster map shows the vehicle's surroundings from a bird's-eye view. A road user to be predicted is, for example, located at the center of the raster map. Subsequently, artificial neural networks, such as convolutional neural networks (CNNs), are used to process information from the raster map and estimate a future trajectory of the road user to be predicted.

[0005] Such a method for predicting the movement of a road user is known from "Henggang Cui et al.: Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks; In: arXiv:1809.10732v2 [cs.RO] 1 Mar 2019". Here, several possible trajectories of road users in the environment of a vehicle or robot are predicted, and their probabilities are estimated. Furthermore, the environmental context of each road user is encoded in a raster image, which is used by deep artificial neural networks to automatically derive a trajectory for the road user. The raster map has a predefined number of grid cells and three layers, corresponding to an RGB image.The layers include both static infrastructure, such as roads, and dynamic information, such as the historical movement of a road user to be predicted and surrounding road users.

[0006] The patent application US 2022 / 144309 A1 relates to a system and method for determining a navigation trajectory using reinforcement learning for an ego-vehicle in a navigation network. A hypothetical vehicle location is represented as a value in a fixed grid aligned with the ego-vehicle's location and direction of travel.

[0007] US Patent 2021 / 347377 A1 discloses techniques for predicting object behavior in a road user's environment, involving inputting data into and receiving output from the model that is a discretized representation. This discretized representation can be associated with a probability that an object will reach a location in the environment at a future time. The road user's environment is represented as a raster map, which is fed into the model as input.

[0008] The invention is based on the objective of providing a novel method for predicting the movement of a road user and a use of a movement of a road user predicted in such a method.

[0009] The problem is solved according to the invention by a method which has the features specified in claim 1, and by a use which has the features specified in claim 6.

[0010] Advantageous embodiments of the invention are the subject of the dependent claims.

[0011] In a method for predicting the movement of a road user in a vehicle environment, the road user's environment is represented in the form of a raster map with a predetermined number of raster cells, the raster map being fed to an artificial neural network as input information, and the neural network predicting a trajectory of the road user from the input information.

[0012] According to the invention, the scale of the grid cells is dynamically scaled depending on the speed of the road user in order to scale the display area of ​​the raster map. Using the raster map, a larger area of ​​the road user's surroundings is displayed at high speeds than at low speeds.

[0013] Using the present method and the dynamic scaling of the display area, also referred to as the viewing area, it is particularly advantageous in raster map-based approaches to avoid the need to define, before training the artificial neural network, the display area of ​​a raster map (e.g., 50 meters x 50 meters) in the real world, given a raster map resolution of, for example, 256 pixels x 256 pixels. Thus, even the display area covered by a single raster cell of the raster map does not need to be fixed.

[0014] This eliminates the need to define a fixed and constant display area, and also avoids the compromises that arise when choosing a fixed display area for the raster map. This results from the fact that, due to the dynamic specification of the display area for high speeds of the road user to be predicted, for example, a vehicle on a highway, the present method covers the largest possible display area, so that road geometries and surrounding road users that are far away from the road user being predicted are also included in the prediction.Due to the dynamic specification of the display area, the method also allows for the selection of a lower display area at low speeds of a road user to be predicted, for example a vehicle in urban traffic, in order to capture all nearby surrounding information in as much detail as possible.

[0015] Existing prediction approaches can be extended to implement the procedure in a particularly advantageous way.

[0016] The present method thus makes it possible to eliminate the main weakness of raster map-based prediction approaches, namely the fixed specification of the raster map's display area and the resulting problems. The method allows the display area to be unrestricted, eliminating the need for compromises between low and high speeds. A further advantage is that higher raster map resolutions are not automatically required for high-speed vehicles.

[0017] In one possible embodiment of the procedure, the display area is according to B = L = max 50 , 2 v 2 2 a Meter where B = width of the display area, L = length of the display area, v = speed of the road user and a = acceleration of the road user determined. The size of the display area scales non-linearly with the braking distance, i.e. v22a of the road user to be predicted. Typical values ​​for the acceleration of the road user are, for example, 3 m / s² for vehicles. 2 up to 5 m / s 2 The "max" operator defines a lower limit of 50 meters.

[0018] In another possible embodiment of the method, the input information is scaled depending on the scaled display area. This allows the method to be implemented by extending existing prediction approaches.

[0019] In another possible embodiment of the method, a scaling factor corresponding to the scale of the grid cells is applied to the neural network and / or a network for further processing the results obtained by the network. This allows for the conversion of the results derived from the potentially scaled input information and thus also enables the method to be implemented by extending existing prediction approaches.

[0020] In another possible embodiment of the procedure, after the prediction of the trajectory, the display area is scaled back to a predetermined value, so that it is, for example, back in an initial state.

[0021] The use according to the invention of a movement of a road user in a vehicle environment predicted in a method according to one of the preceding claims for the operation of an automated vehicle function enables, due to the advantageous prediction of the road users in the vehicle environment, a particularly precise and reliable operation of the automated vehicle function, for example an automated longitudinal and / or lateral control of the vehicle.

[0022] Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

[0023] This shows: Fig. 1 schematically shows a top view of a traffic situation with several road users, Fig. 2 schematically shows a top view of another traffic situation with several road users and a display area of ​​a raster map, Fig. 3 schematically shows a top view of another traffic situation with several road users and a display area of ​​a raster map, Fig. 4 schematically shows a block diagram of a device for predicting the movement of a road user in a vehicle environment according to the prior art, and Fig. 5 schematically shows a block diagram of a device for predicting the movement of a road user in a vehicle environment.

[0024] Corresponding parts are marked with the same reference symbols in all figures.

[0025] In Figure 1 is a top view of a traffic situation with several road users V1 to V5, for example motor vehicles.

[0026] For automated, and especially highly automated or autonomous, driving of a vehicle not shown in detail, it is necessary to predict the future behavior and thus the future movements of road users V1 to V5 present in the vehicle's vicinity. Possible movements of road user V1 are represented by various possible trajectories T1 to Tn.

[0027] Raster map-based approaches are used for such predictions. Here, the environment, or parts of it, such as road geometry and the movement history of surrounding road users V1 to V5 (also referred to as agents), are represented in a format that is... Figures 2, 3 and 5The raster map RK, shown in more detail below, is presented from a bird's-eye view. The road user V1 to V5 to be predicted, which is, for example, a vehicle, pedestrian, cyclist, or other road user V1 to V5, is located, for example, in the center of the raster map RK. Subsequently, in the Figures 4 and 5 Artificial neural networks 2, described in more detail, for example so-called Convolutional Neural Networks, or CNNs for short, are used to process the information from the raster map RK and to estimate a future trajectory T1 to Tn of the road user V1 to V5 to be predicted.

[0028] Figure 2 The diagram shows a top view of another traffic situation with several road users V1 to V3 and a display area DB of a raster map RK. The road users V1 to V3 are primarily vehicles.

[0029] To eliminate compromises in choosing the display area DB of the raster map RK when predicting the movement of the respective road users V1 to V3 using a raster map-based approach, it is planned to dynamically scale the display area DB. This eliminates the need to define a constant display area DB, thus avoiding the need to find compromises for different traffic scenarios, such as in a motorway environment or an urban environment.

[0030] This scaling is achieved by dynamically scaling the scale of raster cells of the raster map RK depending on the speed of the road user V1 to V3 to be predicted.

[0031] The resolution of the raster map RK, for example 256 pixels x 256 pixels, remains fixed. However, during operation, the display area DB covered by the raster map RK is dynamically determined. This dynamic determination is based on the instantaneous speed of the road user V1 to V3 being predicted.

[0032] For higher speeds of the respective road user V1 to V3, a larger display area DB is covered than for lower speeds. Thus, the display area DB covered by a single grid cell also changes.

[0033] One possible equation for determining the dimensions of the display area DB is B = L = max 50 , 2 v 2 2 a Meter , where B = width of the display area DB, L = length of the display area DB, v = speed of the road user to be predicted V1 to V3 and a = acceleration of the road user to be predicted V1 to V3.

[0034] The size of the display area DB scales non-linearly with a braking distance, i.e. v 2 2 a of the road user V1 to V3 to be predicted. Typical values ​​for the acceleration a of road user V1 to V3 are, for example, 3 m / s² to 5 m / s² for vehicles. The "max" operator defines a lower limit of 50 meters.

[0035] In the illustrated embodiment according to Figure 2The road users V1 to V3 are located in an urban environment, with road user V1 being the vehicle to be predicted and road users V2 and V3 being vehicles located in the vicinity of road user V1. An image B1 depicting the traffic situation is rotated such that one direction of movement of the road user to be predicted, V1, is directed upwards.

[0036] Due to the urban environment and the associated relatively low speeds of road users V1 to V3, especially of the road user V1 to be predicted, the display area DB of the raster map RK is chosen to be relatively small.

[0037] In Figure 3The diagram shows a top view of a traffic situation with several road users V1 to Vm and a display area DB of a raster map RK, where the road users V1 to Vm are located on a motorway. Road user V1 is the vehicle to be predicted, and road users V2 to Vm are vehicles located in the vicinity of road user V1. A diagram B2 depicting the traffic situation is rotated such that one direction of movement of the road user to be predicted, V1, is upwards.

[0038] Unlike the one in Figure 2 In the illustrated embodiment, the display area DB of the raster map RK is chosen to be larger due to the higher speeds of the road users V1 to Vm on the motorway.

[0039] Figure 4shows a block diagram of a device 1' for predicting the movement of a road user V1 to Vm in a vehicle environment according to a raster map-based approach according to the state of the art.

[0040] The device 1' comprises an artificial neural network 2, for example a Convolutional Neural Network, to which data from a defined grid map RK' of a fixed size are supplied as input information.

[0041] Network 2 forms grid features RM from the input information and transmits them to a network 3, which links the grid features RM with state information sEl, which relates to, for example, speed, position, angle, etc., and as a result predicts trajectories T1 to Tn of road users V1 to Vm.

[0042] The results are stored in a memory 4 and output by means of an output unit 5.

[0043] In Figure 5 Figure 1 shows a block diagram of a device 1 for predicting the movement of a road user V1 to Vm in a vehicle environment.

[0044] Device 1 includes, in addition to the one described in Figure 4 The device 1' shown includes a scaling module 6 for determining a scaling factor SF for the Figures 2 and 3 described scaling of the raster map RK.

[0045] First, a speed of the road user to be predicted, V1 to Vm, is determined from the state information sEI.

[0046] The scaling factor SF for scaling the display area DB of the raster map RK is then determined using scaling module 6, and the raster map RK is subsequently scaled accordingly. The data from the scaled raster map RK are then fed into network 3.

[0047] Furthermore, the state information sEl is scaled depending on the determined display area DB, and the scaling factor SF is fed into network 3 as additional input information. Subsequently, the corresponding trajectory T1 to Tn is predicted, and based on this prediction, the display area DB is scaled back to a predefined value.

Claims

1. Method for predicting the movement of a road user (V1 to Vm) in a vehicle environment, - the environment of the road user (V1 to Vm) being displayed in the form of a grid map (RK) containing a predefined number of grid cells, - the grid map (RK) being supplied to an artificial neural network (2) as input information and - a trajectory (T1 to Tn) of the road user (V1 to Vm) being predicted from the input information by means of the neural network (2), characterized in that - a scale of the grid cells is dynamically scaled on the basis of a speed of the road user (V1 to Vm) in order to scale a display region (DB) of the grid map (RK), - a larger region of the environment of the road user (V1 to Vm) being displayed by means of the grid map (RK) in the case of a high speed of the road user (V1 to Vm) than in the case of a low speed.

2. Method according to claim 1, characterized in that the display region (DB) is determined in accordance with W = L = max 50 , 2 v 2 2 a meters where W = width of the display region (DB), L = length of the display region (DB), v = speed of the road user (V1 to Vm) and a = acceleration of the road user (V1 to Vm).

3. Method according to claim 1 or claim 2, characterized in that the input information is scaled on the basis of the scaled display region (DB).

4. Method according to any of the preceding claims, characterized in that a scaling factor (SF) of the scale of the grid cells is supplied to the neural network (2) and / or to a network for further processing results determined by means of the network (2).

5. Method according to any of the preceding claims, characterized in that after the prediction of the trajectory (T1 to Tn), the display region (DB) is scaled back to a predefined value.

6. Method according to any of the preceding claims, characterized in that the predicted movement of a road user in a vehicle environment is used to operate an automated vehicle function.