Prediction of a road alignment based on radar data

Description

[0001] The present invention relates to a method, a computer program with instructions and a device for predicting a road course based on radar data, in particular radar data from a radar sensor of a motor vehicle.

[0002] The number of road accidents can be significantly reduced through the use of driver assistance systems. These systems can take over control when dangerous situations arise. Early attempts to make driving safer involved simple assistance systems, such as brake assist. Now, the trend is moving towards fully autonomous vehicles. The goal of this development is to minimize accidents.

[0003] Besides increasing safety, driver assistance systems can also enhance comfort by relieving the driver of some driving tasks. Examples of such systems include lane keeping assist and adaptive cruise control. Other systems can help reduce fuel consumption by anticipating acceleration and braking maneuvers. In the future, fully autonomous vehicles could also manage traffic flow through systems designed to reduce congestion.

[0004] To create systems capable of making decisions, a comprehensive view of the environment must be modeled. The increasing complexity of such assistance systems demands ever more accurate models. Therefore, real-time data from various types of sensors integrated into the vehicle are fused into a comprehensive and consistent representation of the environment.

[0005] Since many driver assistance systems and autonomous vehicles require information about the road ahead and their relative position, lane and road edge detection systems are of great importance. This information can be used to assist the driver with navigation tasks and to locate the vehicle within its surroundings.

[0006] Several technologies already exist for detecting the lane and the edges of the road, based on the evaluation of camera data, lidar data or radar data.

[0007] For example, the article "Vanishing point detection for road detection" by H. Kong et al. [1] describes an approach for segmenting a road area from a single image. The approach is based on estimating a vanishing point of the road using a locally adaptive soft-voting algorithm. The estimated vanishing point is then used as a boundary condition to determine two dominant edges for segmenting the road area.

[0008] DE 10 2018 131 477 A1 describes a method for controlling a vehicle. In this method, image data from a multitude of the vehicle's cameras is received and combined. Based on this combined image data, lane features are classified and localized using an artificial neural network. Based on this classified and localized lane data, fused lane feature data is generated using a data fusion process. The vehicle is then controlled based on this fused lane feature data.

[0009] The article "Lidar-based road and road-edge detection" by W. Zhang [2] describes a lidar-based method for detecting roads and road edges. The lidar data is decomposed into signals related to height and signals projected onto the ground plane. First, the height signals are processed using filtering techniques to identify a candidate region for a road. Pattern recognition techniques are then used to determine whether the candidate region is a road segment. Next, a line representation of the signals projected onto the ground plane is identified and compared to a simple road model in plan view to determine whether the candidate region is a road segment with associated road edges.

[0010] The article "Efficient occupancy grid computation on the GPU with lidar and radar for road boundary detection" by F. Homm et al. [3] describes an approach for the rapid computation of occupancy grid maps using laser rangefinders and radar sensors. This approach utilizes a GPU to overcome the limitations of classical occupancy grid computation in the automotive industry. Using a histogram-based approach, road edges can be detected with the help of these occupancy grid maps.

[0011] The document by Christian Lundquist et al., "Road Intensity Based Mapping Using Radar Measurements With a Probability Hypothesis Density Filter," published in IEEE Transactions on Signal Processing, Vol. 59, No. 4, April 1, 2011, describes a method for environmental modeling for vehicles using a PHD filter based on radar data. Stationary objects such as guardrails are detected through clustering and polynomial modeling. The method uses a mixture of Gaussian components for map generation.

[0012] Many newer approaches rely on manually created features, predefined models, or heuristics to generate a meaningful description of road edges. These systems are designed for use on highways and can fail in complex scenarios, such as urban areas. Approaches based on occupancy grids and open spaces are more flexible in the road geometry they support. Open space models, however, can identify all drivable areas and are not limited to the road itself. Therefore, parking lots, highway ramps, and other undesirable areas could also be included in the open space.

[0013] Machine learning-based approaches mainly focus on camera-based imaging systems due to the increasing popularity of convolutional neural networks (CNNs).

[0014] It is an object of the present invention to provide improved solutions for predicting a road course based on radar data from a radar sensor of a motor vehicle.

[0015] This problem is solved by a method having the features of claim 1, by a computer program with instructions having the features of

[0016] Claim 5 and by a device having the features of claim 6. Preferred embodiments of the invention are the subject of the dependent claims.

[0017] According to a first aspect of the invention, a method for predicting a road course based on radar data from a radar sensor of a motor vehicle comprises the following steps: Receiving radar data; accumulating radar data in a measurement grid; forming clusters for objects in the measurement grid; processing the clusters to determine polynomials to describe the road edges; and outputting the polynomials.

[0018] According to another aspect of the invention, a computer program comprises instructions which, when executed by a computer, cause the computer to perform the following steps to predict a road course based on radar data from a radar sensor of a motor vehicle: Receiving radar data; accumulating radar data in a measurement grid; forming clusters for objects in the measurement grid; processing the clusters to determine polynomials to describe the road edges; and outputting the polynomials.

[0019] The term "computer" is to be understood broadly. In particular, it also includes control units, controllers, embedded systems, and other processor-based data processing devices.

[0020] The computer program can, for example, be made available for electronic retrieval or be stored on a computer-readable storage medium.

[0021] According to another aspect of the invention, a device for predicting a road course based on radar data from a radar sensor of a motor vehicle is set up to perform the following steps: Receiving radar data; accumulating radar data in a measurement grid; forming clusters for objects in the measurement grid; processing the clusters to determine polynomials to describe the road edges; and outputting the polynomials.

[0022] For this purpose, the device preferably has: a receiver module for receiving the radar data; a preprocessing module for accumulating the radar data in a measurement grid and for forming clusters for objects in the measurement grid; a processing module for processing the clusters to determine polynomials for describing the road edges; and an output module for outputting the polynomials.

[0023] In the solution according to the invention, radar data from a moving vehicle is preprocessed and grouped into related objects. This leads to a significant reduction in complexity and noise, as the entire measurement grid can be abstracted by a set of clusters representing the objects. The resulting radar clusters are then processed to estimate the road's course. The road's course is modeled as a polynomial for the left and right sides of the road. The described solution allows for a relatively simple modeling of the problem without having to rely on manually designed heuristics. As a result, this reduces the number of potential errors in the code and allows for a simpler program structure with improved performance, depending on data availability.With a large amount of available data, higher detection accuracy is achieved, along with better handling of complex scenarios. The more data available, the better the predictions become. Since manual processing of all special cases is unnecessary, the solution according to the invention is simpler and more flexible than known solutions.

[0024] Cluster descriptions are generated for the clusters. Preferably, each cluster description includes a center of the cluster and a description of the cluster's boundary in the form of a boundary polygon. A boundary polygon allows for an approximate description of the shape of an object or the cluster itself. Additionally, a bounding box for the cluster can be defined. Further information can be included in the cluster descriptions, such as a probability indicating the likelihood of an object moving. Another piece of information can be the probability of a detected object existing.

[0025] The clusters are transformed into a feature space using a receptive field. The clusters can be folded with the individual receptors of the receptive field. A receptive field, consisting of, for example, 50 to 500 receptors, serves as the feature representation for further regression. The receptive field of a single receptor is defined by the probability density function (PDF) of a 2D normal distribution with finite extent. This extent is defined by a threshold for the maximum distance to the receptor center. Stimuli that occur closer to the center of the receptor elicit a stronger response. In other words, the clusters, or rather their envelope descriptions, are folded with the individual receptors and thereby transformed into a feature space with a fixed number of dimensions. Each receptor represents one dimension in the feature space.In contrast to typical convolutional neural networks that operate on images, the receptive field generates only a few hundred values. A convolutional neural network, on the other hand, usually has many feature maps of varying sizes, each containing 2D arrays of features, and therefore requires more processing power and memory. Evaluating receptive fields is particularly efficient when performed in parallel, for example, on a graphics card.

[0026] According to one aspect of the invention, the centers of the receptors and the covariance matrices of the receptors are determined manually or by evolutionary algorithms. The quality of feature extraction depends on the location, orientation, and shape of the individual receptors that comprise the receptive field. Intuitively, for example, a regular grid or a regular hexagonal grid can be implemented. Alternatively, circular arrangements can also be used. Evolutionary strategies are particularly advantageous when applied to the receptive field, since the evolution of the receptors and the resulting optimization of the road detection quality can achieve an improved feature space representation of the radar clusters.

[0027] The description of the clusters serves as input for a neural network in the feature space. The neural network is trained to regress the parameters of the polynomials used to describe the road edges. The receptive field and the neural network can also be advantageously implemented as a pipeline. The neural network considers the object clusters detected by the radar from a 2D top-down view of the world around the vehicle. For example, guardrails can be identified in the radar data. The course of the guardrails along the roadway is used by the neural network to estimate the road's alignment. The system's capabilities depend primarily on the chosen training data. New, unknown scenarios or special cases can be learned by processing more training data, i.e., simply by generating reference data that includes radar data plus the corresponding road alignment. No adjustment of the algorithm is necessary.Using a neural network achieves a special kind of dynamic. The neural network typically ensures smooth, consistently changing outputs. Compared to other approaches, there are fewer abrupt changes in the output.

[0028] To train the neural network, an error function can be used that minimizes the error of the output polynomials, i.e., the prediction, in geometric space, i.e., in the vehicle's ego-coordinate system, instead of directly optimizing the polynomial parameter vector. This ensures that the network is better able to generalize in unknown scenarios. The neural network is advantageously implemented using existing libraries such as Keras, TensorFlow, Caffe, Torch, Theano, etc.

[0029] Preferably, a method or device according to the invention is used in a motor vehicle, e.g., in a semi-autonomous, autonomous, or motor vehicle equipped with at least one driver assistance system. For example, a solution according to the invention can be used to predict the road course on highways, rural roads, and, with sufficient data, also in urban areas. The predicted road course can be used for a variety of driver assistance systems, such as lane keeping assist systems.

[0030] Further features of the present invention will become apparent from the following description and the attached claims in conjunction with the figures. Figure overview

[0031] Fig. 1 schematically shows a method for predicting a road's course based on radar data from a radar sensor of a motor vehicle; Fig. 2 schematically shows a first embodiment of a device for predicting a road's course based on radar data from a radar sensor of a motor vehicle; Fig. 3 schematically shows a second embodiment of a device for predicting a road's course based on radar data from a radar sensor of a motor vehicle; Fig. 4 schematically shows a motor vehicle in which a solution according to the invention is implemented; Fig. 5 shows clusters of radar data in a measurement grid; Fig. 6 schematically shows an example of a receptor; Fig. 7 illustrates various sampling models for the clusters; Fig. 8 shows the effect of the sampling models. Fig. 7 Fig. 9 schematically shows different arrangements of receptors; Fig. 10 shows examples of output models for describing road edges; Fig. 11 illustrates a loss function in geometric space; Fig. 12 shows two separate networks for the left and right road edges; Fig. 13 shows a network with shared layers and task-specific layers; Fig. 14 shows a network with exclusively shared layers; Fig. 15 shows a neural convolutional network that directly uses the output of the receptive field as input; Fig. 16 shows a modeling of road edges with polygons in an interactive tool; and Fig. 17 shows an example scenario for a polynomial fit. Character description

[0032] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. The same reference numerals are used in the figures for identical or equivalently acting elements and are not necessarily described again for each figure. It is understood that the invention is not limited to the embodiments shown and that the described features can also be combined or modified without departing from the scope of protection of the invention as defined in the appended claims.

[0033] Fig. 1 Figure 1 schematically illustrates a method for predicting a road's course based on radar data from a vehicle's radar sensor. In a first step (S1), the radar data to be processed is received and then accumulated in a measurement grid (S2). Next, clusters are formed for objects within the measurement grid (S3). Cluster descriptions are then generated for these clusters (S4). These descriptions include, for example, the cluster's center, a description of the cluster's envelope in the form of an envelope polygon, and a bounding box. The resulting clusters are then processed (S5) to determine polynomials for describing the road's edges. During processing (S5), the clusters can first be transformed into a feature space using a receptive field. This transformation involves, for example, convolving the clusters with the individual receptors of the receptive field.The centers of the receptors and the covariance matrices of the receptors can be determined manually or by evolutionary algorithms. The description of the clusters in the feature space then serves as input for a neural network trained to regress the parameters of the polynomials used to describe the road edges. The determined polynomials are then output for further use (S6).

[0034] Fig. 2 Figure 1 shows a simplified schematic representation of a first embodiment of a device 20 for predicting a road's course based on radar data RD from a radar sensor of a motor vehicle. The device 20 has an input 21 through which a receiver module 22 receives the radar data RD to be processed. A preprocessing module 23 accumulates the radar data RD in a measurement grid and forms clusters C for objects in the measurement grid. The preprocessing module 23 can also generate cluster descriptions B for the clusters C. The cluster descriptions B each include, for example, a center of the cluster C, a description of the envelope of the cluster C in the form of an envelope polygon, and a bounding box. A processing module 24 processes the clusters C to determine polynomials P for describing the road edges.The processing module 24 can first transform the clusters C into a feature space using a receptive field, for example by convolving the clusters C with the individual receptors of the receptive field. The centers of the receptors and the...

[0035] The covariance matrices of the receptors can be determined manually or by evolutionary algorithms. The description of the clusters C in the feature space then serves as input for a neural network trained to regress the parameters of the polynomials P used to describe the road edges. The polynomials P determined by the processing module 24 can be output by an output module 25 via an output 28 of the device 20 for further use.

[0036] The various modules 22-25 can be controlled by a control module 26. Settings of modules 22-26 can be changed, if necessary, via a user interface 29. Data generated in the device 20 can be stored in a memory 27 of the device 20, for example, for later evaluation or for use by the components of the device 20. Modules 22-26 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or fully combined or implemented as software running on a suitable processor, such as a GPU or a CPU. The input 21 and the output 28 can be implemented as separate interfaces or as a combined bidirectional interface.

[0037] Fig. 3 Figure 3 shows a simplified schematic representation of a second embodiment of a device 30 for predicting a road's course based on radar data from a radar sensor of a motor vehicle. The device 30 comprises a processor 32 and a memory 31. For example, the device 30 is a control unit or a controller. Instructions are stored in the memory 31 which, when executed by the processor 32, cause the device 30 to perform the steps according to one of the described methods. The instructions stored in the memory 31 thus embody a program executable by the processor 32, which implements the method according to the invention. The device 30 has an input 33 for receiving radar data. Data generated by the processor 32 is provided via an output 34. Furthermore, it can be stored in the memory 31.Input 33 and output 34 can be combined into a bidirectional interface.

[0038] The processor 32 can comprise one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.

[0039] The storage devices 27, 31 described can have both volatile and non-volatile storage areas and can include a wide variety of storage devices and storage media, such as hard disks, optical storage media or semiconductor storage.

[0040] Fig. 4 Figure 40 schematically shows a motor vehicle 40 in which a solution according to the invention is implemented. The motor vehicle 40 has a radar sensor 41 with which radar data RD of the motor vehicle 40's surroundings are acquired. The motor vehicle 40 also has a device 20 for predicting a road course based on the radar data RD. Polynomials P determined by the device 20 for describing the road edges are output to at least one assistance system 42 of the motor vehicle 40. The device 20 can, of course, also be integrated into the assistance system 42. Further components of the motor vehicle 40 in this example are a navigation system 43, which can provide position information, and a data transmission unit 44. A connection to a backend can be established, for example, by means of the data transmission unit 44, e.g., to obtain updated software. A memory 45 is provided for storing data.Data exchange between the various components of the motor vehicle 40 takes place via a network 46.

[0041] The following section will provide further details of the invention based on Fig. 5 bis Fig. 17 be described.

[0042] Clothoids are used in road construction to model road geometry. A clothoid is a plane curve whose curvature is uniquely determined in the plane, up to similarity, by the property that the curvature at any point on the curve is proportional to the length of its arc up to that point. The curvature thus changes linearly along the curve. Clothoids allow for smooth transitions between circles and straight line segments. They can be approximated piecewise by polynomials, preferably third-order polynomials. At the transition point between the polynomials, the first and second derivatives are set to zero. Therefore, using polynomials is advantageous for predicting the road's alignment.

[0043] The starting point for predicting the road's path is a pre-processed representation of the vehicle's surroundings. The vehicle's radar sensor continuously acquires measurement data, which is accumulated in a measurement grid. This grid serves to track and smooth the successive radar scans and combine them into a coherent model of the environment. Objects within the grid are then grouped into clusters C, for each of which a cluster description is generated. The cluster description can include information about the center of cluster C, a label (obstacle, etc.), a bounding box of cluster C aligned with the axes of the measurement grid, and a representation of the cluster C's boundary in the form of a 2D polygon. Fig. 5 The figure shows such clusters C of radar data in a measurement grid along a trajectory T of the vehicle. The clusters C are represented in world coordinates, taking only stationary objects into account. Fig. 5 For illustrative purposes, the data from the measurement grid is displayed in the world coordinate system over a longer period, e.g., the last 10 seconds, not the view of a vehicle at a given time. While data is also accumulated at a single point in time, this occurs over significantly shorter periods. The representation clearly shows a road geometry. The goal is to predict the position and shape of the road edges based on the cluster data. Challenges arise because the measurement grids, as input variables for further processing, contain a variable number of clusters C, and these clusters C, in turn, have a variable number of points for modeling the envelope polygon.

[0044] According to the invention, the input data is first processed, transforming the radar clusters into a more easily manageable representation. This transformation is based on a receptive field, i.e., an arrangement of receptors. Receptive fields originate in biology, where they are found, for example, in the visual cortex of animals and humans. A receptive field represents a set of quasi-sensors, for example, with a Gaussian shape and response curve. The individual receptive field of each receptor is defined in this case as a 2D multivariate Gaussian probability density function with a finite extent. In general, receptive fields can have arbitrary shapes. For example, they become increasingly complex from stage to stage in the human visual system. Fig. 6 The diagram schematically shows an example of a receptor. The ellipse represents the sensitive area. The individual points illustrate the sensitivity at different positions. Using such Gaussian receptors, individual point-like stimuli can be transformed into an n-dimensional feature space, where n is the number of receptors in the entire receptive field.

[0045] However, the radar clusters C are represented by their hull polygons, which consist of individual connected points. The radar clusters C must therefore first be transformed into a set of individual stimuli S. Different approaches based on various sampling models can be used for this purpose. Fig. 7 Figure 1 illustrates three such sampling models for the cluster C. The individual stimuli S are represented by the black circles. Fig. 8 shows the effect of the sampling models from Fig. 7 .

[0046] In the first model, which was in Fig. 7a As depicted, the points of the envelope polygon are used directly; that is, the connectivity information of the cluster envelopes is ignored, and the envelope points of the clusters at a given time in the measurement grid are fed into the receptive field. This approach is a very crude approximation of the input data and completely discards the connectivity information of the clusters. Furthermore, it has two structural weaknesses. A first problem is that receptors located entirely within a single cluster cannot perceive the cluster itself, since only the envelope points are considered. A second problem is that the envelope polygons do not necessarily have a constant sampling rate along the boundary. Different clusters with the same shape and location can therefore lead to different receptor responses, and the response is not proportional to the shape and size of the cluster.This problem can be solved by re-scanning the polygon hull along the boundary.

[0047] In a second model, which is in Fig. 7b As shown in the diagram, a cluster C is represented by its bounding box BB, which is then sampled along the x-axis and y-axis with a constant step size of, for example, 50 cm. In this case, the receptor responses to a cluster are proportional to the size of its bounding box. Furthermore, small receptors do not miss a stimulus S if they are located within a cluster. A disadvantage of this approach is that an axis-oriented bounding box BB is, in turn, a crude approximation for arbitrary polygon boundaries.

[0048] A third model, which is in Fig. 7c As shown in the diagram, the area of ​​the envelope polygon HP is sampled, for example, again with a constant step size of 50 cm along the x-axis and the y-axis. The set of stimuli S then includes all sample points that lie within the polygon. This approach increases the plausibility of the transformation of the radar clusters into the feature space, but also increases the computational complexity, since hundreds or even thousands of envelope polygons have to be rendered and the respective points extracted.

[0049] This task is comparable to the preprocessing required for the direct application of CNNs. One approach is to first render the radar clusters as a binary or grayscale image at a specific intensity. All pixel coordinates corresponding to the clusters are extracted from the image and transformed back into the ego coordinate system, in which the receptors are located. The receptors' response to the resulting set of stimuli can then be calculated.

[0050] The receptors can be arranged in different ways. Fig. 9 schematically shows four different arrangements of receptors. Fig. 9a ) shows a regular grid, Fig. 9b ) a regular grid with a hexagonal arrangement, Fig. 9c ) a circular arrangement and Fig. 9d ) a circular hexagonal arrangement.

[0051] The regular grid in Fig. 9a The system consists of identically shaped receptors with constant dimensions in the x and y directions and a fixed spacing along the x and y axes. The variances are chosen to cover the entire input space, resulting in slight overlaps between neighboring receptors. This approach is not orientation-selective, but it can be used in combination with CNN-based systems because the feature channels can be transformed into a 2D image.

[0052] The regular grid with hexagonal arrangement in Fig. 9b This is a variation of the regular grid where the layout is hexagonal instead of rectangular. Every second row of receptors is offset along the x-axis by half the fixed receptor spacing. This reduces aliasing artifacts that can occur when stimuli are located between two adjacent receptors.

[0053] The circular arrangement in Fig. 9c The system consists of receptors located on a series of lines sharing a common origin, e.g., 30 meters behind the vehicle. The lines are spaced at equal intervals between ±45°. The receptor spacing is proportional to the logarithm of the distance to the origin, ensuring a higher sampling rate closer to the origin. The variances are chosen to fill all areas between the receptors. This design increases the resolution closer to the vehicle but reduces the sampling rate at greater distances. It also results in receptors with selectivity regarding a preferred orientation.

[0054] The circular hexagonal arrangement in Fig. 9d This is a variation of the circular arrangement. In this approach, the same receptor spacing is used along lines radiating from the common origin, and the same circular arrangement is employed, but every second ring of receptors is rotated around the origin by an angle equal to half the angle between any two adjacent lines. This results in a scanning pattern resembling a hexagonal arrangement, which in turn reduces aliasing artifacts.

[0055] Regardless of the chosen arrangement, all responses are evaluated in a receptive field to form a feature vector. This feature vector can then be used as input for any regression model.

[0056] Instead of manually generating receptive fields, genetic algorithms can be used to create them. For this purpose, the receptive fields are encoded in a vector of real values ​​representing a genome. The fitness of the receptive field is defined by the sum of the captured input variance of all receptors, plus any negative training loss. The receptive fields are then developed by applying genetic operators, specifically a mutation operator and a crossing operator. The mutation operator utilizes Gaussian noise, while the crossing operator is preferably based on single-point crossover. Higher-performing individuals have a greater chance of reproduction; therefore, fitness increases with each generation.

[0057] Different output models for the polynomials are suitable for predicting road edges using a neural network. Some examples of output models for describing road edges are given in Fig. 10 The parameters used to describe the polynomials are highly nonlinear and influence the geometric shape in various ways. This makes learning a regression model more difficult, so it is advisable to make some simplifications. For example, the starting point of the prediction window is set to 0 for all models. This simplification has no negative impact on the results.

[0058] The solid line illustrates a description using piecewise-defined polynomials, where a variable or fixed window size can be used. This is a very complex representation with limitations, which also requires a suitable loss function for training the neural network. Predicting the window size solely based on receptor responses is difficult. Therefore, the window size is preferably set to a constant value.

[0059] Describing a system using piecewise-defined polynomials can be relatively difficult for a neural network to learn. Therefore, an alternative description using a single polynomial can be implemented, which is defined in Fig. 10 This is illustrated by the dashed line. Again, a variable or fixed window size can be used, although a fixed window size is preferred. The representation is also relatively complex, but has fewer limitations. However, in this case too, a suitable loss function is required for training the neural network.

[0060] Another alternative is a description as a polygonal chain consisting of linearly interpolated points with a constant distance between the control points on the x-axis. Such a polygonal chain is described in Fig. 10 This is illustrated by the dashed line. The window size is preferably set to a constant value. This approach simplifies the representation but increases the dimensionality of the output space. The regression model determines the y-coordinates for a list of fixed x-coordinates in front of the vehicle. While this sacrifices the implicit smoothness compared to polynomial representations, it increases the model's flexibility and simplifies the prediction representation compared to more complex polynomial models. If a small number of sampled coordinates are used, a polynomial can be fitted to the determined points using a simple least-squares solver to reconstruct a polynomial description of the road edges.

[0061] The regression model is preferably trained directly on the parameter vector. However, similar solutions in the parameter space are not necessarily similar in the geometric space. Therefore, a fitted loss function is preferably used. Instead of calculating the prediction errors in the parameter space, a loss is defined in the geometric space. This can be achieved by sampling polynomials in the ego coordinate system at equidistant points along the x-axis. Fig. 11 This illustrates such a loss function, where the average length of the arrows representing the deviation between the basic truth (dashed line) and the prediction (dash line) is minimized.

[0062] To learn how to map input and output data, various regression models can be implemented using network architectures. These are described in Fig. 12 bis Fig. 15 The box below represents the feature space 51, obtained by applying the receptive field to the input data. The box above represents the output layer 52, which generates a prediction for the left / right / combined road geometry.

[0063] Fig. 12 Figure 50 shows two separate networks for the left and right edges of the road. In this approach, two simple, independent, fully connected neural networks predict the left and right edges of the road separately, without the possibility of exchanging information between the networks. Each of the neural networks has a set of dense or fully connected layers. The feature space is a one-dimensional feature vector.

[0064] Fig. 13 Figure 50 shows a network with some shared dense or fully connected layers and some task-specific dense or fully connected layers. This network builds shared and task-specific features for the left and right edges of the road. Two separate output layers predict the individual road edges. The feature space is a one-dimensional feature vector.

[0065] Fig. 14 Figure 50 shows a network with exclusively shared dense or fully connected layers 53. The output of the output layer 52 is, in this case, a list of concatenated predictions for the left and right edges of the road. The feature space 51 is also a one-dimensional feature vector.

[0066] Fig. 15 Figure 50 shows a network 50 in which a convolutional neural network 54 with a series of convolutional layers 55 and subsampling layers 56 (max pooling layers) is used. Several dense or fully connected layers 53 are attached to the convolutional neural network 54. The feature space 51 consists of a grid of N×M values ​​derived from the one-dimensional feature vector provided by the receptive field. The 1D feature vector is essentially transformed into a low-resolution 2D image of the environment.

[0067] To train the neural network, basic truths about the training data must be provided. This can be done using an interactive tool. The road edges are modeled as a set of polygons. An example of such a model is shown below. Fig. 16 In addition to the trajectory T, the left road edge RL and the right road edge RR are visible, each represented by a polygon. Users of the interactive tool can modify these polygons by moving, adding, or removing points, splitting lines into segments, or merging adjacent segments. If available, the underlying data can be initialized by an existing application, eliminating the need to create the data from scratch.

[0068] The marked polygons are time-independent and cannot be directly used as frame-related basic truths. Therefore, additional steps are required. For each frame of the vehicle's self-motion, a local basic truth in the form of a polynomial is generated. The polynomial is then fitted to a subset of the points of the polygon. The window size and the transition point for the polynomial are determined iteratively. Fig. 17 Figure 1 shows an example scenario for polynomial fitting. Polylines are transformed into the Ego coordinate system, and the first point behind vehicle 40 along the x-axis and within a local neighborhood is found. The corresponding points are marked by the unfilled circles. Polynomial fitting within a window W is implemented as a hierarchical approach for each of the two independent road boundaries. The dashed lines indicate a possible solution generated by the fitting algorithm.

[0069] The modeling results in a baseline with variable window size. A frame-specific baseline is available. The data can be used to evaluate or train models using examples. Preferably, the data is further filtered manually or automatically. This allows, for example, the removal of very short baseline segments. Segments can also be removed if there are too few radar clusters, if the baseline for the right or left side of the road is unavailable, if the adaptation algorithm fails, or if they cannot be labeled. Referenzen

[0070] [1] H. Kong et al.: "Vanishing point detection for road detection", 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009), S. 96-103. [2] W. Zhang: "Lidar-based road and road-edge detection", 2010 IEEE Intelligent Vehicles Symposium (2010), S. 845-848. [3] F. Homm et al.: "Efficient occupancy grid computation on the GPU with lidar and radar for road boundary detection", 2010 IEEE Intelligent Vehicles Symposium (2010), S. 1006-1013.

Claims

1. A method for predicting a road route based on radar data from a radar sensor (41) of a motor vehicle (40), comprising the steps: - receiving (S1) the radar data (RD); - accumulating (S2) the radar data (RD) in a measurement grid; - forming (S3) clusters (C) for objects in the measurement grid; - processing (S5) the clusters (C) to determine polynomials (P) for describing the edges of the road, wherein cluster descriptions (B) for the clusters (C) are generated (S4), wherein the clusters (C) are transformed into a feature space by means of a receptive field, wherein the description of the clusters (C) in the feature space serves as an input variable for a neural network, wherein the neural network is trained for the regression of parameters of the polynomials (P) for describing the edges of the road; and - outputting (S6) the polynomials (P).

2. The method according to claim 1, wherein the cluster descriptions (B) each comprise a centre of the cluster (C) and a description of a hull of the cluster (C) in the form of a hull polygon.

3. The method according to one of the preceding claims, wherein the clusters (C) are convolved with the individual receptors of the receptive field for transformation into the feature space.

4. The method according to claim 3, wherein the centres of the receptors and the covariance matrices of the receptors are determined manually or by evolutionary algorithms.

5. A computer program comprising instructions which, when executed by a computer, cause the computer to execute the steps of a method according to one of claims 1 to 4 for predicting a road route based on radar data (RD) of a radar sensor (41) of a motor vehicle (40).

6. A device (20) for predicting a road route based on radar data (RD) of a radar sensor (41) of a motor vehicle (40), wherein the device (20) is configured to execute the steps of a method according to one of claims 1 to 4.