Method for detecting objects in a multidimensional radar spectrum

Abstract

The invention relates to a method for detecting objects in a multidimensional radar spectrum, in which object hypotheses are formed on the basis of the signal distribution (10) in the spectrum, the object hypotheses are checked for plausibility on the basis of a number of predefined criteria and selected reflection points (16, 18) belonging to the plausibility-checked objects are represented in a point cloud, characterised in that the object hypotheses comprise hypotheses for extended objects for which the signal distribution in the spectrum extends over a plurality of adjacent bins (12), and in that, in order to check the plausibility of these objects, the signal distributions (10) in the spectrum are compared with stored model distributions for standard objects.

Description

[0001] R.415845

[0002] - 1 -

[0003] Description

[0004] title

[0005] Methods for detecting objects in a multidimensional radar spectrum

[0006] Description

[0007] The invention relates to a method for detecting objects in a multidimensional radar spectrum, in which object hypotheses are formed based on the signal distribution in the spectrum, the object hypotheses are made plausible based on a number of predetermined criteria, and selected reflection points belonging to the made plausible objects are represented in a point cloud.

[0008] In particular, the invention relates to a detection method for radar sensors used in driver assistance systems or autonomous driving systems for motor vehicles.

[0009] State of the art

[0010] In an FMCW radar sensor commonly used in driver assistance systems, a frequency-modulated radar signal is transmitted, consisting of a sequence of steep frequency ramps (so-called chirps). The signal reflected by objects and received back by the radar sensor is mixed with a portion of the signal transmitted at the time of reception, resulting in an intermediate frequency signal whose (beat) frequency corresponds to the frequency difference between the transmitted and received signals. The frequency of this signal depends on the signal propagation time and thus on the distance between the detected objects. A Fourier transform of the intermediate frequency signal yields a spectrum in which each detected object is distinguished by a local maximum (a peak), where not R.415845

[0011] - 2 - It is excluded that two or more adjacent peaks are generated by different reflection points of the same physical object. The frequency axis is divided into so-called distance bins, the size of which is determined by the resolution of the radar sensor. The distance bin in which a local maximum is located then indicates the distance of the corresponding reflection point.

[0012] As a sequence of received signals, acquired at corresponding sampling times in successive frequency ramps, the phase of the complex amplitude is a periodic function of the relative velocity. A second Fourier transform in the so-called Doppler dimension, i.e., across the successive frequency ramps, then yields a two-dimensional spectrum in which each peak indicates the distance and relative velocity of an object. The velocity axis is also divided into bins according to the resolution.

[0013] However, since the signals received in the successive chirps have a certain time interval between them, the assignment between the velocity bins and the relative velocities is not unambiguous. Each velocity bin is associated with several relative velocities, which are closer together the greater the time interval between the frequency ramps. These ambiguities are then resolved in a subsequent processing step using suitable criteria. This resolution of ambiguities takes place within the framework of object plausibility checks.

[0014] An azimuth-angle-resolving radar sensor has several horizontally spaced receiving antennas. The phases of the signals received in the different receiving channels are a periodic function of the object's azimuth angle. A further discrete Fourier transform across the different receiving channels yields a three-dimensional spectrum with the dimensions of distance, relative velocity, and azimuth angle. The interpretation of the peaks is also ambiguous in the angular dimension, since the object represented by a single peak can actually be in several angular positions, which are closer together the larger the distances between them.

[0015] - 3 -

[0016] Antenna to antenna. These ambiguities must also be resolved through suitable plausibility checks of the object hypotheses.

[0017] If the radar sensor is also angle-resolving in elevations, a four-dimensional spectrum is obtained analogously, with the elevation angle as an additional dimension.

[0018] In practice, the received radar signal is more or less noisy, so that even in cells or bins of the up to four-dimensional spectrum where no object is actually located, a signal with a certain signal strength is obtained. These signals form the so-called noise background, from which the peaks generated by real objects stand out only by their greater signal strength. So far, this noise background has been filtered out by only considering those peaks in the spectrum whose signal strength exceeds a certain threshold during further analysis.So-called CFAR algorithms are known, with which the threshold is adjusted to the local noise background in such a way that the signals of all real objects are highly likely to be above the threshold and thus detected, while there is only a small probability that fluctuations in the noise background are falsely detected as objects.

[0019] Object detection can be performed in several steps. For example, initial object detection can be carried out in the two-dimensional distance-velocity spectrum (dv spectrum) by selecting the local maxima that lie above the threshold value. The spectrum in the angular dimension(s) is then calculated only for the detected objects. It may turn out that a detected peak is actually a superposition of two peaks generated by objects with the same distance and relative velocity but different angular positions. Consequently, these objects can only be separated from each other after resolving the angular dimension.

[0020] Angle measurement or estimation often involves a maximum likelihood estimation, where the complex amplitudes of the various R.415845

[0021] - 4 -

[0022] The signals received by the receiving channels are compared with an antenna pattern that indicates the angular dependence of these amplitudes. If even the best-fitting antenna pattern shows only a low degree of agreement with the measured amplitude distribution, this may indicate that peak overlap is present, which then needs to be resolved with special multi-target algorithms.

[0023] In other known methods, object detection only occurs in the three- or four-dimensional spectrum, where the local maxima within this multidimensional spectrum are located and compared with the threshold value. Generally, the number of local maxima in a higher-dimensional spectrum is smaller than in a lower-dimensional spectrum because, for example, a peak that is a local maximum in two dimensions may turn out not to be a local maximum in the third dimension.

[0024] The plausibility check of object hypotheses, as used here, encompasses all processing steps involved in determining whether a signal that might prima facie indicate an object is in fact a real object. This includes resolving ambiguities and peak overlaps, as well as eliminating spurious objects, which can arise, for example, from reflections of the radar signal off planar objects such as guardrails, buildings, or even the road surface. One criterion for detecting such reflections is the degree of agreement in relative speed between hypothetical objects that differ in distance and detection angle. Other well-known criteria for object plausibility checks include confidence indicators, such as those derived from angle or velocity estimation, and determining the radar cross-section of the supposed objects.

[0025] All local maxima that have been plausibly verified as real objects based on the aforementioned criteria are then represented in a point cloud by determining the position coordinates of the corresponding object for each local maximum based on its position in the distance and angle spectrum and representing it as a point in a vehicle-fixed spatial R.415845

[0026] - 5 -

[0027] The coordinate system (with two or three dimensions) can be displayed. Optionally, at least one velocity coordinate can also be specified for each point.

[0028] The point cloud obtained in this way then forms the basis for further processing at the so-called object level, where the objects are classified, for example, as pedestrians, two-wheelers, cars, trucks, buildings or stationary road infrastructure objects (traffic light poles, traffic signs, etc.).

[0029] The point clouds obtained in this way using known methods are relatively "thin", i.e., they contain only a relatively small number of points and thus provide only a relatively unclear and low-detail image of the actual vehicle environment.

[0030] Disclosure of the invention

[0031] The object of the invention is to provide a method by which the informative value and density of the point cloud can be improved.

[0032] This problem is solved according to the invention by including the object hypotheses as hypotheses for extended objects for which the signal distribution in the spectrum extends over several neighboring bins, and by comparing the signal distributions in the spectrum with stored model distributions for standard objects to make these objects plausible.

[0033] The invention is based on the consideration that a large part of the information content of the radar spectrum remains unused if the detection of hypothetical objects is limited to local maxima. According to the invention, therefore, not only the tip of a peak is treated as a hypothetical object, but the signal distribution in the vicinity of the local maximum is also taken into account in at least one dimension. The hypothetical object is then not treated as a point-like object in a single bin of the spectrum, but as a quasi-continuum of radar reflections extending over several contiguous bins. R.415845

[0034] - 6 - and has a shape that depends on the type of object. This shape, i.e., the distribution of the signal in at least one dimension, can then be compared with model distributions that were previously calculated or measured for certain standard objects. These standard objects can be, for example, constellations of nearly point-like reflectors, such as triple mirrors, which are also used to calibrate radar sensors, or extended objects such as panels, rods, cuboids, cylinders, etc., of different shapes and sizes. By comparing the signal distribution of the hypothetical extended object with the model distributions, it is then possible to determine which standard object the hypothetical object most closely resembles.Based on the known shape of the standard object, additional points can then be identified in the spectrum that lie away from the local maximum and correspond to the positions of the triple mirrors or, in the case of extended standard objects, e.g., the corners or edges of this object. In this way, the point cloud is enriched with additional points, resulting in a more complete and detailed picture of the environment. At the same time, comparing the measured signal distribution with the model distributions provides useful information for object classification.

[0035] Advantageous embodiments and further developments of the invention are set out in the dependent claims.

[0036] Feature-based filtering procedures can be used to validate the plausibility of extended objects, for example, a threshold comparison of the correlation between the measured distribution and the model distribution. Artificial intelligence methods can also be employed.

[0037] The following section explains an exemplary embodiment in more detail with reference to the drawing. The drawing shows:

[0038] Fig. 1 An example of a signal distribution for an extended object in a two-dimensional spectrum; R.415845

[0039] - 7 -

[0040] Fig. 2 shows the signal distribution according to Fig. 1 in a section along the

[0041] Line 11-12 in Fig. 1 ,

[0042] Fig. 3 shows a model distribution that can be compared with the signal distribution according to Fig. 1;

[0043] Fig. 4 shows an enlarged representation of the signal distribution according to Fig. 1 with measurement points used to create a point cloud;

[0044] Fig. 5 shows an example of an environmental scenario to be detected with a radar sensor;

[0045] Fig. 6 shows a product produced according to the inventive method.

[0046] Point cloud for the scenario shown in Fig. 5; and

[0047] Fig. 7 shows a point cloud created according to the inventive method in a distance-velocity space for the scenario shown in Fig. 5.

[0048] Fig. 1 shows an example of a two-dimensional radar spectrum of an object located by a radar sensor. A signal distribution 10 is represented in the form of contour lines, indicating the amplitude (the magnitude of the complex amplitude) of the signal received by the radar sensor as a function of the distance d and the velocity v. The distance-velocity space (dv space) is divided into cells or bins 12, as usual. The signal distribution 10 extends over several contiguous bins and thus represents an object that is extended in both the distance and velocity dimensions. For example, the object could be a tire located on the optical axis of the radar sensor and rolling away from the radar sensor.The upper and lower vertices of the tire are at the greatest distance (d) from the radar sensor, but generate only a relatively weak radar echo, while the middle part of the tire is at the shortest distance to the radar sensor and generates a significantly stronger radar echo. The lower vertex has a velocity of zero relative to the radar sensor. The velocity of the middle part is the velocity v. r , with which the tire identifies itself as R.415845

[0049] - 8 -

[0050] The entire area is removed from the radar sensor, and the top of the tire is moving away at twice the speed 2v. r The signal distribution therefore has a crescent shape with a single local maximum 14 at the smallest value of the distance.

[0051] Fig. 2 shows the signal distribution 10 in a section along line 11-11 in Fig. 1. On the vertical axis, the amplitude a at velocity v is shown. rspecified, with which the tire moves away.

[0052] Figure 3 shows a model distribution 10' that was calculated or measured for the constellation considered here (rolling tire). The actual signal distribution 10 in Figure 1 is distorted by noise and other interference effects and therefore only approximates, but does not exactly, correspond to the model distribution 10'. However, the agreement is sufficient to interpret the measured signal distribution 10 with some certainty as a rolling tire.

[0053] In practice, rolling tires can of course have different diameters, and they can also roll at different speeds. It would therefore not be practical to calculate a model distribution for each of these different scenarios. Before comparing the measured distribution 10 with the model distribution 10', the measured distribution will therefore be scaled so that its spectral extent in distance and speed matches the extent of the model distribution 10'.

[0054] In a conventional object localization method, one would only determine the distance and velocity bin in which the local maximum 14 of the measured distribution 10 is located. In a point cloud, the tire would consequently be represented by only a single point indicating the distance of the part of the tire closest to the radar sensor. However, the method proposed here selects from a multitude of stored model distributions the one that best matches the measured distribution 10, thereby allowing for a more precise characterization of the object that produced the measured distribution. Specifically, the measured distribution 10 is assigned to a standard object for which the R.415845

[0055] - 9 -

[0056] Model distribution 10' calculated or measured. Since the shape of this standard object is known, additional reflection points can be added to model distribution 10' that lie away from the local maximum and further characterize the contour and velocity behavior of the standard object. In Fig. 3, therefore, in addition to a reflection point 16 that marks the local maximum 14, further

[0057] Reflection points 18 are entered to further characterize the model object. After appropriate downscaling, these reflection points 18 can be transferred to the measured signal distribution 10, as shown on a larger scale in Fig. 4. The agreement between the measured signal distribution 10 and the model distribution 10' not only substantiates the hypothesis that the located object is a rolling tire, but also further describes the shape of this object using the additional reflection points 18. When generating the point cloud, each reflection point 18 then provides an additional point in the point cloud, making the shape of the tire more clearly defined.

[0058] If none of the stored model distributions is found where the correlation with the measured signal distribution 10 is above a certain threshold, the object is treated as a point-like, i.e., non-extended, object and its plausibility is checked using the criteria known for point objects.

[0059] In the simplified example shown in Figures 1 to 4, only the spectrum in the two-dimensional distance-velocity space was considered. However, with a radar sensor capable of resolving angles in both azimuth and elevation, the spectrum would have a total of four dimensions, meaning each reflection point would also have two angular coordinates. These coordinates can be converted into Cartesian coordinates within the point cloud's coordinate system, allowing the contour of the detected object in three-dimensional space to be more clearly defined in the point cloud. Based on the measured relative velocities of the reflection points, a four-dimensional point cloud could also be generated, which would additionally provide velocity information.

[0060] Figure 5 shows an example of a three-dimensional scenario, such as might be recorded by a video camera in a motor vehicle. In this R.415845

[0061] - 10 - In a simplified example, the scenario shows a vehicle 22 driving ahead on a road 20, a lamppost 24 and a traffic sign 26 on the right side of the road, and a guardrail 28 and a building 30 on the left side of the road. If this scenario is captured by a radar sensor with angle resolution in azimuth and elevation, a highly detailed point cloud can be constructed using the method described above, in which the objects are much more clearly recognizable than in point clouds constructed using conventional methods.

[0062] As a simplified example, Fig. 6 shows a two-dimensional point cloud 32 in the spatial dimensions x (perpendicular to the direction of travel) and y (in the direction of travel). The contours of the guardrail 28 and the building 30 are clearly recognizable. The vehicle 22 ahead is also represented by several points, which indicate the approximate shape and size of the vehicle. The traffic sign 26 and the traffic light pole 24 are also represented in the point cloud, albeit only by a few points, since these objects are essentially extended in the third dimension, which is not shown here.

[0063] Fig. 7 shows a point cloud 34 for the same scenario in dv space. The vehicle ahead is represented by a cluster 22' of points that detail the vehicle's distance / speed profile. All other objects are represented by a cluster 36 of points at zero speed and with varying distance values.

Claims

R.415845 - 11 - Claims 1. Method for detecting objects in a multidimensional radar spectrum, in which object hypotheses are formed based on the signal distribution (10) in the spectrum, the object hypotheses are made plausible based on a number of predefined criteria, and selected reflection points (16, 18) belonging to the made plausible objects are represented in a point cloud (32, 34), characterized in that the object hypotheses include hypotheses for extended objects for which the signal distribution in the spectrum extends over several adjacent bins (12), and that, for the purpose of making these objects plausible, the signal distributions (10) in the spectrum are compared with stored model distributions (10') for standard objects.

2. Method according to claim 1, wherein the plausibility check of extended objects includes a filter procedure by which those object hypotheses for which the correlation between the measured signal distribution (10) and each of the model distributions (10') is below a predetermined threshold are rejected as hypotheses for extended objects and replaced by hypotheses for point objects.

3. A method according to claim 1 or 2, wherein artificial intelligence is used to validate the extended objects.

Images