Method and system for radar data processing

Abstract

Methods and systems for radar data processing. A computer-implemented method for radar data processing, comprising the following steps performed by computer hardware components: acquiring radar data from a radar sensor mounted on a vehicle; determining at least one of a vehicle speed or a steering wheel angle of the vehicle; determining a subset of the radar data to be processed based on the at least one of the vehicle speed or the steering wheel angle of the vehicle.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to methods and systems for radar data processing, in particular adaptive radar field of view. BACKGROUND

[0002] Processing radar data is a fundamental task for various applications, such as (at least partially) autonomous driving vehicles. However, the computational cost of processing radar data is high.

[0003] Therefore, there is a need to provide methods and systems for efficiently processing radar data. SUMMARY

[0004] The present disclosure provides computer-implemented methods, computer systems and non-transitory computer-readable media. Embodiments are given in the description and drawings.

[0005] In one aspect, the present disclosure relates to a computer-implemented method for radar data processing, the method comprising the following steps performed by computer hardware components: acquiring (1002) radar data from a radar sensor mounted on a vehicle; determining at least one of a vehicle speed or a steering wheel angle of the vehicle; and determining a subset of the radar data to be processed based on the at least one of the vehicle speed or the steering wheel angle of the vehicle.

[0006] According to embodiments, the radar data comprises data having a range dimension and an angle dimension; and, the subset comprises a subset along the range dimension and / or along the angle dimension.

[0007] According to embodiments, the range dimension comprises a range of up to 60 m, or up to 100 m, or up to 135 m, or up to 150 m, or up to 200 m, or up to 210 m, or up to 300 m; and / or the angle dimension comprises an angle range of 30°, or 35°, or 45°, or 60°, or 75°, or 90°.

[0008] According to embodiments, the subset comprises a subset along a range dimension based on the vehicle speed.

[0009] According to embodiments, for a speed below a first speed threshold, the range dimension is limited to a first range limit.

[0010] According to embodiments, the subset comprises a subset along an angle dimension based on the vehicle speed.

[0011] According to embodiments, for a speed between the first speed threshold and a second speed threshold, the range dimension is limited to a second range limit and the angle dimension is limited between a first angle limit and a second angle limit.

[0012] According to an embodiment, for speeds above the second speed threshold, the angular dimension is limited between a third angular limit and a fourth angular limit.

[0013] For example, the distance and speed can be in the following ranges:

[0014] 1) 50m distance for speeds between 0 and 20 km / h (with a 10 km / h hysteresis);

[0015] 2) 105m distance for speeds between 20 km / h and 70 km / h (with a 10 km / h hysteresis); and

[0016] 3) 200m distance for speeds above 70 km / h (with a 10 km / h hysteresis) (e.g. corresponding to German Autobahn / rural maximum speed), including hysteresis overlap.

[0017] According to an embodiment, the subset comprises a subset along an angular dimension based on a steering wheel angle of the vehicle.

[0018] According to an embodiment, the subset along the distance dimension is determined based on filtering out input beam vectors for angular search.

[0019] According to an embodiment, the subset along the angular dimension is determined based on limiting the output of an angular search method.

[0020] According to an embodiment, the vehicle speed and / or the steering wheel angle of the vehicle are determined by a bus system of the vehicle.

[0021] In another aspect, the disclosure is directed to a computer system comprising a plurality of computer hardware components configured to perform some or all steps of a computer- implemented method as described herein.

[0022] The computer system can comprise a plurality of computer hardware components, such as a processor, e.g. a processing unit or a processing network, at least one memory, e.g. a memory unit or a memory network, and at least one non-transitory data storage. It is understood that further computer hardware components can be provided and used for performing steps of a computer-implemented method in the computer system. The non-transitory data storage and / or the memory unit can comprise a computer program for instructing the computer to perform several or all steps or aspects of a computer-implemented method as described herein, e.g. using the processing unit and the at least one memory unit.

[0023] In another aspect, the disclosure relates to a vehicle comprising a computer system as described herein; and a radar sensor.

[0024] In another aspect, the present disclosure is directed to a non-transitory computer readable medium comprising instructions for performing some or all of the steps or aspects of the computer-implemented methods described herein. The computer readable medium can be configured as an optical medium, such as a compact disc (CD) or a digital versatile disc (DVD), a magnetic medium, such as a hard disk drive (HDD), a solid-state drive (SSD), a read-only memory (ROM), such as a flash memory, or the like. Further, the computer readable medium can be configured as a data store accessible over a data connection, such as an internet connection. For example, the computer readable medium can be an online data repository or a cloud storage.

[0025] The present disclosure is also directed to a computer program for instructing a computer to perform some or all of the steps or aspects of the computer-implemented methods described herein.

[0026] With the methods and systems as described herein, it is possible to provide an adjustment of the processing of signals within a selected distance / field of view of a vehicle radar system based on operational parameters of the vehicle and / or the driving environment. Thus, it is possible to reduce the hardware requirements for machine learning (ML) radar recognition. BRIEF DESCRIPTION OF DRAWINGS

[0027] Exemplary embodiments and functions of the present invention are described herein in conjunction with the following drawings, in which:

[0028] FIG. 1A is a schematic illustration of full radar range;

[0029] FIG. 1B shows a full range full angle area;

[0030] FIG. 2A and FIG. 2B illustrates a low speed situation according to various embodiments.

[0031] FIG. 3A and FIG. 3B illustrates a medium speed situation according to various embodiments;

[0032] FIG. 4A and FIG. 4B illustrates a high speed situation according to various embodiments;

[0033] FIG. 5A and FIG. 5B illustrates an adjustment of the angle of interest area depending on the steering wheel angle according to various embodiments;

[0034] FIG. 6 is a schematic illustration of a radar sensor on a vehicle;

[0035] FIG. 7 is a schematic illustration of a maximum curvature on a highway;

[0036] FIG. 8 is a schematic illustration of a blocked field of view;

[0037] FIG. 9 is a schematic illustration of a system according to various embodiments; and

[0038] FIG. 10 illustrates a flowchart of a method for radar data processing according to various embodiments. DETAILED DESCRIPTION

[0039] In various embedded systems, methods of machine learning (ML) are used to solve perception tasks. The product of performing these tasks can be a deep neural network (DNN).

[0040] ML techniques can have advantages in many fields, but can lead to a large amount of computational demand. Since most techniques are developed for servers and web applications, the embedded resource demand is not always the initial demand. The strategy for embedded deployment of DNNs is to optimize techniques and specialized hardware. While these techniques are a prerequisite for computing these methods, reducing resources can still be important because of the large system cost they generate.

[0041] According to various embodiments, to limit the computational requirements, the input data can be delimited so that only relevant data is selected. In terms of perception, this can mean reducing the spatial region to the region where the detected object or obstacle is relevant to the vehicle.

[0042] According to various embodiments, by creating a region of interest (ROI), data based on irrelevant regions can be excluded from expensive computations.

[0043] In the case of radar sensors, this can be important because the spatial location of the input data is not defined. Furthermore, radars are designed to meet the expected requirements. According to various embodiments, certain driving parameters can be considered to effectively solve this problem.

[0044] In commonly used methods, the input data used is not limited by an effective input data reduction. Therefore, the computational complexity remains high, and thus the expected product cost also remains high.

[0045] In the following, creating an adaptive region of interest for a front-facing radar sensor according to various embodiments will be described.

[0046] A front-facing radar sensor can be designed to cover a large distance while producing an aperture angle that can overlook the driving situation in front of the motor vehicle close by.

[0047] FIG. 1AAn illustration 100 of the full radar range is shown. The ego vehicle 104 is shown as a black rectangle surrounded by a short-range field of view 106, which can be used for 360-degree surround recognition. The field of view 102 is shown in the example of a front-facing radar sensor. Since the whole area is covered, the ML radar recognition approach can have to be applied to the whole range and angle area 152 (as indicated by reference sign 150), resulting in higher computational requirements. FIG. 1B

[0048] According to various embodiments, at least one of the following can be used to provide an adaptive region of interest:

[0049] 1) adjusting the range region of interest according to the ego vehicle speed; and / or

[0050] 2) adjusting the angle region of interest according to the steering wheel angle.

[0051] Both values, the ego vehicle speed and the steering wheel angle, are available in a vehicle (e.g. a car or truck) and can be obtained from the available bus systems (e.g. CAN, LIN or Ethernet).

[0052] Depending on the complete system requirements, further possibilities to limit the input data can be applied additionally. Two examples of such further approaches are:

[0053] • an inner region of interest for short-range traffic recognition; and / or

[0054] • a filter for static objects depending on the ego vehicle speed, filtering against static Doppler speed (providing a“customized data cube”)

[0055] Both further approaches cannot solve the case of a front-facing radar for the recognition of moving objects.

[0056] According to various embodiments, the range region of interest can be adjusted according to the ego vehicle speed, as shown in FIGS. 2A-4B

[0057] FIG. 2A and FIG. 2B Illustrations 200 and 250 of a low-speed case according to various embodiments are shown.

[0058] In a low-speed case, it can be sufficient to limit the range of a front-facing radar sensor to a certain distance 202, since objects in this area can have an impact on the driving decision, while objects further away can not be relevant yet. With this limitation, the range dimension of the input data can be reduced to the area 252, and a wide field of view angle can be maintained.

[0059] FIG. 3A and FIG. 3B ​​Illustrations 300 and 350 show medium-speed conditions according to various embodiments.

[0060] At medium speeds, the required region 302 for the distance dimension side may become larger. However, limiting the angle region effectively restricts the input data, thus ignoring the sides of the angle region that the vehicle will not enter. The resulting region 352 is... FIG. 3B The distance versus angle is shown in the diagram.

[0061] FIG. 4A and FIG. 4B Illustrations 400 and 450 show high-speed scenarios according to various implementations.

[0062] At high speeds, looking as far ahead in the driving direction as possible can be important. This greater distance allows the angle subtended by 402 to be further reduced. The resulting region 452 is... FIG. 4B The distance versus angle is shown in the diagram.

[0063] To minimize the angle difference without overlooking relevant objects, the angle area of ​​interest can be adjusted based on the steering wheel angle, such as by referring to... FIG. 5A and FIG. 5B As described.

[0064] FIG. 5A and FIG. 5B Illustrations 500 and 550 show the adjustment of the area of ​​interest based on the steering wheel angle according to various embodiments.

[0065] The vehicle's travel path can be calculated based on the steering wheel angle. Therefore, the area of ​​focus can be directed towards the driving direction.

[0066] Therefore, the corner areas excluded can be saved to the maximum extent. According to regulatory rules, a curvature that can be expected to not exceed a certain limit is defined.

[0067] FIG. 6 Illustration 600 shows a radar sensor on a vehicle.

[0068] FIG. 7 The diagram 700 shows the maximum curvature on a highway.

[0069] FIG. 8 Illustration 800 shows an obstructed field of vision.

[0070] Use cases might include exemplary calculations of road curvature to generate the angle. These angles / curvatures, and their dependence on the target speed, can vary depending on feature requirements and are therefore not explicitly limited.

[0071] For example, a feature considering only lane objects (AEB) or potential lane change candidates of adjacent lanes can have requirements for a safety zone plus two sides of the area. A feature relying on bridge identification can also need to consider the side areas of the road where the bridge boundaries can occur.

[0072] Use cases (e.g. as FIGS. 6-8 shown) can ensure that also in quantitative measures, the methods and systems according to various embodiments can result in a significant reduction of hardware requirements.

[0073] To understand the quantitative impact of the method, a real use case is exemplified in FIGS. 6-8 .

[0074] The front radar of the vehicle 602 can be installed in the center of the front of the vehicle and can have an opening angle of 60° and a distance (outer circle) of 210 m.

[0075] The inner circle 604 shows an exemplary medium speed limit with a radius of 105 m (inner circle).

[0076] FIG. 7 A curvature limit of a typical German inner-city highway is shown, with a speed limit of 100 km / h. Exemplary angle limits for medium and high speed are shown in dashed lines. It can be seen that there is still distance considering a reasonable driving path area. This distance can be optimized according to requirements and feature needs.

[0077] The curvatures 702, 704 show the respective limits of a highway (Autobahn) in FIG. 7 .

[0078] In addition to the limits of the curvatures, it can also be seen that due to the near field obstacles of the road, a part of the outer circle 706 can actually still be blocked from direct perception (as FIG. 8 shown).

[0079] FIG. 9 A diagram 900 showing a system according to various embodiments is shown.

[0080] In the following, the implementation of the adaptive computation will be described.

[0081] To implement the computation steps within the architecture of the radar recognition network, it can be important to base on the given or at least possible architecture and the modules affected by the adaptive region of interest.

[0082] FIG. 9 A high-level overview of the currently existing lateral radar signal processing is shown on the left, and a hypothesis of the front radar signal processing is shown in the middle.

[0083] Depending on the module affected by the adaptive radar look- ahead zone restriction, the strategy of how to implement this can differ. The following can provide details and alternatives; however, it should be understood that other details and alternatives are possible.

[0084] Distance-dependent adjustment:

[0085] - Adaptively filter the input beam vectors for angle finding depending on the distance;

[0086] - This way:

[0087] 1) Strongly limit the number of beam vectors in low-speed cases;

[0088] 2) Less restriction, but still impact the computational demand, in medium distance; and

[0089] 3) No filtering in long-range cases.

[0090] Angle-dependent adjustment:

[0091] - In the front-compression step, the angle of arrival is known from the module, so the angle adjustment can limit the output of the angle finding early, before the module starts processing the input;

[0092] - This way:

[0093] 1) Strongly limit the angular dimension of the output tensor in high-speed cases;

[0094] 2) Less restriction, but still impact the computational demand, in medium distance; and

[0095] 3) No filtering in low-speed cases.

[0096] Machine learning (ML) object recognition:

[0097] - The implementation of the detection and classification itself can be based on a grid structure;

[0098] - This grid (for short range) can represent the vehicle coordinate system in space;

[0099] - A possible alternative can be to stay in polar coordinates;

[0100] - In all cases, the restriction of the angle and distance can limit the grid, so the computational demand can be greatly reduced, because:

[0101] 1) Regarding memory, this part of the network is the bottleneck:

[0102] 2) To reach long distances with the front radar sensor, the grid area can even exceed the demand seen from the look-ahead zone of interest.

[0103] As described herein, implementations can be provided for three fixed driving speed regimes (low, medium, and high). An alternative approach can be to slide the regions of interest to warp these regions in a continuous fashion. Implementation of this variant can imply:

[0104] - only one variable filtering limit for both initial steps: distance and angle dependent adjustment;

[0105] - constant grid size but with distortion from different distance / angle inputs.

[0106] Yet another alternative can be to implement a fixed grid:

[0107] - Radar needs to balance long and short range perception. At high speeds, it can be expected that perception needs to be focused on highways with limited curvature and curvature change. Only roads with limited curvature can really offer free fields of view of hundreds of meters or more. Due to this fact, radar processing can handle short range perception over the entire field of view, but limit long range perception of >80m to a region of ±30°-45°, which can reduce the computational effort needed by 25% to 50% (for longer distances) with no perceivable performance degradation.

[0108] - Fixed grid structure can ensure gradient propagation in E2E schemes and can create trainable systems with less need to structure the training systematically into motion sub-classes or as a two-stage scheme to train the system.

[0109] FIG. 10 A flowchart 1000 is shown, which illustrates a method for radar data processing according to various embodiments. At 1002, radar data can be acquired from a radar sensor mounted on a vehicle. At 1004, at least one of a vehicle speed or a steering wheel angle of the vehicle can be determined. At 1006, a subset of the radar data for processing can be determined based on the at least one of the vehicle speed or the steering wheel angle of the vehicle.

[0110] According to various embodiments, the radar data can include or can be data having a distance dimension and an angle dimension; and the subset can include or can be a subset along the distance dimension and / or along the angle dimension.

[0111] According to various embodiments, the distance dimension can comprise or can be a distance of at most 100 m, or at most 150 m, or at most 200 m, or at most 210 m, or at most 300 m; and / or the angle dimension can comprise or can be an angle range of 30°, or 45°, or 60°, or 75°, or 90°. It will be appreciated that these distance and angle dimensions are merely examples, and that other values can be used for these dimensions; for example, the distance dimension can be a distance of at most 135 m and the angle dimension can be 35°, or the distance dimension can be a distance of at most 60 m and the angle dimension can be 45°.

[0112] According to various embodiments, the subset can comprise or can be a subset along a distance dimension based on a speed of the vehicle.

[0113] According to various embodiments, for a speed below a first speed threshold, the distance dimension can be limited to a first distance limit.

[0114] According to various embodiments, the subset can comprise or can be a subset along an angle dimension based on a steering wheel angle of the vehicle.

[0115] According to various embodiments, for a speed between the first speed threshold and a second speed threshold, the distance dimension can be limited to a second distance limit and the angle dimension can be limited between a first angle limit and a second angle limit.

[0116] According to various embodiments, for a speed above the second speed threshold, the angle dimension can be limited between a third angle limit and a fourth angle limit.

[0117] According to various embodiments, the subset can comprise or can be a subset along an angle dimension based on a steering wheel angle of the vehicle.

[0118] According to various embodiments, the subset along the distance dimension can be determined based on filtering input beam vectors for angle finding.

[0119] According to various embodiments, the subset along the angle dimension can be determined based on limiting an output of an angle finding method.

[0120] According to various embodiments, the speed of the vehicle and / or the steering wheel angle of the vehicle can be determined from a bus system of the vehicle.

[0121] The various steps in steps 1002, 1004, 1006, as well as the additional steps described above, can be performed by computer hardware components.

Claims

1. A computer-implemented method for radar data processing, the method comprising the following steps performed by computer hardware components: acquiring (1002) radar data from a radar sensor mounted on a vehicle; determining (1004) at least one of a vehicle speed or a steering wheel angle of the vehicle; and determining (1006) a subset of the radar data to be processed based on the at least one of the vehicle speed or the steering wheel angle of the vehicle, wherein the radar data comprising data having a range dimension and an angle dimension; and wherein the subset comprises a subset along the range dimension; wherein the subset comprises a subset along an angle dimension based on the vehicle speed; wherein for speeds below a first speed threshold, the range dimension is limited to a first range limit; for speeds between the first speed threshold and a second speed threshold, the range dimension is limited to a second range limit greater than the first range limit and the angle dimension is limited between a first angle limit and a second angle limit; for speeds above the second speed threshold, the angle dimension is limited between a third angle limit and a fourth angle limit narrower than between the first angle limit and the second angle limit.

2. The computer-implemented method of claim 1, wherein, the range dimension comprises a range of at most 60 m, or at most 100 m, or at most 135 m, or at most 150 m, or at most 200 m, or at most 210 m, or at most 300 m; and / or wherein the angle dimension comprises an angle range of 30°, or 35°, or 45°, or 60°, or 75°, or 90°.

3. The computer-implemented method of claim 1, wherein the subset comprises a subset along an angle dimension based on the steering wheel angle of the vehicle.

4. The computer- implemented method of claim 1, wherein, the subset along the range dimension is determined based on filtering out input beam vectors for angle finding.

5. The computer-implemented method of claim 1, wherein the subset along the angle dimension is determined based on limiting output of an angle finding method.

6. The computer- implemented method of claim 1, wherein, the vehicle speed and / or the steering wheel angle of the vehicle are determined by a bus system of the vehicle.

7. A computer system comprising a plurality of computer hardware components configured to perform the steps of the computer-implemented method of at least one of claims 1 to 6.

8. A vehicle comprising: the computer system of claim 7; and a radar sensor.

9. A non-transitory computer readable medium comprising instructions for performing the computer-implemented method of at least one of claims 1 to 6. ​

Images