METHODS AND SYSTEMS FOR RADAR DATA PROCESSING

Description

FIELD

[0001] The present disclosure relates to methods and systems for radar data processing, in particular adaptive radar sight.BACKGROUND

[0002] Processing radar data is an essential task for various applications, for example for (at least partially) autonomously driving vehicles. However, processing radar data is computationally expensive.

[0003] Accordingly, there is a need to provide methods and systems for efficient processing of radar data.

[0004] GB 2 422 499 A discloses a vehicular object detection system comprising an object sensor and a controller. The controller is configured to compare a sensed range of an object within a range gate based on the azimuth angle of the object. The object sensor is switchable between a plurality of sensing fields, wherein a range gate for each sensor sensing field is programmable to form a contiguous sensing region closely matching the shape of the desired detection zone. The object sensor may be a radar, ultrasonic, laser or infrared sensor. The range gate and parameters of the sensing field are each programmable through the controller and may be automatically varied based on vehicle parameters such as vehicle speed, steering wheel angle and steering wheel rate.

[0005] EP 3 296 762 A1 discloses a monitoring-target-region setting device including an object detector that detects objects present around an own vehicle, a scene discriminator that discriminates on the basis of position information of the own vehicle and map information around the own vehicle whether the own vehicle is located near a place where an assumed blind spot region is present, a monitoring region setter that sets, when the scene discriminator discriminates that the own vehicle is located near the place, while the own vehicle passes near the place, a region including at least the assumed blind spot region when viewed from the own vehicle as a monitoring region where alarm is performed when objects are detected, and an object selector that selects, in the monitoring region, according to whether the detected objects satisfy the determined condition, whether the detected objects are set as a target for which alarm is performed.

[0006] US 2010 / 165323 A1 discloses a device for recording a geometry of an environment of a device in a detection field with the aid of laser scanning which includes a laser beam controlled by an oscillating micromechanical mirror. The detection field is specifiable in the vertical and horizontal directions by adapting an amplitude of oscillation of the micromechanical mirror.SUMMARY

[0007] The present disclosure provides a computer implemented method, a computer system and a non-transitory computer readable medium according to the independent claims. Preferred embodiments are given in the dependent claims.DRAWINGS

[0008] Exemplary embodiments and functions of the present disclosure are described herein in conjunction with the following drawings, showing schematically: Fig. 1Aan illustration of a full radar range; Fig. 1Ba full range and full angle area; Fig. 2A and 2Billustrations of a low-speed situation according to various embodiments.; Fig. 3A and 3Billustrations of a mid-speed situation according to various embodiments; Fig. 4A and 4Billustrations of a high-speed situation according to various embodiments; Fig. 5A and 5Billustrations of an adaption of the angular region of interest dependent on the steering wheel angle according to various embodiments; Fig. 6an illustration of a radar sensor on a vehicle; Fig. 7an illustration of maximum curvatures on highways; Fig. 8an illustration of a blocked view; Fig. 9an illustration of a system according to various embodiments; and Fig. 10a flow diagram illustrating a method for radar data processing according to various embodiments. DETAILED DESCRIPTION

[0009] In various embedded systems, method for machine learning (ML) are used to solve perception tasks. The product which is performing these tasks may be deep neural networks (DNN).

[0010] The ML technology may be superior in many fields, but may lead to large computational requirements. As a large part of technology was developed for server and web applications, embedded resource requirements are not always an initial requirement. Strategies for embedded deployment of DNNs are optimization techniques and special purpose hardware. Even though those technologies are a precondition to computing those methods, it may still be important to reduce the resource as it creates large systems cost.

[0011] According to various embodiments, in order to limit the computational requirements, the input data may be delimited, such that only relevant data is chosen. Regarding perception, this may mean to reduce the spatial area to one in which detected objects or obstacles are relevant for the vehicle.

[0012] According to various embodiments, by creation of a region of interest (ROI), the data according to areas which are irrelevant may be excluded from expensive computation.

[0013] In case of a radar sensor, this may be non-trivial, since a spatial position of an input data is not defined. Furthermore, radars are already designed to fit to the expected requirements. According to various embodiments, certain driving parameters may be taken into account for effectively solving this problem.

[0014] In commonly used methods, the input data which is used is not limited by an effective input data reduction. Thereby, the computational complexity stays high and thereby the expected product cost stay high as well.

[0015] In the following, creating an adaptive region of interest for front radar sensors according to various embodiments will be described.

[0016] Front radar sensors may be designed to reach a long range and in the same time create an aperture angle which enables overlooking driving situations in the front near to the car.

[0017] Fig. 1A shows an illustration 100 of a full radar range. The ego vehicle 104 is illustrated as a black rectangle surrounded by a near range field of view 106, which may be used for 360 degree surround recognition. The field of view 102 is illustrated in an example of a front radar sensor. As the full area is covered, the ML radar recognition method may have to be applied on the full range and angle area 152 (as illustrated in illustration 150 of Fig. 1B), creating high computational requirements.

[0018] According to various embodiments, an adaptive region of interest may be provided using the following: 1) Adapting the range region of interest dependent on ego vehicle speed; and preferably further 2) Adapting the angular region of interest dependent on the steering wheel angle-

[0019] Both values (the ego vehicle speed and the steering wheel angle) may be available in the vehicles (for example cars or trucks) and may be obtained from a bus system (for example CAN, LIN, or Ethernet) available.

[0020] Further possibilities to limit the input data may be applied additionally or not, dependent on the full system requirements. Two examples of such further methods are: Inner region of interest for near range traffic recognition; and / or Static object filtering dependent on ego speed and thereby filtering for static doppler speed (which may provide a "tailored datacube")

[0021] Both further methods do not solve the case of a front radar intended to recognize moving objects.

[0022] According to various embodiments, the range region of interest is adapted dependent on ego vehicle speed, as illustrated in Fig. 2A to Fig. 4B.

[0023] Fig. 2A and Fig. 2B show illustrations 200 and 250 of a low-speed situation according to various embodiments.

[0024] In low-speed situations, it may be sufficient to limit the range of the front radar sensor to a certain range 202, as objects in that area would have impact on the driving decisions, and objects further away may not yet be relevant. By this limitation, the range dimension of the input data can be reduced to area 252, and the angle field of view may be kept wide.

[0025] Fig. 3A and Fig. 3B show illustrations 300 and 350 of a mid-speed situation according to various embodiments.

[0026] In mid-speed situations, the area 302 needed from range dimension side gets bigger. But a limitation of the angular area limits the input data effectively, leaving out the sides of the angular field in which the vehicle will not enter. The resulting area 352 is illustrated in the range vs. angle diagram of Fig. 3B.

[0027] Fig. 4A and Fig. 4B show illustrations 400 and 450 of a high-speed situation according to various embodiments.

[0028] In high-speed situations, it is important to look far forward into the driving direction, as far as possible. In favor of this large range, the angular opening angle 402 is further be reduced. The resulting area 452 is illustrated in the range vs. angle diagram of Fig. 4B.

[0029] In order to maximize the reduction of angular opening angle without missing relevant objects, the angular region of interest may further be adapted dependent on the steering wheel angle, like will be described with reference to Fig. 5A and Fig. 5B. Fig. 5A and Fig. 5B show illustrations 500 and 550 of an adaption of the angular region of interest dependent on the steering wheel angle according to various embodiments.

[0030] According to the steering wheel angle, it may be possible to calculate the driving path of the ego vehicle. By this, the region of interest may be aimed into the direction of driving.

[0031] Thus, the savings for excluded angular area may be maximized. According to regulatory rules the curvature which can be expected not to be exceeded is defined.

[0032] Fig. 6 shows an illustration 600 of a radar sensor on a vehicle.

[0033] Fig. 7 shows an illustration 700 of maximum curvatures on highways.

[0034] Fig. 8 shows an illustration 800 of a blocked view.

[0035] Use cases may include exemplary calculation of road curvature and thereby resulting opening angle. These angles / curvatures and depending on that target speeds may depend on feature requirements and are thereby not explicitly defined.

[0036] For example, a feature which only considers in lane objects (AEB) or neighboring lane potential candidates to lane change may have a requirement targeting that area plus a safety area at the sides. A feature depending on bridge recognition may have requirements also considering the road's side areas, where a bridges boundaries might be expected.

[0037] Use cases (for example as illustrated in Fig. 6 to Fig. 8) may assure that also in quantitative measures, the methods and systems according to various embodiments can lead to a significant reduction of hardware needs.

[0038] In order to understand the quantitative impact of that method, a realistic use case is illustrated in Fig. 6 to Fig. 8.

[0039] A front radar of a vehicle 602 may be mounted at a central position in the vehicle front and may have an opening angle of 60° and a range of 210m (outer circle)

[0040] The inner circle 604 shows an exemplary mid speed limit with a radius of 105m (inner circle).

[0041] Fig. 7 shows the curvature limitation on a typical German inner city highway with 100 kph speed limitation. With dotted lines the exemplary angular limitation for mid and high speed are shown. It may be seen that there is still a distance taking into account a reasonable driving path area. This distance may be objective to optimizations according to requirements and feature demands.

[0042] The curvature 702, 704 shows the respective limitation for highway (high speed) in Fig. 7.

[0043] Besides the limitation in curvature, it may also be seen that a part of the outer circle 706 may be realistically still blocked from direct perception due to near field barriers of the road (as for example illustrated in Fig. 8).

[0044] Fig. 9 shows an illustration 900 of a system according to various embodiments.

[0045] In the following, realization of adaptive computation will be described.

[0046] In order to realize the computation steps within the architecture of the radar recognition network, it is important to base on the given or at least possible architectures and modules influenced by the adaptive region of interest.

[0047] Fig. 9 shows an high level overview of the currently existing side radar signal processing on the left and an assumption of how the front radar signal processing will look like in the middle.

[0048] Depending on the module impacted by the adaptive radar sight region of interest limitations, the strategy how to realize this may be different. The following may provide details and alternatives; however, it will be understood that other details and alternatives are possible.

[0049] Range dependent adaptation: Filter out the input beamvectors for angle finding according to the range adaptation; By this: 1) strongly limit the number of beam vectors in the low-speed case; 2) limit less but still impacting computational demand in the mid range; and 3) do not filter in the long range case

[0050] Angle dependent adaptation: in the front part compression step the angle of arrival is known from the module, so the angle adaptation can limit the output of angle finding already earlier, before this module starts to process the input; By this: 1) strongly limit the number of angle dimension of the output tensor in the high-speed case; 2) limit less but still impacting computational demand in the mid range; and 3) do not filter in the low speed case.

[0051] ML (machine learning) object recognition: the implementation of the detection and classification itself may be based on a grid structure; that grid (for near range) may be spatially representing the vehicle coordinate system; a possible alternative solution may be to stay in polar coordinate system; in all cases the limitation of the angle and range may limit the grid and by this may achieve a massive reduction in computational needs because: 1) regarding memory this part of the network is a bottle neck: 2) in order to reach high distance with the front radar sensor the grid area would exceed even the demand seen from the near range area of interest.

[0052] As described herein, a realization of 3 fixed states of driving speed (low, mid and high) is provided. An alternative may be to have a sliding adaptation of the region of interest, distorting the areas in a continuous manner. The realization of such a variant may imply: only a variable filtering limit for both initial steps: range and angle dependent adaptation; a constant grid size but distorted input from different ranges / angles.

[0053] A further alternative may be to implement a fixed grid: A radar needs to balance long range & short range perception. At high speeds, one can expect the perception needs to be focused on high speed roads with limited curvature and curvature changes. Only roads with limited curvatures may truly provide a free field of view over several hundred meters. Due to this fact, a radar processing may process a short range perception over the full field of view, but to limit long range perception >80m to an area of +-30°- 45 degree, which may cut required calculation effort by 25% to 50% for longer ranges while there should be no perceivable performance drop. Fixed grid structures may ensure gradient propagation in E2E solutions and may create a trainable system with less need to structure training systematically into motion subclasses or to train the system as two stage approach.

[0054] Fig. 10 shows a flow diagram 1000 illustrating a method for radar data processing according to various embodiments. At 1002, radar data may be acquired from a radar sensor mounted on a vehicle. At 1004, at least one of a speed of the vehicle or a steering wheel angle of the vehicle may be determined. At 1006, a subset of the radar data for processing is determined based on the at least one of the speed of the vehicle or the steering wheel angle of the vehicle.

[0055] The radar data includes data with a range dimension and an angle dimension; and the subset comprises a subset along the angle and range dimension based on the speed of the vehicle.

[0056] According to various embodiments, the range dimension may include or may be a range up to 100 m, or up to 150 m, or up to 200 m, or up to 210 m, or up to 300 m; and / or the angle dimension may include or may be an angle range of 30°, or 45°, or 60°, or 75°, or 90°. It will be understood that these range dimension and angle dimensions are merely examples, and that other values may be used for these dimensions; for example, the range dimension may be a range up to 135 m and the angle dimension may be 35°, or the range dimension may be a range up to 60 m and the angle dimension may be 45°.

[0057] According to various embodiments, for a speed below a first speed threshold, the range dimension may be limited to a first range limit.

[0058] According to the invention, for a speed below between the first speed threshold and a second speed threshold, the range dimension may be limited to a second range limit and the angle dimension may be limited between a first angle limit and a second angle limit.

[0059] According to the invention, for a speed higher than the second speed threshold, the angle dimension may be limited between a third angle limit and a fourth angle limit.

[0060] According to various embodiments, the subset may include or may be a subset along the angle dimension based on the steering wheel angle of the vehicle.

[0061] According to various embodiments, the subset along the range dimension may be determined based on filtering out input beamvectors for angle finding.

[0062] According to various embodiments, the subset along the angle dimension may be determined based on limiting an output of an angle finding method.

[0063] According to various embodiments, the speed of the vehicle and / or the steering wheel angle of the vehicle may be determined from a bus system of the vehicle.

[0064] Each of the steps 1002, 1004, 1006 and the further steps described above may be performed by computer hardware components.

Claims

1. Computer implemented method for radar data processing, the method comprising the following steps carried out by computer hardware components: - acquiring (1002) radar data from a radar sensor mounted on a vehicle; - determining (1004) at least one of a speed of the vehicle or a steering wheel angle of the vehicle; and - determining (1006) a subset of the radar data for processing based on the at least one of the speed of the vehicle or the steering wheel angle of the vehicle; wherein the radar data comprises data with a range dimension and an angle dimension; and wherein the subset comprises a subset along the range dimension; wherein the subset comprises a subset along the angle dimension based on the speed of the vehicle; wherein for a speed below a first speed threshold, the range dimension is limited to a first range limit; characterized in that for a speed between the first speed threshold and a second speed threshold, the range dimension is limited to a second range limit bigger than the first range limit, and the angle dimension is limited between a first angle limit and a second angle limit; wherein for a speed higher than the second speed threshold, the angle dimension is limited between a third angle limit and a fourth angle limit, narrower than the first angle limit and the second angle limit.

2. The computer implemented method of claim 1, wherein the range dimension comprises a range up to 60 m, or up to 100 m, or up to 135 m, or up to 150 m, or up to 200 m, or up to 210 m, or up to 300 m; and / or wherein the angle dimension comprises an angle range of 30°, or 35°, or 45°, or 60°, or 75°, or 90°.

3. The computer implemented method of at least one of claims 1 to 2, wherein the subset comprises a subset along the angle dimension based on the steering wheel angle of the vehicle.

4. The computer implemented method of at least one of claims 1 to 3, wherein the subset along the range dimension is determined based on filtering out input beamvectors for angle finding.

5. The computer implemented method of at least one of claims 1 to 4, wherein the subset along the angle dimension is determined based on limiting an output of an angle finding method.

6. The computer implemented method of at least one of claims 1 to 5, wherein the speed of the vehicle and / or the steering wheel angle of the vehicle is determined from a bus system of the vehicle.

7. Computer system, the computer system comprising a plurality of computer hardware components configured to carry out steps of the computer implemented method of at least one of claims 1 to 6.

8. Vehicle comprising: the computers system of claim 7; and the radar sensor.

9. Non-transitory computer readable medium comprising instructions for carrying out the computer implemented method of at least one of claims 1 to 6.