Method and system for determining a beam vector list for object detection

Abstract

A computer-implemented method for determining a list of beam vectors for object detection may include the following steps implemented by a computer hardware component: receiving radar data from at least one radar sensor; determining distance information and speed information based on the radar data; and determining at least one beam vector list based on the distance information and the velocity information using a neural network.

Description

Technical Field

[0001] The present disclosure relates to methods and systems for determining a list of beam vectors for object detection. Background Art

[0002] Various sensors, such as cameras, radar sensors, or lidar sensors, can be used in automotive applications to monitor the environment of a vehicle. Driver assistance systems can utilize data captured from sensors, for example, by analyzing the data to detect objects. Object detection can be based on the strength of the signal received with the sensor. However, the received signal can be difficult to distinguish from noise, which can also be captured by the sensor. Therefore, reliable object detection can be a cumbersome task.

[0003] Therefore, there is a need to provide methods and systems for object detection that efficiently lead to accurate results. Summary of the invention

[0004] The present disclosure provides a computer implemented method, a computer system, a vehicle and a non-transitory computer readable medium according to the independent claims. Embodiments are given in the dependent claims, the detailed description and the drawings.

[0005] In one aspect, the present disclosure relates to a computer-implemented method for determining a list of beam vectors for object detection, wherein the method includes the following steps performed by computer hardware components: receiving radar data from at least one radar sensor; determining range information and velocity information based on the radar data; and determining at least one list of beam vectors based on the range information and the velocity information using a neural network.

[0006] In other words, the computer-implemented method described herein can determine a sub-selection of beam vectors or beamforming vectors of a radar sensor that may be important for a specific task. The specific task can be, for example, object detection, such as vulnerable road users (VRUs), such as cyclists or pedestrians. It should be understood that the systems and methods described herein can also be used for other tasks. The sub-selection of beam vectors can be described by a beam vector list. The beam vector list can include an ordered or unordered list, wherein each entry of the beam vector list can correspond to a beam vector of a radar sensor. The beam vector list can be determined based on radar data received by the radar sensor. The radar data can include raw radar data, such as analog-to-digital converter (ADC) data. The radar sensor can be mounted on a vehicle and configured to transmit and receive radar data. Specifically, the radar sensor can include multiple antennas, wherein each of the multiple antennas can be configured to transmit and receive radar data. The transmitted radar data can include electromagnetic waves emitted by the radar sensor. The received radar data can include a portion of the transmitted electromagnetic waves reflected from an object near the vehicle.

[0007] The computer-implemented method may determine distance information of an object in the environment of the vehicle based on the radar data of the radar sensor. The distance information may describe the distance between the object and the vehicle. A Fast Fourier Transform (FFT) method may be used to determine the distance information. In addition, the computer-implemented method may determine velocity information of an object in the environment of the vehicle based on the radar data of the radar sensor. The velocity information may be radial velocity information or Doppler information of the object.

[0008] The neural network may be used to determine a list of beam vectors based on distance information of objects in the vehicle environment and speed information of objects in the vehicle environment. The neural network may be a trained neural network. For example, a plurality of appropriate training data may be used to train the neural network. In other words, the training data may be input data of the neural network, and output data of the neural network may be compared with valid true data, and then the weights of the neural network may be adjusted so that the output data is close to the valid true data. The neural network may be a convolutional neural network (CNN) or a recurrent neural network (RNN). It should be understood that other types of neural networks may also be used to determine at least one list of beam vectors based on the distance information and speed information of the radar sensor.

[0009] According to an embodiment, the method may further include the following steps performed by a computer hardware component: determining a signal strength of the radar data based on the distance information and the speed information; transmitting the signal strength as an input to a neural network; and determining at least one list of beam vectors based on the distance information, speed information and signal strength of the radar data using the neural network. The signal strength (e.g., norm) of the radar data may be the signal strength of the received radar data. Specifically, the signal strength may be the signal strength of a reflected electromagnetic wave received by a radar sensor from an object near the vehicle. The signal strength may depend on the size of the object in the vehicle environment. In addition, the signal strength may depend on the distance (or range) between the object and the radar sensor.

[0010] According to an embodiment, the method may further include the following steps performed by the computer hardware component: determining at least one angle based on the distance information and the velocity information; transmitting the at least one angle as an input to the neural network; and determining at least one list of beam vectors based on the distance information, the velocity information, and the at least one angle using the neural network. The at least one angle may include an estimate of an azimuth and / or an elevation angle. The azimuth and / or elevation angle may describe an angle between the radar sensor and an object or target from which the radar sensor may receive radar data.

[0011] According to an embodiment, a Discrete Fourier Transform (DFT) method may be used to determine the at least one angle.

[0012] According to an embodiment, the method may further include the following steps performed by the computer hardware component: determining at least one numerical value based on the distance information and the speed information; transmitting the at least one numerical value as an input to the neural network; and using the neural network to determine at least one list of beam vectors based on the distance information, the speed information and the at least one numerical value. The at least one numerical value may include context information, for example, an average value over a range region of the range-Doppler map and / or an average value over a Doppler region of the range-Doppler map.

[0013] According to an embodiment, at least one numerical value may be an average value of a distance region and / or an average value of a Doppler region of a range-Doppler map. The range-Doppler map may include a two-dimensional matrix. One dimension may describe the distance information of the received radar data of the radar sensor. Another dimension may describe the Doppler information or speed information of the received radar data of the radar sensor. The range-Doppler map may be divided into a plurality of range-Doppler regions (in other words, range-Doppler cells). A plurality of range-Doppler regions may include a plurality of range regions and a plurality of Doppler regions (in other words: speed regions). The range-Doppler map may be a dense map. The range-Doppler map may visualize the received radar signals of objects in the vehicle environment.

[0014] According to an embodiment, the beam vector corresponding to the entry of at least one beam vector list may include an azimuth and an elevation angle. Therefore, in addition to the distance information of the object in the vehicle environment and the speed information of the object in the vehicle environment, the azimuth and elevation angle of the object may also be known, which may improve the positioning of the object relative to the radar sensor or the vehicle.

[0015] According to an embodiment, the method may further include the following steps performed by a computer hardware component: determining a selection value based on an output of the neural network using a sigmoid function; and determining at least one list of beam vectors based on the selection value. For each of the plurality of range-Doppler units, a selection value may be determined. Each of the plurality of selection values ​​may include a value between 0 and 1. For example, the selection value may describe how important the corresponding range-Doppler unit may be for a particular task. A high selection value may mean that the corresponding range-Doppler unit is more important for a particular task than a range-Doppler unit with a low selection value.

[0016] According to an embodiment, at least one beam vector list may correspond to a plurality of cells in a range-Doppler map. The number of cells of the range-Doppler map or the number of range-Doppler cells may be an integer. The number of entries of at least one beam vector list may correspond to the number of range-Doppler cells in the range-Doppler map. For example, at least one beam vector list may include 1, 2, 3, 4 or 5 entries. Thus, the number of entries of at least one beam vector list is 1, 2, 3, 4 or 5. Thus, the number of cells in the range-Doppler map is 1, 2, 3, 4 or 5, respectively. The number of cells corresponding to the number of entries in at least one beam vector list may determine the cells in the range-Doppler map that are most important for a specific task. The specific task may be detecting pedestrians, cyclists, etc. in an environment of a vehicle.

[0017] According to an embodiment, the number of cells may be a predefined fixed number of cells in the range-Doppler map or a dynamic number of cells in the range-Doppler map. Thus, it may be predetermined how many cells in the range-Doppler map may be selected for further processing. For example, the predefined fixed number of cells in the range-Doppler map may be 1, 2, or 3. Thus, only 1, 2, or 3 cells in the range-Doppler map that may correspond to 1, 2, or 3 entries in at least one beam vector list may be selected for further processing. Further processing may include object detection using another neural network. The number of cells may depend on a specific task. The number of cells corresponding to the entries of at least one beam vector list may be different for different specific tasks. For example, detecting pedestrians in a vehicle environment may include a higher (or lower) number of cells than detecting cyclists in a vehicle environment. Thus, a higher (or lower) number of entries of at least one beam vector list may be selected for further processing. The number of cells may also be a dynamic number. For example, the number of cells may depend on the number of objects detected in the vehicle environment.

[0018] According to an embodiment, a neural network may be trained end-to-end. A neural network may be trained for a specific task using appropriate training data and corresponding valid truth data. For example, a neural network may be trained to determine at least one list of beam vectors for pedestrian detection in a vehicle environment. The neural network may be a deep neural network (DNN) comprising a plurality of layers, wherein each layer may address a portion of a specific task, e.g., extracting relevant features from input data of the neural network. The neural network may be trained end-to-end with another neural network, e.g., the other neural network may be a neural network for object detection.

[0019] According to an embodiment, the method may further comprise the following steps performed by the computer hardware component: transmitting the at least one beam vector list to another neural network. The another neural network may be used to further process the at least one beam vector list. In particular, to further process the entries of the at least one beam vector list or the corresponding beam vectors. The another neural network may be used for object detection based on the at least one beam vector list. It should be understood that the tasks of determining the at least one beam vector list and object detection may be performed using one neural network, wherein the one neural network may include the neural network and the another neural network. The neural network and the another neural network may be trained end-to-end.

[0020] In another aspect, the present disclosure relates to a computer system comprising a plurality of computer hardware components, the computer system being configured to carry out several or all of the steps of the computer-implemented methods described herein.The computer system may be part of a vehicle.

[0021] The computer system may include a plurality of computer hardware components (e.g., a processor, such as a processing unit or a processing network, at least one memory, such as a memory unit or a memory network, and at least one non-transient data storage). It should be understood that further computer hardware components may be provided and used to carry out the steps of the computer-implemented method in the computer system. The non-transient data storage and / or the memory unit may include a computer program for instructing a computer, such as using a processing unit and at least one memory unit, to perform some or all steps or aspects of the computer-implemented method described herein.

[0022] In another aspect, the disclosure relates to a vehicle comprising a computer system as described herein and at least one radar sensor. The vehicle may be a car or a truck, and the sensor may be mounted on the vehicle. The sensor may be directed to an area at the front or rear or side of the vehicle.

[0023] In another aspect, the present disclosure relates to a non-transitory computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform some or all steps or aspects of the computer-implemented methods described herein. The computer-readable medium may be configured as an optical medium, such as a compact disc (CD) or a digital versatile disc (DVD); a magnetic medium, such as a hard disk drive (HDD); a solid-state drive (SSD); a read-only memory (ROM), such as a flash memory; or the like. In addition, the computer-readable medium may be configured as a data storage accessible via a data connection, such as an Internet connection. The computer-readable medium may be, for example, an online data repository or a cloud storage.

[0024] The present disclosure also relates to a computer program for instructing a computer to execute several or all steps or aspects of the computer-implemented method described herein.

[0025] It should be understood that features described in conjunction with the method may be implemented in a computer system as well as a non-transitory computer readable medium, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Exemplary embodiments and functionality of the present disclosure are described herein in conjunction with the following drawings, which schematically illustrate:

[0027] Figure 1 According to the solution of the embodiment;

[0028] Figure 2 According to another embodiment;

[0029] Figure 3 A flowchart illustrating a method for determining a list of beam vectors for object detection according to various embodiments; and

[0030] Figure 4 According to various embodiments, a computer system having a plurality of computer hardware components configured to perform the steps of a computer-implemented method for determining a list of beam vectors for object detection. DETAILED DESCRIPTION

[0031] In automotive applications using radar sensors, signal processing can typically start with analog-to-digital converter (ADC) data. The radar sensor may include multiple antennas. The ADC data may be three-dimensional. The ADC data may be raw radar data. The analog-to-digital converter may change an analog signal that is continuous in time and amplitude into a digital signal that is discrete in time and amplitude to determine the ADC data. The analog-to-digital converter data may be transformed along fast time (e.g., a time index within a chirp) and slow time (e.g., a time index across a chirp). There may be data points for multiple sampling points within each antenna of the radar sensor, each chirp (slow time), and each chirp (fast time). A chirp may be a radar signal in which the frequency of the radar signal increases or decreases over time.

[0032] Range information can be retrieved from the ADC data by running a Fast Fourier Transform (FFT) method on the fast time dimension of the chirp. Radial velocity information or "Doppler" information can be determined by subsequently running an FFT on the slow time dimension of the chirp. Additionally, a direction of arrival module can be used along the antenna dimension to determine the azimuth and / or elevation of the detection based on the ADC data.

[0033] The first two steps, running a corresponding FFT method on the fast time dimension of the chirp and running a corresponding FFT on the slow time dimension of the chirp, can be performed (in other words: executed) with relatively little computational effort. The final step of determining at least one of the detected azimuth or elevation angle may require more computational cost than the first two steps. Therefore, this final step may be expensive, extended, dynamic and / or complex.

[0034] In order to reduce the computational cost for determining the azimuth or elevation of the detection, a thresholding algorithm may be used on the output of the first two steps. In other words, a thresholding algorithm may be performed on the range information and the radial velocity information, which may be determined using the corresponding FFT of the chirp. The thresholding algorithm may be, for example, a constant false alarm rate (CFAR) algorithm. Thus, using the CFAR algorithm, the direction of arrival algorithm of the direction of arrival module may be performed only on a selection of the range-Doppler cells of the range-Doppler map, rather than on all the range-Doppler cells of the range-Doppler map. The selection of the range-Doppler cells may be determined using the CFAR algorithm.

[0035] The range-Doppler map may represent the signal strength of a signal received from a radar sensor in a coordinate system having an ordinate axis showing range information and an abscissa axis showing possible detected Doppler information (or radial velocity information). The range-Doppler map may include a plurality of range-Doppler cells. This means that a two-dimensional range-Doppler map may contain the signal strength of each range-Doppler cell.

[0036] The classic CFAR algorithm can be performed on a range-Doppler plot where the antenna dimensions are compressed by the signal strength (where the signal strength can be based on a norm or can be a norm) or an approximation thereof. The CFAR algorithm can be used to find a local threshold that separates the noise from the detected signal (in other words: the target).

[0037] However, for object detection, especially for detecting vulnerable road users (VRUs), a threshold based purely on signal strength may be suboptimal. There may be no guarantee that an object of interest (e.g., a pedestrian) has a higher signal strength than a potentially uninteresting object (e.g., a utility pole or a curb). Weak reflectors like pedestrians in the local environment may be represented by a signal strength in the range-Doppler map that is below the threshold of the CFAR algorithm, and thus the information of the weak reflector may be lost.

[0038] Neural networks can be used in the field of radar signal processing and for high-level tasks such as object detection or semantic segmentation. However, neural networks (deep learning or machine learning) can also be used for thresholding tasks. Specifically, neural networks can be used as a thresholding algorithm or a filtering algorithm to determine a list of beam vectors or a sub-selection of a list of beam vectors.

[0039] A range-Doppler-antenna cube including range information and Doppler information may be provided as input to a neural network. The neural network may output a list of beam vectors (in other words: a list of beam vectors), as will be described in more detail further below. The use of a neural network may enable the algorithm to be adapted to the optimization goal of a subsequent neural network in an end-to-end training manner for high-level tasks such as object detection or semantic segmentation. Thus, the method described herein may be optimized for different goals, for example, to determine a list of beam vectors based on radar data for pedestrian detection, utility pole detection, or both, pedestrian detection and utility pole detection, etc.

[0040] Classical algorithms may only consider information within a range, for example, the CFAR thresholding algorithm may be purely based on Doppler bins in a neighborhood in the Doppler map. In addition, the classical CFAR algorithm may be mainly based on the norm along the antenna dimension, for example, the signal strength in one range-Doppler bin.

[0041] Figure 1 A scheme 100 according to an embodiment is shown. Radar data 102 may be received from at least one radar sensor. The at least one radar sensor may include at least one antenna configured to transmit and receive radar signals. The radar data 100 may include analog-to-digital converter (ADC) data. The ADC data or radar data 100 may include a fast time dimension of a chirp and a slow time dimension of a chirp for each antenna of the radar sensor, wherein the chirp may be a radar signal of an antenna of the at least one radar sensor, wherein the frequency of the radar signal increases or decreases over time.

[0042] A Fast Fourier Transform (FFT) method 104 (which may also be referred to as a range and Doppler FFT) may be performed on the fast time dimension of the chirp to determine range information of the target or detection in the field of view of at least one radar sensor. In addition, a Fast Fourier Transform method 104 may be performed on the slow time dimension of the chirp to determine radial velocity information (in other words: Doppler information) of the target or detection in the field of view of at least one radar sensor.

[0043] The range information and radial velocity information may be represented in a range-Doppler map 106 (which may also be referred to as a range-Doppler antenna cube) for each antenna of the at least one radar sensor. The range-Doppler map 106 including the range information and radial velocity information for each antenna of the radar sensor may be provided as an input to a neural network-based sub-selection module 108. The neural network-based sub-selection module 108 may include an end-to-end trained neural network, such as the neural network 214, as will be described in more detail below. Furthermore, the neural network-based sub-selection module 108 may determine at least one beam vector list 110 based on the range information and radial velocity information determined by a corresponding FFT applied to the radar data 102 for each antenna of the at least one radar sensor. The at least one beam vector list 110 may include a plurality of beam vectors that may be used for further processing, such as for object detection using another neural network.

[0044] Figure 2 Another solution 200 according to an embodiment is shown. Figure 1 The radar data 102, the fast Fourier transform method 104, the range-Doppler map 106, the neural network-based sub-selection module 108 and the beam vector list 110 described in the embodiment can also be used in this embodiment. In this embodiment, at least one input data of the plurality of input data (in other words: input features) can be used as an input to the neural network 214 provided in the neural network-based sub-selection module 108. The neural network 214 can be a convolutional neural network (CNN) in the range-Doppler dimension. It should be understood that other architectures of the neural network 214 are also possible.

[0045] The neural network based sub-selection module 108 may determine a signal strength 202 or signal strength data of the radar data 102 based on information of the range-Doppler map 106. Specifically, the signal strength 202 of the radar signal (or chirp) received by at least one radar sensor may be determined based on the range information and the speed information. The signal strength 202 may be transmitted as an input to the neural network 214. The neural network based sub-selection module 108 may determine at least one beam vector list 110 based on the range information, the speed information, and the signal strength 202 of the radar data 102.

[0046] According to another embodiment, at least one angle 204 may be determined based on information of the range-Doppler map 106. At least one angle 204 may be an estimate of an azimuth and / or elevation angle of a possible detection of an object in the field of view of the at least one radar sensor. In particular, the estimate of angle 204 may be determined based on the range information and the velocity information. A discrete Fourier transform (DFT) method may be used to determine angle 204 or an estimate of angle 204. In particular, a DFT method having a number of output bins that may be approximately equal to the number of antennas of the at least one radar sensor may be used to determine angle 204 or rough angle information (in other words: an estimate of angle 204).

[0047] The at least one angle 204 may be transmitted as an input to the neural network 214. The neural network based sub-selection module 108 may determine the at least one beam vector list 110 based on the distance information, the speed information, and the at least one angle 204. The neural network based sub-selection module 108 may determine the at least one beam vector list 110 based on the distance information, the speed information, and the signal strength 202 of the radar data 102.

[0048] According to another embodiment, at least one numerical value 206 may be determined based on information of the range-Doppler map 106. The at least one numerical value 206 may be an average value of a range region (in other words: an average value of a range cell) in the range-Doppler map 106 and / or an average value of a Doppler region (or radial velocity cell) of the range-Doppler map 106. Specifically, the at least one numerical value 206 may be determined based on the range information and the velocity information. The at least one numerical value 206 may be transmitted as an input to the neural network 214. The neural network-based sub-selection module 108 may determine at least one beam vector list 110 based on the range information, the velocity information, and the at least one numerical value 206. In another embodiment, the neural network-based sub-selection module 108 may determine at least one beam vector list 110 based on the range information, the velocity information, the signal strength 202, the at least one angle 204, and the at least one numerical value 206 of the radar data 102.

[0049] According to an embodiment, a sigmoid function (in other words: sigmoid activation) may be used to determine the selection value 208 based on the output of the neural network 214. For each range-Doppler cell of the range-Doppler map 106, the selection value 208 may be a value between 0 and 1. The at least one beam vector list 110 may be determined based on the selection value 208. For example, the selection module 210 may select a dynamic number or a fixed number of cells in the range-Doppler map 106 based on the selection value 208 to determine the at least one beam vector list 110. Thus, the at least one beam vector list 110 may correspond to a plurality of cells in the range-Doppler map 106. Thus, the at least one beam vector list 110 may include only a portion or a plurality of portions of the range-Doppler map 106. The selection of the number of cells in the range-Doppler map 106 may be implemented using a threshold, for example, the number of cells may be the number of the "K" highest values ​​in the range-Doppler map 106. It should be understood that more complex selection methods may also be used to select a plurality of cells in the range-Doppler map 106. For example, a neural network based sub-selection module 108 may be used to select the number of cells in the range-Doppler map 106 .

[0050] According to an embodiment, at least one beam vector list 110 may be determined using a combination module 212 based on the output of the selection module 210 (e.g., a selected dynamic number or a fixed number of cells in the range-Doppler map 106, and information of the range-Doppler map 106 (range information and radial velocity information for each antenna of at least one radar sensor)). The combination module 212 may select information of a range-Doppler cell in the range-Doppler map 106 based on the output of the selection module 210. For example, the combination module 212 may select a range-Doppler cell based on a cell index of a range-Doppler cell corresponding to a dynamic number or a fixed number of cells as described by the output of the selection module 210. The selected range-Doppler cell may be a sub-selection of a range-Doppler cell or a beam vector that may indicate an object in the vehicle environment. The at least one beam vector list 110 may include a sub-selection.

[0051] The beam vectors corresponding to the at least one beam vector list 110 may be forwarded or transmitted to another neural network for further processing. For example, the other neural network may be used for object detection based on the at least one beam vector list 110. The neural network 214 and the other neural network ( Figure 2 104 (not shown) can be trained end-to-end, for example by providing training data that includes input data similar to radar data 102 (ADC data) and valid true value data corresponding to the expected output of the other neural network when the ADC data is provided as input, and training neural network 214 and the other neural network simultaneously.

[0052] Figure 3 A flow chart of a method for determining a list of beam vectors for object detection according to various embodiments is shown. At 302, radar data can be received from at least one radar sensor. At 304, range information and velocity information can be determined based on the radar data. At 306, at least one list of beam vectors can be determined based on the range information and velocity information using a neural network.

[0053] According to an embodiment, the method may further include the following steps performed by computer hardware components: determining a signal strength of the radar data based on the distance information and the speed information; transmitting the signal strength as an input to a neural network; and using the neural network to determine at least one beam vector list based on the distance information, speed information and signal strength of the radar data.

[0054] According to an embodiment, the method may further include the following steps performed by computer hardware components: determining at least one angle based on distance information and speed information; transmitting the at least one angle as input to a neural network; and determining at least one beam vector list based on the distance information, speed information and at least one angle using the neural network.

[0055] According to an embodiment, a Discrete Fourier Transform (DFT) method may be used to determine the at least one angle.

[0056] According to an embodiment, the method may further include the following steps performed by computer hardware components: determining at least one numerical value based on the distance information and the speed information; transmitting the at least one numerical value as input to the neural network; and using the neural network to determine at least one beam vector list based on the distance information, the speed information and the at least one numerical value.

[0057] According to an embodiment, the at least one value may be an average value of a range region and / or an average value of a Doppler region of the range-Doppler map.

[0058] According to an embodiment, a beam vector corresponding to an entry of the at least one beam vector list may include an azimuth angle and an elevation angle.

[0059] According to an embodiment, the method may further include the following steps performed by computer hardware components: determining a selection value based on an output of the neural network using a sigmoid function; and determining at least one beam vector list based on the selection value.

[0060] According to an embodiment, at least one beam vector list may correspond to a plurality of cells in the range-Doppler map.

[0061] According to an embodiment, the number of cells may be a predefined fixed number of cells in the range-Doppler map or a dynamic number of cells in the range-Doppler map.

[0062] According to an embodiment, the neural network may be trained end-to-end.

[0063] According to an embodiment, the method may further comprise the following step performed by the computer hardware component: transmitting at least one list of beam vectors to another neural network.

[0064] Each of steps 302, 304, 306 and the further steps described above may be performed by computer hardware components, such as those described in reference to Figure 4 As described.

[0065] Figure 4 A computer system 400 having multiple computer hardware components configured to perform the steps of a computer-implemented method for determining a list of beam vectors for object detection is shown in accordance with various embodiments. The computer system 400 may include a processor 402, a memory 404, and a non-transitory data store 406. A radar sensor 408 may be provided as part of the computer system 400 (e.g., Figure 4 ), or may be provided external to the computer system 400.

[0066] Processor 402 may execute instructions provided in memory 404. Non-transitory data storage 406 may store computer programs including instructions that may be transferred to memory 404 and then executed by processor 402. Radar sensor 408 may be used to determine radar data, such as radar data provided to methods as described herein.

[0067] The processor 402, the memory 404 and the non-transitory data storage 406 may be coupled to each other, for example, via an electrical connection 410 (such as, for example, a cable or a computer bus) or via any other suitable electrical connection to exchange electrical signals. The radar sensor 408 may be coupled to the computer system 400, for example, via an external interface, or may be provided as part of the computer system (in other words: inside the computer system 400, for example coupled via the electrical connection 410).

[0068] The terms “coupling” or “connection” are intended to include a direct “coupling” (eg, via a physical link) or a direct “connection” as well as an indirect “coupling” or an indirect “connection” (eg, via a logical link), respectively.

[0069] It should be understood that what is described with respect to one of the above methods may be similarly applicable to the computer system 400 .

[0070] The methods described herein may involve a learnable CFAR approach based on range-Doppler-antenna information and may propose a solution for determining a list of beam vectors based on radar data that may overcome a complete reliance on the signal strength of the radar data, which may be more adaptable to surrounding or following algorithms, for example, for object detection or semantic segmentation.

[0071] In contrast to most classical algorithms that can only consider information in one range (i.e., thresholding algorithms like CFAR can be purely based on neighborhoods in Doppler), the methods and systems described herein can use antenna information of the radar sensor and neighborhoods in range. The methods and systems described herein can also utilize signal strength in a spatial region (which can also be called a signature), i.e., distance and (roughly estimated) angle plus Doppler information. Thus, this information combination can be more flexible to also detect targets that may not have the highest signal strength.

[0072] In summary, the methods and systems described herein can solve the problem that the classical CFAR method is not adaptable to further processing and relies entirely on signal strength. The methods described herein can be trained end-to-end and can find signatures in radar data to select the best beam vector for closed optimization goals, such as detecting a specific class of objects. To this end, the methods described herein can also consider information from different antennas, coarse angle information, and neighboring range cells separately. Therefore, the methods and systems described herein may not be limited to the signal strength of each range-Doppler cell.

[0073] Reference numerals list

[0074] 100 Solutions

[0075] 102 Radar Data

[0076] 104 Fast Fourier Transform (FFT) Method

[0077] 106 Range-Doppler map

[0078] 108 Neural Network-Based Subselection Module

[0079] 110 beam vector list

[0080] 200 Solution

[0081] 202 Signal Strength

[0082] 204 Angle

[0083] 206 Value

[0084] 208 Select Value

[0085] 210 Select Module

[0086] 212 Combination Modules

[0087] 214 Neural Networks

[0088] 300 illustrates a method for determining at least one list of beam vectors for object detection.

[0089] Flowchart

[0090] 302 Step of receiving radar data from at least one radar sensor

[0091] 304 Step of determining distance information and speed information based on radar data

[0092] 306 Determine at least one beam direction based on the distance information and the speed information using a neural network

[0093] Steps to Quantity List

[0094] 400 Computer system according to various embodiments

[0095] 402 Processor

[0096] 404 Memory

[0097] 406 Non-transient data storage

[0098] 408 Radar Sensor

[0099] 410 connection.

Claims

1. A computer-implemented method for determining a list of beam vectors (110) for object detection, The method comprises the following steps performed by computer hardware components: - receiving (302) radar data (102) from at least one radar sensor; - determining (304) distance information and speed information based on the radar data (102); - determining (306) at least one list of beam vectors (110) based on the distance information and the speed information using a neural network (214).

2. The method of claim 1 further comprising the following steps performed by computer hardware components: - determining a signal strength (202) of the radar data (102) based on the distance information and the speed information; - transmitting the signal strength (202) as input to the neural network (214); and - using the neural network (214) to determine the at least one list of beam vectors (110) based on the range information, the velocity information and the signal strength (202) of the radar data (102).

3. The method of claim 1 further comprising the following steps performed by computer hardware components: - determining at least one angle based on the distance information and the speed information (204); - transmitting the at least one angle (204) as input to the neural network (214); and - using the neural network (214) to determine the at least one list of beam vectors (110) based on the distance information, the velocity information and the at least one angle (204).

4. The method according to claim 3, It is characterized in that The at least one angle is determined (204) using a discrete Fourier transform (DFT) method.

5. The method of at least one of claims 1 to 4, further comprising the following steps performed by computer hardware components: - determining at least one value based on the distance information and the speed information (206); - transmitting the at least one value (206) as input to the neural network (214); and - using the neural network (214) to determine the at least one list of beam vectors (110) based on the distance information, the speed information and the at least one numerical value (206).

6. The method according to claim 5, It is characterized in that The at least one value (206) is a mean value of a range region and / or a mean value of a Doppler region of the range-Doppler map (106).

7. The method according to at least one of claims 1 to 4, It is characterized in that The beam vector corresponding to an entry of the at least one beam vector list (110) includes an azimuth angle and an elevation angle.

8. The method of at least one of claims 1 to 4, further comprising the following steps performed by computer hardware components: - determining a selection value (208) based on an output of the neural network (214) using a sigmoid function; and - determining said at least one list of beam vectors (110) based on said selection value (208).

9. The method according to at least one of claims 1 to 4, It is characterized in that The at least one beam vector list (110) corresponds to a plurality of cells in the range-Doppler map (106).

10. The method according to claim 9, It is characterized in that The number of cells is a predefined fixed number of cells in the range-Doppler map (106) or a dynamic number of cells in the range-Doppler map (106).

11. The method according to at least one of claims 1 to 4, It is characterized in that The neural network (214) is trained end-to-end.

12. The method of at least one of claims 1 to 4, further comprising the following steps performed by computer hardware components: - transmitting the at least one list of beam vectors (110) to another neural network.

13. A computer system, the computer system (400) comprising a plurality of computer hardware components configured to carry out the steps of the computer-implemented method of at least one of claims 1 to 12.

14. A vehicle comprising the computer system (400) of claim 13 and the at least one radar sensor.

15. A non-transitory computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the computer-implemented method of any one of claims 1 to 12.

Images