Radar system for identifying number of objects using machine learning model

Abstract

This document describes techniques and systems related to a radar system that uses a machine learning model to identify the number of objects within each range-Doppler bin. For example, a radar system includes a processor that obtains radar data associated with an object and processes the radar data to generate a beam vector. The processor then uses a machine learning model to identify the number of objects within each range-Doppler bin using the extracted features of the beam vectors (e.g., amplitude variation, signal-to-noise ratio, sub-array beam vector correlation). The processor selects a particular angle determination technique based on identifying a single object or multiple objects. In this manner, the described systems and techniques more accurately identify the number of objects in each range-Doppler bin, thereby improving computational efficiency and robustness of subsequent angle measurements.

Description

Background Art

[0001] Many vehicles use radar systems to detect and track objects in the surrounding environment. Advanced driver assistance systems (ADAS) (e.g., adaptive cruise control, lane change assistance, and emergency braking) rely on radar data (including the distance and azimuth associated with the object) to monitor and respond to detected objects. Some radar systems use different angle determination algorithms according to the number of objects in the same range-Doppler bin. For example, the fast Fourier transform (FFT) is used to determine the azimuth associated with a single object, and the FFT algorithm is computationally cheaper than many other angle determination algorithms. When multiple objects are located in the same range-Doppler bin, other computationally more expensive algorithms (e.g., super-resolution techniques) can be used. The techniques for determining whether a specific range-Doppler bin includes a single object or multiple objects generally have lower classification accuracy, and therefore may result in computational inefficiency and inaccurate angle determination, which may reduce the response time or efficacy of ADAS features. Summary of the invention

[0002] This document describes techniques and systems related to a radar system that uses a machine learning model to identify the number of detected objects. For example, the radar system includes one or more processors that obtain radar data associated with objects that reflect radar signals and process the radar data to generate multiple beam vectors. The processor then uses the machine learning model to use characteristics of the beam vectors (e.g., amplitude variation, signal-to-noise ratio, subarray beam vector correlation) to identify the number of objects within each range-Doppler bin. The processor then selects a specific angle determination technique based on whether a single object or multiple objects are detected within a particular range-Doppler bin. In this way, the described systems and techniques can more accurately identify the number of objects within each range-Doppler bin, thereby improving the computational efficiency and robustness of subsequent angle determinations.

[0003] This document also describes methods performed by the above-summarized techniques and components and other configurations of radar systems set forth herein, as well as apparatuses for performing these methods.

[0004] This summary introduces simplified concepts related to a radar system using a machine learning model for identifying the number of detected objects, which are further described in the detailed description and accompanying drawings. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Details of one or more aspects of a radar system using a machine learning model for identifying a number of objects are described in this document with reference to the following figures. The same numbers are generally used throughout the figures to reference similar features and components: Figure 1 An example environment is shown in which a radar system may use a machine learning model to identify a quantity of objects in accordance with techniques of this disclosure; Figure 2 An example configuration of a vehicle having a radar system that uses a machine learning model to identify a number of objects is shown; Figure 3 An example conceptual diagram showing a radar system that uses a machine learning model to identify a number of objects; Figure 4 An example block diagram of processing radar signals as part of a radar system that uses a machine learning model to identify a number of objects is shown; Figure 5 An example conceptual diagram showing a decision tree machine learning model for identifying a quantity of objects according to the techniques of this disclosure; Figure 6 An example flow chart illustrating a method for training a machine learning model for a radar system to identify a number of objects in accordance with techniques of this disclosure; and Figure 7 A flow chart illustrating an example method for a radar system using a machine learning model for identifying a number of objects in accordance with techniques of this disclosure. DETAILED DESCRIPTION Overview

[0006] Radar systems can be configured as an important sensing technology that ADAS features (e.g., adaptive cruise control, lane change assist, emergency braking) rely on to obtain information about the surrounding environment. Specifically, radar systems can provide accurate distance, speed, and angle information of objects detected in the host vehicle's surroundings. For example, a vehicle-based system can use a radar system to detect objects in or near the road and take necessary actions (e.g., reduce speed, change lanes) when necessary to avoid a collision.

[0007] The radar system may use various angle determination processes or techniques to estimate angles (e.g., azimuth and / or elevation) associated with detected objects. For example, when a single object is detected in a particular range-Doppler bin, the radar system may use an FFT technique to determine the azimuth. Such FFT techniques are typically computationally cheaper than many other angle determination techniques (e.g., super-resolution techniques). When multiple objects are detected in the same range-Doppler bin, computationally more expensive super-resolution techniques are typically used to determine the azimuth.

[0008] Radar systems typically use predetermined thresholds for each range-Doppler bin to distinguish between single object scenes and multiple object scenes. For example, the amplitude variation of the beam vector and the subarray beam vector correlation are compared to a signal-to-noise ratio (SNR) related threshold to determine whether a particular range-Doppler bin includes a single object or multiple objects. However, this approach is typically not very accurate and may be biased toward multiple object scenes. For example, inaccurate beam vector calibration, out-of-field-of-view detection, and poor fascia integration may cause many single object beam vectors to be incorrectly classified as having multiple objects. Classifying a single object scene as a multiple target scene may result in false detections, angle jumps between consecutive frames, and / or computational inefficiencies.

[0009] In contrast, this document describes techniques and systems for accurately identifying multiple objects using machine learning models. For example, a radar system includes one or more processors that can accurately identify the number of objects within a specific range-Doppler bin. The processor obtains radar data associated with an object that reflects a radar signal and processes the radar data to generate a beam vector. Each beam vector includes radar data from multiple channels of the radar system for a specific range-Doppler bin. The processor then uses a machine learning model to identify the number of objects in each range-Doppler bin. Features of the radar data associated with the beam vector (e.g., amplitude variation, SNR, subarray beam vector correlation) are used to determine whether there is a single object or multiple objects in each range-Doppler bin. The processor then uses a specific angle determination technique for each range-Doppler bin based on whether there is a single object or multiple objects. In this way, the described systems and techniques can increase the object quantity classification accuracy score (e.g., from 60% to 96%), thereby improving the computational efficiency of subsequent angle determinations and reducing false detections and angle jumps.

[0010] This is just one example of the techniques and systems described for a radar system that uses a machine learning model to identify a number of objects. This document describes other examples and implementations. Operating Environment

[0011] Figure 1 An example environment 100 is shown in which a radar system 104 uses a machine learning model to identify a number of objects in accordance with techniques of this disclosure. In the depicted environment 100, the radar system 104 is mounted to or integrated within a vehicle 102 traveling on a road 106. Within an instrument field of view 108, the radar system 104 can detect one or more objects 110 (e.g., object 110-1 and object 110-2) near the vehicle 102.

[0012] Although shown as a truck, vehicle 102 may represent other types of motorized vehicles (e.g., automobiles, motorcycles, buses, tractors, semi-trailers), non-motorized vehicles (e.g., bicycles), rail vehicles (e.g., trains), watercraft (e.g., boats), aircraft (e.g., airplanes), or spacecraft (e.g., satellites). Manufacturers may mount radar system 104 to any mobile platform, including mobile mechanical or robotic equipment.

[0013] In the depicted implementation, the radar system 104 is mounted on the front of the vehicle 102 and illuminates the object 110. The radar system 104 can detect the object 110 from any exterior surface of the vehicle 102. For example, the vehicle manufacturer can integrate, install or attach the radar system 104 to the front, bumper, side mirror, headlight, taillight, or any other interior or exterior location where the object 110 needs to be detected. In some cases, the vehicle 102 includes multiple radar systems 104, such as a first radar system 104-1 and a second radar system 104-2 that provide a larger instrument field of view 108. In general, the vehicle manufacturer can design the position of one or more radar systems 104 to provide a specific field of view 108 that includes an area of ​​interest. Example fields of view 108 include a 360-degree field of view, one or more 180-degree fields of view, one or more 90-degree fields of view, etc., which can overlap to form a field of view 108 of a specific size.

[0014] Objects 110-1 and 110-2 include one or more materials that reflect radar signals. Depending on the application, objects 110 may represent targets of interest. For example, objects 110 may include parked vehicles, moving vehicles, roadside signs, road obstacles, other moving objects, or debris on road 106.

[0015] Radar system 104 transmits electromagnetic (EM) radiation by transmitting EM signals or waveforms via antenna elements. Radar system 104 may detect and track objects 110 in environment 100 by transmitting and receiving one or more radar signals. For example, radar system 104 may transmit EM signals between 100 and 400 gigahertz (GHz), between 4 and 100 GHz, or between approximately 70 and 80 GHz.

[0016] The radar system 104 may include a transmitter 112 including at least one antenna to transmit the EM signal. The radar system 104 may also include a receiver 114 including at least one antenna to receive a reflected version of the EM signal. The transmitter 112 includes one or more components for transmitting the EM signal. The receiver 114 includes one or more components for detecting the reflected EM signal. Manufacturers may incorporate the transmitter 112 and the receiver 114 together on the same integrated circuit (e.g., configured as a transceiver) or separately on different integrated circuits.

[0017] The radar system 104 also includes one or more processors 116 (e.g., energy processing units, electronic control units) and a computer readable storage medium (CRM) 118. The processor 116 may be a microprocessor or a system on a chip. The processor 116 may execute computer executable instructions stored in the CRM 118. For example, the processor 116 may process the EM energy received by the receiver 114 and determine the number of objects 110 within each range-Doppler bin of the radar data using the object quantity identification module 120. The processor 116 may then use the number of identified objects 110 for each range-Doppler bin to determine which angle determination technique to use to determine the relative position (e.g., azimuth, elevation) of each object 110. The object quantity identification module 120 may detect and use various features associated with the radar data or its beam vector (e.g., amplitude variation, subarray beam vector correlation, SNR). In this way, the processor 116 may improve the classification accuracy between single target scenes and multiple target scenes, and use the corresponding angle determination techniques to improve the accuracy and computational efficiency of the angle determination process.

[0018] Processor 116 may also generate radar data for at least one automotive system. For example, processor 116 may control an autonomous or semi-autonomous driving system of vehicle 102 based on the processed EM energy from receiver 114. For example, the autonomous driving system may control the operation of vehicle 102 to maneuver around object 110, or to slow down or stop to avoid a collision with object 110. As another example, a semi-autonomous driving system may warn an operator of vehicle 102 that object 110 is in road 106.

[0019] The object population identification module 120 receives radar data (e.g., beam vectors associated with EM energy received by the receiver 114) and uses features associated with the beam vectors to determine whether one or more objects are in a particular range-Doppler bin. The object population identification module 120 uses a machine learning model 122 to assist in the described operations and functions. The radar system 104 may implement the object population identification module 120 and the machine learning model 122 as computer executable instructions executed by the processor 116 in the CRM 118, hardware, software, or a combination thereof.

[0020] The machine learning model 122 can use the features extracted from the beam vector to identify the number of objects 110. The machine learning model 122 is trained to receive the beam vector and the features extracted from the beam vector to perform object quantity identification. The output of the machine learning model 122 includes an identification of the number of objects 110 for each range-Doppler bin.

[0021] In some implementations, the machine learning model 122 can perform classification, clustering, tracing, and / or other tasks. For classification, supervised learning techniques can be used to train the machine learning model 122. For example, the object population identification module 120 can train the machine learning model 122 using training data (e.g., valid ground truth data) including beam vectors of radar data labeled as single object scenes or multiple object scenes. The labels can be applied manually by an engineer or provided by other techniques (e.g., based on data from other sensor systems). The training data set can include beam vectors of radar data similar to those of radar data input to the machine learning model 122 during operation of the vehicle 102.

[0022] The machine learning model 122 can be trained offline, for example, in a training computing system, and then provided for storage and implementation on one or more computing devices. For example, the training computing system can include a model trainer. The training computing system can be included in the computing device that implements the machine learning model 122 or can be separate from the computing device. Figure 6 The training of the machine learning model 122 is described in more detail.

[0023] The machine learning model 122 can implement logistic regression, decision tree or multi-layer perceptron. If the processor 116 has relatively limited processing power, these types of machine learning models can be selected. If the processor 116 has greater processing power, other types of models can be used. Logistic regression usually uses a sigmoid function to map the final output to an interval between 0 and 1, indicating the probability of input in each category (e.g., a single object scene or multiple object scenes). The extracted features and polynomial features can be used to improve the performance of logistic regression. The decision tree uses a white box model to show which extracted features are important in classification. For example, the depth of the decision tree of the machine learning model 122 can be set to three. The multi-layer perceptron is a feedforward neural network that uses a linear combination of features and a nonlinear activation function to generate a classification output. During the training phase, the weights of the activation function are iteratively updated or learned using a gradient-based optimization method (e.g., via back propagation). For example, the machine learning model 122 can use a hidden layer with five neurons. The complexity of the selected machine learning model (e.g., the number of polynomial features, the depth of the decision tree, or the number of neurons) may be optimized based on the processing power of the processor 116 and the desired classification accuracy.

[0024] As an illustrative implementation, Figure 1 Vehicle 102 is shown traveling on road 106. Radar system 104 detects object 110. Radar system 104 processes radar data to generate a beam vector of the radar data and extracts features associated with the beam vector. As described above, vehicle 102 may also include at least one automotive system that relies on data from radar system 104, such as a driver assistance system, an autonomous driving system, or a semi-autonomous driving system. Radar system 104 may include an interface to automotive systems that rely on the data. For example, processor 116 outputs a signal based on the EM energy received by receiver 114 via the interface.

[0025] Typically, automotive systems use radar data provided by radar system 104 to perform functions. For example, a driver assistance system can provide blind spot monitoring and generate an alert indicating a potential collision with object 110. Radar data can also indicate when it is safe or unsafe to change lanes. An autonomous driving system can move vehicle 102 to a specific location on road 106 while avoiding a collision with object 110. Radar data provided by radar system 104 can also provide information about the distance to object 110 and the location of object 110 to enable the autonomous driving system to perform emergency braking, perform lane changes, or adjust the speed of vehicle 102. Transportation Configuration

[0026] Figure 2An example configuration of a vehicle 102 with a radar system 104 that uses a machine learning model 122 to identify a number of objects is shown. Figure 1 As depicted, vehicle 102 may include radar system 104, processor 116, CRM 118, object population identification module 120, and machine learning model 122. Vehicle 102 may also include one or more communication devices 202, angle determination modules, and one or more vehicle-based systems 210.

[0027] The communication device 202 may include a sensor interface and a vehicle-based system interface. For example, when individual components of the object quantity identification module 120 are integrated within the vehicle 102, the sensor interface and the vehicle-based system interface may transmit data over a communication bus of the vehicle 102.

[0028] The object quantity identification module 120 may also include an input processing module 204 and a feature extraction module 206. The input processing module 204 may receive radar data as input from the receiver 114. Typically, the radar data is received as raw data output from an analog-to-digital converter. The radar data may be processed by the output processing module 204 (e.g., interference detection, interference mitigation, fast time processing, slow time processing, integration, thresholding, peak detection, and / or peak estimation) to provide better detection resolution and generate beam vectors to facilitate extraction of features associated with the radar data of the beam vectors. Specifically, the input processing module 204 may process the radar data to generate beam vectors from which features for the machine learning model 122 may be extracted. For example, a beam vector associated with a single object may differ in amplitude variation, SNR, and subarray beam vector correlation compared to a beam vector associated with multiple objects.

[0029] The feature extraction module 206 can process the beam vectors to extract features associated with the radar data. As described above, the features can include at least one of amplitude variation, SNR, and subarray beam vector correlation. The feature extraction module 206 can also perform additional processing to extract additional features associated with the beam vector. The feature extraction module 206 then inputs the features into the machine learning model 122 to determine the number of detected objects for each range-Doppler bin of the range-Doppler map. The machine learning model 122 outputs the number of detected objects for each range-Doppler bin to the angle determination module 208. In other implementations, the feature extraction module 206 is implemented as part of the machine learning model 122.

[0030] The angle determination module 208 may perform additional processing to determine the angle (e.g., azimuth, elevation) associated with each object 110. The angle determination module 208 may utilize different angle determination techniques based on the number of identified objects for a particular range-Doppler bin. For example, if the machine learning model 122 identifies a single object for a particular range-Doppler bin, the angle determination module 208 may use FFT beamforming techniques to determine the angle associated with the single object. If the machine learning model 122 identifies multiple objects for a particular range-Doppler bin, the angle determination module 208 may use super-resolution techniques to determine the angles associated with multiple objects. The angle determination module 208 may then provide the angle determination to the vehicle-based system 210.

[0031] The vehicle 102 also includes a vehicle-based system 210 (such as an assisted driving system 212 and an autonomous driving system 214) that relies on data from the radar system 104 (including the angle provided by the angle determination module 208) to control the operation of the vehicle 102 (e.g., braking, lane changes). In general, the vehicle-based system 210 can use the data provided by the object quantity identification module 120 to control the operation of the vehicle 102 and perform certain functions. For example, the assisted driving system 212 can warn the driver of the object 110 and perform an evasive maneuver to avoid a collision with the object 110. As another example, the autonomous driving system 214 can navigate the vehicle 102 to a specific destination to avoid a collision with the object 110. Object quantity identification

[0032] Figure 3 An example conceptual diagram 300 of a radar system 104 using a machine learning model 122 to identify a number of objects is shown. Figure 2 As described, the vehicle 102 may include the input processing module 204, the feature extraction module 206, and the angle determination module 208 of the object quantity identification module 120. The conceptual diagram 300 shows example inputs, outputs, and operations of the object quantity identification module 120, but the object quantity identification module 120 is not necessarily limited to the order or combination of inputs, outputs, and operations shown here. In addition, any one or more operations may be repeated, combined, or reorganized to provide other functions.

[0033] Radar system 104 provides time series frames of EM energy to input processing module 204 as radar data 302. Radar data 302 may be low-level radar data or point cloud radar data. Radar data 302 includes information associated with object 110 in multiple dimensions, including range space, Doppler space, elevation space, and azimuth space.

[0034] At operation 304, the input processing module 204 performs initial processing on the radar data 302 to generate a beam vector (e.g., a Doppler beam vector). The beam vector may include all ranges and Doppler intervals and include intensity data associated with nearby objects (e.g., both stationary and moving objects) fused together. Specifically, the beam vector includes radar from multiple channels of multiple range-Doppler bins of the range-Doppler map. The input processing module 204 may perform additional processing on the beam vector to separate the intensity data associated with the various detected objects.

[0035] At operation 306, the feature extraction module 206 extracts features from the beam vector. For example, the feature extraction module 206 may analyze the beam vector to determine the amplitude change (err 变化 ) and subarray beam vector correlation (BV 相关性 ) value. The amplitude change can be determined using equations (1) and (2): in and n represents the number of elements in the corresponding beam vector, and BV i represents the i-th element in the corresponding beam vector. The subarray beam vector correlation can be determined using equation (3): Among them BV 子阵列,j and BV 子阵列,k Represents the subarray beam vector of a channel. For example, BV 相关性,1 represents the correlation between the subarray from the first channel (e.g., TX1) and the subarray from the second channel (e.g., TX2). Similarly, BV 相关性,2 represents the correlation between the subarray from the first channel (e.g., TX1) and the subarray from the third channel (e.g., TX3). Because the amplitude variation and subarray beam vector correlation vary with SNR, SNR is also extracted as a feature for the machine learning model 122 to determine the number of objects detected in the corresponding range-Doppler bin.

[0036] At operation 308, the machine learning model 122 may then determine whether a single object or multiple objects are present in a particular range-Doppler bin. To perform binary classification between a single object scene or multiple object scenes, the machine learning model 122 may use logistic regression techniques, decision tree methods, or multi-layer perceptron methods.

[0037] The logistic regression method uses a sigmoid function to map the final output to an interval between 0 and 1, representing the probability of each beam vector in each category so that the correct output category label can be determined. The logistic regression method can use the weights associated with each feature and polynomial feature to improve the performance of the machine learning model 122.

[0038] A decision tree is a white-box model that shows which features are important in classification. In this case, the maximum depth of the decision tree can be set to three, as sufficient accuracy is obtained with up to three decision levels. The decision tree method compares various feature values ​​of the corresponding beam vector with the conditional value to determine a single object or multiple object classification.

[0039] The multilayer perceptron method is a feedforward neural network that uses linear combinations of features and nonlinear activation functions to generate classifications through layers, where weights are iteratively updated or learned using a gradient-based optimization method (e.g., via backpropagation during a training phase). In this case, the multilayer perceptron method can use one hidden layer with five neurons to achieve the desired level of accuracy. If greater accuracy or complexity is needed or desired, additional layers or neurons can be implemented.

[0040] The machine learning model 122 may also use a pre-trained deep learning model within a convolutional neural network to extract features and feed the extracted features to one of the above-mentioned models or methods trained to identify the number of objects present.

[0041] If there is a single object for a particular range-Doppler bin, the angle determination module 208 uses an FFT beamforming method to determine an angle (e.g., azimuth) associated with the single object at operation 310. If there are multiple objects, the angle determination module 208 uses a super-resolution method to determine an angle (e.g., azimuth) associated with each of the multiple objects at operation 312.

[0042] Figure 4 An example block diagram 400 of signal processing as part of a radar system that uses a machine learning model to identify a number of objects is shown. For example, Figure 4 The radar system can be Figures 1 to 3 of radar system 104.

[0043] The radar system 104 provides the time series frames of EM energy as radar data 402 to the input processing module 204. The radar data 402 may be low-level radar data, which typically includes more information than point cloud data. Since the object population identification module 120 uses the radar data 402, it does not require additional input data from other sensors (e.g., a camera or a lidar system) to identify the number of objects in a particular range-Doppler bin. The radar data 402 includes information associated with the object 110 in multiple dimensions, including range space, Doppler space, elevation space, and azimuth space.

[0044] At operation 404, input processing module 204 performs interference detection and mitigation on radar data 402. Interference in radar data 402 may be introduced by other radar systems having a center frequency similar to the center frequency of the signal transmitted by radar system 104. Input processing module 204 may analyze radar data 402 to detect such interference and apply different techniques to mitigate the interference.

[0045] At operation 406, the input processing module 204 performs fast time processing on the radar data 402. For example, the input processing module 204 may apply a Fourier transform (e.g., a fast Fourier transform or a discrete Fourier transform) in a fast time dimension (e.g., a dimension of sampling points within the radar data 402 within a chirp) to account for frequency variations.

[0046] At operation 408, the input processing module 204 performs slow-time processing on the radar data 402. For example, the input processing module 204 may apply a Fourier transform (e.g., a fast Fourier transform or a discrete Fourier transform) in the slow-time dimension (e.g., the dimension of the chirp index within a frame of radar data 402) to account for phase variations. As a result of the fast-time and slow-time processing, the input processing module 204 may obtain a complex-valued data matrix that may be referred to as a range-Doppler map.

[0047] At operation 410, the input processing module 204 performs integration and thresholding. For example, the input processing module 204 can coherently integrate the range-Doppler map along the virtual receiver dimension to increase the SNR. Thresholding can also be applied to remove certain data or noise.

[0048] At operation 412, the input processing module 204 performs peak detection and estimation. For example, the input processing module 204 may apply a constant false alarm rate detector to detect peaks in the range-Doppler map(s). The input processing module 204 may then apply a Fourier transform (e.g., a fast Fourier transform) on the channel dimension of the radar data 402 for a particular range-Doppler bin to generate a beam vector 414. The beam vector 414 may correspond to a complex response of an antenna array element or a virtual array element.

[0049] The feature extraction module 206 then uses the beam vector 414 to extract features 416 of the radar data 402. As described above, the features 416 include amplitude variation, SNR, first subarray beam vector correlation, and second subarray beam vector correlation. At operation 418, the machine learning model 122 then uses the features 416 and the beam vector 414 to classify each range-Doppler bin as a single object scene or a multiple object scene, thereby identifying the number of objects for each range-Doppler bin.

[0050] At operation 420, the angle determination module 208 performs angle determination and outputs an angle 422 associated with the object 110. If a particular range-Doppler bin is identified as a single object scene, the angle determination module 208 applies an FFT technique to determine the angle 422 (e.g., an azimuth) associated with the object 110. If the range-Doppler bin is identified as a multiple object scene, the angle determination module 208 applies a super-resolution technique to determine the angle 422 associated with the object 110. In other implementations, the angle determination module 208 may use different angle determination techniques for single object scenes and / or multiple object scenes.

[0051] Figure 5 An example conceptual diagram of a decision tree machine learning model 500 for identifying the number of objects in each range-Doppler bin is shown. The decision tree machine learning model 500 shown may be, for example, Figures 1 to 4 The machine learning model 122. This conceptual diagram illustrates example operations, nodes, edges, and decisions (or leaves) of a decision tree machine learning model 500, but the decision tree machine learning model 500 is not necessarily limited to the order or combination of operations, nodes, edges, and decisions shown here.

[0052] The input to the decision tree machine learning model 500 includes beam vectors 414 and features 416. Figure 5As shown, the decision tree machine learning model 500 includes up to three levels or has a maximum depth of three. The number of levels can be reduced or increased depending on processing power or desired accuracy. The object quantity identification module 120 uses the decision tree machine learning model 500 to classify the input beam vector 414 of a particular range-Doppler bin as a single object scene or a multiple object scene. The decision tree machine learning model 500 is designed to divide each node into two additional nodes or leaves.

[0053] At the first node 502-1, the decision tree machine learning model 500 considers a first condition 504-1 of the feature 416. For example, the first condition 504-1 may be a first subarray beam vector correlation (BV 相关性，1 ) is less than or equal to a first threshold value (eg, 0.2). In other implementations, the first condition 504-1 may evaluate different features, use different threshold values, or use different mathematical operators (eg, greater than).

[0054] If the first subarray beam vector correlation (BV 相关性，1 ) is less than or equal to the first threshold, the decision tree machine learning model 500 proceeds to the second node 502-2. At the second node 502-2, the decision tree machine learning model 500 considers the second condition 504-2 of the feature 416. For example, the second condition 504-2 may be the second subarray beam vector correlation (BV 相关性，2 ) is less than or equal to a second threshold value (eg, 0.2). In other implementations, the second condition 504-2 may evaluate different features, use different thresholds, or use different mathematical operators (eg, greater than).

[0055] If the second subarray beam vector correlation (BV 相关性，2 ) is less than or equal to the second threshold, the decision tree machine learning model 500 proceeds to the first leaf 506-1 indicating that the associated beam vector 414 is a single object scene.

[0056] Returning to the second node 502-2, if the second subarray beam vector correlation (BV 相关性，2) is greater than the second threshold, the decision tree machine learning model 500 proceeds to a fourth node 502-4. At the fourth node 502-4, the decision tree machine learning model 500 considers a fourth condition 504-4 of the feature 416. For example, the fourth condition 504-4 may be whether the SNR is less than or equal to a fourth threshold (e.g., 20). In other implementations, the fourth condition 504-4 may evaluate different features, use different thresholds, or use different mathematical operators (e.g., greater than). If the SNR is less than or equal to the fourth threshold, the decision tree machine learning model 500 proceeds to a second leaf 506-2 indicating that the associated beam vector 414 is a single object scene. If the SNR is greater than the fourth threshold, the decision tree machine learning model 500 proceeds to a third leaf 506-3 indicating that the associated beam vector 414 is a multiple object scene.

[0057] Returning to the first node 502-1, if the first subarray beam vector correlation (BV 相关性，1 ) is greater than the first threshold, the decision tree machine learning model 500 proceeds to a third node 502-3. At the third node 502-3, the decision tree machine learning model 500 considers a third condition 504-3 of the feature 416. For example, the third condition 504-3 may be whether the SNR is less than or equal to a third threshold (e.g., 14). In other implementations, the third condition 504-3 may evaluate different features, use different thresholds, or use different mathematical operators (e.g., greater than). If the SNR is less than or equal to the third threshold, the decision tree machine learning model 500 proceeds to a fourth leaf 506-4 indicating that the associated beam vector 414 is a single object scene.

[0058] If the SNR is greater than the third threshold, the decision tree machine learning model 500 proceeds to the fifth node 502-5. At the fifth node 502-5, the decision tree machine learning model 500 considers the fifth condition 504-5 of the feature 416. For example, the fifth condition 504-5 may be a magnitude change (err 幅度 ) is less than or equal to a fifth threshold value (e.g., 0.1). In other implementations, the fifth condition 504-5 may evaluate different features, use different thresholds, or use different mathematical operators (e.g., greater than). 幅度 ) is less than or equal to the fifth threshold, the decision tree machine learning model 500 proceeds to the fifth leaf 506-5 (eg, decision, classification) indicating that the associated beam vector 414 is a single object scene. 幅度 ) is greater than the fifth threshold, the decision tree machine learning model 500 proceeds to the sixth leaf 506-6 (e.g., decision, classification) indicating that the associated beam vector 414 is a multiple object scene.

[0059] Figure 6 An example flowchart 600 for training a machine learning model 122 to identify the number of objects is shown. Specifically, flowchart 600 illustrates a feature extraction and labeling process of an object quantity identification module 120 or another module of a computer system for training a machine learning model 122. Flowchart 600 illustrates example inputs, outputs, and operations, but flowchart 600 is not necessarily limited to the order or combination of inputs, outputs, and operations shown here. In addition, any one or more operations may be repeated, combined, or reorganized to provide other functionality.

[0060] At operation 602, an engineer prepares or generates a single target beam vector. For example, a radar system can be used in a chamber and an open space to collect high SNR beam vectors at different azimuths and elevations of a single corner reflector. The elevation angle can vary from negative twelve degrees to positive twelve degrees relative to the radar system. The azimuth angle can vary from negative ninety degrees to positive ninety degrees relative to the radar system. Then, a single target beam vector of different SNRs (e.g., from 8dB to 70dB) can be generated by interfering with the high SNR beam vector with Gaussian noise.

[0061] At operation 604, the engineer prepares or generates multiple object beam vectors. For example, the radar system can be used in a chamber and an open space to collect high SNR beam vectors with different separations (e.g., different azimuths and elevations) and dynamic ranges for two corner reflectors. In other implementations, additional corner reflectors can be used to generate multiple object beam vectors of additional types. The elevation angle can vary between negative four degrees and positive four degrees relative to the radar system. The azimuth of the first object or target can vary from negative seventy-five degrees to positive seventy-five degrees relative to the (multiple) radar systems. The azimuth (e.g., separation angle) of the second object or target relative to the first object can vary from four degrees to fifteen degrees. Then, two target beam vectors of different SNRs (e.g., from 16dB to 70dB) can be generated by interfering with the high SNR beam vector with Gaussian noise. The dynamic range of the two object beam vectors can vary from 12dB to 12dB. Subsets of the single object beam vectors can also be linearly combined to create two object beam vectors, thereby creating additional samples or training data.

[0062] At operation 606, the single object and multiple object beam vectors are analyzed to perform feature engineering. For example, an engineer or computer system may analyze the single object and two object beam vectors to identify distinguishing features or characteristics between different types of beam vectors. As described above, features 416 may include amplitude changes (err 幅度 ), SNR, first subarray beam vector correlation (BV 相关性,1 ) and the second subarray beam vector correlation (BV 相关性,2). Additional features associated with single-object and multi-object beam vectors can also be extracted.

[0063] At operation 608, the machine learning model 122 is trained and constructed. For example, the machine learning model 122 can be implemented as a logistic regression model, a decision tree model, or a multi-layer perceptron model. A subset (e.g., 70%) of the beam vectors generated in operations 602 and 604 can be used to train the machine learning model. Another subset (e.g., 30%) of the beam vectors can be used to test or validate the machine learning model 122.

[0064] The machine learning model 122 can be trained offline or online. In offline training (e.g., batch learning), the machine learning model 122 is trained using a static training data set. In online training, the machine learning model 122 is continuously trained (or retrained) as new training data becomes available (e.g., when the machine learning model is used to perform object quantity identification).

[0065] Centralized training of multiple machine learning models 122 (e.g., based on a centrally stored data set) can be performed. In other implementations, the trainer can use decentralized training techniques, including distributed training or federated learning, to train, update, or personalize the machine learning model 122.

[0066] At operation 610, the test data set is used to evaluate the machine learning model(s) 122. For example, the machine learning model(s) 122 may be evaluated against an existing algorithm using the same single object and two object beam vectors to analyze the accuracy of the machine learning model 122. In the illustrative example, the accuracy of correctly identifying the number of objects in a particular range-Doppler bin is improved from about 60% to about 90%.

[0067] At operation 612, the machine learning model(s) 122 are employed. For example, the machine learning model(s) 122 are implemented, integrated, or employed in the radar system 104 of the host vehicle 102. In this way, the machine learning model(s) 122 improve the classification accuracy between single object scenes and multiple object scenes, thereby improving the computational efficiency of the angle determination module 208 and reducing false detections and angle jumps. In addition, the machine learning model 122 can be implemented in low-end processors with limited processing power. Example Method

[0068] Figure 7A flow chart of an example method 700 for a radar system using a machine learning model 122 for identifying multiple objects is shown. The method 700 is shown as multiple sets of operations (or actions) being performed, but is not necessarily limited to the order or combination of operations shown herein. In addition, any one or more operations may be repeated, combined, or reorganized to provide other methods. In the following discussion, reference may be made to Figure 1 Environment 100 and Figures 1 to 6 The entities detailed in the foregoing are referenced for purposes of example only. The technology is not limited to being performed by one entity or multiple entities.

[0069] At operation 702, radar data associated with one or more detected objects is obtained. For example, processor 116 is configured to identify the number of objects 110 within each range-Doppler bin. Processor 116 obtains radar data associated with objects 110 that reflect radar signals from radar system 104. As described above, radar system 104 and processor 116 are integrated into vehicle 102 or installed in vehicle 102.

[0070] At operation 704, a plurality of beam vectors are generated using the radar data. For example, processor 116 processes the radar data to generate beam vectors 414. Each beam vector 414 includes radar data from a plurality of channels of a corresponding range-Doppler bin. As part of the processing, processor 116 may perform at least one of interference detection and environment, fast time processing, slow time processing, integration and thresholding, or peak estimation, such as with respect to Figure 4 Described in more detail.

[0071] At operation 706, a plurality of features associated with the beam vectors of the radar data are extracted. For example, the feature extraction module 206 analyzes the beam vectors 414 to extract features 416 of each respective beam vector 414 in the plurality of beam vectors 414. The features 416 may include amplitude variation, signal-to-noise ratio, first subarray beam vector correlation, and second subarray beam vector correlation. The feature extraction module 206 may be implemented as a separate module or as part of the machine learning model 122.

[0072] At operation 708, the number of detected objects within each range-Doppler bin is identified using a machine learning model. For example, the machine learning model 122 uses the features 416 to identify the number of objects 110 detected within each range-Doppler bin. The machine learning model 122 uses the plurality of features 416 associated with the corresponding beam vector 414 to determine whether a single object or multiple objects are present in a particular range-Doppler bin. In one example implementation, the machine learning model 122 may be a logistic regression model having a weight associated with each feature 416 in the plurality of features 416 and other weights associated with polynomial features in the plurality of features 416. In another example implementation, the machine learning model 122 is a decision tree model that compares the feature value of the corresponding beam vector 414 with a conditional value at each decision or node of the decision tree. In yet another implementation, the machine learning model 122 is a multi-layer perceptron model using a linear combination of the plurality of features and a nonlinear activation function. The complexity type of the machine learning model 122 may be determined or selected based on the processing power of the processor 116 of the radar system 104.

[0073] As about Figure 6 As described, the machine learning model 122 is trained using valid truth data. The valid truth data includes beam vectors for single object scenes and multiple object (e.g., two object) scenes. The valid truth data can be collected using corner reflectors in a controlled environment (e.g., a chamber or an open space), or collected from data collected by a radar system in a real-world driving scene. The truth data can be feature engineered to determine relevant features for classifying the beam vector as a single object scene or a multiple object scene.

[0074] At operation 710, in response to the output of the machine learning model classifying the corresponding range-Doppler bin as having a single object, a first angle determination technique (e.g., a fast Fourier transform technique) is used to determine an angle associated with the single object of the corresponding range-Doppler bin. For example, in response to the machine learning model 122 classifying the radar data associated with a particular range-Doppler bin as a single object scene, the angle determination module 208 uses the FFT technique to determine the angle 422 associated with the object 110. The determined angle 422 may be an azimuth and / or elevation angle of the object 110.

[0075] At operation 712, in response to the output of the machine learning model classifying the corresponding range-Doppler bin as having multiple objects, a second angle determination technique (e.g., a super-resolution technique) different from the first angle determination technique is used to determine the angle associated with the multiple objects of the corresponding range-Doppler bin. For example, in response to the machine learning model 122 classifying the radar data associated with a particular range-Doppler bin as a multiple object scene, the angle determination module 208 uses a super-resolution technique to determine the angle 422 associated with the object 110. The determined angle 422 may be an azimuth and / or elevation angle of the object 110. The super-resolution technique may include at least one of signal parameter estimation via rotation invariant technique (ESPRIT), multiple signal classification (MUSIC), iterative adaptive method (IAA), or a two-step FFT method.

[0076] The radar system 104 or the processor 116 may also output the angle(s) 422 to the assisted driving system 212 or the autonomous driving system 214 of the vehicle 102. The assisted driving system 212 or the autonomous driving system 214 may then operate the vehicle 102 based on the angle(s) 422 associated with the object(s) 110.

[0077] The method 700 may optionally retrain or update the machine learning model 122 with additional valid truth data. For example, the implementation of the machine learning model 122 may be updated based on additional or new valid truth data obtained and processed at another computing system and / or location during operation of the vehicle 102. Additional Examples

[0078] In the following sections, additional examples are provided.

[0079] Example 1: A method for identifying the number of objects within each range-Doppler bin of a range-Doppler map, the method comprising: obtaining radar data associated with one or more objects reflecting radar signals; generating a plurality of beam vectors using the radar data, each respective beam vector comprising radar data from a plurality of channels of a particular range-Doppler bin; extracting a plurality of features of the radar data for each respective beam vector in the plurality of beam vectors; identifying the number of detected objects within a particular range-Doppler bin using a machine learning model for each respective beam vector in the plurality of beam vectors, the machine learning model being configured to determine whether a single object or multiple objects are present in the particular range-Doppler bin using the plurality of features associated with the respective beam vector; and in response to an output of the machine learning model classifying the corresponding range-Doppler bin as having a single object, determining an angle associated with the single object of the corresponding range-Doppler bin using a first angle determination technique; or in response to an output of the machine learning model classifying the corresponding range-Doppler bin as having multiple objects, determining an angle associated with the multiple objects of the corresponding range-Doppler bin using a second angle determination technique, the second angle determination technique being different from the first angle determination technique.

[0080] Example 2: The method of Example 1, the first angle determination technique is a fast Fourier transform (FFT) beamforming method; and the second angle determination technique is a super-resolution technique.

[0081] Example 3: The method of Example 2, wherein the super-resolution technique comprises at least one of a multiple signal classification (MUSIC) technique, signal parameter estimation via rotational invariance technique (ESPRIT), or an iterative adaptive approach (IAA).

[0082] Example 4: The method of any of the previous examples, wherein the machine learning model is a logistic regression model having a weight associated with each of the plurality of features and an additional weight associated with a polynomial feature of the plurality of features.

[0083] Example 5: The method of any of the previous examples, wherein the machine learning model is a decision tree model that compares the feature value of the corresponding beam vector with the conditional value at each decision.

[0084] Example 6: The method of any of the previous examples, wherein the machine learning model is a multilayer perceptron model that uses linear combinations of multiple features and a non-linear activation function.

[0085] Example 7: The method of any of the previous examples, wherein the plurality of features comprises amplitude variation of the plurality of beam vectors, a signal-to-noise ratio, a first subarray beam vector correlation, and a second subarray beam vector correlation.

[0086] Example 8: The method of any of the previous examples, wherein the plurality of features are extracted by a machine learning model.

[0087] Example 9: The method of any of the previous examples, wherein: the machine learning model is trained offline using valid truth data, the valid truth data comprising beam vectors for single object scenes and multiple object scenes; and the multiple features are identified by performing feature engineering on the valid truth data.

[0088] Example 10: The method of any of the previous examples, wherein the angle is at least one of an elevation angle or an azimuth angle.

[0089] Example 11: The method of any of the previous examples, wherein the method further includes: outputting an angle associated with a single object or an angle associated with multiple objects to an assisted driving system or an autonomous driving system of a main vehicle; and operating the main vehicle based on the angle associated with the single object or the angle associated with multiple objects.

[0090] Example 12: The method of any of the previous examples, wherein the type and complexity of the machine learning model is determined based on the processing capabilities of one or more processors of the radar system.

[0091] Example 13: A system, comprising: a radar system, the radar system comprising one or more processors, the one or more processors being configured to identify the number of objects within each range-Doppler bin of a range-Doppler map by executing the method of any one of Examples 1 to 12.

[0092] Example 14: The system of Example 13, wherein the system is configured to be integrated into or installed in a vehicle.

[0093] Example 15: A computer-readable storage medium comprising computer-executable instructions, which, when executed, cause a processor of a radar system to identify the number of objects within each range-Doppler bin of a range-Doppler map by performing the method of any one of Examples 1 to 12. Conclusion

[0094] Although various embodiments of the present disclosure are described in the foregoing description and shown in the accompanying drawings, it should be understood that the present disclosure is not limited thereto, but can be implemented in various ways within the scope of the following claims. From the foregoing description, it will be apparent that various changes can be made without departing from the scope of the present disclosure defined by the following claims.

Claims

1. A system, wherein the system include: A radar system comprising one or more processors configured to identify a number of objects within each range-Doppler bin of a range-Doppler map by: obtaining radar data associated with one or more objects reflecting the radar signal; generating a plurality of beam vectors using the radar data, each respective beam vector comprising the radar data from a plurality of channels of a particular range-Doppler bin; extracting a plurality of features of the radar data for each corresponding beam vector in the plurality of beam vectors; identifying the number of the objects within the particular range-Doppler bin using a machine learning model for each respective beam vector of the plurality of beam vectors, the machine learning model being configured to determine whether a single object or multiple objects are present in the particular range-Doppler bin using the plurality of features associated with the respective beam vector; as well as responsive to an output of the machine learning model classifying the corresponding range-Doppler bin as having the single object, determining an angle associated with the single object of the corresponding range-Doppler bin using a first angle determination technique; or In response to the output of the machine learning model classifying the corresponding range-Doppler bin as having the plurality of objects, determining angles associated with the plurality of objects of the corresponding range-Doppler bin using a second angle determination technique, the second angle determination technique being different from the first angle determination technique.

2. The system according to claim 1, Features: The first angle determination technique is a Fast Fourier Transform (FFT) beamforming method; and The second goniometric technique is a super-resolution technique.

3. The system according to claim 2, It is characterized in that The super-resolution technique includes at least one of a multiple signal classification (MUSIC) technique, an estimation of signal parameters via rotational invariance technique (ESPRIT), or an iterative adaptive approach (IAA).

4. The system according to claim 1, It is characterized in that The machine learning model is a logistic regression model having a weight associated with each feature of the plurality of features and an additional weight associated with a polynomial feature of the plurality of features.

5. The system according to claim 1, It is characterized in that The machine learning model is a decision tree model that compares the feature value of the corresponding beam vector with a conditional value at each decision.

6. The system of claim 1, It is characterized in that The machine learning model is a multi-layer perceptron model, which uses a linear combination of the multiple features and a non-linear activation function.

7. The system of claim 1, It is characterized in that The plurality of characteristics include amplitude variation, signal-to-noise ratio, first subarray beam vector correlation, and second subarray beam vector correlation of the plurality of beam vectors.

8. The system of claim 1, It is characterized in that The multiple features are extracted by the machine learning model.

9. The system of claim 1, It is characterized in that The machine learning model is trained offline using valid ground-truth data, which includes beam vectors for single-object scenes and multiple-object scenes.

10. The system of claim 1, It is characterized in that The angle is at least one of an elevation angle or an azimuth angle.

11. The system of claim 1, Features: The one or more processors are further configured to output the angle associated with the single object or the angle associated with the plurality of objects to an assisted driving system or an autonomous driving system of a host vehicle; and The system further includes the assisted driving system or the autonomous driving system configured to operate the host vehicle based on the angle associated with the single object or the angles associated with the multiple objects.

12. The system of claim 1, It is characterized in that The system is configured to be integrated in or installed in a vehicle.

13. The system of claim 1, It is characterized in that The type and complexity of the machine learning model is determined based on the processing capabilities of the one or more processors of the radar system.

14. A method for identifying the number of objects within each range-Doppler bin of a range-Doppler map, the method include: obtaining radar data associated with one or more objects reflecting the radar signal; generating a plurality of beam vectors using the radar data, each respective beam vector comprising the radar data from a plurality of channels of a particular range-Doppler bin; for each respective beam vector in the plurality of beam vectors, extracting a plurality of features of the radar data; identifying a number of detected objects within the particular range-Doppler bin using a machine learning model for each respective beam vector of the plurality of beam vectors, the machine learning model being configured to determine whether a single object or multiple objects are present in the particular range-Doppler bin using the plurality of features associated with the respective beam vector; as well as responsive to an output of the machine learning model classifying the corresponding range-Doppler bin as having the single object, determining an angle associated with the single object of the corresponding range-Doppler bin using a first angle determination technique; or In response to the output of the machine learning model classifying the corresponding range-Doppler bin as having the plurality of objects, determining angles associated with the plurality of objects of the corresponding range-Doppler bin using a second angle determination technique, the second angle determination technique being different from the first angle determination technique.

15. The method of claim 14, Features: The first angle determination technique is a Fast Fourier Transform (FFT) beamforming method; and The second angle determination technique is a super-resolution technique including at least one of a multiple signal classification (MUSIC) technique, an estimation of signal parameters via rotational invariance technique (ESPRIT), or an iterative adaptive approach (IAA).

16. The method of claim 14, It is characterized in that The plurality of characteristics include amplitude variation, signal-to-noise ratio, first subarray beam vector correlation, and second subarray beam vector correlation of the plurality of beam vectors.

17. The method of claim 14, It is characterized in that The method further comprises: outputting the angle associated with the single object or the angles associated with the plurality of objects to an assisted driving system or an autonomous driving system of a host vehicle; and The host vehicle is operated based on the angle associated with the single object or the angles associated with the plurality of objects.

18. The system of claim 14, It is characterized in that The plurality of features are identified by performing feature engineering on ground truth data, the ground truth data comprising beam vectors for single object scenes and multiple object scenes.

19. The method of claim 14, It is characterized in that The machine learning model includes: a logistic regression model having a weight associated with each feature of the plurality of features and other weights associated with polynomial features of the plurality of features; A decision tree model that compares the eigenvalue of the corresponding beam vector with a conditional value at each decision; or A multi-layer perceptron model is provided, wherein the multi-layer perceptron model uses a linear combination of the plurality of features and a non-linear activation function.

20. A computer-readable storage medium comprising computer-executable instructions which, when executed, cause a processor of a radar system to identify a number of objects within each range-Doppler bin of a range-Doppler map by: obtaining radar data associated with one or more objects reflecting the radar signal; generating a plurality of beam vectors using the radar data, each respective beam vector comprising the radar data from a plurality of channels of a particular range-Doppler bin; for each respective beam vector in the plurality of beam vectors, extracting a plurality of features of the radar data; identifying a number of detected objects within the particular range-Doppler bin using a machine learning model for each respective beam vector of the plurality of beam vectors, the machine learning model being configured to determine whether a single object or multiple objects are present in the particular range-Doppler bin using the plurality of features associated with the respective beam vector; as well as responsive to an output of the machine learning model classifying the corresponding range-Doppler bin as having the single object, determining an angle associated with the single object of the corresponding range-Doppler bin using a first angle determination technique; or In response to the output of the machine learning model classifying the corresponding range-Doppler bin as having the plurality of objects, determining angles associated with the plurality of objects of the corresponding range-Doppler bin using a second angle determination technique, the second angle determination technique being different from the first angle determination technique.

Images