RADAR CALIABILITY SYSTEMS AND PROCEDURES

Abstract

Method (400) for calibrating a radar system of a vehicle (10), comprising: Receiving radar data (66) based on at least one scan of a controlled calibration environment of the radar system; Generating a beamforming matrix for each scan based on the received radar data (66); Generating (410) a training set (68) of data based on the generated beamforming matrices by a processor (44); Processing (420) by the processor (44) of the training set (68) using a machine learning method to obtain centroids (340) of interdependent clusters within the training set (68), wherein the machine learning method is a convolutional neural network (CNN, 300) comprising an input stage (310) that receives the training set (68), a convolution stage (320) that filters the training set (68) to extract interdependent features, and a subsampling stage (330) that reduces the computational complexity to generate the centroids (340); Generating (430) an optimized path (70) based on the centroids (340) of the interdependent clusters; Generating (460) a dense matrix (76) of sampled data based on the optimized path (70); and Calibrating (470) the radar system based on the dense matrix (76) generated by the processor (44).

Description

INTRODUCTION

[0001] The present disclosure relates generally to radars for vehicles, and more particularly to systems and methods for calibrating a radar system for a vehicle.

[0002] An autonomous vehicle is a vehicle capable of sensing its environment and navigating with little or no user input. An autonomous vehicle senses its environment using sensing devices such as radar, lidar, image sensors such as cameras, and the like. The autonomous vehicle system further utilizes information from global positioning systems (GPS), navigation systems, vehicle-to-vehicle communications, vehicle infrastructure technologies, and / or wired systems to navigate the vehicle.

[0003] While significant progress has been made in radar systems for autonomous vehicles and other vehicles in recent years, these systems could still be improved in several aspects. For example, in some cases, calibrating a single radar with sufficient resolution involves sampling up to 11,011 radar samples for a duration of up to 2.6 seconds per sample; and using the sampled radar returns to calibrate the radar using a common calibration procedure. This calibration process, based on the measurement and analysis of the transmitted and received pulses, is a very time-consuming process that must be performed during the production process when calibrating a radar system to a vehicle environment.

[0004] Accordingly, it is desirable to provide improved systems and methods for calibrating a vehicle's radar system. Further desirable features and functions of the present disclosure will become apparent from the following detailed description and the appended claims, taken in conjunction with the accompanying drawings, as well as the foregoing technical field and background.

[0005] US 6,366,236 B1 discloses a radar system for processing a radar signal using a neural network. US 2017 / 0060254 A1 discloses an apparatus and method for gesture recognition. US 2016 / 0375592 A1 discloses an apparatus and method for navigating a robot based on projected and received patterns used to determine the presence of objects during the robot's navigation. SUMMARY

[0006] Systems and methods are provided for calibrating a radar system of an autonomous vehicle. In one embodiment, a method includes: receiving radar data based on at least one scan of a controlled calibration environment of the radar system; generating a beamforming matrix for each scan based on the received radar data; generating, by a processor, a training set of data based on the generated beamforming matrices; processing, by the processor, the training set with a machine learning method to obtain centroids of interdependent clusters within the training set, wherein the machine learning method is a convolutional neural network that includes an input phase that receives the training set, a convolution phase that filters the training set to extract interdependent features, and a subsampling phase that reduces computational complexity to generate the centroids.Generating an optimized path based on the centroids of the interdependent clusters; generating a dense matrix of sample data based on the optimized path; and calibrating, by the processor, the radar system based on the generated dense matrix.

[0007] In one embodiment, a system includes: a first non-transitory module configured to receive radar data based on at least one scan of a controlled calibration environment of the radar system and generate a beamforming matrix for each scan based on the received radar data and generate, by a processor, a training set of data based on the generated beamforming matrices;a second non-volatile module that, through the processor, processes the training set with a machine learning method to obtain centroids of interdependent clusters within the training set, generates an optimized path based on the centroids of the interdependent clusters, and generates a dense matrix of sample data based on the optimized path, wherein the machine learning method is a convolutional neural network that includes an input phase that receives the training set, a convolution phase that filters the training set to extract interdependent features, and a subsampling phase that reduces computational complexity to generate the centroids; and a third non-volatile module that, through the processor, calibrates the radar system based on the generated dense matrix. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The exemplary embodiments are described below in conjunction with the following drawings, wherein like reference numerals designate like elements, and wherein: Fig. 1 is a functional block diagram illustrating an autonomous vehicle with an image sensor calibration system according to various embodiments; Fig. 2 is a data flow diagram illustrating a calibration system associated with the autonomous vehicle according to various embodiments; Fig. 3 is an exemplary convolutional neural network obtained through the calibration system of Fig. 2 can be implemented according to various embodiments, and Fig. 4 is a flowchart illustrating a calibration method for calibrating a radar system of the autonomous vehicle according to various embodiments. DETAILED DESCRIPTION

[0009] The following detailed description is provided for illustrative purposes only and is not intended to limit the application and uses in any way. Furthermore, there is no intention to be bound by any theory expressed or implied in the foregoing technical field, background, brief summary, or the following detailed description. As used herein, the term "module" refers to any hardware, software, firmware products, electronic control components, processing logic, and / or processor devices, individually or in combination, including, but not limited to, an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or grouped), and memory that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality.

[0010] Embodiments of the present disclosure may be described herein as functional and / or logical block components and various processing steps. It should be noted that such block components may be constructed from any number of hardware, software, and / or firmware components configured to perform the required functions. For example, an embodiment of the present disclosure of a system or component may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, look-up tables, or the like, capable of performing multiple functions under the control of one or more microprocessors or other control devices.Furthermore, those skilled in the art will recognize that the exemplary embodiments of the present disclosure may be used in connection with any number of systems, and that the system described herein is merely one exemplary embodiment of the present disclosure.

[0011] For the sake of brevity, conventional techniques associated with signal processing, data transmission, signaling, control, and other functional aspects of the systems (and the individual control elements of the systems) may not be described in detail herein. Furthermore, the connecting lines depicted in the various figures are intended to represent exemplary functional relationships and / or physical connections between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in an embodiment of the present disclosure.

[0012] As mentioned with reference to Fig. 1, a calibration system, generally depicted at 100, is associated with a vehicle 10 according to various embodiments. As explained in more detail below, the calibration system 100 calibrates a radar system of the vehicle 10 using an optimized calibration method for sampling radar returns. In various embodiments, the optimized calibration method is based on the output of a machine learning process that minimizes the number of measurement points and optimizes the path for acquiring the measurement points.

[0013] As in Fig. 1, the vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially encloses the other components of the vehicle 10. The body 14 and the chassis 12 may together form a frame. The wheels 16-18 are each pivotally connected to the chassis 12 near a respective corner of the body 14.

[0014] In various embodiments, the vehicle 10 is an autonomous vehicle, and the calibration system 100 is integrated with or associated with the autonomous vehicle 10 (hereinafter referred to as the autonomous vehicle 10). The autonomous vehicle 10 is, for example, a vehicle that is automatically controlled to transport passengers from one location to another. The vehicle 10 is shown as a passenger car in the illustrated embodiment, but it should be noted that any other vehicle, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), vessels, aircraft, etc., may be used. In one exemplary embodiment, the autonomous vehicle 10 is a so-called Level Four or Level Five automation system.A level four system indicates "high automation" by referring to the drive mode-specific performance by an automated driving system of all aspects of the dynamic driving task, even when a human driver fails to adequately respond to a request to intervene. A level five system indicates "full automation" by referring to the full-time performance of an automated driving system of all aspects of the dynamic driving task under all road and environmental conditions manageable by a human driver. As can be appreciated, in various embodiments, the vehicle 10 may assume other levels of automation or may not be autonomous.

[0015] As illustrated, the autonomous vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. The propulsion system 20, in various embodiments, may include an internal combustion engine, an electric machine such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transfer power from the propulsion system 20 to the vehicle wheels 16-18 according to selectable gear ratios. According to various embodiments, the transmission system 22 may include a stepped-ratio automatic transmission, a continuously variable transmission, or other suitable transmission. The braking system 26 is configured to provide braking torque to the vehicle wheels 16-18.The braking system 26 may, in various embodiments, include friction brakes, brake-by-wire, a regenerative braking system such as an electric machine, and / or other suitable braking systems. The steering system 24 influences the position of the vehicle wheels 16-18. While illustrated as a steering wheel in some embodiments within the scope of the present disclosure, the steering system 24 may not include a steering wheel.

[0016] The sensor system 28 includes one or more sensor devices 40a-40n that detect observable conditions of the external environment and / or the internal environment of the autonomous vehicle 10. The detection devices 40a-40n may include, but are not limited to, radars, lidars, global positioning systems, optical cameras, thermal imaging cameras, ultrasonic sensors, inertial measurement units, and / or other sensors. In various embodiments, the detection devices are equipped with the radar system.

[0017] The actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle features, such as, but not limited to, the drive system 20, the transmission system 22, the steering system 24, and the braking system 26. In various embodiments, the vehicle features may further include, but are not limited to, interior and / or exterior vehicle features, such as doors, a trunk, and interior features such as air, music, lighting, etc. (unnumbered).

[0018] The communication system 36 is configured to wirelessly transmit information to and from other units 48, such as, but not limited to, other vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems, and / or personal devices (relating to Fig. 2). In an exemplary embodiment, the wireless communication system 36 is configured to communicate over a wireless local area network (WLAN) using the IEEE 802.11 standard, via Bluetooth, or via mobile data communication. However, additional or alternative communication methods, such as a dedicated short-range communication (DSRC) channel, are also contemplated within the scope of the present disclosure. DSRC channels refer to one-way or two-way short- to mid-range radio communication channels specifically designed for automotive applications and a corresponding set of protocols and standards.

[0019] The data storage device 32 stores data for use in automatically controlling the autonomous vehicle 10. In various embodiments, the data storage device 32 stores defined maps of the navigable environment. In various embodiments, the defined maps are predefined and retrieved from a remote system (in further detail with respect to Fig. 2). For example, the defined maps may be assembled by the remote system and communicated to the autonomous vehicle 10 (wirelessly and / or wired) and stored in the data storage device 32. As can be seen, the data storage device 32 may be part of the controller 34, separate from the controller 34, or part of the controller 34 and part of a separate system.

[0020] The controller 34 includes at least one processor 44 and a computer-readable storage device or media 46. The processor 44 may be a custom-made or off-the-shelf processor, a central processing unit (CPU), a graphics processing unit (GPU) among multiple processors connected to the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally any device for executing instructions. The computer-readable storage device or media 46 may include volatile and non-volatile memory in read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operating variables while the processor 44 is powered off.The computer-readable storage device or media 46 may be implemented using any of a number of known storage devices, such as PROMs (programmable read-only memory), EPROMs (electrical PROMs), EEPROMs (electrically erasable PROMs), flash memory, or any other electrical, magnetic, optical, or combination storage devices capable of storing data, some of which represent executable instructions used by the controller 34 in controlling the autonomous vehicle 10.

[0021] The instructions may include one or more separate programs, each comprising an ordered list of executable instructions for implementing logical functions. The instructions, when executed by processor 44, receive and process signals from sensor system 28, perform logic, calculations, methods, and / or algorithms to automatically control the components of autonomous vehicle 10, and generate control signals to actuator system 30 to automatically control the components of autonomous vehicle 10 based on the logic, calculations, methods, and / or algorithms. Although in Fig. 1 only one controller 34 is shown, embodiments of the autonomous vehicle 10 may include any number of controllers 34 that communicate and cooperate via a suitable communication medium or combination of communication media to process the sensor signals, perform logic, calculations, methods and / or algorithms, and generate control signals to automatically control the functions of the autonomous vehicle 10.

[0022] In various embodiments, one or more instructions from controller 34 are embodied in calibration system 100, and when executed by processor 44, the scanning radar returns according to the optimized calibration method, thereby minimizing the scanning of radar returns. As can be appreciated, in various embodiments, these instructions may also be embodied in a calibration controller (not shown) independent of vehicle 10, for example, in conjunction with a production process or partially in a calibration controller and partially in vehicle controller 34.

[0023] As can be seen, the subject matter disclosed herein provides certain enhanced features and functionality for what may be considered a standard or baseline autonomous vehicle 10 and / or an autonomous vehicle-based transportation system. To this end, an autonomous vehicle-based transportation system may be modified, enhanced, or otherwise supplemented to provide the additional functionality described in more detail below.

[0024] With reference to Fig. 2 and continue to Fig. 1 illustrates a data flow diagram of the calibration system 100 according to various embodiments. It will be appreciated that various embodiments of the calibration system 100 according to the present disclosure may include any number of submodules. Fig. The sub-modules shown in Figure 2 can, for example, be combined and / or further subdivided to uniformly calibrate the radar system of the vehicle 10. As explained above, the inputs to the calibration system include data obtained from the radar system's radar returns in a controlled calibration environment. In various embodiments, the calibration system 100 includes an offline calibration system 50 and an online calibration system 52. The offline calibration system 50 includes a data acquisition module 54, a machine learning module 56, and a script generation module 58. In various embodiments, the functions of the modules 50, 54 of the offline calibration system 50 are performed offline or once for all radar systems, for example, as a pre-calibration for any number of vehicles.

[0025] In various embodiments, the online calibration system 52 includes a data sampling module 60, a data interpolation module 62, and a calibration module 64. In various embodiments, the functions of modules 58-64 of the online calibration system 52 are performed online, for example, as a calibration process for the vehicle 10.

[0026] As in Fig. 2, the data acquisition module 54 receives radar data 66 generated by scanning through a controlled calibration environment (e.g., in a test environment or an anechoic chamber) in one-degree increments of angle between minus sixty degrees and sixty degrees in azimuth and minus forty-five degrees and forty-five degrees in elevation. The radar data 66 may include any number of scans by the radar of the vehicle 10 or other vehicles (not shown) in a test pool. For each scan, the data acquisition module 54 generates a beamforming matrix of complex in-phase and quadrature (IQ) data based on the received radar data 66. In various embodiments, the matrix includes 11,011 data points. As can be seen, the matrix may include any number of data points. The data acquisition module 54 collects and stores the generated matrices as a training set of matrices 68.

[0027] The machine learning module 56 receives the training set of matrices 68. The machine learning module 56 processes the set of matrices 68 with a machine learning method to identify a minimum number of required angle combinations, also called centroids, and to find an optimized rotation path between the angle combinations. As can be seen, examples of machine learning techniques include artificial neural networks (ANNs) (e.g., recurrent neural networks (RNNs) and convolutional neural networks (CNNs)), decision tree models (e.g., classification and regression trees (CARTs)), ensemble learning models (e.g., boosting, bootstrapped aggregation, gradient boost machines, and random forest), Bayesian network models (e.g., naive Bayes), principal component analysis (PCA), support vector machines (SVMs), clustering models (such as k-neighbor, k-means, expectation maximization, hierarchical clustering, etc.), and linear discriminant analysis models.

[0028] In various embodiments, the machine learning method is a convolutional neural network (CNN). As in Fig. 3, an exemplary CNN 300 generally includes an input stage 310 that receives the training set of matrices 68. The CNN 300 further includes a convolution stage 320 and a subsampling stage 330. The convolution stage 320 includes a series of hidden layers that learn patterns in the IQ data points, group interdependent points, and create centroids for each created cluster. Each layer is a filter that performs a convolution to extract independent features. The subsampling stage 930 then subsamples the extracted features to reduce computational complexity and thereby output centroids 340 representing interdependent clusters. In various embodiments, the subsampling may include an average value or a maximum value over a sample of the inputs 68.

[0029] In general, this can Fig. 3 can be trained in an unsupervised mode, for example, by using backpropagation 350 to refine the learning of the CNN 300.

[0030] With reference back to Fig. 2, the resulting model is then fed into the machine learning module 56 of Fig. 2. Subsequently, the training set 68 is processed in normal operation using the learned CNN 300. The machine learning module 56 then evaluates the resulting centroids 340 to provide an optimized path 70 for acquiring the scan data.

[0031] The script generation module 58 receives the identified centroids 340 and the optimized path 70. Based on the identified centroids 340 and the path 70, the script generation module 58 generates a script with the steps to be performed by the calibration system 100. The steps instruct the calibration system 100 to scan the environment with the radar at the identified cluster points in the order identified by the path.

[0032] The data sampling module 60 receives the script 72 and the uncalibrated radar 73 and executes the steps of the script to obtain the IQ data 74 for each centroid. The data interpolation module 62 receives the acquired IQ data 74 for each centroid and interpolates the sampled IQ data to input the IQ data for the remaining data points (the data points outside the centroid). Thus, the data sampling module 60 generates a dense matrix 76 from the acquired IQ data 74 and the interpolated data.

[0033] The calibration module 64 receives the dense matrix 76 and calibrates the radar system 10 based thereon. For example, the calibration module 64 loads the dense matrix 76 as calibration 78 onto a data storage device of the radar system.

[0034] The calibration system 100 implemented in this way uses pre-calibration data to limit the travel time by finding a minimum number of angle combinations required to produce a comparable, sparse beamforming matrix, and it uses reinforcement learning to optimize the path the radar system's rotator must take to scan all combinations.

[0035] With reference now to Fig. 4 and continued reference to the Fig. 1-3, a flowchart illustrates a calibration method 400 performed by the system 100 of Fig. 1 according to the present disclosure. As will be readily understood in light of the disclosure, the order of operation within the method is not limited to sequential execution as in Fig.4, but may be executed in one or more varying orders, as applicable and in accordance with the present disclosure. In various embodiments, method 400 may be scheduled for execution based on one or more predefined events and / or may be executed continuously during operation of autonomous vehicle 10.

[0036] In one example, the method may begin 405. The training set of matrices 68 is collected at 410. The machine learning process is performed on the training set of matrices 68 to train the CNN 300 to locate the centroids 340 at 420. The optimized path 70 is generated from the identified centroids 340 at 430. The script 72 is then generated from the centroids 340 and the optimized path 70 at 440. The optimized scan is then performed by the calibration system on an individual radar to obtain IQ data 74 at 450. Interpolation is performed on the obtained IQ data 74 at 460 to generate the dense matrix 76. The dense matrix 76 is then integrated into the radar at 470, and the radar is now calibrated. After that, the method may end at 480.

[0037] While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that numerous variations exist. It should further be understood that the exemplary embodiment or embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of this disclosure in any way. Rather, the foregoing detailed description provides those skilled in the art with a convenient roadmap for implementing the exemplary embodiment or embodiments. It should be understood that various changes in the function and arrangement of elements may be made without departing from the scope of the disclosure as recited in the appended claims and their legal equivalents.

Claims

[1] A method (400) for calibrating a radar system of a vehicle (10), comprising: Receiving radar data (66) based on at least one scan of a controlled calibration environment of the radar system; Generating a beamforming matrix for each scan based on the received radar data (66); generating (410), by a processor (44), a training set (68) of data based on the generated beamforming matrices; Processing (420), by the processor (44), the training set (68) with a machine learning method to obtain centroids (340) of interdependent clusters within the training set (68), the machine learning method being a convolutional neural network (CNN) (300) that includes an input phase (310) receiving the training set (68), a convolution phase (320) filtering the training set (68) to extract interdependent features, and a subsampling phase (330) reducing computational complexity to generate the centroids (340); Generating (430) an optimized path (70) based on the centroids (340) of the interdependent clusters; Generating (460) a dense matrix (76) of sample data based on the optimized path (70); and Calibrating (470) the radar system based on the generated dense matrix (76) by the processor (44). [2] The method (400) of claim 1, further comprising generating (440), by the processor (44), a script (72) for sampling data based on the centroids (340); and wherein the calibration is performed based on the script (72). [3] The method (400) of claim 1, wherein the training set (68) includes complex in-phase and quadrature (IQ) data. [4] The method (400) of claim 1, further comprising performing a scan by the radar system based on the centroids (340) and the optimized path (70) to obtain sample data; and interpolating between the sample data to obtain the dense matrix (76) including the sample data and the interpolated data. [5] The method (400) of claim 1, wherein the convolutional neural network (300) further includes backpropagation based on the centroids (340). [6] The method (400) of claim 1, wherein the steps of obtaining and processing are performed offline, and wherein the step of calibrating is performed online. [7] System (100) for calibrating a radar system of a vehicle (10), comprising: a first non-volatile module configured to receive radar data (66) based on at least one scan of a controlled calibration environment of the radar system and to generate a beamforming matrix for each scan based on the received radar data (66) and to generate, by a processor (44), a training set (68) of data based on the generated beamforming matrices; a second non-transitory module that, through the processor (44), processes the training set (68) with a machine learning method to obtain centroids (340) of independent clusters within the training set (68), generate an optimized path (70) based on the centroids (340) of the interdependent clusters, and generate a dense matrix (76) of sample data based on the optimized path (70), wherein the machine learning method is a convolutional neural network, CNN, (300) that includes an input phase (310) that receives the training set (68), a convolution phase (320) that filters the training set (68) to extract interdependent features, and a subsampling phase (330) that reduces computational complexity to generate the centroids (340); and a third non-volatile module that calibrates the radar system based on the generated dense matrix (76) through the processor (44).

Images