Increasing radar angular resolution with motion-extended aperture

Abstract

A vehicle and a system and method of operating the vehicle. The system includes an extended radar array, a processor, and a controller. The extended radar array is formed by moving a radar array of the vehicle a selected distance. The processor is configured to receive a plurality of observations of an object from the extended radar array, operate a neural network based on the plurality of observations to generate a network output signal, and determine an object parameter of the object relative to the vehicle from the network output signal. The controller operates the vehicle based on the object parameter of the object.

Description

Technical Field

[0001] This subject matter discloses vehicle radar systems, and more particularly, systems and methods for using the motion of a vehicle to increase the angular resolution of a vehicle radar array. Background Technology

[0002] Autonomous vehicles can navigate relative to objects in their environment by detecting objects and determining trajectories to avoid them. Detection can be performed by various detection systems, one of which is a radar system employing one or more radar antennas. The angular resolution of a radar antenna is limited by its aperture size, which is typically a few centimeters. Angular resolution can be improved by using an antenna array spanning a wider aperture. However, the size of the vehicle limits the size of the antenna array, thus limiting its angular resolution. Therefore, it is desirable to provide a system and method for operating a vehicle antenna array that extends its angular resolution beyond the limitations imposed by the vehicle's size. Summary of the Invention

[0003] In one exemplary embodiment, a method for operating a vehicle is disclosed. Multiple observations of an object are received at an extended radar array formed by moving the vehicle's radar array at a selected distance. The multiple observations are input into a neural network to generate a network output signal. Object parameters of the object relative to the vehicle are determined from the network output signal. The vehicle operates based on the object parameters.

[0004] In addition to one or more features described herein, the method includes acquiring multiple observations at each of multiple locations on the radar array as the radar array moves through a selected distance. The method also includes feeding the multiple observations into a neural network to generate multiple features and combining these features to obtain a network output signal. The neural network includes multiple convolutional networks, each receiving a corresponding observation from the multiple observations and generating a corresponding feature of the multiple features. The method further includes training the neural network by determining weights that minimize a loss function comprising the network output signal and a reference signal. Based on the known relative distance between the radar array and the object during relative motion between the vehicle and the object, a reference signal is generated by coherently combining the multiple observations over time. The reference signal comprises the product of observations received from the extended radar array and a synthetic response based on the angle and distance range recorded for the observations.

[0005] In another exemplary embodiment, a system for operating a vehicle is disclosed. The system includes an extended radar array, a processor, and a controller. The extended radar array is formed by moving the vehicle's radar array a selected distance. The processor is configured to receive multiple observations of an object from the extended radar array, operate a neural network based on the multiple observations to generate a network output signal, and determine object parameters of the object relative to the vehicle from the network output signal. The controller operates the vehicle based on the object parameters of the object.

[0006] In addition to one or more features described herein, as the radar array moves through a selected distance, the extended radar array acquires multiple observations at each of multiple locations within the radar array. The processor is also configured to operate a neural network to generate multiple features based on the multiple observations and to operate a concatenation module to combine the multiple features to obtain a network output signal. This neural network includes multiple convolutional networks, each configured to receive a corresponding observation from the multiple observations and generate a corresponding feature of the multiple features. The processor is also configured to train the neural network by determining weights that minimize a loss function comprising the network output signal and a reference signal. The processor is further configured to generate a reference signal by coherently combining the multiple observations over time based on the known relative distance between the radar array and the object during relative motion between the vehicle and the object. The processor is also configured to generate the reference signal from the product of observations received from the extended radar array and a synthetic response based on the angles and ranges recorded for the observations.

[0007] In yet another exemplary embodiment, a vehicle is disclosed. The vehicle includes an extended radar array, a processor, and a controller. The extended radar array is formed by moving the vehicle's radar array a selected distance. The processor is configured to receive multiple observations of an object from the extended radar array, operate a neural network to generate a network output signal, and determine object parameters of the object relative to the vehicle from the network output signal. The controller operates the vehicle based on the object parameters of the object.

[0008] In addition to one or more features described herein, as the radar array moves through a selected distance, the extended radar array acquires multiple observations at each of multiple locations within the radar array. The processor is also configured to operate the neural network based on the inputs of the multiple observations to generate multiple features, and to operate a cascaded module to combine the multiple features to obtain the network output signal. The processor is also configured to train the neural network by determining weights that minimize a loss function comprising the network output signal and a reference signal. The processor is further configured to generate a reference signal by coherently combining the multiple observations over time based on the known relative distance between the radar array and the object during relative motion between the vehicle and the object. The processor is also configured to generate the reference signal from the product of observations received from the extended radar array and a synthetic response based on the angles and ranges recorded for the observations.

[0009] The above-described features and advantages, as well as other features and advantages, of this disclosure will become apparent when taken in conjunction with the accompanying drawings and the following detailed description. Attached Figure Description

[0010] Other features, advantages, and details appear by way of example only in the following detailed description, which is described in detail with reference to the accompanying drawings, in which:

[0011] Figure 1 An autonomous vehicle is shown in one embodiment;

[0012] Figure 2 It shows Figure 1 Autonomous vehicles, which include radar arrays of radar systems suitable for detecting objects in their environment;

[0013] Figure 3 It shows how to... Figure 2 The radar array moves by selecting a distance to generate an extended radar array;

[0014] Figure 4 A schematic diagram illustrating side-to-side motion as an autonomous vehicle moves forward to generate an extended radar array is shown.

[0015] Figure 5 A schematic diagram illustrating a method for training a neural network to determine an angular position, the resolution of which is insensitive to the lateral or sideways motion of the vehicle, is shown.

[0016] Figure 6 A block diagram illustrating a method for training a deep neural network according to an embodiment is shown;

[0017] Figure 7 It shows the corresponding Figure 6 The neural network architecture for the feature generation process;

[0018] Figure 8 A block diagram illustrating a method for determining the angular position of an object using a trained deep neural network is shown.

[0019] Figure 9 A graph showing the angular resolution obtained using the method disclosed herein is shown; and

[0020] Figure 10 A top view of the autonomous vehicle is shown, illustrating the angular resolution of the three radar arrays at different angles relative to the vehicle. Detailed Implementation

[0021] The following description is merely exemplary in nature and is not intended to limit this disclosure, its application, or use. It should be understood that in all the accompanying drawings, corresponding reference numerals denote similar or corresponding parts and features. As used herein, the term module refers to processing circuitry, which may include application-specific integrated circuits, electronic circuitry, processors (shared, dedicated, or grouped), and memory executing one or more software or firmware programs, combinational logic circuitry, and / or other suitable components providing the described functionality.

[0022] According to an exemplary embodiment, Figure 1 An autonomous vehicle 10 is illustrated. In an exemplary embodiment, the autonomous vehicle 10 is a so-called Level 4 or Level 5 automation system. A Level 4 system signifies "high automation," referring to the autonomous driving system's performance in a specific driving mode for all aspects of a dynamic driving task, even if the driver does not respond appropriately to intervention requests. A Level 5 system signifies "full automation," referring to the autonomous driving system's full-time performance for all aspects of a dynamic driving task under all road and environmental conditions that the driver can manage. It should be understood that the systems and methods disclosed herein can also be used with autonomous vehicles operating at any of Levels 1 to 5.

[0023] Autonomous vehicle 10 typically includes at least a navigation system 20, a propulsion system 22, a transmission system 24, a steering system 26, a braking system 28, a sensor system 30, an actuator system 32, and a controller 34. Navigation system 20 determines a trajectory plan for autonomous driving of autonomous vehicle 10. Propulsion system 22 provides power to generate propulsion for autonomous vehicle 10 and, in various embodiments, may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system. Transmission system 24 is configured to transmit power from propulsion system 22 to two or more wheels 16 of autonomous vehicle 10 according to a selectable speed ratio. Steering system 26 affects the position of two or more wheels 16. Although described for illustrative purposes as including a steering wheel 27, in some embodiments contemplated within the scope of this disclosure, steering system 26 may not include a steering wheel 27. Braking system 28 is configured to provide braking torque to two or more wheels 16.

[0024] Sensor system 30 includes radar system 40, which senses objects in the external environment of autonomous vehicle 10 and provides various radar parameters for determining object parameters of one or more objects 50, such as the position and relative speed of various remote vehicles in the autonomous vehicle environment. These radar parameters can be provided to navigation system 20. In operation, transmitter 42 of radar system 40 emits a radio frequency (RF) source signal 48, which is reflected back at autonomous vehicle 10 by one or more objects 50 in the field of view of radar system 40 as one or more reflected echo signals 52, which are received at receiver 44. The one or more echo signals 52 can be used to determine various object parameters of one or more objects 50, such as the object's range, Doppler frequency or relative radial velocity, and azimuth. Sensor system 30 includes additional sensors, such as digital cameras, for identifying road features, etc.

[0025] The navigation system 20 establishes a trajectory for the autonomous vehicle 10 based on radar parameters from the radar system 40 and any other relevant parameters. The controller 34 can provide the trajectory to the actuator system 32 to control the propulsion system 22, the transmission system 24, the steering system 26, and / or the braking system 28, thereby navigating the autonomous vehicle 10 relative to the object 50.

[0026] The controller 34 includes a processor 36 and a computer-readable storage device or computer-readable storage medium 38. The computer-readable storage medium includes a program or instructions 39 that, when executed by the processor 36, operate the autonomous vehicle based at least on radar parameters and other relevant data. The computer-readable storage medium 38 may also include a program or instructions 39 that, when executed by the processor 36, determine the state of the object 50 to allow the autonomous vehicle to move relative to the object.

[0027] Figure 2 It shows Figure 1 A plan view 200 of an autonomous vehicle 10 is provided, the autonomous vehicle including a radar array 202 of a radar system 40 adapted to detect objects within its environment. The radar array 202 includes individual radars (202a, 202b, 202c) positioned along the front end of the autonomous vehicle 10. In various embodiments, the radar array 202 may be located at any selected position on the autonomous vehicle 10. The radar array 202 is operated to generate a source signal 48 and, in response, to receive an echo signal 52 by reflecting the source signal from an object such as object 50. The radar system 40 may operate the radar array 202 to perform beam control of the source signal. Comparison of the echo signal and the source signal produces information about object parameters of object 50, such as its range, azimuth position, elevation, and relative radial velocity relative to the autonomous vehicle 10. Although the radar array 202 is shown as having three radars (202a, 202b, 202c), this is for illustrative purposes only and is not intended to be limiting.

[0028] The radars (202a, 202b, 202c) are aligned substantially along the baseline 204 of the radar array 202. The length of the baseline 204 is defined by the distance from one end of the radar array 202 to the opposite end of the radar array. Although the baseline 204 can be a straight line, in other embodiments, the radars (202a, 202b, 202c) are positioned along a baseline that is a curved surface, such as the front surface of the autonomous vehicle 10.

[0029] Figure 3 The autonomous vehicle 10 is shown moving. Figure 2 The radar array 202 selects a distance to form a planar view 300 of the extended radar array 302. In various embodiments, the radar array 202 moves in a direction perpendicular to or substantially perpendicular to the baseline 204. Radar observations (X1, ..., X...) n ) is obtained at different times during the selected distance movement process, resulting in Figure 3 The different radar array positions shown (L1, ..., L) n The echo signal is detected by the radar array. The forward motion of the autonomous vehicle 10 generates a two-dimensional extended radar array 302. The forward aperture 304 of the extended radar array 302 is defined by the length of the baseline 204 of the radar array 200. The side aperture 306 of the extended radar array 302 is defined by the distance traveled by the autonomous vehicle 10 within a selected time period.

[0030] Figure 4A schematic diagram 400 is shown, illustrating the lateral motion as the autonomous vehicle 10 moves forward to generate an extended radar array. Velocity vectors 402a, 402b, 402c, and 402d shown for the autonomous vehicle 10 reveal that even when the vehicle is moving in a "straight" direction, a lateral component of the velocity exists due to this lateral motion. The angular resolution of the extended radar array 302 generated by the vehicle's forward motion is sensitive to this lateral motion.

[0031] Figure 5 A schematic diagram 500 illustrates a method for training a neural network to determine angular positions with resolution insensitive to the lateral or sideways motion of the autonomous vehicle 10. The training phase of the neural network utilizes ground-based knowledge relating to the relative distance between radar array 202 and object 50 during relative motion between them. Observations (X1, ..., X2) recorded by the extended radar array 302 are used. n The signal is fed into a neural network such as a deep neural network (DNN) 510. The DNN 510 outputs an intensity image (I1, ..., I...). n From this image, various object parameters can be determined, such as the object's angular position and extent. Each observation (X1, ..., X...) n Intensity image (I1, ..., I) n The images (I1, ..., I) show the regions defined by range (x) and cross-range (y) coordinates, respectively, in relation to angular position. n The DNN510 can be trained by comparing its weighted sum coefficients with ground-based images to update the DNN510's values ​​for later use in the inference phase of operations. The intensity images (I1, ..., I...) n The intensity peaks of the image appear at different locations within the region. For example, the intensity peaks in intensity image I2 are closer to the peaks in other intensity images, while remaining essentially within the same lateral range. The trained DNN 510 is able to determine the angular position of an object with a higher angular resolution than the radar array's angular resolution.

[0032] Figure 6 A block diagram 600 illustrating a method for training a DNN 510 according to an embodiment is shown. In block 602, at time (T1, ..., T... N ) Obtain observations (X1, ..., X N In box 604, the DNN 510 processes each observation (X1, ..., X) independently. N ), and from the observed values ​​(X1, ..., X N Generate a set of features (Q1, ..., Q)N In box 606, the network combines features (Q1, ..., Q). N To generate network output signals It is a coherently combined reflection intensity image.

[0033] Meanwhile, in box 608, each observation (X1, ..., X...) is recorded. N The radar array positions (L1, ..., L) N In box 610, given the radar array position for each observation point, the observation point (X1, ..., X...) is... N The observations are coherently combined. The combined observations generate a reference signal Z, as shown in equation (1):

[0034]

[0035] Among them, a H (θ n φ n R n X is a synthetic response array based on the angle and range recorded from the nth observation. n It is the nth observation received from the extended radar array.

[0036] In box 612, based on the network output signal The loss is calculated using a loss function, as disclosed in equation (2) below, along with a reference signal Z.

[0037]

[0038] Where p is a value between 0.5 and 2, and E represents the average operator over a set of examples (e.g., the training set). Therefore, the loss is the network output signal. The average of the differences between the loss function and the reference signal Z. The loss calculated in box 612 is used in box 604 to update the weighted sum coefficients of the neural network. Updating the weighted sum coefficients involves determining whether to minimize the loss function or minimize the network output signal. The values ​​of the weighted sum coefficients of the neural network for the difference between reference signals Z.

[0039] Figure 7 It shows the corresponding Figure 6 The feature generation process (i.e., boxes 604 and 606 of block diagram 600) is represented by a neural network architecture 700. This neural network architecture includes multiple convolutional neural networks (CNNs) 702a, ..., 702N. Each CNN 702a receives an observation (X1, ..., X...). N ), and generate one or more features (Q1, ..., Q) based on the observation. N ).like Figure 7As shown, CNN702a receives observation X1 and generates feature Q1, CNN702b receives observation X2 and generates feature Q2, and CNN702n receives observation X... N And generate feature Q N Cascading module 704 cascading features (Q1, ..., Q) N Cascaded features are sent via a CNN 706, which generates a network signal that includes a focused radar image with enhanced resolution.

[0040] Figure 8 Block diagram 800 illustrates a method for determining the angular position of an object using a trained DNN 510. In block 802, at times (T1, ..., T...)... N Obtain antenna array observations (X1, ..., X...) N In box 804, the trained DNN 510 processes each observation (X1, ..., X) independently. N ), and from the observed values ​​(X1, ..., X N Generate a set of features (Q1, ..., Q) N In box 806, coherent matched filtering is used to combine features (Q1, ..., Q). N The combination is then processed by a trained CNN to generate the network output signal.

[0041] Figure 9 An angular resolution map obtained using the method disclosed herein is shown. The results are from an autonomous vehicle 10 with three radars (202a, 202b, 202c), which move at a rate sufficient to produce a 5-meter side aperture. Each radar includes an antenna array, each having an angular resolution of 1.5 degrees when operating independently of the method disclosed herein. The azimuth (θ) of the object is shown along the abscissa, with 0 degrees representing the direction directly in front of the vehicle and 90 degrees representing one side of the vehicle. The angular resolution (R) is shown along the ordinate axis. Multiple observations (X1, ..., X2) are obtained using a single radar (e.g., radar 202a). N Radar 202a can achieve the angular resolution shown in curve 902. For an object in front of the vehicle (zero degrees), the resolution of a single radar is the same as the standard resolution of a single radar (e.g., 1.5 degrees), as shown in curve 902 at 0 degrees. As the object angle increases, the angular resolution of a single radar decreases, resulting in an improvement to approximately 0.4 degrees at a distance of 10 degrees in front of the vehicle. At higher object angles, the angular resolution of a single radar steadily increases, resulting in an angular resolution of approximately 0.1 degrees at 45 degrees.

[0042] Curve 904 illustrates the angular resolution of the extended radar array 302 based on radar array 202 with three radars (202a, 202b, 202c). For an object in front of the vehicle (zero degrees), the resolution is the same as that of a single antenna in the antenna array (e.g., 1.5 degrees), as shown in curve 904. As the object angle increases, the angular resolution of radar array 202 decreases, resulting in an improvement of approximately 0.1 degrees at a distance of 10 degrees from the vehicle. At higher object angles, the angular resolution of radar array 202 steadily increases, resulting in an angular resolution of approximately 0.02 degrees at 45 degrees.

[0043] Figure 10 A top view 1000 of the autonomous vehicle 10 is shown, illustrating the angular resolution of the radar array 202 with three radars (202a, 202b, 202c) relative to the vehicle at different angles. The angular resolution is 1.5 degrees at 0 degrees. The angular resolution is 0.1 degrees at 10 degrees. The angular resolution is 0.04 degrees at 25 degrees. The angular resolution is 0.02 degrees at 45 degrees.

[0044] While the foregoing disclosure has been described with reference to exemplary embodiments, those skilled in the art will understand that various changes can be made and elements can be substituted with equivalents without departing from the scope of the invention. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of this disclosure without departing from the basic scope of the disclosure. Therefore, it is intended that this disclosure be limited to the specific embodiments disclosed, but will include all embodiments falling within its scope.

Claims

1. A method of operating a vehicle, comprising: Multiple observations of an object are received at an extended radar array formed by moving the vehicle's radar array at a selected distance; Multiple observations are input into a neural network to generate the network output signal; Determine the object parameters relative to the vehicle from the network output signal; and Vehicle operations based on object parameters. The extended radar array is formed by moving the radar array in a direction perpendicular to the radar array baseline as the vehicle moves forward, and the multiple observations are obtained at different time points during the vehicle's movement, used to characterize the reflection characteristics of the object at different locations. The multiple observations are input into the neural network to generate multiple features, and these features are combined to obtain the network output signal. The neural network is trained by determining weights that minimize a loss function that includes the network output signal and a reference signal. The reference signal is generated by coherently combining multiple observations over time based on the known relative distance between the radar array and the object during the relative motion between the vehicle and the object. The reference signal includes the product of the observations received from the extended radar array and the synthetic response based on the angles and distances recorded for the observations.

2. The method of claim 1 further includes obtaining multiple observations at each of the multiple locations of the radar array as the radar array moves through the selected distance.

3. The method according to claim 1, wherein, The neural network includes multiple convolutional networks, each of which receives corresponding observations from the multiple observations and generates corresponding features from the multiple features.

4. A system for operating a vehicle, comprising: An extended radar array, which is formed by moving the vehicle's radar array by a selected distance; The processor is configured as follows: Receive multiple observations of the object from the extended radar array; Based on the multiple observations, the neural network is operated to generate the network output signal; Determine the object parameters relative to the vehicle from the network output signal; and A controller is used to operate a vehicle based on object parameters. The extended radar array is formed by moving the radar array in a direction perpendicular to the radar array baseline as the vehicle moves forward, and the multiple observations are obtained at different time points during the vehicle's movement, used to characterize the reflection characteristics of the object at different locations. The multiple observations are input into the neural network to generate multiple features, and these features are combined to obtain the network output signal. The neural network is trained by determining weights that minimize a loss function that includes the network output signal and a reference signal. The reference signal is generated by coherently combining multiple observations over time based on the known relative distance between the radar array and the object during the relative motion between the vehicle and the object. The reference signal includes the product of the observations received from the extended radar array and the synthetic response based on the angles and distances recorded for the observations.

5. The system according to claim 4, wherein, As the radar array moves through a selected distance, the extended radar array acquires multiple observations at each of the multiple locations on the radar array.

6. The system according to claim 4, wherein, The neural network comprises multiple convolutional networks, each configured to receive corresponding observations from multiple observations and generate corresponding features of multiple features.

Images