Target tracking during acceleration events

Abstract

A vehicle and a method for tracking an object and controlling the vehicle based on the tracked object. A radar Doppler (RD) map is received from a radar sensing system of the vehicle, and relative acceleration of the object relative to the vehicle is detected based on the RD map to provide acceleration data. A current frame of detected object data is received from the vehicle's sensing system. When relative acceleration is detected, a tracking algorithm is adapted to reduce the influence of a predicted motion model or the object's historical state, and an adapted tracking algorithm is used to track the object, thereby providing adapted estimated object data based on the object tracking. One or more vehicle actuators are controlled based on the adapted estimated object data.

Description

Technical Field

[0001] TECHNICAL FIELD The present invention relates generally to vehicles and, more particularly, to methods and systems for more accurately tracking a target object during an acceleration event. Background Art

[0002] Vehicles utilize object tracking algorithms in a variety of vehicle control applications, including advanced driver assistance systems (ADAS) and autonomous driving tasks, to name just two of many possible examples. Such vehicle control applications may require the use of sensing systems to detect and track objects for emergency braking, collision avoidance, path planning, and other automated vehicle control functions.

[0003] Some vehicles utilize camera vision systems, radar systems, and / or lidar systems for object detection and tracking. Object tracking utilizes tracking algorithms that may rely on a motion model of the target vehicle. Certain motion models may not properly account for sudden anomalies in the target vehicle's motion, such as sudden stops, accelerations, or sharp turns. That is, the object information output from the tracker may not match the true object information, for example due to delays introduced by the tracking algorithm. As a result, the tracker may experience large errors during acceleration events and may even lose lock on the target. This can be troublesome because, especially during object acceleration events, accurate object position and velocity information is required for proper automatic control of the vehicle.

[0004] Therefore, it is desirable to reduce errors in object position estimates (and other state data) during acceleration events and to reduce the likelihood of false detections during motion anomalies (e.g., acceleration events). Additionally, it is desirable to enhance automated vehicle control applications using more accurate object tracking data without requiring significant additional hardware expenditures. Furthermore, other desirable features and characteristics of the present invention will become apparent from the following detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background. Summary of the Invention

[0005] In a first aspect, a method for object tracking and vehicle control is provided. The method includes, via a processor: receiving a radar Doppler RD map from a radar sensing system of a vehicle; detecting a relative acceleration of an object relative to the vehicle based on the RD map, and providing acceleration data based thereon; and receiving a current frame of detected object data from the sensing system of the vehicle. When relative acceleration has not yet been detected, the object is tracked using a tracking algorithm based on the current frame of detected object data, a predicted motion model, and a historical state of the object, and estimated object data is provided based on the object tracking. When relative acceleration has been detected, the tracking algorithm is adapted to reduce the influence of the predicted motion model and the historical state of the object, and the object is tracked using the adapted tracking algorithm, and adapted estimated object data is provided based on the object tracking. Vehicle features are controlled based on the adapted estimated object data or the estimated object data.

[0006] In an embodiment, a neural network is used to detect the relative acceleration of an object relative to a vehicle. The neural network may be a deep neural network comprising a convolutional neural network, a fully connected layer, and a logistic regression operation. The deep neural network is trained to classify RD maps captured while the vehicle is traveling at a constant speed, and to classify RD maps captured while the vehicle is accelerating.

[0007] In an embodiment, the acceleration data includes a binary value indicating an accelerated state and a non-accelerated state; a plurality of output levels corresponding to different acceleration states; or a continuously variable output representing a degree of acceleration.

[0008] In an embodiment, the acceleration data is determined based on detecting ambiguity of the RD map.

[0009] In an embodiment, receiving the current frame of detected object data from a sensing system of the vehicle includes receiving the current frame of detected object data from a radar system.

[0010] In an embodiment, receiving the current frame of detected object data from a sensing system of the vehicle includes receiving the current frame of detected object data from a lidar system or a camera vision system.

[0011] In an embodiment, the tracking algorithm comprises a Kalman filter.

[0012] In an embodiment, adjusting the tracking algorithm includes replacing a historical state of the object with a current frame of detected object data; adjusting a covariance matrix and / or adjusting parameters of a predictive motion model.

[0013] In an embodiment, the vehicle characteristics include steering commands, braking commands, and propulsion commands.

[0014] In another aspect, a vehicle is provided. The vehicle includes: a radar sensing system; one or more vehicle actuators, including at least one of a steering, braking, and propulsion actuator; and at least one processor in operable communication with the radar sensing system and the vehicle actuators. The processor is configured to execute program instructions to perform the steps outlined below. A radar Doppler RD map is received from the vehicle's radar sensing system. Relative acceleration of an object relative to the vehicle is detected based on the RD map, and acceleration data is provided based on the relative acceleration detection. A current frame of detected object data is received from the vehicle's sensing system. When relative acceleration has not been detected, the object is tracked using a tracking algorithm based on the current frame of detected object data, a predicted motion model, and a historical state of the object, and estimated object data is provided based on the object tracking. When relative acceleration has been detected, the tracking algorithm is adapted to reduce the influence of the predicted motion model and / or the historical state of the object, the object is tracked using the adapted tracking algorithm, and adapted estimated object data is provided based on the object tracking. One or more vehicle actuators are controlled based on the adapted estimated object data or the estimated object data.

[0015] In one embodiment, a neural network is used to detect the relative acceleration of an object relative to a vehicle. The neural network is a deep neural network that includes a convolutional neural network, a fully connected layer, and a logistic regression operation. The deep neural network is trained to classify RD maps captured while the vehicle is traveling at a constant speed, and also to classify RD maps captured while the vehicle is accelerating.

[0016] In an embodiment, the acceleration data includes: a binary value indicating an accelerated state and a non-accelerated state; a plurality of output levels corresponding to different acceleration states; or a continuously variable output representing a degree of acceleration.

[0017] In an embodiment, the acceleration data is determined based on a measure of the ambiguity of the RD map.

[0018] In an embodiment, receiving the current frame of detected object data from a sensing system of the vehicle includes receiving the current frame of detected object data from a radar system.

[0019] In an embodiment, receiving the current frame of detected object data from a sensing system of the vehicle includes receiving the current frame of detected object data from a lidar system or a camera vision system.

[0020] In an embodiment, the tracking algorithm comprises a Kalman filter.

[0021] In an embodiment, adjusting the tracking algorithm includes replacing a historical state of the object with a current frame of detected object data; adjusting a covariance matrix and / or adjusting parameters of a predictive motion model.

[0022] In an embodiment, the detected object data is generated based on the raw radar data from the radar device of the radar system based on the range Doppler, beamforming and reflection point detection processes. The RD map is generated on the raw radar data through the range Doppler process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Hereinafter, exemplary embodiments will be described with reference to the following drawings, wherein like reference numerals denote like elements, and wherein:

[0024] Figure 1 is a functional block diagram of a vehicle including a radar system and a control system according to an exemplary embodiment;

[0025] Figure 2 According to an exemplary embodiment Figure 1 Functional block diagram of the vehicle's control system;

[0026] Figure 3 According to an exemplary embodiment, Figure 1 vehicles and Figure 2 a data flow diagram executed by a control system for providing estimated object data of a tracked object;

[0027] Figure 4 shows an exemplary RD graph for detecting acceleration of an object according to an exemplary embodiment; and

[0028] Figure 5 is a method according to an exemplary embodiment for combining Figure 1 vehicles and Figure 2 Flowchart of a method for implementing object tracking using an adaptable tracking algorithm in a control system. DETAILED DESCRIPTION

[0029] The following detailed description is merely exemplary in nature and is not intended to limit the application and use. In addition, there is no intention to be bound by any express or implied theory presented in the preceding technical field, background technology, summary of the invention or the following detailed description. As used herein, the term module refers to an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated or group) and memory that executes one or more software or firmware programs, a combinational logic circuit and / or other suitable components that provide the described functionality.

[0030] Figure 1A vehicle 100 or automobile according to an exemplary embodiment is shown. As described in further detail below, the vehicle 100 includes a control system 102 for utilizing radar data to detect whether an acceleration event is occurring. Such an acceleration event includes relative acceleration between the vehicle 100 and an object. During such an acceleration event, the acceleration typically appears as a more ambiguous radar Doppler (RD) map than when the RD map is captured under constant relative velocity conditions. Therefore, when an acceleration event is detected, the tracking algorithm of the control system 102 that tracks the object is adapted to place more weight on the current frame of object sensing data and / or place less weight on the predicted vehicle motion model compared to historical object sensing data. In this way, when an acceleration event is detected, the tracking algorithm will provide an object position estimate that is more reflective of instantaneous sensing and less influenced by historical and predicted model factors.

[0031] like Figure 1 As shown, in addition to the control system 102 described above, the vehicle 100 also includes a chassis 112, a body 114, four wheels 116, an electronic control system 118, a steering system 150, and a braking system 160. The body 114 is disposed on the chassis 112 and substantially surrounds the other components of the vehicle 100. The body 114 and the chassis 112 may together form a frame. The wheels 116 are each rotationally coupled to the chassis 112 near a respective corner of the body 114. In various embodiments, the vehicle 100 may be configured to Figure 1 For example, in some embodiments, the number of wheels 116 may vary. As another example, in various embodiments, vehicle 100 may not have a steering system, may be steered by differential braking, and various other possible differences may also be present.

[0032] exist Figure 1 In the exemplary embodiment shown, vehicle 100 includes an actuator assembly 120. Actuator assembly 120 includes at least one propulsion system 129 mounted on chassis 112 that drives wheels 116. Actuator assembly 120 also includes a steering system 150 and a braking system 160, as well as associated steering and braking actuators. In the depicted embodiment, actuator assembly 120 includes an engine 130. In one embodiment, engine 130 includes an internal combustion engine. In other embodiments, actuator assembly 120 may include one or more other types of engines and / or motors, such as an electric motor / generator, instead of or in addition to an internal combustion engine.

[0033] Still refer to Figure 1, the engine 130 is coupled to at least some of the wheels 116 via one or more drive shafts 134. In some embodiments, the engine 130 is mechanically coupled to a transmission. In other embodiments, the engine 130 may alternatively be coupled to a generator that is used to power an electric motor that is mechanically coupled to the transmission. In certain other embodiments (e.g., electric vehicles), the engine and / or transmission may not be required.

[0034] The steering system 150 is mounted on the chassis 112 and controls the steering of the wheels 116. The steering system 150 includes a steering wheel and a steering column (not shown). The steering wheel receives input from the driver of the vehicle 100. The steering column generates a desired steering angle for the wheels 116 via the drive shaft 134 based on the input from the driver. Similar to the discussion above regarding possible variations of the vehicle 100, in some embodiments, the vehicle 100 may not include a steering wheel and / or a steering column. In addition, in some embodiments, the autonomous vehicle may utilize steering commands generated by a computer without the involvement of a driver.

[0035] Braking system 160 is mounted on chassis 112 and provides braking for vehicle 100. Braking system 160 receives input from the driver via a brake pedal (not shown) and provides appropriate braking via a brake unit (also not shown). The driver also provides input regarding the desired speed or acceleration of the vehicle via an accelerator pedal (not shown), as well as various other inputs for various vehicle devices and / or systems, such as one or more vehicle radios, other entertainment systems, environmental control systems, lighting units, navigation systems, etc. (also not shown). Similar to the discussion above regarding possible variations of vehicle 100, in some embodiments, steering, braking, and / or acceleration may be commanded by a computer rather than by the driver (in one such embodiment, the vehicle's computer may use input from a radar system to steer, brake, and / or accelerate the vehicle).

[0036] exist Figure 1 In an embodiment of the present invention, the vehicle 100 includes a plurality of radar devices 132a to 132g mounted to the vehicle body 114. The radar devices (radars) 132a to 132g include forward looking radar devices 132a to 132c, side looking radar devices 132d, 132e, and / or rear looking radar devices 132f, 132g. The various radar devices 132a to 132g may be configured as part of a parking assist system, a rear collision warning system, a cross traffic alert system, an emergency braking system, a pedestrian detection system, a forward collision avoidance system, and / or a blind spot detection system, among other possible systems that utilize radar data as input to various vehicle outputs. Although in Figure 1Seven radar devices 132a through 132g are shown, but fewer or more radar devices may be provided. Radar devices 132a through 132g may be short-, medium-, or long-range radar devices. Radar devices 132a through 132g each generate a range-Doppler beam (RDB) map after pre-processing the raw radar data, as described further below. Forward-looking radar devices 132a through 132c are particularly useful for providing RD maps to detect acceleration of objects and track them, as described further below.

[0037] The control system 102 is mounted on the chassis 112. The control system 102 provides object tracking using a tracking algorithm and adapts the tracking algorithm based on whether the object is detected to be accelerating. The control system 102 implements a method for detecting motion anomalies (e.g., acceleration) from a range Doppler spectrum using a deep neural network (DNN), and an object tracking algorithm that adapts based on the anomaly detection status. The object tracking algorithm can be based on other sensor modalities besides radar, such as camera vision and lidar. In one example, the control system 102 provides object tracking based on the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the following in combination with the control system 102. Figure 5 and Figure 3 The related data conversion is further described in the method 500 to provide these functions.

[0038] Control system 102 is depicted as including Figure 2 1 and 2. The radar system 202 in FIG. 1 is shown in FIG. 1 . However, it will be appreciated that in certain embodiments, these features may include two or more systems. Additionally, in various embodiments, the control system 102 may include all or a portion of various other vehicle devices and systems and / or may be coupled thereto, such as the actuator assembly 120 and / or the electronic control system 118.

[0039] refer to Figure 2 , according to an exemplary embodiment, is Figure 1 The control system 102 provides a functional block diagram. Figure 2 As shown, the control system 102 includes a radar system 202 and a controller 204. The radar system 202 is included as part of the vision system 103, which may include one or more additional sensors 104. In the illustrated embodiment, the sensors 104 include one or more cameras 210 (as part of a camera vision system) and one or more light detection and ranging (LIDAR) systems 212. The cameras 210, the LIDAR system 212, and the radar system 202 obtain respective sensor information that identifies objects on or near a roadway on which the vehicle 100 is traveling, such as moving or stationary vehicles, pedestrians, bicycles, animals, buildings, trees, guardrails, medians, and / or other objects on or near the roadway.

[0040] Likewise Figure 2As shown, radar system 202 includes multiple radar devices 132a through 132g. In one embodiment, each radar device 132a through 132g includes a transmitter (or transmitter antenna), a receiver (or receiver antenna), and is coupled to a pre-processing module 226. In another embodiment, a common pre-processing module 226 may be provided. The transmitter transmits radar signals in the form of time-separated, frequency-modulated chirps. After the transmitted radar signals come into contact with one or more objects on or near the road where vehicle 100 is located (stationary or moving) and are reflected / redirected back to radar system 202, the redirected radar signals are received by the receivers of the respective radar devices 132a through 132g.

[0041] like Figure 2 As shown, the controller 204 is coupled to the radar system 202 and the sensor 104 (which are part of the additional sensing system). Similar to the discussion above, in some embodiments, the controller 204 can be disposed in whole or in part within or as part of the radar system 202. In addition, in some embodiments, the controller 204 is also coupled to one or more other vehicle systems (such as Figure 1 The controller 204 receives information sensed or determined from the radar system 202 and the sensor 104. In one embodiment, the controller 204 receives raw radar data from the radar system 202 and pre-processes the raw radar data in the controller's pre-processing module 226 to provide one or more RD maps. The controller 204 uses a trained neural network classifier to detect relative object acceleration based on the RD map. When relative object acceleration has been detected, the tracking algorithm is adapted to be more biased towards the current frame of detected object data from the vision system 103. The controller 204 typically detects the object based on the RD map. Figure 5 Method 500 and Figure 3 The data flow diagram performs these functions.

[0042] like Figure 2 As shown, the controller 204 includes a computer system. In certain embodiments, the controller 204 may also include one or more of the radar system 202, the sensor 104, one or more other systems and / or components thereof. Additionally, it will be understood that the controller 204 may be other than Figure 2 For example, the controller 204 may be coupled to or may otherwise utilize one or more remote computer systems and / or other control systems, such as Figure 1 Electronic control system 118.

[0043] In the depicted embodiment, the computer system of the controller 204 includes a processor 230, a memory 232, an interface 234, a storage device 236, and a bus 238. The processor 230 performs the computational and control functions of the controller 204 and may include any type of processor or multiple processors, such as a single integrated circuit of a microprocessor, or any suitable number of integrated circuit devices and / or circuit boards working in conjunction to perform the functions of a processing unit. During operation, the processor 230 executes one or more programs 240 contained in the memory 232 and, therefore, generally controls the general operation of the controller 204 and the computer system of the controller 204 in performing the processes described herein, such as those described below in conjunction with Figure 5 and Figure 3 The data flow process is further described in the method 500. The one or more programs 240 include a pre-processing module 226, an acceleration detection module 241, a tracker module 243, and a tracker update module 245 for performing the steps of the method 500 described in detail below.

[0044] The processor 230 is capable of executing one or more programs (i.e., running software) to perform various tasks encoded in the programs, particularly the pre-processing, acceleration detection, tracker, and tracker update modules 226, 241, 243, 245. The processor 230 may be a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), or other suitable device recognized by those skilled in the art.

[0045] The memory 232 can be any type of suitable memory. This would include various types of dynamic random access memory (DRAM) such as SDRAM, various types of static RAM (SRAM), and various types of non-volatile memory (PROM, EPROM, and flash memory). In some examples, the memory 232 is located on the same computer chip and / or co-located with the processor 230. In the depicted embodiment, the memory 232 stores the program 240 described above and one or more stored values ​​242 used to make the determination.

[0046] The bus 238 is used to transfer programs, data, status, and other information or signals between the various components of the computer system of the controller 204. The interface 234 allows communication to the computer system of the controller 204, for example, from a system driver and / or another computer system, and can be implemented using any suitable methods and devices. The interface 234 may include one or more network interfaces to communicate with other systems or components. The interface 234 may also include one or more network interfaces for communicating with technicians and / or one or more storage interfaces for connecting to storage devices such as storage device 236.

[0047] Storage device 236 can be any suitable type of storage device, including direct access storage devices such as hard drives, flash memory systems, floppy disk drives, and optical disk drives. In an exemplary embodiment, storage device 236 includes a program product from which memory 232 can receive program 240 (including computer modules 226, 241, 243, and 245) that performs one or more embodiments of one or more processes of the present disclosure, such as the steps of method 500 (and any sub-processes thereof). In another exemplary embodiment, the program product can be stored directly in and / or accessed by memory 232 and / or a disk (e.g., disk 244), such as referenced below.

[0048] The bus 238 may be any suitable physical or logical means of connecting computer systems and components. This includes, but is not limited to, direct hard-wired connections, fiber optics, infrared, and wireless bus technologies. During operation, the program 240 is stored in the memory 232 and executed by the processor 230.

[0049] It will be understood that although the exemplary embodiment is described in the context of a fully functional computer system, those skilled in the art will recognize that the mechanisms of the present disclosure are capable of being distributed as a program product having one or more types of non-transitory computer-readable signal-bearing media for storing the program and its instructions and performing its distribution, such as a non-transitory computer-readable medium that carries the program and contains computer instructions stored therein for causing a computer processor (such as processor 230) to perform and execute the program. Such a program product may take a variety of forms, and the present disclosure applies equally regardless of the particular type of computer-readable signal-bearing medium used to perform the distribution. Examples of signal-bearing media include recordable media such as floppy disks, hard drives, memory cards, and optical disks, and transmission media such as digital and analog communication links. Similarly, it will be understood that the computer system of controller 204 may also be connected to Figure 2 Unlike the illustrated embodiment, a computer system such as controller 204 may be coupled to or may utilize one or more remote computer systems and / or other control systems.

[0050] Additional references Figure 3 In data flow diagram 300, the or each radar device 132 provides raw radar data 302. In one embodiment, there are eight channels corresponding to the respective antennas associated with each radar device 132. Each channel provides corresponding raw radar data 302 for each time frame. Each radar device 132 may include fewer or more channels. Raw radar data 302 (or radar signals) may be provided in the form of I / Q data. For example, each frame of raw data may be collected every 20 to 50 milliseconds.

[0051] Raw radar data 302 is received by pre-processing module 226, which performs range-Doppler (RD) processing 330, beamforming 332, and reflection point detection 334. Pre-processing module 226 outputs RD maps 304 and provides detected object data 308, which represents objects detected by radar system 202. RD processing 330 provides a two-dimensional (2D) matrix of range and energy (Doppler) bins, where each pixel represents an intensity value for a sub-range of range and energy readings. In one embodiment, raw radar data 302 includes eight channels of raw radar data 302, meaning that range-Doppler processing 330 provides eight RD maps 304. RD processing 330 for each frame assumes that the relative velocity of the vehicle and any objects remains constant within the short frame time of radar capture, which can be 20 to 50 milliseconds as described above. In one embodiment, RD processing 330 includes discrete Fourier transform (DFT) processing that assumes a constant velocity for each frame. The RD processing 330 assumes linear phase variation, which is correct for constant velocity, but not during significant acceleration events. When the vehicle has a constant velocity within the radar frame time, then the constant velocity assumption is correct, and the RD processing 330 matches the phase variation. However, when the vehicle motion is not constant velocity (e.g., during acceleration / deceleration / turning), then the constant velocity assumption relied upon by the range-Doppler processing 330 is incorrect, the phase variation is not linear, and the RD processing 330 has a mismatch.

[0052] Brief reference Figure 4 , it can be seen that the RD processing 330 produces a relatively sharp RD map 402 during periods when the vehicle and the objects captured in the raw radar data maintain a constant relative velocity, while producing a relatively blurred or unfocused RD map 404 during relative acceleration events (which include positive relative acceleration, deceleration, and turning). As will be further described, the present disclosure provides for detecting motion anomalies from the RD map 304 using a DNN trained to classify different levels of blur in the RD map 304 to quickly detect acceleration events. When a motion anomaly is detected, the tracking history is reset and / or the tracking model is updated.

[0053] Continue to refer Figure 3, the RD map 304 is output to the acceleration detection module 241 (which will be described further below) and the beamforming process 332. Beamforming 332 combines the RD maps 304 from multiple channels and adds an angular dimension to the data. That is, beamforming 332 causes the eight (e.g.) channels of the RD map 304 for the current frame to be converted into a single three- or four-dimensional matrix that includes range and Doppler information as well as azimuth and / or elevation information, thereby defining the angular direction of an object, the distance to the object, and the velocity of the object. Beamforming process 332 outputs the RDB map 306 to the reflection point detection process 334. The reflection point detection process 334 identifies objects in the RDB map 306 based on intensity values ​​greater than a threshold value to distinguish between areas where objects or targets are present or absent. The resulting detected object data 308 is passed to the tracker module 243 to identify and track each object.

[0054] In an embodiment, the tracker module 243 tracks objects and reduces errors in position estimate errors by essentially averaging instantaneous position estimates and combining them with position predictions based on the history of the object's position (represented by the historical object state 314) and the predicted motion model 336. The present disclosure recognizes that the predicted motion model 336 is desirably different for normal driving (e.g., during a constant relative velocity between the object and the vehicle 100) and abnormal driving (e.g., during relative lateral and / or longitudinal acceleration between one or more objects and the vehicle 100) to reduce the latency inherent in the tracker module 243 outputting the object's position during acceleration events. Normal driving is the most common scenario, which is why conventional object tracking systems are designed based on the assumption of normal driving motion. However, if a relative acceleration event can be detected early enough, the tracker module 243 can be adapted to rely less on the historical object state 314 and the predicted motion model 336 than during constant relative velocity conditions, which will make the tracker more accurate relative to the ground truth. The present disclosure allows for nearly instantaneous detection (e.g., on a frame-by-frame basis) of such acceleration events by analyzing the ambiguity of the RD graph 304 using a neural network, where the ambiguity indicates all relevant kinds of acceleration. Figure 4 It should be understood that the tracker module 243 can operate based on the detected object data 308 from the radar system 202, but can additionally or alternatively operate based on other detected object data from other sensing modes, as described with respect to the additional sensors 104. For example, the tracker module 243 can track objects detected by the lidar system 212 and / or by the camera 210. The tracking principles described further below are applicable regardless of the sensing mode, and acceleration event detection can be performed based on the RD map 304 even if object tracking is performed based on the other detected object data 310.

[0055] Acceleration detection module 241 receives RD map 304 from radar system 202 and performs acceleration detection based thereon. In one embodiment, acceleration detection module 241 processes RD map 304 via DNN 340 to determine acceleration data 318, which indicates the relative acceleration levels between vehicle 100 and any objects in RD map 304. In one embodiment, the DNN processes RD map 304 in parallel across all eight (e.g., channels) on a per-frame basis. Acceleration data 318 can be a binary output representing acceleration and constant velocity conditions, a multi-level output (e.g., greater than three) representing different acceleration levels from constant velocity to a maximum specified acceleration level, or a continuous acceleration measurement representing the output of acceleration readings. Similarly, tracker module 243 can adapt in a binary manner, to multiple different levels, or continuously depending on the type of output from acceleration detection module 241. Acceleration detection module 241 determines acceleration data 318 based on a measure of the ambiguity of RD map 304. In an embodiment, DNN 340 has been trained using labeled training RD maps to classify ambiguities in RD map 304. The training RD map can be simulated using a random phase jitter function on raw radar data, or it can be acquired from radar system 202 during vehicle operation. In one embodiment, DNN 340 includes a convolutional neural network (CNN) 342 with decimation, a fully connected layer 344, and a logistic regression or classification layer 346. As part of a supervised training process, the CNN and fully connected layers are trained based on a large number of labeled examples of RD map 304.

[0056] In an embodiment, tracker update module 245 responds to acceleration data 318 to output tracker adaptation data 320 describing the adaptations to be made to tracker module 243, and in particular, to the tracking algorithm. Tracker adaptation data 320 may include commands such as replacing historical object state 314 with the current frame of object state represented by detected object data 308; adapting the covariance matrix of the tracking algorithm and / or adapting parameters of the predicted motion model. This allows the current frame of detected object data 308 to have a greater impact on estimated object data 312 than would be the case if no acceleration were detected, compared to the prediction or historical terms when acceleration was detected. In some embodiments, acceleration data 318 provides multiple acceleration levels or a continuous measurement of acceleration, and the tracking algorithm is adapted proportionally based on the quantitative measurements included in acceleration data 318.

[0057] An example of a tracking algorithm implemented by the tracker module 243 is a Kalman filter. The purpose of the Kalman filter is to solve for the current state parameter vector of each object d at time n, which is defined as:

[0058] d n =[x n ,yn ,v x ,v y ] T (Equation 1)

[0059] x n and y n Represents the position coordinates of the tracked object, v x and v y represents the velocity along the x and y axes. The tracker module 243 includes a dynamic prediction motion model 336 and a measurement model 338. The dynamic prediction motion model 336 is defined as:

[0060] d n =[x n ,y n ,v x ,v y ] T (Equation 2)

[0061] In Equation 2, Φ represents the Kalman filter applied to the previous state d n-1 The state transition model Φ can be determined based on the physical model of the object. n is a term that intentionally introduces noise. n Represents the historical object state d n-1 The measurement model is defined as:

[0062] y n =Hd n +v n (Equation 3)

[0063] In Equation 3, y n Represents the true state d n Obtain state measurement y from the detected object data 308 n , the detected object data 308 may include measured position and velocity vectors. n is the assumed measurement or observation noise. The Kalman filter output is based on the predicted motion model 336 and the measurement model 338, and therefore takes into account the current state of the detected object, the historical state of the detected object, and the object motion model. In one embodiment, the Kalman filter output is described as:

[0064]

[0065] In Equation 4, K is the Kalman gain, which depends on the predicted motion model noise w n When the covariance Q is high, the uncertainty in the predicted motion model 336 is high, which means that A lower weight or bias is placed on the predicted motion model 336.

[0066] In an embodiment, the tracker update module 245 determines the tracker adaptation data 320, which adapts the covariance Q (which adapts the Kalman gain K) based on the acceleration data 318, and converts the historical object state 314 into 314. The parameters of the adapted state transition model Φ are set to the current frame of detected object data 308. Thus, the tracker module is biased to more closely reflect the current measured state of the object and to reduce the weight of the predicted motion model 336 and the historical object state 314 as the relative acceleration of the object increases. The acceleration data 318 will define when the acceleration detection module 241 does not detect relative acceleration (i.e., detects relative constant velocity), which will cause the tracker update module 245 to revert to a tracking algorithm that fully considers the predicted motion model 336, the historical object state 314, and the measurement model 338. The adaptation of the covariance matrix Q or the change of the state transition model Φ can be performed by multiplying the relative constant velocity setpoint by a scalar multiplier greater than 1, such as 10 to 100.

[0067] In an embodiment, the electronic control system 118 or some other automatic vehicle control application responds to the estimated object data 312 from the tracker module 243 to determine actuator control commands 316, such as braking, steering, and propulsion actuator commands. Depending on whether the acceleration detection module 241 determines an acceleration event and / or a quantitative measurement of acceleration during an acceleration event, the estimated object data 312 will be determined based on different tracking algorithms.

[0068] Figure 5 is a flow chart of a method 500 for tracking an object using an acceleration-adapted tracking algorithm and automatically controlling at least one vehicle function based on the tracked object. According to an exemplary embodiment, the method 500 may be combined with Figure 1 100 vehicles and Figure 2 The method 500 may be implemented by the control system 102. The method 500 may be performed continuously during vehicle operation or intermittently in response to detection of certain vehicle conditions, such as a threshold acceleration being breached.

[0069] like Figure 5 As shown, the method includes step 510 of receiving RD map 304 from radar system 202. RD map 304 may have varying degrees of discernible ambiguity. At step 520, acceleration of an object is detected based on RD map 304, particularly based on a measure of the ambiguity. At step 520, the acceleration may be detected as a binary output, at multiple levels of acceleration, or continuously. Detection of acceleration based on RD map 304 may be performed by DNN 340.

[0070] At step 530, a current frame of detected object data 308 is received from a radar system or from an additional sensor 104, such as a camera 210 or a lidar system 212. The current frame of detected object data 308 represents a measurement of the current state of one or more objects as seen by the sensor. At step 540, the objects described in detected object data 308 are tracked. Specifically, the tracking algorithm considers predicted motion model 336, which predicts the future state of an object based on a physics model. The tracking algorithm also considers the object's historical state and the object's current state. The object's state includes position and velocity data. The tracking algorithm can consider more or less historical state data and more or less deviation from predicted motion model 336. According to step 550, when acceleration is detected at step 520, the tracking algorithm is adapted to reduce the influence of predicted motion model 336 and / or reduce the time step or exclude the presence of historical object states 314, thereby increasing the influence of measurement model 338 and the current object state. Thus, the tracking algorithm is adapted at step 550 based on the results of object acceleration detection step 520.

[0071] At step 560, the object is tracked using the adapted tracking algorithm. When acceleration detection ceases, the object can be tracked using the unadapted tracking algorithm. At step 570, one or more vehicle actuators are controlled based on the tracked object, as described in the estimated object data 312, for example, to avoid a collision with the tracked object. In one embodiment, steering, braking, and / or propulsion commands are controlled based on the estimated object data 312.

[0072] It will be understood that the disclosed methods, systems, and vehicles may vary from those depicted in the figures and described herein. For example, the vehicle 100 and control system 102 and / or its various components may vary from those depicted in the figures and described herein. Figure 1 and 2 In addition, it will be appreciated that certain steps of method 500 may be different from those described in and in conjunction with the description thereof. Figure 5 It will be similarly appreciated that certain steps of the above methods may occur simultaneously or in conjunction with Figure 5 The sequences described in the preceding text may not occur in the same order.

[0073] Although at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that there are a large number of variations. It should also be understood that one or more exemplary embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of the present disclosure in any way. On the contrary, the foregoing detailed description will provide those skilled in the art with a convenient roadmap for implementing one or more exemplary embodiments. It should be understood that various changes may be made to the function and arrangement of elements without departing from the scope of the present disclosure as set forth in the appended claims and their legal equivalents.

Claims

1. A method for object tracking and vehicle control, the method comprising: receiving, via at least one processor, at least one radar Doppler RD map from a radar sensing system of a vehicle; detecting, via at least one processor, a relative acceleration of the object relative to the vehicle based on the RD map and providing acceleration data based thereon; receiving, via at least one processor, a current frame of detected object data from a sensing system of the vehicle; tracking the object using a tracking algorithm based on a current frame of detected object data, the predicted motion model, and a historical state of the object, and providing estimated object data based on the object tracking, when relative acceleration has not been detected, via the at least one processor; When relative acceleration has been detected, adapting, via the at least one processor, a tracking algorithm to reduce an influence of at least one of a predicted motion model and a historical state of the object, and tracking the object using the adapted tracking algorithm, and providing adapted estimated object data based on the object tracking; as well as At least one vehicle feature is controlled based on the adapted estimated object data or the estimated object data via at least one processor.

2. The method according to claim 1, wherein Detecting the relative acceleration of an object with respect to the vehicle is performed using a neural network.

3. The method according to claim 2, wherein: The neural network is a deep neural network and includes a convolutional neural network, a fully connected layer, and a logistic regression operation, wherein the deep neural network is trained to classify RD maps captured when the vehicle is traveling at a constant speed and to classify RD maps captured when the vehicle is accelerating.

4. The method according to claim 1, wherein The acceleration data includes: A binary value representing the accelerated and non-accelerated states.

5. The method according to claim 1, wherein The acceleration data is determined based on detecting ambiguity of the RD map.

6. The method according to claim 1, wherein Receiving a current frame of detected object data from a sensing system of the vehicle includes receiving a current frame of detected object data from a radar system.

7. The method according to claim 1, wherein Receiving a current frame of detected object data from a sensing system of the vehicle includes receiving a current frame of detected object data from a lidar system or a camera vision system.

8. The method according to claim 1, wherein The tracking algorithm includes a Kalman filter.

9. The method according to claim 1, wherein Adapting the tracking algorithm includes at least one of the following: Replace the historical state of the object with the current frame of detected object data; Adjusting the covariance matrix; and Adjust the parameters of the predicted motion model.

10. A vehicle comprising: radar sensing systems; one or more vehicle actuators including at least one of a steering, braking, and propulsion actuator; as well as At least one processor in operable communication with the radar sensing system and the vehicle actuators, the at least one processor configured to execute program instructions, wherein the program instructions are configured to cause the at least one processor to: receiving at least one radar Doppler RD map from a radar sensing system of the vehicle; detecting a relative acceleration of an object relative to the vehicle based on the RD graph, and providing acceleration data based on the relative acceleration detection; receiving a current frame of detected object data from a sensing system of the vehicle; When relative acceleration has not been detected, tracking the object using a tracking algorithm based on a current frame of detected object data, the predicted motion model, and a historical state of the object, and providing estimated object data based on object tracking; When relative acceleration has been detected, adapting a tracking algorithm to reduce an influence of at least one of a predicted motion model and a historical state of the object, and tracking the object using the adapted tracking algorithm, and providing adapted estimated object data based on the object tracking; as well as One or more vehicle actuators are controlled based on the adapted estimated object data or the estimated object data.

Images