System and method for tracking the augmented state of a moving object using an online adaptive composite measurement model - Patents.com

Abstract

A tracking system for tracking an object's expansion state is provided. The tracking system runs a probability filter that iteratively tracks beliefs about the object's expansion state over a predetermined period of time, where the beliefs are predicted using a motion model of the object and further updated using a composite measurement model of the object. After the predetermined period of time, the updated beliefs are smoothed to generate an online batch of state-separated training data. The composite measurement model includes a plurality of probability distributions constrained to lie around the object's contour with a predetermined relative geometric mapping to the center of the object. The composite measurement model is updated using the online batch of training data. The tracking system further tracks the object's expansion state based on the updated composite measurement model.

Description

[Technical field]

[0001] The present disclosure relates generally to automotive object tracking, and more particularly to a system and method for tracking the growth of an object using measurements of the object. [Background technology]

[0002] Control systems utilized by vehicles, such as autonomous and semi-autonomous vehicles, predict the safe motion or path of the vehicle to avoid collisions with obstacles, such as other vehicles or pedestrians. In some scenarios, the vehicle is also configured to sense surrounding conditions, such as road shoulders, pedestrians, and other vehicles, with the aid of one or more sensors in the vehicle. Some of these sensors include ultrasonic sensors, cameras, and LIDAR sensors used in existing Advanced Driver Assistance Systems (ADAS).

[0003] A vehicle's control system tracks the object states (the object states include kinematic states) of other vehicles based on automotive radar measurements in order to control the vehicle. Extended Object Tracking (EOT) with multiple measurements per scan has been shown to improve object tracking over traditional point object tracking that includes only one measurement per scan by augmenting the object state from only kinematic states to both kinematic and extended states. The extended state provides the dimensions and orientation of the object being tracked. To achieve this, it is necessary to capture the spatial distribution (i.e., how the automotive radar measurements are spatially distributed around the object) along with the sensor noise. Current methods include a framework of fixed point sets on a rigid body, which requires non-scalable data association between the fixed point sets and automotive radar detections even in the case of single object tracking. Spatial models such as contour and surface models avoid the cumbersome data association step.

[0004] In automotive radar measurements, contour models reflect the measurement distribution along the contour of an object (e.g., a rigid body), while surface models assume that radar measurements are generated from the interior surface of a two-dimensional shape. Examples of contour models include simple rectangular shapes and more general star-shaped convex shapes modeled by random hypersurface models or Gaussian process models. Some surface models, such as Gaussian-based ellipse models and hierarchical Gaussian-based ellipse models, are computationally much easier to describe more complex shapes than contour models, which require many more degrees of freedom. However, object measurements are subject to noise and reflections are received only from the object's surface. Thus, the above models do not capture real-world automotive radar measurements.

[0005] Therefore, there is a need for a system and method for tracking both the kinematic and extensional states of an object by capturing real-world automotive radar measurements. Summary of the Invention

[0006] It is an object of some embodiments to provide a system and method for tracking the expansion state of an object. The expansion state of an object includes a kinematic state indicating one or a combination of the position and velocity of the center of the object, and an expansion state indicating one or a combination of the dimensions and orientation of the object. The center of the object is one or a combination of an arbitrarily selected point, the geometric center of the object, the center of gravity of the object, the center of the rear axle of the wheels of a vehicle, etc. To track an object (such as a vehicle), a sensor is used, for example, an automotive radar. In one embodiment, the automotive radar can provide a direct measurement of the radial velocity, a long operating range, a small size in the millimeter or sub-terahertz frequency band, and a high spatial resolution.

[0007] In point object tracking, one measurement is received from the vehicle per scan. Point object tracking provides only the kinematic state (position) of the vehicle. To further track the vehicle, a probabilistic filter with a measurement model having a distribution of the kinematic state is utilized. In extended object tracking (EOT), multiple measurements are received per scan. The multiple measurements are spatially structured around the vehicle. Extended object tracking provides both the kinematic and extended state states of the vehicle. To track the vehicle, a probabilistic filter with a measurement model having a distribution of the extended state is utilized.

[0008] However, real-world automotive radar measurement distributions show that the multiple reflections from the vehicle are complex. This complexity makes it complicated to design a suitable measurement model. Therefore, the usual measurement model can only be applied to the kinematic state, but not to the extended state.

[0009] To this end, in some embodiments, spatial models such as contour and surface models are used to capture real-world automotive radar measurements. In particular, in one embodiment, a composite measurement model (a type of surface volume model) is determined based on the principles of contour and surface models. The composite measurement model includes multiple probability distributions that are constrained to lie on the object's contour by a predetermined relative geometric mapping to the center of the object. The multiple probability distributions are used to cover the spread of measurements along the object's contour.

[0010] A composite measurement model is composite in multiple senses. For example, a composite measurement model has a composite structure, i.e., multiple probability distributions. Also, a composite measurement model has a composite composition, i.e., a function of multiple probability distributions, a function of contours, and their relationships. Furthermore, a composite measurement model has a composite nature, i.e., the multiple probability distributions are based on measurements and therefore represent a data-driven approach to model generation, while the contours are based on modeling the shape of an object, e.g., the shape of a vehicle, using physics-based modeling principles.

[0011] Furthermore, the composite measurement model leverages various extended state modeling principles; that is, the composite measurement model combines the principles of the contour model and the surface model. As a result, the composite measurement model better represents the physics of object tracking while simplifying measurement assignment. Also, the multiple probability distributions of the composite measurement model are more flexible than the single distribution of the surface model, which can better describe the contours of an object and more flexibly describe measurements from different angles or views of the object.

[0012] The composite measurement model is trained offline, i.e., in advance. The composite measurement model may be trained in a unit coordinate system or a global coordinate system. Some embodiments are based on the recognition that training the composite measurement model in a unit coordinate system is beneficial as it simplifies the calculations and makes the composite measurement model independent of the object dimensions. Each of a plurality of probability distributions (represented as ellipses) can be assigned measurements in a probabilistic manner. Measurements associated with an ellipse may be referred to as ellipse-assigned measurements.

[0013] According to some embodiments, the composite measurement model learned offline is used for online tracking of the object's expanded state, i.e., real-time tracking of the object's expanded state. However, there may be a mismatch in terms of automotive radar specifications between the on-board automotive radar used by the vehicle to acquire measurements and that used for offline data collection (offline data collection (also referred to as "offline training data") is used to train the composite measurement model).

[0014] Furthermore, the offline training data contains coarse vehicle labels. Therefore, training the composite measurement model using only the offline training data may lead to over-smoothing of the offline-learned composite measurement model, which is averaged across different vehicle models. For example, a coarsely labeled dataset may contain sedans and SUVs in the same class. Therefore, it may happen that the composite measurement model cannot accurately classify between different types of objects (in this case, vehicles such as trucks, cars, and tractors).

[0015] To that end, the present disclosure proposes online adaptation of a composite measurement model that refines the offline learned composite measurement model and further improves online state estimation performance using a more customized composite measurement model that fits the on-board automotive radar measurements.

[0016] According to some embodiments, the offline learned composite measurement model is run for a predetermined period of time and updated zonal states of the object, predicted zonal states of the object, and measurements performed by the automotive radar of the object are accumulated to form an online batch of training data.

[0017] Some embodiments are based on the recognition that the online batch of training data includes data accumulated only within a predetermined period of time, which may be a few seconds or minutes. Therefore, the online batch of training data includes much less training data compared to the training data used to train the composite measurement model offline. In order to improve the accuracy of the composite measurement model trained using the online batch of training data, it is important to obtain the relationship between data in the accumulated data and use the relationship to update the parameters of the composite measurement model.

[0018] To this end, the accumulated update beliefs are smoothed using the covariance between the accumulated update beliefs and the predicted beliefs, as well as backward and forward recursion. The smoothed update beliefs are used to generate online batches of training data. In the backward recursion, the accumulated update beliefs are smoothed backwards from a particular time within a predetermined time period based on measurements at that particular time. Alternatively, in the forward recursion, the accumulated update beliefs are smoothed forwards from a particular time within a predetermined time period based on measurements at that particular time.

[0019] In some embodiments, Bayesian smoothing customized to the offline trained composite measurement model is applied to the measurements to obtain the smoothed state.

[0020] Further, the online batch of training data includes state-separated measurements. To state-separate the online batch of training data, the measurements in the global coordinate system are transformed to a unit coordinate system that is positioned at the center of the object and oriented such that the x-axis of the unit coordinate system points to the object front surface using the azimuth angle and the object center. Finally, the measurements transformed to the unit coordinate system are normalized by the range states, i.e., length and width.

[0021] The online batches of state-separated training data are then used for online learning of a composite measurement model, which updates the composite measurement model by updating one or more parameters of the offline-trained composite measurement model, where the parameters of the composite measurement model include the number of probability distributions in the composite measurement model, a control point that determines the center of the probability distributions, and a covariance of each probability distribution.

[0022] However, one or more parameters of the composite measurement model are updated such that a predetermined relative geometric mapping of the multiple probability distributions to the center of the tracked object is maintained. To that end, while updating the parameters of the composite measurement model, control points corresponding to the multiple probability distributions are maintained, and a penalty function, such as a log-likelihood function, that implements a maximum allowable change to the control points is used to maintain the control points of the multiple probability distributions.

[0023] Accordingly, one embodiment discloses a tracking system for tracking an expansion state of an object, the expansion state including a kinematic state indicative of a combination of a position and velocity of a center of the object, and an extension state indicative of a combination of a size and an orientation of the object, the tracking system comprising at least one processor and a memory having instructions stored thereon, the instructions, when executed by the at least one processor, causing the tracking system to receive measurements associated with at least one sensor, the at least one sensor configured to explore a scene including the object by one or more signal transmissions to generate one or more measurements of the object per transmission, the instructions, when executed by the at least one processor, further causing the tracking system to execute a probabilistic filter that iteratively tracks beliefs regarding the expansion state of the object, the beliefs being predicted using a kinematic model of the object and updated using a composite measurement model of the object, the composite measurement model including a plurality of probability distributions constrained to lie around a contour of the object by a predetermined relative geometric mapping to the center of the object, the iterative In each iteration of tracking, the beliefs regarding the expansion state are updated based on a difference between predicted beliefs and updated beliefs, the updated beliefs being estimated based on a probability of the measurements taken within the predetermined period fitting each of the multiple probability distributions and mapped to the expansion state of the object based on the corresponding geometric mapping, and the composite measurement model is pre-trained offline using offline training data, and the instructions, when executed by the at least one processor, further cause the tracking system to accumulate updated beliefs, predicted beliefs and measurements over the predetermined period to generate online batches of training data including state separation measurements, update the composite measurement model by updating parameters of the composite measurement model based on the online batches of training data, and track the expansion state of the object based on the updated composite measurement model.

[0024] Accordingly, another embodiment discloses a tracking method for tracking an expansion state of an object, the expansion state including a kinematic state indicative of one or a combination of a position and a velocity of a center of the object, and an extension state indicative of one or a combination of a size and an orientation of the object, the tracking method including receiving measurements associated with at least one sensor, the at least one sensor configured to explore a scene including the object by one or more signal transmissions to generate one or more measurements of the object per transmission, the tracking method further including executing a probabilistic filter to iteratively track a belief regarding the expansion state of the object, the belief being predicted using a kinematic model of the object and updated using a composite measurement model of the object, the composite measurement model including a plurality of probability distributions constrained to lie around a contour of the object by a predetermined relative geometric mapping to the center of the object, and at each iteration of the iterative tracking, the belief regarding the expansion state is a function of the predicted belief and the updated belief. and updating the composite measurement model based on a difference between the measurements taken within the predetermined time period, the updated beliefs being estimated based on a probability of the measurements taken within the predetermined time period fitting each of the plurality of probability distributions and mapped to the expanded state of the object based on the corresponding geometric mapping, the composite measurement model being pre-trained offline using offline training data, and the tracking method further includes accumulating updated beliefs, predicted beliefs and measurements over the predetermined time period to generate online batches of training data including state separation measurements, updating the composite measurement model by updating parameters of the composite measurement model based on the online batches of training data, and tracking the expanded state of the object based on the updated composite measurement model.

[0025] A non-transitory computer-readable storage medium having embodied thereon a program executable by a processor to execute a method for tracking an expansion state of an object, the expansion state including a kinematic state indicative of one or a combination of a position and a velocity of a center of the object, and an extension state indicative of one or a combination of a size and an orientation of the object, the method including receiving measurements associated with at least one sensor configured to explore a scene including the object by one or more signal transmissions to generate one or more measurements of the object per transmission, the method further including executing a probabilistic filter to iteratively track a belief regarding the expansion state of the object, the belief being predicted using a kinematic model of the object and updated using a composite measurement model of the object, the composite measurement model including a plurality of probability distributions constrained to lie around a contour of the object by a predetermined relative geometric mapping to the center of the object, and at each iteration of the iterative tracking, the belief regarding the expansion state is updated based on a correlation between the predicted belief and the updated belief. and the updated beliefs are estimated based on a probability of the measurements taken within the predetermined time period fitting each of the plurality of probability distributions and mapped to the expanded state of the object based on the corresponding geometric mapping, and the composite measurement model is pre-trained offline using offline training data, and the method further includes accumulating update beliefs, prediction beliefs and measurements over the predetermined time period to generate online batches of training data including state separation measurements, updating the composite measurement model by updating parameters of the composite measurement model based on the online batches of training data, and tracking the expanded state of the object based on the updated composite measurement model.

[0026] The presently disclosed embodiments will be further described with reference to the accompanying drawings, in which the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the presently disclosed embodiments. [Brief description of the drawings]

[0027] [Figure 1A] FIG. 1 shows a schematic overview of the principles for tracking the magnification state of an object, according to some embodiments. [Figure 1B] FIG. 1 shows a schematic overview of the principles for tracking the magnification state of an object, according to some embodiments. [Figure 1C] FIG. 1 shows a schematic overview of the principles for tracking the magnification state of an object, according to some embodiments. [Diagram 2] FIG. 1 is a block diagram of a tracking system for tracking the magnification state of an object according to some embodiments. [Figure 3A] 1A-1D illustrate various schematics for tracking beliefs about the expansion state of an object using a composite measurement model, according to some embodiments. [Figure 3B] 1A-1D illustrate various schematics for tracking beliefs about the expansion state of an object using a composite measurement model, according to some embodiments. [Figure 3C] 1A-1D illustrate various schematics for tracking beliefs about the expansion state of an object using a composite measurement model, according to some embodiments. [Figure 4] FIG. 1 illustrates a workflow for tracking the expansion state of an object using a composite measurement model, according to some embodiments. [Diagram 5] FIG. 2 illustrates a flowchart of a method for offline learning of parameters of a composite measurement model according to some embodiments. [Figure 6] FIG. 1 illustrates a flowchart of a method for online learning of parameters of a composite measurement model according to some embodiments. [Figure 7A] FIG. 1 illustrates a schematic diagram of transforming training data collected from different motions of different objects into a common unit coordinate system, according to some embodiments. [Figure 7B]FIG. 1 illustrates a schematic diagram of transforming training data collected from different motions of different objects into a common unit coordinate system, according to some embodiments. [Figure 8] FIG. 2 is a block diagram of an Expectation-Maximization (EM) method for offline learning of parameters of a composite measurement model, according to some embodiments. [Figure 9] FIG. 1 is a block diagram of an EM method for online learning of parameters of a composite measurement model according to some embodiments. [Figure 10A] FIG. 2 illustrates a flowchart of an Unscented Kalman Filter-Probabilistic Multi-Hypothesis Tracking (UKF-PMHT) algorithm, according to some embodiments. [Figure 10B] FIG. 2 is a block diagram of steps performed to calculate predicted measurements and covariance matrices according to some embodiments. [Figure 10C] FIG. 2 is a block diagram of steps performed to calculate composite measurements and a composite covariance matrix according to some embodiments. [Figure 10D] FIG. 1 is a block diagram of steps performed to update the augmentation state and covariance matrix according to some embodiments. [Figure 10E] FIG. 2 is a block diagram of steps performed for online training of a composite measurement model according to some embodiments. [Figure 10F] FIG. 2 is a diagram illustrating a schematic illustrating backward and forward recursion according to some embodiments. [Figure 11A] FIG. 1 shows a schematic diagram of a vehicle including a controller in communication with a system utilizing the principles of some embodiments. [Figure 11B] FIG. 11B illustrates a schematic diagram of the interaction between a controller of the system of FIG. 11A and a vehicle controller, according to some embodiments. [Figure 11C]FIG. 1 shows a schematic diagram of an autonomous or semi-autonomous controlled vehicle in which control inputs are generated using some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure may be practiced without these specific details. In other instances, the devices and methods are shown in block diagram form solely to avoid obscuring the present disclosure.

[0029] As used in the present specification and claims, the words "for example," "for instance," and "such as," as well as the verbs "comprising," "having," "including," and other forms of these verbs, when used in conjunction with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be considered to exclude other additional components or items. The word "based on" means based at least in part on. Furthermore, it should be understood that the phraseology and terminology used herein are for the purpose of description and should not be considered limiting. Any headings used in this description are for convenience only and do not have any legal or restrictive effect.

[0030] 1A, 1B and 1C are diagrams showing a schematic overview of some principles used by some embodiments to track the extended state of an object. The extended state of an object includes a kinematic state indicating one or a combination of the position and velocity of the center of the object, and an extended state indicating one or a combination of the dimensions and orientation of the object. The center of the object is one or a combination of an arbitrarily selected point, the geometric center of the object, the center of gravity of the object, the center of the rear axle of the vehicle's wheels, etc. To track an object (such as a vehicle 106), a sensor 104 (e.g., an automotive radar) is used. In point object tracking 100, one measurement 108 per scan is received from the vehicle 106. The point object tracking 100 provides only the kinematic state (position) of the vehicle 106. Furthermore, a probabilistic filter with a measurement model having a distribution of the kinematic states is utilized to track the vehicle 106. In extended object tracking (EOT) 102, multiple measurements 110 are received per scan. The multiple measurements 110 are spatially structured around the vehicle 106. The EOT 102 provides both the kinematic and extended state states of the vehicle 106. To track the vehicle 106, a probabilistic filter with a measurement model having a distribution of the extended states is utilized.

[0031] However, the real-world automotive radar measurement 112 distribution as shown in FIG. 1B shows that the multiple reflections from the vehicle 106 are complex. This complexity complicates the design of the measurement model. Therefore, the normal measurement model can only be applied to the kinematic state, but not to the extended state.

[0032] To this end, in some embodiments, spatial models such as a contour model 114 and a surface model 116 as shown in FIG. 1C are used to capture the real-world automotive radar measurements 112. However, the spatial models are inaccurate. Some embodiments are based on the recognition that the real-world automotive radar measurements 112 are distributed around the edges of an object (vehicle 106) having a certain volume, resulting in a surface volume model. To this end, some embodiments are based on the objective of formulating a surface volume model 118 that resembles and captures the real-world automotive radar measurements 112. The surface volume model 118 strikes a balance between the contour model 114 and the surface model 116 with more realistic features while keeping the EOT accurate.

[0033] In particular, in one embodiment, a composite measurement model 120 (a type of surface volume model) is determined based on the principles of the contour model 114 and the surface model 116. The composite measurement model 120 includes a number of probability distributions 122 that are geometrically constrained to the contour 124 of the object. In FIG. 1C, the geometric constraint is that the centers of the multiple probability distributions are located on the contour. And the composite measurement model has a predetermined relative geometric mapping to the center of the object. The multiple probability distributions 122 are used to cover the spread of measurements along the contour 124 of the object.

[0034] The composite measurement model 120 is composite in multiple senses. For example, the composite measurement model 120 has a composite structure, i.e., multiple probability distributions 122. Also, the composite measurement model 120 has a composite composition, i.e., functions of the multiple probability distributions 122, functions of the contours 124, and the relationships between them. Furthermore, the composite measurement model 120 has a composite nature, i.e., the multiple probability distributions 122 are based on measurements and therefore represent a data-driven approach to model generation, while the contours 124 are based on modeling the shape of an object, e.g., the shape of a vehicle, using principles of physics-based modeling.

[0035] Furthermore, the composite measurement model 120 leverages various extended state modeling principles; that is, the composite measurement model 120 combines the principles of the contour model 114 and the surface model 116. As a result, the composite measurement model 120 better represents the physics of object tracking while simplifying measurement assignment. Also, the multiple probability distributions 122 of the composite measurement model 120 are more flexible than the single distribution of the surface model 116 and can be configured to better describe the contour 124, as well as flexibly describe measurements from different angles or viewpoints of the object.

[0036] Some embodiments are based on the understanding that, assuming that there are no constraints on the shape of the contour 124, the plurality of probability distributions 122 can theoretically be located on the contour 124. However, in practice, such an assumption is inaccurate and does not help in tracking the enlargement state. In contrast, the contour 124 of the object is predefined, and instead of the contour 124 being fitted to the plurality of probability distributions 122, the plurality of probability distributions 122 are fitted to the contour 124. This allows the physical structure of the object to be reflected during the update phase of the probabilistic filter.

[0037] The composite measurement model 120 is trained offline, i.e., in advance. The composite measurement model 120 may be trained in a unit coordinate system or a global coordinate system. Some embodiments are based on the recognition that training the composite measurement model 120 in a unit coordinate system is beneficial as it simplifies the calculations and makes the composite measurement model 120 independent of the object dimensions. Each of the multiple probability distributions 122 (represented as ellipses) may be assigned measurements in a probabilistic manner. Measurements associated with an ellipse may be referred to as ellipse-assigned measurements.

[0038] Some embodiments are based on the recognition that the object's growth state can be tracked online, i.e., in real-time, using the composite measurement model 120. In particular, various embodiments track the object's growth state using a probabilistic filter that tracks beliefs about the object's growth state, which are predicted using a motion model of the object and updated using the composite measurement model 120 of the object.

[0039] Some embodiments are based on the recognition that there may be a mismatch in terms of radar sensor specifications between the on-board sensors used by the vehicle 106 to obtain measurements and those used for offline data collection (which is also referred to as “offline training data”) used to train the composite measurement model 120).

[0040] Some embodiments are based on the recognition that the offline training data has coarse vehicle labels, which may lead to over-smoothing of the offline learned composite measurement model 120, which is averaged across different vehicle models. For example, a coarsely labeled dataset may contain sedans and SUVs in the same class. 。

[0041] To that end, the present disclosure proposes online adaptation of the composite measurement model 120 (also referred to as “online composite measurement model adaptation”) to improve the offline learned composite measurement model 120 and further enhance online state estimation performance using a more customized composite measurement model 120 that is adapted to onboard automotive radar measurements.

[0042] FIG. 2 is a block diagram of a tracking system 200 for tracking the expansion state of an object using a composite measurement model 120 (shown in the previous figure) according to some embodiments. The object may be a vehicle, such as but not limited to a car, a motorcycle, a bus or a truck. Also, the vehicle may be an autonomous vehicle or a semi-autonomous vehicle. The expansion state includes the kinematic state and the expansion state of the object. The composite measurement model 120 is learned offline using offline training data (FIG. 5). First, the learned composite measurement model 120 is run for a predetermined period T to track the expansion state of the object. Furthermore, the learned composite measurement model 120 is updated / refinement with the corresponding expansion state based on the measurements acquired within the predetermined period T.

[0043] According to some embodiments, the kinematic state corresponds to the motion parameters of the object, such as speed, acceleration, heading and turn rate. In some other embodiments, the kinematic state corresponds to the position of the object having the motion parameters. The tracking system 200 may include a sensor 202 or may be operatively connected to a set of sensors to survey the scene by one or more signal transmissions. Further, the one or more signal transmissions are configured to generate one or more measurements of the object per transmission. According to some embodiments, the sensor 202 may be an automotive radar. In some embodiments, the scene includes a moving object. In some other embodiments, the scene may include one or more objects, including both moving and stationary objects.

[0044] The tracking system 200 may have several interfaces that connect the tracking system 200 with other systems and devices. For example, a network interface controller (NIC) 214 is adapted to connect the tracking system 200 via a bus 212 to a network 216 that connects the tracking system 200 with a set of sensors. Via the network 216, wirelessly or wired, the tracking system 200 receives data of reflections of one or more signal transmissions to generate one or more measurements of the object per transmission. Additionally or alternatively, the tracking system 200 includes an output interface 220 configured to input a control input to a controller 222.

[0045] The tracking system 200 also includes a processor 204 configured to execute stored instructions and a memory 206 storing instructions executable by the processor 204. The processor 204 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 206 may include a Random Access Memory (RAM), a Read Only Memory (ROM), a flash memory, or other suitable memory system. The processor 204 is connected to one or more input and output devices via a bus 212. Furthermore, the tracking system 200 includes a storage device 208 adapted to store different modules including instructions executable by the processor 204. The storage device 208 may be realized using a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof.

[0046] The storage device 208 is configured to store the object motion model 210a, the object composite measurement model 210b (e.g., the composite measurement model 120), and the refinement module 210c. The processor 204 is configured to iteratively execute a probability filter for iteratively tracking the beliefs regarding the object's expansion state over a predetermined period T, where the beliefs are predicted using the object motion model 210a and updated using the object composite measurement model 210b. After the predetermined period T, the refinement module 210c refines / updates the composite measurement model 210b based on the measurements acquired during the predetermined period T and the corresponding predicted beliefs and updated beliefs. The tracking of the beliefs regarding the object's expansion state based on the offline learning of the composite measurement model 120 will be described in detail below with reference to Figures 3A, 3B, and 3C.

[0047]

number

[0048] The predicted expansion state 302 of the object may be referred to as the predicted belief of the expansion state since this prediction is probabilistic. Some embodiments are based on the recognition that the predicted expansion state 302 of the object may be inaccurate to generate predicted measurements of the expansion state because an accurate spatial model of the automobile radar measurements is required. For this purpose, in some embodiments, a composite measurement model 304 in the unit coordinate system that is learned offline is used. In order to align the composite measurement model 304 in the unit coordinate system with the predicted expansion state 302, it is necessary to transform the composite measurement model 304 from the unit coordinate system to the global coordinate system for the predicted expansion state 302. In particular, it is necessary to transform the ellipse-allocated measurements in the unit coordinate system to the global coordinate system.

[0049] Some embodiments are based on the recognition that such a transformation can be realized using an unscented transformation function 308. To that end, in one embodiment, the processor 204 generates a sigma point for the ellipse 306 (i.e., the probability distribution of the composite measurement model 304). "Ellipse" and "probability distribution" may be used interchangeably and will mean the same thing. Furthermore, the sigma point is propagated into the unscented transformation function 308, which is a function of the predicted state 302, so that predicted measurements in the global coordinate system corresponding to the ellipse-assigned measurements of the ellipse 306 in the unit coordinate system are determined. Furthermore, covariances corresponding to these predicted measurements are determined based on the predicted measurements. Similarly, measurements in the global coordinate system corresponding to the ellipse-assigned measurements associated with the remaining ellipses are determined. To that end, a predicted expansion state model 310 is obtained in which the composite measurement model 304 is aligned according to the predicted expansion state 302. Furthermore, a composite measurement is determined for each probability distribution of the predicted expansion state model 310, as described below with reference to FIG. 3B.

[0050] FIG. 3B is a diagram illustrating a schematic for determining a composite measurement for each probability distribution of the predicted expansion state model 310, according to some embodiments. The processor 204 receives the measurement values ​​312 (represented by cross marks) at the current time step. Some embodiments are based on the recognition that the multiple probability distributions 314a-h (ellipses) can be processed independently of each other, e.g., in parallel. Such independent processing allows taking into account different viewing angles for probing the expansion state of the object. To account for such independent processing, some embodiments consider different probability distributions of the multiple probability distributions 314a-h as belonging to different objects. Furthermore, some embodiments are based on the recognition that soft probabilistic assignment, i.e., probabilistic assignment of the measurements 312 to different probability distributions, is more advantageous than hard deterministic assignment. The soft probabilistic assignment can avoid the catastrophic assignment of hard assignment, while keeping the association dimension linear with the number of ellipses and measurements.

[0051] To that end, the processor 204 assigns the measurement 312 to a probability distribution 314 having an association probability. Similarly, the processor 204 assigns the measurement 312 to each of the probability distributions 314a-314h having a corresponding association probability. A measurement having a corresponding association probability associated with each of the plurality of probability distributions 314a-314h is referred to as a "composite measurement."

[0052] Further, for probability distribution 314a, processor 204 determines a composite centroid 316a and a composite covariance matrix defining spread 316b based on the composite measurements associated with probability distribution 314a. Similarly, for probability distribution 314e, processor 204 determines a composite centroid 318a and a composite covariance matrix defining spread 318b based on the composite measurements associated with probability distribution 314e. Similarly, for probability distribution 314h, processor 204 determines a composite centroid 320a and a composite covariance matrix defining spread 320b based on the composite measurements associated with probability distribution 314h. In this manner, a composite centroid and a composite covariance matrix are determined for each probability distribution. Further, the composite measurements associated with each probability distribution are used to update a predictive belief regarding the spread state, as described below with reference to FIG. 3C.

[0053] 3C is a diagram illustrating a schematic for updating predictive beliefs about the expansion state 302, according to some embodiments. The processor 204 updates predictive beliefs about the expansion state 302 using a probabilistic filter, such as a Kalman filter, together with the composite measurements associated with each probability distribution to generate an updated expansion state x of the object. k|k 322. The updated expanded state of the object x k|k 322 may be referred to as an updated belief about the expansion state. Furthermore, the updated belief about the expansion state is used to update the tracked belief. In one embodiment, the tracked belief is updated based on the difference between the predicted belief and the updated belief. Furthermore, the processor 204 tracks the expansion state of the object based on the updated tracked belief about the expansion state.

[0054] The processor 204 is configured to run the composite measurement model 210b only for a predetermined period T to predict beliefs about the extended states 302 and to update the predicted beliefs about the extended states 302. After the predetermined period T, the processor 204 is further configured to accumulate the predicted beliefs, the updated beliefs, and measurements obtained within the predetermined period to create an online batch of state separation training data, and to update the composite measurement model 210b based on the online batch of state separation training data.

[0055] Therefore, the composite measurement model 304 used for tracking the zoomed-in state of the object as described above is first trained offline. The offline training and characteristics of the composite measurement model 304 are described below.

[0056]

number

[0057] Some embodiments are based on the recognition that the probability distribution of the composite measurement model 304 can be represented using a Gaussian distribution to better match the probabilistic filter. For example, in some embodiments, the probability distribution is defined as a Random Matrix Model (RMM) in a probability space (Ω, P, F), where the sample space Ω is a set of matrices. Random matrices are advantageous for representing multi-dimensional probability distributions, and the parameters of the probability distributions represented as RMMs can be shown using elliptical shapes. According to one embodiment, the L random matrix models are defined as follows, taking into account the measurement-ellipse assignments for all L ellipses:

number

[0058]

number

[0059] FIG. 4 illustrates a workflow for tracking an object's zoom state using a composite measurement model 210b, according to some embodiments. FIG. 4 is described below in conjunction with FIG. 2. The proposed tracking system (in FIG. 2) uses a composite measurement model 210b for tracking an object's zoom state. To that end, the composite measurement model 210b is configured to learn 400 offline. The offline learning 400 of the composite measurement model 210b is described below in detail with respect to FIG. 5. Based on the offline learning, the composite measurement model 210b is used by the tracking system to track 402 the object's zoom state online over a predefined period T. To track 402 the object's zoom state online, the tracking system predicts beliefs about the object's zoom state using the motion model 210a and updates the predicted beliefs using the offline-trained composite measurement model 210b. The online tracking 402 of the zoom state is described in detail with respect to FIGS. 10A-10D.

[0060] After a predetermined period T, the prediction beliefs, the updated prediction beliefs, and the measurements within the period T are accumulated to form an online batch of training data including state separation measurements. This online batch of training data is then used for online learning 404 of the composite measurement model 210b, which updates / refines the composite measurement model 210b by updating one or more parameters of the composite measurement model 210b. The parameters of the composite measurement model 210b include the number of measurements for each probability distribution of the multiple probability distributions configured by the composite measurement model 210b, the control points corresponding to the multiple probability distributions, and the covariance between the multiple probability distributions. The online learning 404 of the composite measurement model 210b is described in more detail below with respect to FIG. 9.

[0061] 5 illustrates a flowchart of a method for offline learning of parameters of a composite measurement model 304, according to some embodiments. The composite measurement model 304 is learned offline using offline training data. At step 500, the method includes accepting training data that includes different measurements of different motions of different objects. At step 502, the method includes transforming the training data into a common coordinate system.

[0062] Some embodiments are based on the recognition that the parameters of the composite measurement model 304 can be learned offline based on the training data and knowledge of the contours of the tracked object using various statistical techniques such as Expectation Maximization (EM). To that end, in step 504, the method comprises learning 504 the parameters of the composite measurement model from the training data using a statistical technique such as EM.

[0063] Some embodiments are based on the recognition that the offline training data used for offline learning by the composite measurement model 304 contains coarse vehicle labels, which may lead to over-smoothing of the offline trained composite measurement model 304, which is averaged across different vehicle models. For example, a coarsely labeled dataset may contain sedans and SUVs in the same class. 。

[0064] Therefore, the present disclosure proposes online adaptation of the composite measurement model 304 (also referred to as "online composite measurement model adaptation"), which improves the offline learned composite measurement model 304 and also improves the online state estimation performance (i.e., real-time tracking of the augmented state of the object). Improving the composite measurement model 304 includes updating parameters of the composite measurement model 304.

[0065] 6 is a diagram illustrating a flowchart of a method for online learning of parameters of the composite measurement model 304, according to some embodiments. The composite measurement model 304 learned offline as shown in FIG. 5 is run for a predetermined period of time to update predictive beliefs regarding the object's growth state, and track the object's growth state based on the updated predictive beliefs. After the predetermined period of time, the method for learning parameters of the composite measurement model 304 includes accumulating predictive beliefs, updated beliefs, and measurements within the predetermined period of time in step 600.

[0066] In step 602, an online batch of state-separated training data is generated based on the accumulated prediction beliefs, the update beliefs, and the measurements within a predetermined time period. Some embodiments are based on the recognition that the composite measurement model 304 can be improved to better track the expansion state of the object by updating the parameters of the composite measurement model 304. The parameters of the composite measurement model 304 are updated based on the online batch of training data and knowledge of the contour of the object to be tracked using various statistical techniques such as EM methods. Thus, in step 604, the method for online learning of the composite measurement model 304 includes the step of improving the composite measurement model 304 by updating the parameters of the composite measurement model 304.

[0067] 7A and 7B are diagrams illustrating an overview of converting training data collected from different motions of different objects into a common unit coordinate system, according to some embodiments. Different measurements collected from tracking different trajectories 700 and 702 are converted into respective object-centered (OC) coordinate systems 704 and 706. The converted measurements are then aggregated (708). In some implementations, measurements are collected for motions of similar types of objects, e.g., from motions of a similar class of vehicles. For example, an embodiment converts measurements from each time step from global coordinates (GC) to object-centered (OC) coordinates for each trajectory, and aggregates OC measurements from all trajectories of vehicles (e.g., sedans) having similar sizes.

[0068] Next, as shown in Figure 7B, embodiments convert the aggregated OC (708) measurements to a Unit Coordinate (UC) system 710. In some implementations, the conversion to the UC system is performed by various normalization techniques that allow the converted training data to be used for machine learning. Furthermore, the measurements in the Unit Coordinate system 710 are used as training data for learning the parameters of the composite measurement model 304.

[0069]

number

[0070]

number

[0071]

number

[0072] The maximization step 806 calculates the model parameters θ={p j ,Σ l} is intended to update

number

[0073]

number

[0074] Furthermore, p and Σ are iterated until the convergence criterion 808 is achieved. l Iterations are performed between estimates of . The convergence criterion 808 can be a predetermined likelihood in (8), a relative change in the estimated parameters over successive iterations being less than a predefined value, or a predetermined maximum number of iterations.

[0075] According to some embodiments, the composite measurement model learned offline is used for online tracking of the object's expanded state, i.e., for real-time tracking of the object's expanded state. Some embodiments are based on the recognition that the probabilistic nature of the composite measurement model can be beneficially matched with the Probabilistic Multi-Hypothesis Tracking (PMHT) method. For example, such matching allows for realizing the probabilistic filter using at least a variant of the Kalman filter. For example, one embodiment uses the Unscented Kalman Filter-Probabilistic Multi-Hypothesis Tracking (UKF-PMHT) method. The Unscented Kalman Filter (UKF) is used to transform the composite measurement model from a unit coordinate system to a global coordinate system. The Probabilistic Multi-Hypothesis Tracking (PMHT) method is then applied to assign the measurements at the current time step to different ellipse components in a probabilistic manner to update the object's expanded state.

[0076]

number

[0077]

number

[0078]

number

[0079]

number

[0080] Furthermore, the enlarged state and measurement value C xz The covariance between is calculated during the UT procedure (10) and the filter gain is calculated as follows:

number

[0081] According to some embodiments, the offline learned composite measurement model 304 is refined using a refinement module 210c, which updates or refines the composite measurement model 304 online (i.e., in real time). To achieve online adaptation of the composite measurement model 304, the expansion state of the object tracked online by the composite measurement model 304 within the T time steps is smoothed to remove noise. To that end, at least one of a backward recursion and a forward recursion is performed to smooth the online composite measurement by filtering the expansion state for all observed measurements within the T time steps. In some embodiments, the smoothing is performed by applying a Bayesian state smoothing technique. In a preferred embodiment, an unscented Rauch-Tung-Striebel (RTS) smoother is applied to calculate the smoother gain, smoothed mean value, and smoothed covariance matrix at each time step k of the T time steps by recursively calculating the posterior state of the expansion state conditional on all observed measurements from the filtered expansion state estimates at the last time step backwards.

[0082] The smoothed augmented state is then used to transform all observed measurements within T time steps in the global coordinate system into a batch of state-separated training data in the unit coordinate system, which is used to improve the composite measurement model 304 by updating its parameters.

[0083] The online learning of the parameters (θ) of the composite measurement model 304 updates the parameters of the offline learned composite measurement model 304 after a predetermined period T. In this way, the online learning improves the offline learned composite measurement model 304. Therefore, the online learning is also referred to as online adaptation of the composite measurement model 304. Since the online batch of training data is state-separated and depends only on the composite measurement model 304, a statistical algorithm such as an EM algorithm can be used to update the model parameters within a regularization range for the distance to the parameters of the offline learned composite measurement model 304. In this way, the parameters of the offline learned composite measurement model (also referred to as a “pre-trained composite measurement model”) are updated based on the regularized distance between the parameters of the updated composite measurement model and the parameters of the offline learned composite measurement model.

[0084] However, the parameters of the offline learned composite measurement model are updated such that the predetermined relative geometric mapping of the multiple probability distributions to the object centroids is preserved. To that end, while updating the parameters of the offline learned composite measurement model, control points corresponding to the multiple probability distributions are preserved, and a penalty function, such as a log-likelihood function, that enforces the maximum allowable change to the control points is used to preserve the control points of the multiple probability distributions.

[0085] For online adaptation of the composite measurement model 304, update state x k|k and the predicted state x k|k-1 and past and future measurements Z (for time k within a predefined time period T) k The online adaptation improves tracking of the object's augmentation states by generating online batches of training data to update the parameters of the composite measurement model 304, where the online batches of training data include state separation measurements.

[0086] The measurements taken during T time steps are used to smooth the updated expansion state of the object to generate an online batch of training data including state separation measurements. An offline learned composite measurement model 304 enables tracking of the expansion state of the object at each time step k, where the expansion state includes kinematic state elements (i.e., the first five elements in equation (19)) and range state elements in terms of length and width in equation (19).

number

[0087]

number

[0088]

number

[0089] Furthermore, to smooth the updated augmentation state, we use the predicted augmentation state x k+1|k and the updated zoom state x k|k The cross-covariance matrix between

number

[0090]

number

[0091]

number

[0092]

number

[0093]

number

[0094]

number

[0095] FIG. 9 is a block diagram of an EM method for online learning of parameters of a composite measurement model 304, according to some embodiments. s ) 600 is input to the EM method for online learning. The EM method includes an expectation step 902 and a maximization step 904.

[0096]

number

[0097]

number

[0098]

number

[0099]

number

[0100]

number

[0101]

number

[0102] 10A illustrates a flow chart of a UKF-PMHT tracking algorithm, according to some embodiments. The UKF-PMHT tracking algorithm is executed by the processor 204. The UKF-PMHT tracking algorithm includes two stages: a prediction stage 1000 and an update stage 1004. In the prediction stage 1000, a motion model is used to predict beliefs about the object's augmented state and the corresponding covariance matrix. Further, in block 1002, an iteration index n=1 and an update stage x l,1 =x k|k-1 and C l,1 =C k|k-1 The iterations of the update step 1004 begin by setting

[0103]

number

[0104] Returning to Figure 10A, at block 1008, composite measurements and a composite covariance matrix are calculated taking into account measurements 1014 at time k. Figure 10C is a block diagram of steps performed to calculate the composite measurements and a composite covariance matrix, according to some embodiments. At block 1028, for the first ellipse, measurement-ellipse association weights are calculated according to equation (16). At block 1030, for the first ellipse, composite measurements and a composite covariance matrix are calculated using equations (17) and (18), respectively.

[0105] Returning to FIG. 10A, in block 1010, the expansion state x l,n and the covariance matrix C l,n where the subscript l is the ellipse index. FIG. 10D shows the expansion state x for the first ellipse (i.e., l=1) according to some embodiments. l,n and the covariance matrix C l,n 10 is a block diagram of the steps performed to update H. At block 1032, the cross-covariance matrix is ​​calculated as follows:

number

[0106] The Kalman filter gain is calculated at block 1034. The Kalman filter gain is expressed as follows:

number

[0107] At block 1036, the expansion state x l,n and the covariance matrix C l,n is updated as follows:

number

[0108]

number

[0109] Once the convergence criterion is achieved, i.e., after a predefined period T, the updated Expanded state x k|k , together with the corresponding measurements taken within the time period T, are used for online training 1020 of the composite measurement model. The online training 1020 performs the offline training of the composite measurement model by updating the parameters of the composite measurement model. Composite Measurement Model 1012 Improve.

[0110] FIG. 10E is a block diagram of steps performed for online learning 1020 of a composite measurement model, according to some embodiments. In step 1038, the updated expanded state is smoothed using measurements acquired within a predetermined number of T time steps using equations (24) and (25) and the covariance (23) between the updated expanded state and the predicted expanded state accumulated within a predetermined number of T time steps. The updated expanded state is smoothed using measurements acquired within a predetermined number of T time steps by using at least one of backward recursion and forward recursion. To this end, some embodiments use a Bayesian smoothing-based RTS smoother. Furthermore, in step 1040, an online batch of training data including state separation measurements is generated based on the smoothed expanded state using equation (26). Finally, the online batch of training data is used to improve the offline-trained composite measurement model by updating parameters using the EM method (FIG. 9).

[0111] FIG. 10F is a schematic diagram illustrating backward recursion 1044 and forward recursion 1046 according to some embodiments. FIG. 10F illustrates accumulated data 1042, which includes update beliefs, prediction beliefs, and measurements taken within a predefined period T seconds. The accumulated data 1042 is used to generate an online batch of training data, which is used to update a composite measurement model, and the magnification state of the object is tracked using the updated composite assumption model. The online batch of training data includes data accumulated only within a predefined period T seconds, where T can be several seconds or minutes. Therefore, the online batch of training data includes much less training data compared to the training data used to train the composite measurement model offline. In order to improve the accuracy of the composite measurement model trained using the online batch of training data, it is important to obtain the relationship between data in the accumulated data 1042 and use the relationship to update the parameters of the composite measurement model.

[0112] To that end, the accumulated updated beliefs are smoothed using backward recursion 1044 and forward recursion 1046 to generate online batches of training data. In the backward recursion 1044, the accumulated updated beliefs are smoothed backwards from a specific time t seconds (e.g., the 5th second) within a predetermined period T seconds (e.g., 10 seconds) based on measurements at the specific time t seconds. Alternatively, in the forward recursion 1046, the accumulated updated beliefs are smoothed forwards from a specific time t seconds (e.g., the 5th second) within a predetermined period T seconds (e.g., 10 seconds).

[0113] 11A is a schematic diagram of a vehicle 1100 including a controller 1102 in communication with a tracking system 200 utilizing the principles of some embodiments. The vehicle 1100 may be any type of wheeled vehicle, such as a car, a bus, or a rover. The vehicle 1100 may also be an autonomous or semi-autonomous vehicle. For example, some embodiments control the motion of the vehicle 1100. An example of the motion includes the lateral motion of the vehicle, which is controlled by a steering system 1104 of the vehicle 1100. In one embodiment, the steering system 1104 is controlled by the controller 1102. Additionally or alternatively, the steering system 1104 may be controlled by a driver of the vehicle 1100.

[0114] In some embodiments, the vehicle 1100 may include an engine 1110 that is controllable by the controller 1102 or other components of the vehicle 1100. In some embodiments, the vehicle may include an electric motor instead of the engine 1110, controllable by the controller 1102 or other components of the vehicle 1100. The vehicle 1100 may also include one or more sensors 1106 for sensing the surrounding environment. Examples of the sensors 1106 include a distance range finder, such as a radar. In some embodiments, the vehicle 1100 includes one or more sensors 1108 for sensing its current motion parameters and internal conditions. Examples of the one or more sensors 1108 include a Global Positioning System (GPS), an accelerometer, an inertial measurement unit, a gyroscope, a shaft rotation sensor, a torque sensor, a deflection sensor, a pressure sensor, and a flow sensor. These sensors provide information to the controller 1102. The vehicle may include a transceiver 1112 that enables the controller 1102's ability to communicate via wired or wireless communication channels with the tracking system 200 of some embodiments. For example, the controller 1102 receives control input from the tracking system 200 via the transceiver 1112 .

[0115] FIG. 11B is a diagram illustrating an overview of the interaction between the controller 1102 and the controller 1114 of the vehicle 1100, according to some embodiments. For example, in some embodiments, the controller 1114 of the vehicle 1100 is a steering control 1116 and a brake / throttle controller 1118 that control the rotation and acceleration of the vehicle 1100. In such a case, the controller 1102 outputs control commands to the controllers 1116 and 1118 based on the control inputs to control the kinematic state of the vehicle. In some embodiments, the controller 1114 also includes a higher level controller, e.g., a lane keeping assist controller 1120, that further processes the control commands of the controller 1102. In either case, the controller 1114 utilizes the output, i.e., the control commands, of the controller 1102 to control at least one actuator of the vehicle 1100, such as the steering wheel and / or brakes of the vehicle 1100, in order to control the motion of the vehicle 1100.

[0116] 11C is a schematic diagram of an autonomous or semi-autonomous controlled vehicle 1122 for which control inputs are generated using some embodiments. The controlled vehicle 1122 may include a tracking system 200. In some embodiments, the expansion state of each obstacle 1124 is tracked by the controlled vehicle 1122, and then the control inputs are generated based on the tracked expansion state of these obstacles. In some embodiments, the control inputs include commands specifying values ​​for one or a combination of the steering angle of the vehicle's wheels and the rotational speed of the wheels, and the measurements include values ​​for one or a combination of the vehicle's turning rate and the vehicle's acceleration.

[0117] The generated control inputs are intended to keep the controlled vehicle 1122 within a certain range of the road 1126 and to avoid obstacles 1124 to other uncontrolled vehicles, i.e., the controlled vehicle 1122. For example, based on the control inputs, the autonomous or semi-autonomous controlled vehicle 1122 may overtake another vehicle, e.g., on the left or right side, or may instead stay behind another vehicle in the current lane of the road 1126.

[0118] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments provides those skilled in the art with an enabling description for implementing one or more exemplary embodiments. It is intended that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter, as set forth in the appended claims.

[0119] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will appreciate that the embodiments may be practiced without these specific details. For example, systems, processes and other elements of the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments. Additionally, like reference numbers and names in the various drawings refer to like elements.

[0120] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of these operations may be performed in parallel or simultaneously. Also, the order of these operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any process specifically described are performed in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, or the like. When a process corresponds to a function, the end of the function may correspond to a return of the function to a calling function or to a main function.

[0121] Furthermore, embodiments of the disclosed subject matter may be implemented at least partially manually or automatically. The manual or automatic implementation may be performed or at least assisted by the use of machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor may perform the necessary tasks.

[0122] The various methods or processes outlined herein may be coded as software executable on one or more processors utilizing any one of a variety of operating systems or platforms. Further, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0123] The embodiments of the present disclosure may be embodied as a method, an example of which is provided. The operations performed as part of this method may be ordered in any suitable manner. Thus, embodiments may be constructed in which operations are performed in an order different from that shown, which may include performing some operations simultaneously even though they are shown as sequential operations in the example embodiment.

[0124] Although the disclosure has been described with reference to certain preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the disclosure. It is therefore the aspect of the appended claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. 1. A tracking system for tracking an extension state of an object, the extension state including a kinematic state indicative of one or a combination of a position and a velocity of a center of the object, and an extension state indicative of one or a combination of a size and an orientation of the object, the tracking system comprising: At least one processor; and a memory having instructions stored thereon, the instructions, when executed by the at least one processor, causing the tracking system to: and receiving measurements associated with at least one sensor over a predetermined period of time, the at least one sensor configured to explore a scene including the object via one or more signal transmissions, the one or more signal transmissions configured to generate one or more measurements of the object per transmission, the instructions when executed by the at least one processor further causing the tracking system to: and executing a probabilistic filter that iteratively tracks beliefs regarding the expansion state of the object, the beliefs being predicted using a motion model of the object and updated using a composite measurement model of the object, the composite measurement model comprising a plurality of probability distributions constrained to lie around a contour of the object by a predetermined relative geometric mapping to the center of the object, at each iteration of the iterative tracking, the beliefs regarding the expansion state are updated based on a difference between predicted beliefs and updated beliefs, the updated beliefs being estimated based on a probability of the measurements taken within the predetermined time period fitting each of the plurality of probability distributions and mapped to the expansion state of the object based on the corresponding geometric mapping, the composite measurement model being pre-trained offline using offline training data, and the instructions when executed by the at least one processor further cause the tracking system to: accumulating the update beliefs, the prediction beliefs, and the measurements over the predetermined time period to generate an on-line batch of training data including state separation measurements; updating the composite measurement model by updating parameters of the composite measurement model based on the online batches of training data; and tracking the expansion state of the object based on the updated composite measurement model. and to generate the online batches of training data, the processor is further configured to smooth the accumulated updated beliefs using a covariance between the updated beliefs and the predicted beliefs, and at least one of a backward recursion and a forward recursion.

2. 2. The tracking system of claim 1, wherein the composite measurement model is updated by updating parameters of the composite measurement model, the parameters of the composite measurement model including a number of probability distributions, control points corresponding to the plurality of probability distributions, and covariances between the plurality of probability distributions.

3. The tracking system of claim 1 , wherein the backward recursion is performed to smooth the accumulated updated beliefs backwards from a particular time based on measurements at the particular time within the predetermined period.

4. The tracking system of claim 1 , wherein the forward recursion is performed to smooth the accumulated updated beliefs forward from a particular time based on measurements at the particular time within the predetermined period.

5. The tracking system of claim 1 , wherein the accumulated updated beliefs are smoothed using a Bayes filter with an unscented smoother.

6. 2. The tracking system of claim 1, wherein the parameters of the pre-trained composite measurement model are updated based on a regularized distance between the parameters of the updated composite measurement model and the parameters of the pre-trained composite measurement model.

7. 7. The tracking system of claim 6, wherein the parameters of the pre-trained composite measurement model are updated such that the predetermined relative geometric mapping of the plurality of probability distributions to the centroid of the object is maintained.

8. The tracking system of claim 7 , wherein the processor is further configured to maintain control points corresponding to the plurality of probability distributions to maintain the geometric mapping of the plurality of probability distributions to the center of the object.

9. 2. The tracking system of claim 1, wherein the processor is further configured to transform the updated composite measurement model from a global coordinate system to a unit coordinate system to generate the online batches of training data including the state separation measurements.

10. The tracking system of claim 1 , wherein the processor is further configured to assign the measurements to different ones of the plurality of probability distributions by considering the different probability distributions as belonging to different objects.

11. 11. The tracking system of claim 10, wherein the processor is further configured to assign the measurements to the different probability distributions using Probabilistic Multiple-Hypothesis Tracking (PMHT) to perform the assignment of the measurements to the different probability distributions for being considered as the different objects.

12. The tracking system of claim 1 , wherein the parameters of the pre-trained composite measurement model are updated using an Expectation-Maximization (EM) method.

13. The tracking system of claim 1 , wherein one or more parameters of each probability distribution are represented by a Random Matrix Model (RMM) in a two-dimensional (2D) probability space.

14. The tracking system of claim 1 , wherein the contour of the object corresponds to a B-spline curve.

15. the processor is configured to determine a control input to a controller of the vehicle based on the augmented state of the tracked object using the updated composite measurement model, and control the vehicle according to the control input; The tracking system of claim 1 , wherein the vehicle is operatively connected to the tracking system of claim 1 .

16. 1. A tracking method for tracking an extension state of an object, the extension state including a kinematic state indicative of one or a combination of a position and a velocity of a center of the object, and an extension state indicative of one or a combination of a size and an orientation of the object, the tracking method comprising: receiving measurements associated with at least one sensor over a predetermined period of time, the at least one sensor configured to explore a scene including the object via one or more signal transmissions, the one or more signal transmissions configured to generate one or more measurements of the object per transmission, the tracking method further comprising: the step of executing a probabilistic filter that iteratively tracks beliefs regarding the expansion state of the object, the beliefs being predicted using a motion model of the object and updated using a composite measurement model of the object, the composite measurement model comprising a plurality of probability distributions constrained to lie around the contour of the object by a predetermined relative geometric mapping to the center of the object, in each iteration of the iterative tracking the beliefs regarding the expansion state are updated based on a difference between predicted beliefs and updated beliefs, the updated beliefs being estimated based on a probability of the measurements taken within the predetermined time period fitting each of the plurality of probability distributions and mapped to the expansion state of the object based on the corresponding geometric mapping, the composite measurement model being pre-trained offline using offline training data, the tracking method further comprising: accumulating update beliefs, prediction beliefs and measurements over said predetermined time period to generate on-line batches of training data including state separation measurements; updating the composite measurement model by updating parameters of the composite measurement model based on online batches of the training data; tracking the expansion state of the object based on the updated composite measurement model; and smoothing the accumulated updated beliefs using a covariance between the updated beliefs and the predicted beliefs, and at least one of a backward recursion and a forward recursion to generate an online batch of the training data.

17. 17. The tracking method of claim 16, wherein the composite measurement model is updated by updating parameters of the composite measurement model, the parameters of the composite measurement model including a number of probability distributions, control points corresponding to the plurality of probability distributions, and covariances between the plurality of probability distributions.

18. 1. A non-transitory computer readable storage medium having embodied thereon a program executable by a processor to execute a method for tracking an expansion state of an object, the expansion state including a kinematic state indicative of one or a combination of a position and a velocity of a center of the object, and an extension state indicative of one or a combination of a size and an orientation of the object, the method comprising: receiving measurements associated with at least one sensor over a predetermined period of time, the at least one sensor configured to survey a scene including the object via one or more signal transmissions, the one or more signal transmissions configured to generate one or more measurements of the object per transmission, the method further comprising: the method further comprising: executing a probabilistic filter that iteratively tracks beliefs regarding the expansion state of the object, the beliefs being predicted using a motion model of the object and updated using a composite measurement model of the object, the composite measurement model comprising a plurality of probability distributions constrained to lie around a contour of the object by a predetermined relative geometric mapping to the center of the object; at each iteration of the iterative tracking, the beliefs regarding the expansion state are updated based on a difference between predicted beliefs and updated beliefs, the updated beliefs being estimated based on a probability of the measurements taken within the predetermined time period fitting each of the plurality of probability distributions and mapped to the expansion state of the object based on the corresponding geometric mapping; the composite measurement model is pre-trained offline using offline training data; the method further comprises: accumulating update beliefs, prediction beliefs and measurements over said predetermined time period to generate on-line batches of training data including state separation measurements; updating the composite measurement model by updating parameters of the composite measurement model based on online batches of the training data; tracking the expansion state of the object based on the updated composite measurement model; and smoothing the accumulated updated beliefs using a covariance between the updated beliefs and the predicted beliefs, and at least one of a backward recursion and a forward recursion to generate the online batches of training data.

Images