Computationally efficient representation of traffic participant trajectories

Abstract

To disclose, in general, an autonomous vehicle, and in particular, a technique for representing a path of an object such as a traffic participant (for example, a vehicle, a pedestrian or a cyclist) in an efficient manner regarding calculation.SOLUTION: An exemplary method of generating a control signal for controlling a vehicle includes: acquiring a parameter representation of a path of a single object in the same environment as that of the vehicle; updating the parameter representation of the path of the single object on the basis of data received by one or multiple sensors of the vehicle, within a framework of multiple objects and multiple hypothetical tracking processes; and generating a control signal for controlling the vehicle on the basis of the updated path of the vehicle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates generally to autonomous vehicles, and more particularly to techniques for representing the trajectories of objects, such as traffic participants (e.g., vehicles, pedestrians, cyclists), in a computationally efficient manner (e.g., for multi-object tracking by autonomous vehicles). [Background technology]

[0002] The safe and comfortable operation of autonomous agents (e.g., vehicles including cars, drones, robots, agricultural equipment, etc.) requires anticipatory movement planning that takes into account the likelihood of future behaviors of other agents and objects in the environment. Because the intentions of other agents are generally not perceptible to an outside observer, autonomous agents can only make inferences about the future behavior of their dynamic environment. Such inferences can only be based on observations of past and future movements in the context of the environment, as well as any signals that other agents actively transmit to indicate their intentions, e.g., directional directions.

[0003] Thus, an autonomous agent needs to use its sensors to continuously track the movement of dynamic objects in its environment. Because there are typically multiple dynamic objects in the agent's environment, this process is referred to as multiple object tracking ("MOT"). Multiple object tracking presents a number of specific challenges: the number of tracked objects may change as objects enter and exit the agent's observation region, erroneous sensor readings may falsely indicate the presence of objects, or present objects may not provide sensor readings due to temporary occlusion. Thus, the problem of how to correctly associate sensor readings with tracked objects in order to update the current estimate of motion state—the so-called data association problem—is raised.

[0004] An essential solution to dealing with data association uncertainty is to simultaneously maintain multiple consistent hypotheses for assigning sensor readouts to tracked objects in a so-called multiple hypothesis tracking algorithm ("MHT"). In effect, this allows considering all possible alternatives, while postponing the final decision on data association until a later point in time when more reliable evidence has been gathered (e.g., occlusions have been resolved). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Granstroem et al (https: / / arxiv.org / abs / 1912.08718, Figure I) Summary of the Invention [Problem to be solved by the invention]

[0006] While the MHT algorithm exhibits excellent tracking performance for MOT, it is computationally expensive due to the potentially large number of associated hypotheses that must be maintained. When applied to real-time tracking systems on embedded hardware, the MHT algorithm is therefore currently limited to maintaining only estimates of the object's current kinematic state, such as its current position, velocity, and acceleration.

[0007] However, it can be very useful to track not only the current kinematic state of an object, but also some of its trajectory history, as the trajectory history may contain valuable information about the agent's intent. For example, a vehicle that has just completed a lane change is less likely to make another lane change than another vehicle in the same kinematic state that has not performed this recent maneuver.

[0008] If the MHT algorithm is used for trajectory tracking, the memory required scales as the product of the number of hypotheses (M) and the length of the trajectory (L), i.e., the number of time steps to be tracked, M*L. Also, in mobile sensor applications, it is necessary to take into account the computational cost of transforming all hypotheses considered for situational understanding, prediction, and planning into a coordinate system fixed to the sensor, which scales as L*m, where m is the number of hypotheses applied to these tasks and is ≦M.

[0009] As a result, it is desirable to generate and maintain a representation of an orbit that reduces both the memory requirements and the computational requirements for coordinate transformations, allowing the MHT algorithm to be used for orbit tracking as well. [Means for solving the problem]

[0010] The present disclosure relates to methods, electronic devices, systems, apparatus, and non-transitory storage media for generating control signals for autonomous vehicles and operating autonomous agents. Embodiments of the present disclosure can represent the trajectories of traffic participants (e.g., e.g., ego-vehicle, other vehicles, pedestrians, cyclists, etc.) in a memory-efficient and computationally efficient manner. In some embodiments, the trajectories of objects are represented as parameterized representations, e.g., Bézier representations. In some embodiments, the trajectories of objects may be generalized Bézier curves, i.e., linear combinations of basis functions (e.g., first basis function, second basis function, third basis function, etc.) that are learned from data while preserving the essential properties of Bézier curves, ensuring a low memory footprint and computational efficiency of coordinate transformations.

[0011] Representing traffic trajectories using parametric representations, such as Bézier curves, has many technical advantages. For example, Bézier curves are well suited to representing typical traffic trajectories (e.g., smooth driving trajectories) observed by autonomous agents. Furthermore, trajectory history together with the current state of the environment can provide rich information about the intent of other agents, significantly reducing uncertainty and simplifying the planning process.

[0012] Additionally, parameterized representations require significantly less memory than naive trajectory representations. A naive trajectory representation contains a time sequence of kinematic states, each defined by multiple kinematic state parameters (i.e., position, velocity, and acceleration). Therefore, in multiple-hypothesis approaches, the computational cost scales in both memory and computation as M*L, the product of the number of hypotheses maintained for an object (M) and the length (L) of the trajectory being tracked (e.g., the number of time steps being tracked). In contrast, Bézier curves or their generalizations are parameterized by multiple control points, allowing the kinematic states observed over a time period of several seconds (e.g., tens to hundreds of timestamps per cycle) to be summarized using a fixed number of parameters (e.g., 8 for a cubic Bézier curve, 12 for a quintic Bézier curve). Bézier curves reduce the computational cost of any trajectory tracking algorithm from scaling with the length (L) of the trajectory being tracked to a constant. In other words, embodiments of the present invention are able to keep the scaling of the trajectory tracking algorithm comparable to that of tracking only the current state, while providing a rich temporal context, thereby achieving a memory footprint reduction of >95%.

[0013] Additionally, a parameterized representation of a trajectory requires significantly fewer computational resources than a naive trajectory representation. For example, to obtain a Bézier curve representation as seen from a different coordinate system (e.g., due to the movement of the ego-vehicle), the system can translate and rotate the control points, i.e., the parameters of the curve, to obtain an accurate representation of the curve in a computationally efficient manner. In some embodiments, the control points may be transformed using an affine transformation. In other words, the control points are transformed similarly to how the static positions of environmental features, e.g., traffic signs, are transformed. This contrasts with, for example, a polynomial representation of some coordinate system, whose parameters cannot be directly transformed. When an object's trajectory is fitted using a polynomial representation, ego-motion compensation is difficult because the polynomials in the ego-vehicle's coordinate system do not remain polynomial under rotation. Therefore, the system is limited to maintaining a list of all measured points, compensating them for ego-motion, and re-fitting the points using polynomials, which can be computationally expensive.

[0014] The parameter representation can also be updated computationally efficiently. The kinematic model required for the prediction step of the tracking algorithm is a time-invariant linear transformation that depends only on the cycle time. When observing the object's position, velocity, and acceleration, the Kalman update equation for the trajectory's parameters is exact because the observation model is linearly time-invariant. The parameters are fully interpretable. The standard kinematic state vector (position, velocity, acceleration) of the object along the trajectory can be recovered using a linear transformation. No data fitting is required; rather, the update step directly integrates new observations into the curve's parameters. Therefore, there is no need to store the sequence of observations to compute the Bézier curve representation or its generalization.

[0015] As described herein, some embodiments of the present disclosure derive a multivariate Gaussian distribution of control points for a Bézier curve, or a generalization thereof, along with an adapted motion model and measurement model as a direct drop-in into the Kalman update equation for the Gaussian distribution of kinematic state vectors used in state tracking. This converts any state tracking algorithm into a trajectory tracking algorithm without the computational and memory costs of maintaining the sequence of states that form the trajectory. At the same time, compared to general state tracking, no information is lost because the latest point of the trajectory always coincides with the object's current state. Because Bézier curves can represent comfortable, smooth trajectories with limited jerk, deviations from actually measured object states can be used to detect abnormalities in the behavior of other traffic participants. Furthermore, because the compact Bézier representation summarizes the agent's past behavior in the context of a traffic scene, it is suitable as the sole input for AI algorithms (e.g., machine learning models, e.g., neural networks) for situation understanding in autonomous agents.

[0016] An exemplary method for generating a control signal to control a vehicle comprises: obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle; updating the parameterized representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple object and multiple hypothesis tracking; and generating a control signal to control the vehicle based on the updated trajectory of the object.

[0017] In some embodiments, the control signal is generated based on an updated trajectory of the object and at least one other object in the same environment as the vehicle.

[0018] In some embodiments, the method further comprises providing a control signal to the vehicle to control movement of the vehicle.

[0019] In some embodiments, the method further comprises determining an intent associated with the object based on the updated trajectory, and the control signal is determined based on the intent.

[0020] In some embodiments, the intent includes exiting a road, entering a road, changing lanes, crossing a road, making a turn, or any combination thereof.

[0021] In some embodiments, the method further comprises inputting the updated trajectory into a trained machine learning model to obtain an output, and the control signal is determined based on the output of the trained machine learning model.

[0022] In some embodiments, the machine learning model is a neural network.

[0023] In some embodiments, obtaining the parameterized representation of the trajectory includes retrieving a plurality of control points from a memory.

[0024] In some embodiments, the method further comprises transforming the obtained parameter representation into a new coordinate system based on a movement of the vehicle.

[0025] In some embodiments, transforming the obtained parameter representation includes transforming a plurality of control points of the parameter representation to a new coordinate system.

[0026] In some embodiments, updating the parameter representation includes: predicting an expected parameter representation based on the obtained parameter representation and a motion model; comparing the expected parameter representation with data received by one or more sensors of the vehicle; and updating the parameter representation based on the comparison.

[0027] In some embodiments, predicting the expected parameter representation includes determining a plurality of control points of the expected parameter representation.

[0028] In some embodiments, determining the plurality of control points of the expected parameter representation comprises taking the mean and / or covariance of the plurality of control points of the expected parameter representation.

[0029] In some embodiments, the motion model is a linear model configured to shift the obtained parameter representation forward by a time period.

[0030] In some embodiments, the parameter representation is updated based on a Kalman filter algorithm.

[0031] In some embodiments, the method further comprises determining whether the object is anomalous based on the comparison.

[0032] In some embodiments, the data is first data and the updated parameter representation is a first parameter curve representation, and the method includes: updating the obtained parameter representation of the trajectory based on second data received by one or more sensors of the vehicle to obtain a second updated parameter representation; update and storing the first updated parameter representation and the second updated parameter representation as hypotheses associated with the object.

[0033] In some embodiments, the object is a traffic participant.

[0034] In some embodiments, an exemplary vehicle comprises: one or more processors; a memory; and one or more programs, the one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle; updating the parameterized representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple object and multiple hypothesis tracking; and generating control signals to control the vehicle based on the updated trajectory of the object.

[0035] In some embodiments, an exemplary system for generating control signals to control a vehicle comprises: one or more processors; a memory; and one or more programs, the one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle; updating the parameterized representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple object and multiple hypothesis tracking; and generating control signals to control the vehicle based on the updated trajectory of the object.

[0036] For a better understanding of the various embodiments described, reference should be made to the following detailed description taken in conjunction with the following drawings, in which like reference numerals indicate corresponding parts throughout. [Brief explanation of the drawings]

[0037] [Figure 1A] FIG. 1A illustrates a scenario in which only the current states of two vehicles are known, according to some embodiments. [Figure 1B] FIG. 1B illustrates a scenario in which only the current states of two vehicles are known, according to some embodiments. [Figure 1C] FIG. 1C illustrates a scenario in which both the current state and trajectory history of the object are known, according to some embodiments. [Figure 1D] FIG. 1D illustrates a scenario in which both the current state and trajectory history of the object are known, according to some embodiments. [Figure 2] FIG. 2 illustrates an exemplary process for generating control signals for a vehicle, according to some embodiments. [Figure 3A] FIG. 3A illustrates an exemplary trajectory representation, according to some embodiments. [Figure 3B] FIG. 3B illustrates an exemplary trajectory representation, according to some embodiments. [Figure 4] FIG. 4 illustrates an example of a computing device, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0038] The following description is presented to enable those skilled in the art to make and use various embodiments. Descriptions of specific devices, techniques, and applications are provided only as examples. Various modifications to the examples described herein will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the various embodiments. Thus, the various embodiments are not intended to be limited to the examples described and presented herein, but are to be accorded the scope consistent with the claims.

[0039] The present disclosure relates to methods, electronic devices, systems, apparatus, and non-transitory storage media for generating control signals for autonomous vehicles and operating autonomous agents. Embodiments of the present disclosure can represent the trajectories of traffic participants (e.g., an ego-vehicle, other vehicles, pedestrians, cyclists, etc.) in a memory-efficient and computationally efficient manner. In some embodiments, the trajectories of the objects are represented as parameterized representations. The parameterized representations may be generalized Bézier curves, i.e., linear combinations of basis functions (e.g., first basis function, second basis function, third basis function, etc.) that are learned from data while preserving the essential properties of Bézier curves, ensuring a low memory footprint and computationally efficient coordinate transformations.

[0040] Representing traffic trajectories using parametric representations, such as Bézier curves, has many technical advantages. For example, Bézier curves are well suited to representing typical traffic trajectories (e.g., smooth driving trajectories) observed by autonomous agents. Furthermore, trajectory history together with the current state of the environment can provide rich information about the intent of other agents, significantly reducing uncertainty and simplifying the planning process.

[0041] Additionally, parameterized representations require significantly less memory than naive trajectory representations. A naive trajectory representation contains a time sequence of kinematic states, each defined by multiple kinematic state parameters (i.e., position, velocity, and acceleration). Therefore, in multiple-hypothesis approaches, the computational cost scales in both memory and computation as M*L, the product of the number of hypotheses maintained for an object (M) and the length (L) of the trajectory being tracked (e.g., the number of time steps being tracked). In contrast, Bézier curves or their generalizations are parameterized by multiple control points, allowing the kinematic states observed over a time period of several seconds (e.g., tens to hundreds of timestamps per cycle) to be summarized using a fixed number of parameters (e.g., 8 for a cubic Bézier curve, 12 for a quintic Bézier curve). Parameterized representations reduce the computational cost of any trajectory tracking algorithm from scaling with the length (L) of the trajectory being tracked to a constant. In other words, embodiments of the present invention are able to keep the scaling of the tracking algorithm comparable to that of tracking only the current state, while providing a rich temporal context, thereby achieving a memory footprint reduction of >95%.

[0042] Additionally, a parameterized representation of a trajectory requires significantly fewer computational resources than a naive trajectory representation. For example, to obtain a Bézier curve representation as seen from a different coordinate system (e.g., due to the movement of the ego-vehicle), the system can translate and rotate the control points, i.e., the parameters of the curve, to obtain an accurate representation of the curve in a computationally efficient manner. In some embodiments, the control points may be transformed using an affine transformation. In other words, the control points are transformed similarly to how the static positions of environmental features, e.g., traffic signs, are transformed. This contrasts with, for example, a polynomial representation of some coordinate system, whose parameters cannot be directly transformed. When an object's trajectory is fitted using a polynomial representation, ego-motion compensation is difficult because the polynomials in the ego-vehicle's coordinate system do not remain polynomial under rotation. Therefore, the system is limited to maintaining a list of all measured points, compensating them for ego-motion, and re-fitting the points using polynomials, which can be computationally expensive.

[0043] The parameter representation can also be updated computationally efficiently. The kinematic model required for the prediction step of the tracking algorithm is a time-invariant linear transformation that depends only on the cycle time. When observing the object's position, velocity, and acceleration, the Kalman update equation for the trajectory's parameters is exact because the observation model is linearly time-invariant. The parameters are fully interpretable. The standard kinematic state vector (position, velocity, acceleration) of the object along the trajectory can be recovered using a linear transformation. No data fitting is required; rather, the update step directly integrates new observations into the curve's parameters. Therefore, there is no need to store the sequence of observations to compute the Bézier curve representation or its generalization.

[0044] As described herein, some embodiments of the present disclosure derive a multivariate Gaussian distribution of control points for a Bézier curve, or a generalization thereof, along with an adapted motion model and measurement model as a direct drop-in into the Kalman update equation for the Gaussian distribution of kinematic state vectors used in state tracking. This converts any state tracking algorithm into a trajectory tracking algorithm without the computational and memory costs of maintaining the sequence of states that form the trajectory. At the same time, compared to general state tracking, no information is lost because the latest point of the trajectory always coincides with the object's current state. Because the Bézier curve represents a comfortable, smooth trajectory with limited jerk, deviations from the actually measured object state can be used to detect abnormal behavior of other traffic participants. Furthermore, the compact Bézier representation summarizes the agent's past behavior in the context of a traffic scene, making it suitable as the sole input for AI algorithms for situational understanding in autonomous agents.

[0045] In the following description, example methods, parameters, etc. are set forth. However, it should be recognized that such description is not intended to limit the scope of the present disclosure, but is provided as a description of example embodiments.

[0046] In the following description, terms such as "first," "second," etc. are used to describe various elements, but these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first graphic representation could be referred to as a second graphic representation, and similarly, a second graphic representation could be referred to as a first graphic representation, without departing from the scope of the various embodiments described. The first graphic representation and the second graphic representation are both graphic representations, but are not the same graphic representation.

[0047] The terminology used in the description of the various embodiments set forth herein is for the purpose of describing particular embodiments only and is not intended to be limiting. When used in the description of the various embodiments set forth and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. As used herein, it will also be understood that the term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items. It will also be understood that as used herein, the terms "includes," "including," "comprises," and / or "comprising" describe the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0048] The word "if" is arbitrarily interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are arbitrarily interpreted to mean "upon determining," "in response to determining," "upon detecting [the stated condition or event]," or "in response to detecting [the stated condition or event]," depending on the context.

[0049] Safe and comfortable navigation of autonomous vehicles requires anticipatory planning, i.e., the ability to form expectations and make predictions about the future behavior of dynamic objects in the environment. The basis for such predictions is an accurate estimation of the dynamic object's current state based on past observations. Naturally, such a state estimator is probabilistic due to uncertainties in the measurement process or unobservable quantities, such as the driver's intent. State-space models are uniquely suited for this task, as they provide a robust probabilistic framework for continuously assimilating observations into an estimator of the dynamic object's current state and tracking its movement over time. A standard technique for this purpose is the Kalman filter or any of its variants and extensions. In the following, we describe the main operational steps of the Kalman filter, some weaknesses of this approach, and how it can be improved.

[0050] Caiman Filter Algorithm

[0051] Single object x t The probability distribution for the current state vector of is P(x t |o t ,..,o0)=P t|t For example, an autonomous vehicle being tracked in two dimensions can be considered as having x position, y position, velocity v, acceleration a, yaw angle ψ, and rotational velocity ψ · A general state vector x with six components t In the simplest case, the probability density of this vector is given by a Gaussian distribution P t|t =N(x t ;μ t|t ,Σ t|t ) and the average is μ t|t , and the covariance is Σ t|t is.

[0052] We are interested in the current state vector x t :P(x t+δt |x t) is given, then it is a further distribution for the future state vector at time t + δt. Also, in the simplest case, it is a Gaussian distribution P(x t+δt |x t )=N(x t+δt ;x ^ t+δt ,Q(δt)) and the mean is a linear function x ^ t+δt =F(δt)x t , the covariance matrix is ​​Q(δt), and the matrices F(δt) and Q(δt) depend only on δt.

[0053] Furthermore, for a given state vector, the observation o t The likelihood P(o t |x t ) is considered. In the simplest case, the Gaussian distribution P(o t |x t )=N(o t ;o ^ t , R), and the mean is a linear function of the given state function o ^ t =Hx t , and the covariance is R. As an example, for the above state vector and a sensor that only returns position and velocity estimates, the matrix

number

[0054]

number

[0055] Observation vector o t consists of three components: x position, y position and velocity v.

[0056] With these available distributions, the system sequentially calculates new observations from time t + δt into the state vector x t+δt can be assimilated into a refined estimator of δt based on all the information obtained previously through iterations of the prediction step and subsequent application of Bayes' rule in the update step for each observation in the sequence.

[0057]

number

[0058] If we assume a Gaussian distribution of the parameter vector and a Gaussian likelihood of the observations, along with a linear motion and observation model, we can solve these equations in closed form, resulting in the standard Kalman filter equation for the prediction step, P t+δt|t =N(x t+δt ;μ t+δt|t ,Σ t+δt|t ) is obtained.

[0059]

number

[0060] In this case, the updated P t+δt|t+δt =N(x t+δt ;μ t+δt|t+δt ,Σ t+δt|t+δt ) is calculated as follows:

[0061]

number

[0062] Using these equations, the current state of a single object can be efficiently tracked from sequential observations.

[0063] When tracking is performed from a moving sensor, as in the case of an autonomous vehicle, the system must compensate for sensor movement in a process called ego-compensation. The system is in a fixed world coordinate system, which must transform observations from the moving vehicle's coordinate system into the fixed coordinate system in which the prediction and update steps are performed. On the other hand, situation interpretation and planning are typically performed in the vehicle's coordinate system, so the system must back-transform the updated state estimates into the vehicle's coordinate system.

[0064] Preferably, tracking is performed directly in the vehicle's coordinate system, and therefore the system needs to transform the state estimates into the vehicle's current coordinate system where the measurements are made. In this case, updates may be performed directly in the vehicle's coordinate system.

[0065] Multiple object tracking

[0066] An autonomous vehicle's environment may contain two or more dynamic objects to be tracked using a multiple object tracking (MOT) algorithm. Therefore, sensors return a set of detections for different objects. For example, a radar sensor may return multiple detected objects based on radar returns, or multiple objects may be detected in a camera image. Typical object detectors for radar returns, camera images, or lidar sensors operate on a frame-by-frame basis, meaning that there is no established one-to-one correspondence between object detection i in a sensor readout at time t and the state vector of object i estimated based on the previous readout. Such a general, direct correspondence cannot be established as objects move in and out of the sensor's range, and the object detector may produce false-positive object detections or fail to detect an object due to occlusion or simple detector malfunctions.

[0067] {o t} is k at time t t Let o be the set of object detections. t,idenotes detection i at time t. The state estimator for a single object relies on accurately associating the object detection with the tracked object in the Kalman equation update step. For example, consider the case t=3, where the state of object j is

number

number

[0068] Additionally, the data association sequences of different tracked objects must be consistent. If an object detection can only be associated with a single tracked object, the data association sequences of different objects must be disjoint, and a particular detection t,i However, this cannot be done in a connected sequence of two distinct objects.

[0069] This data association consistency is ensured in the multiple object tracking algorithm before the update step. We define the likelihood of any single object detection i arising from the predicted state of any object j as

number

[0070] Multiple hypothesis tracking

[0071] On the other hand, at any time step, there may be several possible data associations that are consistent with comparable likelihoods. This ambiguity arises especially in cluttered scenes with many occlusions, as is typical of, for example, inner-city traffic.

[0072] In general, it can be seen that since the sequential correlation of observations to state estimators cannot correct errors in the data correlation, choosing an incorrect data correlation risks potentially misestimating the current state of the environment and consequently being unable to plan safe actions.

[0073] This risk can be mitigated by lifting the restriction to working with only a single consistent data association. The Multiple Hypothesis Tracking (MHT) algorithm maintains a list of the highest M most likely consistent data associations at each time instant and uses each of them independently in the branching process. In this way, similarly high likelihood data associations, and the resulting estimates of the state of the dynamic environment, are maintained until further observations are collected that resolve the ambiguity.

[0074] The consistent data associations of all tracked objects maintained by the MHT algorithm are called the "global hypothesis." The data association sequence of a single tracked object and the resulting state estimate are called the "local hypothesis." Thus, the global hypothesis comprises a set of consistent local hypotheses. A single local hypothesis may appear in several global hypotheses as long as consistency is not lost. Typically, the number of global hypotheses maintained is in the hundreds, and the number of local hypotheses is in the same order of magnitude.

[0075] To reduce the memory footprint, local hypotheses that are not part of any global hypothesis are pruned from memory.

[0076] Difficulties with this method

[0077] While good in theory, the above approach has several drawbacks, which the present disclosure aims to address.

[0078] In general, the choice of the state vector x is motivated by prior knowledge of the kinematics of the object being tracked, e.g., for a vehicle, the state variables are typically chosen to be position, velocity, acceleration, yaw angle, and roll rate, as described above. While intuitive, this definition of state implies a nonlinear motion model that requires approximations to be introduced into the update equations. For example, the x position at time t+δt is nonlinear in the state variables:

number

number

[0079] Furthermore, such a definition of the state vector is primarily intended to accurately represent the current kinematic state of the vehicle, not to represent information from past observations relevant to future predictions. For example, in FIG. 1A, two vehicles are observed on a highway on-off ramp with their kinematic state vectors complete in their current state. However, this kinematic state does not contain information that reduces uncertainty about the future. Because both vehicles appear equally likely to enter or exit the highway, an observer traveling behind these two vehicles must accommodate four equally likely future developments of the scene ahead, as shown in FIG. 1B.

[0080] Since driver intentions are generally unobservable, having all or at least part of a traffic participant's past trajectory available reduces the uncertainty of predictions about the traffic participant's future behavior and thus facilitates planning. This is illustrated in Figures 1C and 1D below. Given the past trajectory, the uncertainty about the possible future development of the scene essentially disappears.

[0081] A naive approach to startup tracking is to simply maintain a list of L most recent estimates of the kinematic state of a dynamic object. If tracking is performed at a frequency of 10 Hz, maintaining such a list for ΔT seconds requires maintaining L = 10ΔT state vectors. In this way, the memory footprint scales linearly with L for every tracked object in multi-object tracking, or for every tracked local hypothesis in multi-hypothesis tracking. See [1] for a discussion of the computational requirements of prior art implementations of MHT tracking.

[0082] Also, if tracking needs to be done in the vehicle's coordinate system, a naive listing of the L-state history as a trajectory requires ego compensation at every single time instant, which is computationally expensive.

[0083] The present disclosure provides: Multiple In L time steps Cross The past orbit of the object Use this object as its state However, instead of using a list, the system uses a parameterized representation of this trajectory that is independent of L and has a small memory footprint, and an observation model that allows estimation of the corresponding linear motion and parameters of this representation using the Kalman update equation without the need for approximations. The small memory footprint allows trajectory tracking to be used with multiple object tracking algorithms in a real-time capable system running in embedded software.

[0084] Also, ego compensation in this representation is computationally very cheap since only the parameters need to be transformed, and therefore the computational cost does not depend on the length of the tracked trajectory.

[0085] The core of any tracking algorithm following the sequential Bayesian data assimilation paradigm is similar, regardless of whether one is tracking a single or multiple objects and a single or multiple data association hypotheses. A state representation, a motion model that propagates the state forward in time, and an observation model that allows evaluation of the likelihood of observations given a state estimator are required. Below, the mathematical details of the proposed representation of a d-dimensional trajectory based on n+1 basis functions and the corresponding motion and observation models are described.

[0086] The system is an n+1-dimensional vector Φ(t)=[Φ0(t),Φ1(t),Φ2(t),…,Φ n (t)]. i Consider the n+1 orbital basis functions of (t).

[0087] Here, the system defines an (n+1)xd-dimensional matrix P of control points, where each row of P corresponds to one control point in d dimensions. The control points are the parameters of the trajectory, c(t)=Φ T (t)P, where c(t) is a d-dimensional vector, gives the position along the trajectory parameterized by P at time t. As usual, the superscript T denotes transpose. It is clear that for all t, P completely specifies c(t), so if we want to track c(t), we only need to estimate and track P.

[0088] Ego-motion compensation of this trajectory can be obtained for any fixed point in space by simply transforming the control points, which transform trivially under sensor translation and rotation.

[0089] The motion model of P is obtained as a time-invariant linear transformation that can be directly calculated from the basis functions.

[0090] The observation model is obtained as follows: A sensor of a dynamic object can generally obtain time derivatives of position information and velocity, etc. Due to the representation chosen in this disclosure, the jth derivative of the trajectory is calculated by the linear transformation d of the control points of this disclosure. j / dt j c(t)=H j PHereH j =d j / dt j Φ T (t), is given as:

[0091] FIG. 2 illustrates an exemplary process for generating a vehicle control signal, according to some embodiments. For example, process 200 may be performed using one or more electronic devices implementing a software platform. In some examples, process 200 may be performed using one or more electronic devices in an autonomous vehicle (e.g., the ego-vehicle). In some embodiments, process 200 may be performed using a client-server system, with blocks of process 200 optionally divided between a server and one or more client devices. Thus, while portions of process 200 are described in this disclosure as being performed by particular devices, it will be understood that process 200 is not limited thereto. In process 200, some blocks may be optionally combined, some blocks may be optionally reordered, and some blocks may be optionally omitted. In some examples, additional steps may be performed in combination with process 200. Thus, the operations illustrated (and described in more detail below) are exemplary in nature and should not be considered limiting in themselves.

[0092] In block 202, (e.g., one or more electronic devices) obtain a set of estimators of a parametric trajectory representation of an object. The object may be a traffic participant in the same environment as the vehicle, such as a vehicle, a pedestrian, a cyclist, a drone, an animal, etc.

[0093] The d-dimensional parametric trajectory representation comprises n+1 time-dependent basis functions and d-dimensional n+1 control points. The n+1 time τ-dependent basis functions provided to the n+1-dimensional vector are as follows:

[0094] Φ(τ)=[φ0(τ)φ1(τ)φ2(τ)... φ n (τ)]

[0095] The system aims to represent trajectories of length ΔT in the interval τ∈[0,1], which is always possible for any ΔT by making the substitution dτ=dt / ΔT.

[0096] At any time t, for a time span ΔT, a point along the past trajectory of an object in d dimensions is computed using n+1 basis functions and an (n+1)xd-dimensional matrix P t and n+1 control points of d dimensions provided in

[0097] c(τ)=Φ T (τ)P t

[0098] where:

number

[0099] An example of such a choice of basis functions is the Bernstein polynomials.

[0100]

number

[0101] In this case, the curve c(τ) is known as a Bézier curve, and the control points P thas a particularly intuitive interpretation. In this disclosure, we refer to this special case as a working example. However, it should be understood by those skilled in the art that all statements are applicable to general basis functions. In particular, basis functions can be optimized to accurately represent empirically measured trajectories.

[0102] The vector σ(σ) can be used to specify a distribution over the parameters of the trajectory, so that the Kalman filter can use this representation of the trajectory.

number

number

[0103] Figure 3A shows an example using a cubic Bézier curve in two dimensions (e.g., n=3, d=2). A cubic Bézier curve is parameterized by four control points p0, p1, p2, and p3. Each control point is a two-dimensional point represented by two values. The state vector x of such a trajectory is t The size of is (n+1)d=8. In the figure, the probabilistic aspect is also shown. The annotation points and the corresponding ellipses are calculated by the mean μ t and x t The solid line indicates the mean trajectory estimate.

number

number

[0104] Then, in block 206, a data assimilation step is performed by updating the set of local hypotheses based on data received by one or more sensors of the autonomous system. In block 206a, ego compensation is performed. For the parametric trajectory representation, only the control points, i.e., the state vector, need to be transformed. This is significantly less computationally expensive than transforming a list of kinematic state vectors. Because this is a linear transformation, in this disclosure it can be directly applied to parameters of the state density. Assuming that the reference frame is translated by a d-dimensional vector Δo and rotated R by a d×d matrix, the system can first transform the covariance matrix of the state density here. The covariance matrix is ​​only affected by the rotation R.

[0105]

number

[0106] The mean of the density of states is first transformed into homogeneous coordinates, i.e., an extra dimension is introduced at the control points which is a constant and equal to 1. The homogenized mean vector is calculated as follows:

[0107] μ h =[μ1,…μ d ,1,μ d+1 ,…μ 2d ,1…μ nd ,…μ (n+1)d ,1]

[0108] Then a d×(d+1) matrix is ​​introduced.

[0109] T=[R-RΔo]

[0110] The mean vector can be transformed as follows:

[0111]

number

[0112] In block 206b, the trajectory extension is predicted using a specific motion model of the trajectory's control points.

[0113] Considering time t, the system is such that τ0=0 and τ m = 1, and the different times τ i ,i∈{1,..,m}, the m>n+1 samples c(τ i ) These samples may be arranged as rows of an mxd matrix C. An mx(n+1) matrix B is such that the i-th row of B has Φ(τ i ), in which case the control points can be estimated as a least-squares fit to the sample of trajectory samples.

[0114] P t =(B T B) -1 B T C

[0115] Here, we consider the motion model of the control point, that is, the movement along the trajectory. For this purpose, the mx (n+1) matrix B' is defined such that the i-th row of B' is Φ(τ i +δ t / ΔT), in which case the system can obtain transformed control points.

[0116]

number

[0117] In this way, the system can obtain a linear motion model F(δt) of the control points that depends only on the choice of basis functions.

[0118]

number

[0119] where I d is the d-dimensional identity matrix.

[0120] In particular, the endpoint of the shifted orbit is c(1)=Φ T (1)P t+δt Figure 3B shows this for an example implementation of a cubic Bezier curve.

[0121] matrix P t , i.e., the control points p0, p1, p2, and p3 that parameterize the trajectory at time t are given by the matrix P t+δt , that is, time t+δ t The control points p that parameterize the trajectory of ′ 0,p ′ 1,p ′ 2, and p ′ 3. With this configuration, the new orbit estimate follows the old one just up to time t.

[0122] In this case, the prediction step of the Kalman filter equation is written as follows:

[0123]

number

[0124] In block 206d, the likelihood j of the current sensor reading is based on the set of local hypothesis predicted trajectory extensions. A typical object detector makes measurements of object position, velocity, and possibly acceleration. These types of kinematic measurements are easily obtained. The i-th derivative is simply:

[0125]

number

[0126] For trajectory tracking, a natural choice is to consider the most recent, i.e., the points of the trajectory at the last time instant τ = 1. Now, depending on the sensor information available for each object, the observation matrix H can be constructed.

[0127] For example, if the system tracks the trajectory of an object in two dimensions and sensors provide the position and respective velocity, then the observation vector o t =[x,y,v x ,v y ] is formed, where the rows are Φ(1) and

number

[0128]

number

[0129] The system now has all the necessary parts to be able to track the trajectory of an object over a time period ΔT by constantly updating the trajectory with the most recent observations.

[0130] In block 206d, M most likely consistent global hypotheses are formed based on the likelihood of the current sensor reading calculated in the previous step.

[0131] In block 206e, any local hypotheses not used in the M most likely consistent global hypotheses are pruned from memory.

[0132] Process 206d then returns the set of M most likely consistent global hypotheses and their corresponding local hypotheses for further processing at block 208.

[0133] In block 208, the system determines a vehicle control signal based on the object's updated trajectory. In some embodiments, the system may determine an intent associated with the object based on the object's updated trajectory and may determine the vehicle control signal accordingly. For example, as described with reference to FIGS. 1A-1D , the trajectory history may be used to determine the traffic participant's intent (e.g., exiting a highway, entering a highway, crossing a road, making a turn). Based on the intent associated with the object, the system may determine a vehicle control signal to avoid a collision with the object. The control signal may be provided or transmitted to the vehicle to control the vehicle (e.g., maintain speed, accelerate, decelerate, change direction, etc.).

[0134] In some embodiments, the system may use the updated trajectory for various downstream analyses. For example, the system may input the updated trajectory into a machine learning model for situation understanding. For example, the machine learning model may be configured to receive the object's trajectory, identify the object's intent, identify anomalous behavior, predict future trajectories, etc. The compact parameter representation summarizes the past behavior of traffic participants in the context of a traffic scene and is therefore well suited to AI algorithms for situation understanding in autonomous vehicles. Due to the compactness of the Bézier representation, compact machine learning models can be computationally efficient to train, and the trained models can provide fast analysis. The machine learning models described in this disclosure include any computer algorithm that improves automatically through experience and with the use of data. The machine learning models may include supervised models, unsupervised models, semi-supervised models, self-supervised models, etc. Exemplary machine learning models include, but are not limited to, linear regression, logistic regression, decision trees, support vector machines (SVMs), naive Bayes, neural networks, k-means, random forests, dimensionality reduction algorithms, gradient boosting algorithms, etc.

[0135] In some embodiments, the system stores multiple hypotheses for the object as described above. Each hypothesis for the object includes a Bézier curve representation (e.g., control points that parameterize the curve). In this way, an updated Bézier curve representation may be stored as one of many hypotheses associated with the object.

[0136] In some embodiments, the system evaluates the object's updated trajectory to determine whether the object is behaving abnormally. Because Bézier curves are smooth and represent typical driving trajectories well (e.g., comfortable, smooth trajectories with limited jerk), deviations from the actual measured object state can be used to detect abnormal behavior of other traffic participants. In some embodiments, the system determines the deviation between the expected curve (e.g., obtained in block 206b) and the actual observed trajectory and compares this deviation with a predetermined threshold. If the threshold is exceeded, the system may determine that the object is exhibiting abnormal behavior (e.g., reckless driving). Based on the detected abnormality, the system may generate a control signal (e.g., to move away from the abnormal object).

[0137] While the technique described with reference to process 200 is configured to use Bezier curves to represent the trajectories of traffic participants other than the ego vehicle, the technique can also be applied to tracking the trajectory of the ego vehicle itself using Bezier curves. Also, while the technique described with reference to process 200 involves the use of Bezier curves, it should be understood that the Bezier curve representation can be replaced by any linear combination of basis functions (e.g., first basis function, second basis function, third basis function, etc.). In some embodiments, the basis functions can be calculated using a machine learning model, e.g., a neural network or a Gaussian process.

[0138] The operations described in this disclosure are optionally performed by the components shown in FIG. 4 . FIG. 4 illustrates an example of a computing device according to one embodiment. Device 400 may be a host computer connected to a network. Device 400 may be a client computer or a server. As shown in FIG. 4 , device 400 may be any suitable type of microprocessor-based device, such as a personal computer, a workstation, a server, or a handheld computing device (portable electronic device), such as a phone or tablet. The device may include, for example, one or more processors 410, input devices 420, output devices 430, storage 440, and communication devices 460. Input devices 420 and output devices 430 may generally correspond to those described above and may be either connectable to or integrated with the computer.

[0139] Input device 420 may be any suitable device for providing input, such as a touchscreen, a keyboard or keypad, or a voice recognition device. Output device 430 may be any suitable device for providing output, such as a touchscreen, a haptic device, or a speaker.

[0140] Storage 440 may be any suitable device for storage, such as electrical, magnetic, or optical memory including RAM, a cache, a hard drive, or a removable storage disk. Communications device 460 may include any suitable device capable of sending and receiving signals over a network, such as a network interface chip or device. The components of the computer may be connected in any suitable manner, such as by a physical bus or wirelessly.

[0141] Software 450 storable in storage 440 and executable by processor 410 may include, for example, programming that embodies the functionality of the present disclosure (eg, as embodied in the devices described above).

[0142] Additionally, software 450 may be stored and / or transported within a non-transitory computer-readable storage medium for use by or in conjunction with an instruction-execution system, apparatus, or device capable of retrieving and executing software-related instructions from such an instruction-execution system, apparatus, or device. In the context of the present disclosure, a computer-readable storage medium may be any medium, such as storage 440, that contains or is capable of storing programming for use by or in conjunction with an instruction-execution system, apparatus, or device.

[0143] Additionally, the software 450 may be propagated in any transport medium for use by or in conjunction with an instruction execution system, apparatus, or device capable of retrieving software-related instructions from and executing the instructions, such as those described above. In the context of this disclosure, a transport medium may be any medium capable of communicating, propagating, or transporting programming for use by or in conjunction with an instruction execution system, apparatus, or device. Transport-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation media.

[0144] Device 400 may be connected to a network, which may be any suitable type of interconnected communications system. The network may implement any suitable communications protocol and may be protected by any suitable security protocol. The network may comprise any configuration of network links capable of transmitting and receiving network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL or telephone lines, etc.

[0145] Device 400 may run any operating system suitable for operating on a network. Software 450 may be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying functionality of the present disclosure may be deployed in various configurations, such as a client / server configuration, a web-based application, or as a web service in a web browser.

[0146] Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art, and such changes and modifications are to be understood as being included within the scope of the present disclosure and examples, as defined by the claims.

[0147] The foregoing description has been made with reference to specific embodiments for purposes of explanation. However, the illustrative description is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described in order to best explain the principles of the technology and their practical application, thereby enabling those skilled in the art to best utilize the technology and various embodiments, with various modifications suitable for the particular use contemplated. This application relates to the invention described in the claims, but may also include the following configurations as other aspects: 1. 1. A method for generating a control signal for controlling a vehicle, comprising: obtaining a parameterized representation of the trajectory of a single object in the same environment as the vehicle; updating the parameter representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple object and multiple hypothesis tracking; generating the control signal for controlling the vehicle based on the updated trajectory of the object. 2. 2. The method of claim 1, wherein the control signal is generated based on the updated trajectories of the object and at least one other object in the same environment as the vehicle. 3. 3. The method of claim 1 or 2, further comprising providing the control signal to the vehicle to control movement of the vehicle. 4. 4. The method according to any one of 1 to 3, further comprising determining an intent associated with the object based on the updated trajectory, wherein the control signal is determined based on the intent. 5. 5. The method of claim 4, wherein the intent includes exiting a road, entering a road, changing lanes, crossing a road, making a turn, or any combination thereof. 6. 6. The method according to any one of 1 to 5, further comprising inputting the updated trajectory into a trained machine learning model to obtain an output, wherein the control signal is determined based on the output of the trained machine learning model. 7. 7. The method according to claim 6, wherein the machine learning model is a neural network. 8. 8. The method according to any one of claims 1 to 7, wherein obtaining the parameter representation of the trajectory comprises retrieving a plurality of control points from a memory. 9. 9. The method of claim 8, further comprising transforming the obtained parameter representation into a new coordinate system based on movement of the vehicle. 10. 10. The method of claim 9, wherein transforming the obtained parameter representation comprises transforming the plurality of control points of the parameter representation to the new coordinate system. 11. updating the parameter representation predicting an expected parametric representation based on the obtained parametric representation and a motion model; comparing the expected parameter representation to the data received by the one or more sensors of the vehicle; 11. The method according to any one of 1 to 10 above, comprising: updating said parameter representation based on said comparison. 12. 12. The method of claim 11, wherein predicting the expected parameter representation comprises determining a plurality of control points of the expected parameter representation. 13. 13. The method of claim 12, wherein determining the plurality of control points of the expected parameter representation comprises obtaining a mean and / or a covariance of the plurality of control points of the expected parameter representation. 14. 14. A method according to any one of claims 11 to 13, wherein the motion model is a linear model configured to shift the obtained parameter representation forward by a time period. 15. 15. A method according to any one of claims 11 to 14, wherein said parameter representation is updated based on a Kalman filter algorithm. 16. 16. The method according to any one of claims 11 to 15, further comprising determining whether the object is abnormal based on the comparison. 17. 17. The method according to any one of 1 to 16, wherein the data is first data and the updated parameter representation is a first parameter curve representation; updating the obtained parameterized representation of the trajectory based on second data received by the one or more sensors of the vehicle to obtain a second updated parameterized representation; The method further comprising storing the first updated parameter representation and the second updated parameter representation as hypotheses associated with the object. 18. 18. The method according to any one of claims 1 to 17, wherein said object is a traffic participant. 19. one or more processors; Memory and and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: obtaining a parameterized representation of the trajectory of a single object in the same environment as the vehicle; updating the parameter representation of the single object's trajectory based on data received by one or more sensors of the vehicle within a multi-object and multi-hypothesis tracking framework; The vehicle includes instructions for generating the control signal to control the vehicle based on the updated trajectory of the object. 20. 1. A system for generating a control signal for controlling a vehicle, comprising: one or more processors; Memory and and one or more programs, the one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: obtaining a parameterized representation of the trajectory of a single object in the same environment as the vehicle; updating the parameter representation of the single object's trajectory based on data received by one or more sensors of the vehicle within a multi-object and multi-hypothesis tracking framework; The system includes instructions for generating the control signal for controlling the vehicle based on the updated trajectory of the object.

Claims

1. 1. A method for generating a control signal for controlling a vehicle, comprising: obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle using past trajectories of the single object over multiple time steps as states of the single object; updating the parameter representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple object and multiple hypothesis tracking; generating the control signal for controlling the vehicle based on the updated trajectory of the object; Updating the parameter representation comprises: predicting an expected parametric representation based on the obtained parametric representation and a motion model; comparing the expected parameter representation to the data received by the one or more sensors of the vehicle; and updating the parameter representation based on the comparison.

2. The method of claim 1 , wherein the control signal is generated based on the updated trajectories of the object and at least one other object in the same environment as the vehicle.

3. The method of claim 1 or 2, further comprising providing the control signal to the vehicle to control movement of the vehicle.

4. The method of claim 1 or 2, further comprising determining an intent associated with the object based on the updated trajectory, wherein the control signal is determined based on the intent.

5. The method of claim 4 , wherein the intent comprises exiting a road, entering a road, changing lanes, crossing a road, making a turn, or any combination thereof.

6. 3. The method of claim 1 or 2, further comprising inputting the updated trajectory into a trained machine learning model to obtain an output, and wherein the control signal is determined based on the output of the trained machine learning model.

7. The method of claim 6 , wherein the machine learning model is a neural network.

8. The method of claim 1 or 2, wherein obtaining the parameter representation of the trajectory comprises retrieving a plurality of control points from a memory.

9. The method of claim 8 , further comprising transforming the obtained parameter representation into a new coordinate system based on movement of the vehicle.

10. The method of claim 9 , wherein transforming the obtained parameter representation comprises transforming the control points of the parameter representation to the new coordinate system.

11. The method of claim 1 or 2, wherein predicting the expected parametric representation comprises determining a plurality of control points of the expected parametric representation.

12. The method of claim 11 , wherein determining the plurality of control points of the expected parameter representation comprises taking a mean and / or a covariance of the plurality of control points of the expected parameter representation.

13. 3. The method of claim 1, wherein the motion model is a linear model configured to shift the obtained parameter representation forward by a time period.

14. The method of claim 1 or 2, wherein the parameter representation is updated based on a Kalman filter algorithm.

15. The method of claim 1 or 2, further comprising determining whether the object is anomalous based on the comparison.

16. 3. The method of claim 1, wherein the data is first data and the updated parameter representation is a first parameter curve representation. updating the obtained parameterized representation of the trajectory based on second data received by the one or more sensors of the vehicle to obtain a second updated parameterized representation; The method further comprising storing the updated parameter representation and the second updated parameter representation as hypotheses associated with the object.

17. 3. The method of claim 1 or 2, wherein the object is a traffic participant.

18. one or more processors; Memory and and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: The method includes instructions for obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle using past trajectories of the single object over multiple time steps as states of the single object, updating the parameterized representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a framework of multiple objects and multiple hypotheses tracking, and generating control signals to control the vehicle based on the updated trajectories of the objects. Updating the parameter representation comprises: predicting an expected parametric representation based on the obtained parametric representation and a motion model; comparing the expected parameter representation to the data received by the one or more sensors of the vehicle; and updating the parameter representation based on the comparison.

19. 1. A system for generating a control signal for controlling a vehicle, comprising: one or more processors; Memory and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: The method includes instructions for: obtaining a parameterized representation of a trajectory of a single object in the same environment as the vehicle using past trajectories of the single object over multiple time steps as states of the single object; updating the parameterized representation of the trajectory of the single object based on data received by one or more sensors of the vehicle within a multiple object and multiple hypothesis tracking framework; and generating the control signal for controlling the vehicle based on the updated trajectory of the object. Updating the parameter representation comprises: predicting an expected parametric representation based on the obtained parametric representation and a motion model; comparing the expected parameter representation to the data received by the one or more sensors of the vehicle; and updating the parameter representation based on the comparison.

Images