Systems and methods for machine learning implemented state transition planning

Abstract

Systems and methods for the generation of a state transition plan by using a diffusion model to predict state representations. The diffusion model can be constrained using at least one constraint function during training where the at least one constraint function models a start and an end state in the reverse diffusion process. The diffusion model is trained to learn a constrained score function corresponding to a score function comprising a score offset based on the at least one constraint function. At inference, the diffusion model can be provided with observed states of one or more agents to generate predicted states for the one or more agents.

Description

SYSTEMS AND METHODS FOR MACHINE LEARNING IMPLEMENTED STATE TRANSITION PLANNING CROSS REFERENCE TO RELATED APPLICATION

[0001] The present disclosure claims priority to and benefit from United States provisional patent application no. 63 / 749,442, filed on January 24, 2025, and entitled “SYSTEMS AND METHODS FOR MACHINE LEARNING IMPLEMENTED STATE TRANSITION PLANNING”, the entirety of which is hereby incorporated by reference herein.TECHNICAL FIELD

[0002] The present disclosure relates to machine learning implemented agent planning and in particular to agent planning using a diffusion model.BACKGROUND

[0003] A plan, for example for one or more agents, is a sequence of actions that, if taken, should lead to the achievement of some goal. Provided that there are no constraints on actions that can be taken, equivalently a plan can be a sequence of states to be visited en route to a goal. A safe plan is such a sequence that also does not violate constraints. A particular example application of safe planning is in autonomous driving. Computing an efficient, perhaps shortest, route from point a to point b on a map is a planning problem. Computing such a route assuming no traversals of an unsafe road segment is a safe planning problem. Typically such “high-level” planning problems are “solved” using search algorithms on road-connectivity graphs. Viewing safe planning as a more granular planning problem, given a route and its features, safe planning aims to find a collision-, off-road-, and other infraction-free path that generally follows the turn-by-turn guidance of the higher-level path planner. Although machine learning has been proposed as an avenue for performing agent planning, the use of machine learning models in agent planning remains limited.

[0004] Accordingly, systems and methods that enable safe agent planning using machine learning remain highly desirable.SUMMARY

[0005] In accordance with one aspect of the present disclosure, a method for generating a state transition plan is disclosed, comprising receiving at least one observed state representation; predicting at least one unobserved state representation from the at least one observed state representation using a diffusion model trained to model state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model, the diffusion model has been trained by constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function, the constrained score function comprising a score offset based on an output of the constraint function, and the predicting comprising processing the at least one observed state representation with the diffusion model.

[0006] In some aspects, the constraint function is a diffusion function expressed as a function comprising a continuously differentiable function and a diffusion gradient term for a state representation.

[0007] In some aspects, the constraint function comprises a plurality of constituent constraint functions.

[0008] In some aspects, the diffusion model has been trained to model state transitions by applying the constraint function at every sampling step of the diffusion model, each sampling step corresponding to an iterative step in the reverse diffusion process.

[0009] In some aspects, the at least one observed state representation comprises at least one agent state representation; and each of the at least one agent state representation and the at least one unobserved state representation comprises a position of an agent and a heading of the agent at a point in time.

[0010] In some aspects, each of the at least one observed state representation and the at least one observed state representation correspond to a transition between states.

[0011] In some aspects, the at least one observed state representation comprises at least one fixed constraint invariant between a time corresponding to the at least one observed state representation and a time corresponding to the at least one unobserved state representation.

[0012] In some aspects, the at least one fixed constraint is a feature map defining one or more constraints on a position defined by the at least one observed state representation and the at least one unobserved state representation.

[0013] In some aspects, the feature map defines one or more infraction zones; and the diffusion model has been trained by penalizing state representations predicted to be in the one or more infraction zones.

[0014] In some aspects, the diffusion model has been trained by penalizing state representations predicted to be overlapping at the same sampling time.

[0015] In some aspects, the constrained score function is a logarithmic gradient probability density function calculated according to a term corresponding to denoised predicted states, a term corresponding to the constraint function evaluated at the denoised predicted states, and noisy states.

[0016] In some aspects, the at least one unobserved state representation is a plurality of unobserved state representations for a plurality of agents; the plurality of unobserved state representations are modeled jointly; and a predicted state representation of each agent is predicted based on a subset of the plurality of agents according to distances thereto from nearest neighbors of each agent.

[0017] In some aspects, the predicted state representation of each agent is predicted in a respective local coordinate frame by the diffusion model, each of the respective local coordinate fixed to a respective agent; and the method further comprises transforming the predicted state into a global coordinate frame shared by all agents.

[0018] In some aspects, the at least one observed state representation comprises one or more observed agent state representations; the at least one unobserved state representation comprises a control state representation in respectof one agent and one or more conditional state representations in respect of remaining agents and corresponding to the one or more observed agent state representations at a future time step; and the predicting comprises predicting the control state representation according to the one or more observed agent state representations at the future time step and the one or more conditional state representations.

[0019] In some aspects, the diffusion model comprises a transformer network comprising a plurality of layers configured to parameterize the constrained score function and a multi-layer perception network configured to encode input to the transformer network and to output the at least one unobserved state.

[0020] In some aspects, at least one of the plurality of layers correspond to a time transformer layer configured to parameterize time variables, an agent transformer layer configured to parameterize state variables, and a feature map transformer layer configured to fixed constraint variables.

[0021] In some aspects, the diffusion model has been trained using a plurality of training state representations comprising a plurality of observed training state representations and a plurality of unobserved training state representations generated by applying an observation mask to the plurality of training state representations, the diffusion model has been conditioned using multiple pluralities of training state representations, and each of the pluralities of training state representations is generated by using a different observation mask selected from the group of a predictive mask, a goal-condition mask, an agent reactive mask, an up-sampling mask, an imputation mask, a windowed mask, an agent-conditioned mask, and an ego-conditioned mask as the observation mask.

[0022] In some aspects, the diffusion model has been fine-tuned by freezing at least one model parameter configured to determine the score offset.

[0023] In some aspects, the diffusion model has been trained by: directly training an untrained diffusion model to learn the constrained score function.

[0024] In some aspects, the diffusion model has been trained by: training a first diffusion model from an untrained diffusion model to learn the score function; andtraining the diffusion model from the first diffusion model to learn the constrained score function as the diffusion model.

[0025] In some aspects, the diffusion model has been conditioned using a target state representation.

[0026] In some aspects, the diffusion model has been conditioned on scene semantics via classifier guidance.

[0027] In some aspects, the predicting further comprises processing a target state representation and wherein the at least one unobserved state representation corresponds to a transition to the target state representation.

[0028] In some aspects, the at least one observed state representation and the at least one unobserved state representation correspond to a plurality of agents; and the plurality of agents are vehicles and the plurality of states corresponds to paths traveled by the vehicles.

[0029] In some aspects, the receiving comprises capturing sensor data comprising the at least one observed state representation at a vehicle.

[0030] In accordance with another aspect of the present disclosure, a method method for generating a state transition plan is disclosed, the method comprising: receiving at least one observed state representation comprising at least one observed vehicle state representation; predicting at least one unobserved state representation comprising at least one unobserved vehicle state from the at least one observed state representation using a diffusion model trained to model state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model, the diffusion model trained by constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function shifting state representation predictions towards a constrained distribution of state representations, and the predicting comprises processing the at least one observed state representation with the diffusion model to predict the at least one unobserved vehicle state representation from the at least one observed vehicle state representation.

[0031] In some aspects, the constraint function is a diffusion function expressed as a function comprising a continuously differentiable function and a diffusion gradient term for a state representation.

[0032] In some aspects, the constraint function comprises a plurality of constituent constraint functions.

[0033] In some aspects, the diffusion model has been trained to model state transitions by applying the constraint function at every sampling step of the diffusion model, each sampling step corresponding to an iterative step in the reverse diffusion process.

[0034] In some aspects, each of the at least one observed vehicle state representation and the at least one unobserved vehicle state representation comprises a position of a vehicle and a heading of the vehicle at a point in time.

[0035] In some aspects, each of the at least one observed state representation and the at least one observed state representation correspond to a transition between states.

[0036] In some aspects, the at least one observed state representation comprises at least one fixed constraint invariant between a time corresponding to the at least one observed state representation and a time corresponding to the at least one unobserved state representation.

[0037] In some aspects, the at least one fixed constraint is a feature map defining one or more constraints on a position defined by the at least one observed vehicle state representation and the at least one unobserved vehicle state representation.

[0038] In some aspects, the feature map defines one or more infraction zones; and the diffusion model has been trained by penalizing vehicle state representations predicted to be in the one or more infraction zones.

[0039] In some aspects, the feature map is a roadgraph defining one or more roads as non-infraction zones and non-road portions as the one or more infraction zones.

[0040] In some aspects, the feature map is a mesh map; each vehicle state representation is defined according to a bounding box; and the penalizing comprises penalizing the vehicle state representations based on a sum of a farthest distance of the bounding box to a nearest mesh of the non-infraction zones for all vehicle state representations over a sampling time.

[0041] In some aspects, the diffusion model has been trained by penalizing vehicle state representations predicted to be overlapping at the same sampling time.

[0042] In some aspects, each vehicle state representation is defined according to a bounding box; and the penalizing comprises penalizing the vehicle state representations based on a sum of overlapping areas of the bounding box for all vehicle state representations over a sampling time.

[0043] In some aspects, the constrained score function is a logarithmic gradient probability density function calculated according to a term corresponding to denoised predicted states, a term corresponding to the constraint function evaluated at the denoised predicted states, and noisy states.

[0044] In some aspects, the at least one unobserved vehicle state representation is a plurality of unobserved vehicle state representations for a plurality of vehicles; the plurality of unobserved vehicle state representations is modeled jointly; and a predicted vehicle state representation of each vehicle is predicted based on a subset of the plurality of vehicles according to distances thereto from nearest neighbors of each vehicle.

[0045] In some aspects, the predicted vehicle state representation of each vehicle is predicted in a respective local coordinate frame by the diffusion model, each of the respective local coordinate fixed to a respective vehicle; and the method further comprises transforming the predicted state into a global coordinate frame shared by all vehicles.

[0046] In some aspects, the at least one unobserved state representation comprises a control vehicle state representation in respect of one vehicle and one or more conditional vehicle state representations in respect of remaining vehicles and corresponding to the one or more observed vehicle state representations at a futuretime step; and the predicting comprises predicting the control vehicle state representation according to the one or more observed vehicle state representations at the future time step and the one or more conditional vehicle state representations.

[0047] In some aspects, the diffusion model comprises a transformer network comprising a plurality of layers configured to parameterize the constrained score function and a multi-layer perception network configured to encode input to the transformer network and to output the at least one unobserved state.

[0048] In some aspects, at least one of the plurality of layers correspond to a time transformer layer configured to parameterize time variables, an agent transformer layer configured to parameterize state variables, and a feature map transformer layer configured to fixed constraint variables.

[0049] In some aspects, the diffusion model has been trained using a plurality of training state representations comprising a plurality of observed training state representations and a plurality of unobserved training state representations generated by applying an observation mask to the plurality of training state representations, the diffusion model has been conditioned using multiple pluralities of training state representations, and each of the pluralities of training state representations is generated by using a different observation mask selected from the group of a predictive mask, a goal-condition mask, an agent reactive mask, an up-sampling mask, an imputation mask, a windowed mask, an agent-conditioned mask, and an ego-conditioned mask as the observation mask.

[0050] In some aspects, the diffusion model has been trained by: directly training an untrained diffusion model to learn the constrained score function.

[0051] In some aspects, the diffusion model has been trained by: training a first diffusion model from an untrained diffusion model to learn the score function; and training the diffusion model from the first diffusion model to learn the constrained score function as the diffusion model.

[0052] In some aspects, the diffusion model has been conditioned using a target state representation.

[0053] In some aspects, the diffusion model has been conditioned on scene semantics via classifier guidance.

[0054] In some aspects, the predicting further comprises processing a target vehicle state representation and wherein the at least one vehicle unobserved state representation corresponds to a transition to the target vehicle state representation.

[0055] In accordance with another aspect of the present disclosure, a method for training a diffusion model to generate a state transition plan is disclosed, the method comprising: obtaining a plurality of state representations, wherein the plurality of state representations comprises a plurality of observed state representations and a plurality of unobserved state representations; inputting to the diffusion model the plurality of state representations; and training the diffusion model to predict the plurality of unobserved state representations from the plurality of observed state representations by modelling state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model; the training comprising constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function, the constrained score function comprising a score offset based on an output of the constraint function.

[0056] In accordance with another aspect of the present disclosure, a method for training a diffusion model to generate a state transition plan is disclosed, the method comprising: obtaining a plurality of state representations, wherein the plurality of state representations comprises a plurality of observed vehicle state representations and a plurality of unobserved vehicle state representations; inputting to the diffusion model the plurality of state representations; and training the diffusion model to predict the plurality of unobserved vehicle state representations from the plurality of observed vehicle state representations by modelling state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model; the training comprises constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function shifting state representation predictions towards a constrained distribution of state representations.

[0057] In accordance with another aspect of the present disclosure, a system is disclosed, comprising one or more processing units configured to perform the method of any one of above aspects.

[0058] In accordance with another aspect of the present disclosure, a non-transitory computer-readable medium having computer readable instructions stored thereon is disclosed, which, when executed by one or more processing units, causes the one or more processing units to perform the method of any one of above aspects.BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Further features and advantages of the present disclosure will become apparent from the following detailed description, taken in combination with the appended drawings, in which:FIG. 1 depicts a system for performing state transition planning, according to an example embodiment.FIG. 2A depicts a method of training a machine learning model to perform state transition planning for the system of FIG. 1, according to an example embodiment.FIG. 2B depicts a method of using a machine learning model to perform state transition planning for the system of FIG. 1, according to an example embodiment.FIG. 3 depicts a representation of a machine learning model for performing state transition planning for the system of FIG. 1, according to an example embodiment.FIG. 4 depicts an architecture of the machine learning model of FIG. 3, according to an example embodiment.FIGS. 5A and 5B depict use of observation masks to generate observed and latent states, according to an example embodiment.FIGS. 6A and 6B depict coordinate frame systems for agent and state representation, according to an example embodiment.

[0060] It will be noted that throughout the appended drawings, like features are identified by like reference numerals.DETAILED DESCRIPTION

[0061] For the purpose of safe planning using machine learning, it is possible to assume that there is also a lower-level controller that can, provided a sequence of dynamically achievable, safe, states, map from those states into a sequence of actions that actually move an agent from one state to the next with near perfect fidelity. Although agent planning is described herein with references to autonomous driving, it should be noted that the present disclosure and agent planning is not limited to autonomous driving, which is provided only as an example. In the context of autonomous driving, an agent can be a vehicle (e.g. a car, truck, etc.), and safe planning can be the planning of a route from a starting position to an ending position for an agent. Planning of the route can be constrained by factors such as other agents (e.g. vehicles, bikes, pedestrians, etc.), movements thereof, environmental constraints (e.g. roads), and rules (e.g. traffic regulations).

[0062] Safe agent planning can be formulated as a “mid-level” safety-critical planning problem. The key features of this problem can include a dynamic world including reactive agents and controls (e.g. other drivers, pedestrians, traffic lights and signs, etc.), potential conflict between achieving a goal (e.g. a destination) and violating constraints (e.g. hitting another car or pedestrian, going offroad), etc. In the present disclosure, discrete time and a continuous state space and / or continuous perfect displacement action model can be assumed. A further assumption can be partial observability (e.g. not knowing goals of all agents) corresponding to a stochastic environment, and a finite-time planning horizon. In some implementations, a high quality model of the world may be utilized (e.g. defining the features of the traversed roads) including the actual generating distribution of the world. Note that it is also assumed that there is computational capacity to replan sufficiently quickly for model predictive control to be practical.

[0063] A wide variety of methods have been proposed to address the problem of trajectory forecasting, similar to agent planning on several levels. Two attributes which divide this area of work are the output representation type and the agents for which predictions are made. The most common class of models deterministically predict the distribution of ego agent trajectories using a weighted trajectory set either with or without uncertainties. Due to the applicability of this representation as the input for real-time self-driving planners, there are numerous prior methods of this type.Some approaches rasterize the scene into a birdview image and use convolutional neural networks to predict a discrete set of future trajectories for the ego agent. The convolutional architecture of these methods captures local information around the agent well, but the birdview image size and resolution limit the ability to capture high speed and long-range interactions. To address these challenges, other prior approaches encode agent states directly either by using recurrent neural networks, polyline encoders or 1D convolutions. Agent features can be combined with mapped world features such as roadgraph information in a variety of ways including graph convolutional networks or attention.

[0064] To control the distribution of predicted trajectories, several methods have utilized mode or goal conditioning. One approach is to directly predict several goal targets before regressing trajectories to those targets. An alternate approach is to condition on trajectory prototypes or latent embeddings. Predicting joint traffic scenes using per-agent marginal trajectory sets is challenging due to the exponential growth of trajectory combinations. Recent approaches aim to rectify this by producing joint weighted sets of trajectories for all agents in a scene. One such method generates joint trajectory sets by producing “reactor” trajectories which are conditioned on marginal “influencer” trajectories. Scene Transformer, as another example, uses a transformer network to jointly produce trajectory sets for all agents in the scene. As an alternative to deterministic predictions, multiple methods propose generative models of agent trajectories. A variety of generative model classes have been employed including Normalizing Flows, GANs or CVRNNs. Joint generative behavior models can either produce entire scenarios in one shot, or produce scenarios by autoregressively “rolling-out” agent trajectories.

[0065] Diffusion models can be utilized in agent planning. In one example, Diffuser uses diffusion models to generate trajectories for offline reinforcement learning tasks. They condition their samples using classifier guidance to achieve high rewards and satisfy constraints. Another approach utilizes diffusion planning for guided pedestrian motion planning. In the vehicle planning domain, Controllable Traffic Generation (CTG) builds on Diffuser, using diffusion models to generate trajectories which satisfy road rule constraints. In CTG, marginal per-agent trajectorysamples are combined into a joint scene representation by “rolling-out” a portion of each agent’s trajectory before drawing new samples per-agent.

[0066] Constraints on the sample space of generative models have been theoretically described. In such a framework, constraints are represented through a black-box oracle function labeling samples as valid or invalid, which has been practically explored for generative adversarial networks (GANs) and diffusion models. The main idea was to improve generative models by incorporating information from the constraints. Recognizing the expressivity of diffusion models, various approaches to incorporating pre-defined constrains into them have emerged in the literature. One such method proposed reflected diffusion models that enforce the whole diffusion sampling trajectory to remain bounded in a convex set. Another method extended reflected diffusion models to support more general constraints. The use of a barrier function based approach for constrained diffusion models and barrier functions to transform constrained domains into unconstrained dual ones were also proposed.

[0067] Diffusion bridges are stochastic processes that are guaranteed to end in a given constraint set and have been used in conjunction with diffusion models. One proposal includes a set of mathematically sufficient conditions for designing diffusion bridges to a given constraint set as well as using bridges to formulate diffusion models on discrete sets. The proposal also provided closed-form bridges for a restricted set of constraints such as product of intervals.

[0068] The present disclosure generally relates to the use of a machine learning model corresponding to a diffusion model to model or predict state transitions, for example in use for agent planning. The diffusion model can be constrained using one or more constraint functions that are similar to diffusion bridges, referred to herein as manual diffusion bridges. The states of agents such as the position and heading thereof, for example corresponding to the movement of the agents, can be modelled by the diffusion model. In particular, the diffusion model can predict, from observed or known states, the unobserved states (referred to herein as latent states), particularly of agents, for example corresponding to states of agents at some point in the future. The diffusion model can comprise a transformer-based neural network that is trained to learn a score function used to perform sampling as to model the agent states over time for predicting agent states. To learn the scorefunction, the neural network can be trained to learn a modified score function where the modification corresponds to the constraint function(s). The score function can also be offset by a term corresponding to the constraint function(s). Once trained, the diffusion model can be used at inference to perform agent planning. As an example, the diffusion model can be implemented for the electronic control unit (ECU) of a vehicle to perform autonomous driving.

[0069] In contrast to models which are conditioned exclusively on the past states of other agents and the map, and models which use classifier-guidance to condition their samples to follow road rules, the disclosed diffusion model can be conditioned on scene semantics via classifier guidance as well as conditioned on arbitrary state observations, including the past or future states of each agent. Further, it is possible to control the strength of conditioning using classifier-free guidance, for example by using one or more observation masks. Another advantage of the present disclosure is that the disclosed diffusion model can avoid generating invalid samples by design. Moreover, many conventional approaches are only evaluated on lowdimensional problems with simple constraints, for example, by only supporting convex constraints. In comparison, the disclosed diffusion model can be applied to higher dimensional problems that includes a plurality of more complex constraints. Furthermore, prior models that uses diffusion bridges, such as closed form bridges, quickly become intractable as the constraint set gets more complex, which can be avoided with the use of the disclosed diffusion model. Additionally, other methods such as reflected diffusion makes the forward process, and consequently training, expensive.

[0070] Embodiments are described below, by way of example only, with reference to Figs. 1-6B.

[0071] FIG. 1 depicts a system for performing agent planning, according to an example embodiment, shown in FIG. 1 as one or more servers 108. The implementation of the servers 108 is not restrictive and servers 108 may be a physical server, cloud-based server, ora hybrid thereof, for example. A user 102 may interact with the servers 108 via a device 104 over a communications network 106 (e.g. the internet). The device 104 may be a computer, as depicted in FIG. 1, but is not restricted to those expressly shown and may be any suitable device known in the artsuch as smart phones and tablets. The servers 108 may provide a graphical user interface (GUI) (e.g. on the device 104) for ease of communication and operation control by the user. The implementation of the GUI is not restrictive and may be, for example, a mobile / computer application or a web page. The GUI can be used to provide input to and receive output from the servers 108. Additionally or alternatively, other user interfaces, such as an audio interface that allows receipt and processing of spoken commands may be used.

[0072] The user 102 may be interested in training a diffusion model to perform agent planning via the servers 108, as described further herein. The diffusion model may be implemented as one or more machine learning models 126. The diffusion model may be trained to generate agent plans 130, for example corresponding to predicted states of agents. To train the diffusion model, the user 102 can provide the servers 108 (e.g. over the communications network 106) with training data 122 comprising a plurality of states, for example corresponding a plurality of agents. The plurality of states may comprise observed states and latent states. The diffusion model is trained to predict the latent states from the observed states by learning a score function modified by one or more constraint functions, as disclosed herein. The machine learning models 126 may comprise or each be an artificial intelligence (AI) model or algorithm, a machine learning model or algorithm, and may comprise, in particular, at least a transformer network and a multilayer perception (MLP) network.

[0073] According to the present disclosure, the training data 122 may be provided to or retrieved by the servers 108, for example, from the device 104. The training data 122 can also be retrieved from one or more external devices and / or one or more databases. The training data 122 may be requested and received using an application programing interface (API) via requests / calls and responses, for example over the communications network 106, although other forms of communication such as Bluetooth and near-field communication are possible as well.

[0074] It should be noted that while FIG. 1 depicts the device 104 and the servers 108 as separate entities coupled over the communication network 106, the device 104 and servers 108 may also be coupled directly / physically using cable(s) for data transfer. In some embodiments, the servers 108 may also be the device 104 or comprise the device 104 (e.g. the servers 108 being implemented as a part of acomputer system). In such an embodiment, the servers 108 may directly retrieve the training data 122 (as well as any other required data) from fixed local storage or removable local storage.

[0075] In some embodiments, the servers 108 can be implemented as a controller for a vehicle 124, for example as an electronic control unit (ECU) of the vehicle 124. The vehicle can comprise or be coupled to one or more sensors such as cameras or Light Detection and Ranging (LiDAR) configured to capture sensor data 128. The sensor data 128 can comprise state data corresponding to observed states of other agents (e.g. vehicles). The ECU can receive and process the sensor data 128, map data (e.g. GPS map data or map models) as well as control the operations (e.g. perform autonomous driving) of the vehicle 124. The one or more machine learning models 126 stored on the servers 108 may be trained, as described above, and used at inference to generate agent plans 130. The agent plans 130 can be provided (e.g. displayed) to a driver of the vehicle 124, for example the user 102. The servers 108 can also utilize the agent plans 130, for example corresponding to predicted movement paths of other vehicles, to autonomously operate the vehicle 124. In particular, the servers 108 can operate the vehicle 124 to travel to a desired destination (e.g. chosen by the user 102) while avoiding traffic violations and other vehicles using the agent plans 130.

[0076] In a particular implementation, the servers 108 each comprise a CPU 110, a non-transitory computer-readable memory 112, a non-volatile storage 114, an input / output interface 116, and graphical processing units (“GPU”) 118. The non-transitory computer-readable memory 112 comprises computer-executable instructions stored thereon at runtime which, when executed by the CPU 110, configure the server to perform the above described processes of sustainability analysis. The non-volatile storage 114 has stored on it computer-executable instructions that are loaded into the non-transitory computer-readable memory 112 at runtime. The input / output interface 116 allows the server to communicate with one or more external devices such as the device 104 (e.g. via network 106). The non-transitory computer-readable memory 112 may also have stored thereon the one or more machine learning models 126. The GPU 118 may be used to control a display for interacting with the user 102 and may be used to operate the machine learningmodels 126, including for example to train and to perform inference using the machine learning models 126, as described further herein. The CPU 110 and the GPU 118 can also be used to process data such as the training data 122 and the sensor data 128. In some embodiments, the neural networks 126 may be stored at one or more separate servers. The servers 108 and the device 104 may each provide a communications interface which allows software and data to be transferred, for example between the servers 108 and the device 104 over the communications network 106.

[0077] The CPU 110 and GPU 118 may be one or more processors or microprocessors, which are examples of suitable processing units, which may additional or alternatively comprise an artificial intelligence accelerator, programmable logic controller, a microcontroller (which comprises both a processing unit and a non-transitory computer readable medium), Al accelerator, neural processing unit (NPU), or system-on-a-chip (SoC). As an alternative to an implementation that relies on processor-executed computer program code, a hardware-based implementation may be used. For example, an application-specific integrated circuit (ASIC), field programmable gate array (FPGA), or other suitable type of hardware implementation may be used as an alternative to or to supplement an implementation that relies primarily on a processor executing computer program code stored on a computer medium.

[0078] FIG. 2A depicts a method for training a diffusion model corresponding to the one or more machine learning models 126 to perform state transition predictions such as agent planning. At 202, training data 122 is obtained. The training data 122 can comprise state data corresponding to a plurality of states for each of a plurality of agents. As used herein, each state can be a state representation. In particular, each state can define a position (e.g. a coordinate in x, y) and a heading (e.g. a vector representing a direction / speed of movement) of an agent at a point in time. Alternatively or additionally, each state can define a transition between two states (e.g. the action taken that would lead to a change from A to B). in particular, the states can be derived from the movement of the agents. For example, in a case where each agent is a vehicle, the states of an agent can correspond to the path traveled by the vehicle over a period of time where each state can be a “snapshot” of the vehicle at aparticular point in time. Alternatively, the states can be a velocity, direction, and acceleration at a particular point in time (e.g. as to travel from point A to point B during the sample time interval). The plurality of states can comprise or be separated into observed states and latent states. The latent states can correspond to states that are not known or unobserved to the model (e.g. states of the agents in the future). The goal is to train the diffusion model to model the states such that it can predict the latent states from the observed states. To separate the states, an observation mask can be applied, as described further herein. For example, according to the observation mask, states belonging to a particular period or periods of time can be selected as the observed states (e.g. start portion of the overall time period, start and end portions of the overall time period, start and one or more middle portions of the overall time period, etc.), with the rest being the latent states. Different observation masks can be applied to the same set of training data to condition the diffusion model using different sets of observed and latent states.

[0079] In addition to the state data, the diffusion model can also be provided with a set of fixed constraints that are invariant over the sample or scenario time. In particular, the set of fixed constraints comprise a feature map constraining the positions of the agents. For example, the feature map can be a map or model of the road system and as such defines positions that are acceptable for the agent to be present in (e.g. the roads) and infraction zones that agents should not be present in (e.g. sidewalks or buildings).

[0080] At 204, the training data 122 is input to the diffusion model fortraining. A diffusion model is a type of generative machine learning model that generates data, in this case the agent plan 130, by simulating a noising and denoising process. The noising process corresponds to a forward diffusion process modeled as a stochastic differential equation (SDE) and the denoising process corresponds to a reverse process for reversing the added noise. To remove the noise, the diffusion model learns or approximates a score function, which is a gradient of the log-probability of the data, in this case the state data. The score function can be used to iteratively refine the noisy data in the reverse diffusion process. By applying the score function, the diffusion model can generate predicted states based on the observed states.

[0081] In accordance with the present disclosure, the diffusion model can be constrained using at least one constraint function (206). The constraint function can be a diffusion function that operate similar to a diffusion bridge. Although various differences exist between the constraint function and a diffusion bridge, the constraint function is referred to herein as a diffusion bridge for simplicity sake. The constraint function can comprise a plurality of constraining functions (e.g. a combination / union of functions). A diffusion bridge can define or model a start state and an end state over a time interval. By constraining the diffusion model using a diffusion bridge, data can be “guided” towards the end state. For example, a diffusion bridge can be used to model a starting and an ending state in the reverse diffusion process. One or more diffusion bridges can be applied at every sampling step of the diffusion model, each corresponding to an iterative step in the reverse diffusion process. Note that a single diffusion bridge be used to model a transition between states for all agents (e.g. at a sampling time step). Alternatively, each agent or group of agents can be modelled with a respective diffusion bridge. The at least one diffusion bridge assist the diffusion model in learning the score function (208). In particular, the diffusion model may be trained to learn a score function that is modified / constrained by the at least one diffusion bridge, where the score function may comprise a term or a score offset that is based on the at least one diffusion bridge (e.g. based on an output of the at least one diffusion bridge). More specifically, the score function can be a logarithmic gradient probability density function based on a term corresponding to denoised predicted states, a term corresponding to diffusion bridges at the denoised predicted states, and noisy states.

[0082] The diffusion model can learn the score function by minimizing loss between the states predicted according to the score function and the latent states (208). In particular, a transformer network comprising a part of the diffusion model can be configured to parameterize and learn the score function. A MLP or similar network comprising a part of the diffusion model can be configured to encode inputs to the diffusion model as vectors and to transform output vectors into predicted states that are used to generate the agent plan 130 at 210. In some embodiments, the diffusion model can be trained to jointly model the predicted states of a plurality of agents. In such cases, for each modeled agent, only a number of nearest neighbors or neighbors within a set distance are considered. Each agent may be modeled in itsown respective coordinate frame, which can be transformed into a global coordinate frame. To train the diffusion model the predict the latent states, the diffusion model can also use a loss function that models an infraction zone loss (e.g. offroad loss) which penalizes states predicted to be in the infraction zones as well as a loss function that models an overlapping loss (e.g. collision loss) which penalizes states predicted to be overlapping with one another at the same sampling time. To fine-tune the diffusion, one or more model parameters for determining the score offset can be frozen during training.

[0083] The diffusion model can also be conditioned on targets / goals. For example, the diffusion model can be conditioned on a target corresponding to a position / coordinate (e.g. at a particular sample time ortime(s)) during training, which a predicted state should match or correspond to. The target can also be included as an observed state that the predicted state should be modelled towards.

[0084] FIG. 2B depicts a method for using a diffusion model trained using the method of FIG. 2A to predict state transitions, for example corresponding to agent planning. At inference, state data is received at 220. The state data can be received for one or more agents corresponding to the plurality of observed states. The state data can correspond to sensor data 128. For example, by monitoring and tracking the position and movement (e.g. path taken) of the agents as sensor data 128, the observed states for the agents can be determined. State data can also comprise a set of fixed constraints that are invariant over time. In some implementations, only the fixed constraints (e.g. a feature map) is received. That is, the diffusion model may predict unobserved states (e.g. for agents) without first having an observed state. This type of inference can be referred to as initialize as the diffusion model is used to predict the initial state(s) of agents from only a set of fixed constraints. At 222, the state data (e.g. observed states) is input to the diffusion model for processing. By applying the learned score function on the state data at 224, the diffusion model can predict the plurality of latent states for generating an agent plan 130. For example, the score function can be used to guide the data sampling to generate the predicted states. In the context of autonomous driving, the agent plan 130 can be a course / path for the vehicle to travel in.

[0085] In some implementations, each of the unobserved states can be classified as either a control state or a conditional state. The control state can be the unobserved state of an “ego”-agent. For example, the vehicle 124 for which autonomous driving is performed. The conditional state(s) can refer to the remaining states (e.g. corresponding to other vehicles), based upon which the control state may be predicted. Note that as time progresses (e.g. as the other vehicles move in real time), further observed states of agents corresponding to the conditional states can be captured, which can be compared with the determined conditional states to better adjust the predicted control state for the “ego”-agent.

[0086] In some implementations, a target state can be provided to the diffusion model, for example corresponding to a desired or goal state for the “ego”-agent. Accordingly, the agent plan can be a sequence of predicted states that would reach the target state. As such, the diffusion model can predict a sequence of unobserved state that would transition from the last observed state to the target state.

[0087] Note that while the diffusion model and use thereof is described with respect to state transitions and agent planning. The diffusion model as disclosed herein can be applied to other diffusion-based applications as well. For example, the diffusion model can predict the change in pressure of a reactor, perform generative image creation, and model movements of celestial bodies.

[0088] Further details with regard to the methods of FIG. 2A and 2B are described below.Diffusion Model

[0089] Broadly, for a problem of generating an agent plan (e.g. agent planning), one can consider a scenario with A agents across T discrete timesteps (e.g. time samples) distributed on a feature map (e.g. map features) M constraining the positions of the agents. More generally, given a set of known parameters, a state vector can be generated. The state vector can comprise a fixed portion (e.g. corresponding to fixed constraints that are invariant such as the feature map), a dynamic but controlled portion (e.g. corresponding to an agent that can be directly controlled) as well as a dynamic and uncontrolled portion (e.g. uncontrollable agentssuch as other vehicles as well as dynamic factors such as traffic lights). A state transition plan such as an agent plan can aim to generate a series of states for the controlled portion (e.g. a safe path for the vehicle to travel) but can also predict and model the uncontrolled portion (e.g. predict the path of other cars). The disclosure herein is described with respect to a traffic scenario where agents can be vehicles and the feature map can be a set of roadgraph features corresponding to a road. Note that the traffic scenario is only provided as an example and other applications such as chemical plants, industrial processes, etc. can also be modelled using the disclosed diffusion model. Each agent a e {1..., A} in the scene at time T E {1,... T can be represented by a state s“ =0“} comprising of its 2D position (xT,yT) and heading 9T. The joint representation of the scene x is the combination of all agents across all timesteps x = {s“|a E {1, E {1, E IR^*7’*3. The scene can be assumed to be distributed according to an unknown distribution pdata(x). The joint representation of the scene x can correspond to the training data 122.

[0090] The diffusion model as described herein can be conditioned on the map features M and can moreover be flexibly conditioned on arbitrary set of observed agent states. For the conditioning of observed agent states, consider a boolean variable O E {0, 1}AXT, where a state in the scene is observedif = 1. Using O, it is possible partition the scene into two components. The observed portion of the scene is defined as xobs= {s“|s“ e x,0“ = 1} while the unobserved, latent portion is xlat= x\xobs. That is, O can be an observation mask configured to separate the state data in the training data 122 into observed states and latent states.

[0091] The goal of the diffusion model is to learn a conditional distribution over the set of all latent agent states xiatgiven the observed states xobsand the feature map M, by modeling p(x(at|xobs, M). Using this probabilistic framework, conditional distributions corresponding to various trajectory forecasting tasks can be represented by modifying the observation mask O and the corresponding conditioning set xobs, generated using the observation mask.

[0092] Example observation masks are shown in FIGS. 5A and 5B. 502a-502e of FIG. 5A depicts agent states being partitioned into observed states 506 and latent states 508 using various observation masks. The observation mask is used tocondition the training data 122, input to the diffusion model. Each element is an agent state with a mask corresponds to the boolean value of 0 for that agent state (where observation mask of 0 is unobserved and 1 is observed). In 502a-502e, individual agents are shown in rows, with timesteps as columns. 502a-502e respectively correspond to the application of predictive, goal-conditioning, agent-reactive, up-sampling, and imputation masks to the training data 122 and 504a-504e respectively correspond to the traffic scene corresponding to the training data 122 in 502a-502e. In 504a-504e, agents are vehicles distributed on a section of a road. Paths traversed by the agents corresponding to the observed and latent states are respectively denoted as 506 and 508. FIG. 5B depicts further examples of traffic scene determined from the training data 122, where scenes 504a-504d respectively correspond to the application of predictive, goal-conditioning, agent-reactive, and up-sampling observation masks.

[0093] The diffusion model can be trained using a variety of observation masks 0 by randomly drawing masks from a training distribution p(O). The observation masks can include: predictive, goal-conditioning, agent-reactive, ego-conditioned, windowed, up-sampling, and imputation observation masks. Table 1 outlines an example training task distribution corresponding to the distribution of observation masks. The length of agent state history observation for each dataset is denoted as tobsand the total number of timesteps is denoted as T. 11 indicates a uniform distribution over integers. The training observation mask 0 can be sampled from this distribution with probabilities given in the rightmost column.T able 1: T ask distribution for diffusion model training.Task Description ProbabilityPredictive Observe states where t e [0, tobs]. 50%Goal- Observe states where t e [0, tobs] and the final state of 25% Conditioned 3 random agents.Agent- Observe states where t e [0, tobs] and the entire 10% Conditioned trajectory of 3 random agents.Ego-Conditioned Observe states where t e [0, tobs] and the entire ego10%agent trajectory.Windowed Observe states where t e [0, tstart] and t e (tstart+ 5%tobsi T] where tstart ~ 1 / (0, toj,s).Up-sampling Observe every tobs / T states, starting from tstart~ 5% u(o,tobs / r).Imputation Randomly sample observing each state with probability 5%tobs / T ■

[0094] Diffusion models are a class of generative models that learns to invert a stochastic process, known as the “forward process” that gradually adds noise to samples from a data distribution q0(x0), as described above. The forward process is formulated as an SDE:dxt= f(t; t)dt + g(t)dw, x0~ q0, (1 )where f and g respectively correspond to pre-defined drift and diffusion functions and w is the standard Wiener process, and t, the “diffusion time”, is a variable that parametrizes the process. Since there are multiple notions of time, as used herein, T denotes the “scenario time” (e.g. corresponding to time sampling steps and time values tracking the motions of the agent over the observation time) and t to represent this “diffusion time” (e.g. corresponding to time sampling steps in the diffusion process). The forward process is designed such that the SDE’s solution at time T is QT(XT) « 7r(xr) for some known n can represent a noise distribution typically equal to J\T(O, I).

[0095] Path measure on the continuous trajectories following the forward process in equation (1) is identical to the one governed by the following time-reversed SDE:dxt= [f(xt; t) - £2(t)Vxlogqt(xt)] dt + g(t) dw, (2)where xT~ n and w is the Wiener process when time flows backwards. Diffusion models can learn a score function se(xt; t) « Vxlogqt(xt) (e.g. at 208) and approximate the reverse process in equation (2) by using the expression:dxt= [f(xt; t) - g2(t)sg(xt; t)] dt + g(t) dw, (3)where xT~ n(xT'). The diffusion model can learn this score function by minimizing the below expected value:Et,x0,xtA(t) II s0(xt; t) - Vxlogq(xt|x0) ||2, (4)where x0~ q0and xt~ qt- |x0) in the expectation and A: [0, T] -» IR+is a weighting function. Once trained, the diffusion model can generate data corresponding to predicted states by sampling xT~ n and simulating the approximated reverse process in equation (3) using the learned score function.

[0096] As used herein, q and pgrespectively denote the probability density function of the forward and reverse process; Pgand Q denote the probability mass functions associated with p and q. These notations apply to the marginals, conditionals, and posteriors as well.

[0097] The diffusion model can model the joint distribution agent states p(xiat\xobs> M). In particular, the diffusion model can directly predict the unobserved states of each agent in the scene xlatgiven xobsand M. The choice of the observation mask O and observation set xobson which the diffusion model is conditioned can affect model performance in generating quality agent plans. As such, a distribution over observation masks p(O) which controls the tasks on which the diffusion model is trained can be used, as described above.

[0098] In the diffusion process, consider f(x.iat, t) = 0 and g(t) = y / 2t. A score function can be parameterized in the form of:Vx,„tlogp(x,„ |x„hs, M, c) =(5)where Dgdenote a neural network which approximates the latent portion of the noise free data x(at 0(e.g. posterior mean), which can be used to approximate the score function. In addition to x(at tand t, the neural network Dgalso receives the feature map context in the form of M, the clean observed states xobsand c, corresponding to a collection of unmodelled agent features per observed agent timestep such asvelocity, vehicle size, or agent type. An objective function of the neural network Dgcan be expressed as:Exo,t,o,xiat,tII Dd(xlat)t, xobs, M, c, t) - xlati0|| \, (6)where, x0~ pdata(x), xt~ p(xt|x0) = N (x, < J2(t)I) and 0 ~ p(O). The loss for the neural network Dgcan be computed over (t) ~ ptrain, which can correspond to a log normal distribution which controls the variance of the noise added to the data (e.g. observed states).

[0099] Considering the problem of “safe” planning as a problem of constrained generative modeling, a safe agent plan satisfies constraints that agents do not collide (e.g. overlap at the same sampling time) and do not drive off road (e.g. are not predicted to be in positions defined as outside of the road by the feature map). More precisely, constrained generative modeling can tackle the problem of learning and generating from a distribution within a bounded domain £1 c iRd. This constrained domain £1 can either be described by explicit constraints such as a set of linear inequalities, or implicitly such as via binary functions taking x e IRdas inputs and indicating whether the constraints are satisfied. The training data 122 in such a problem should satisfy the given constraints. Formally, the dataset D = {xj,}"tfollows a data distribution q(x), with a probability mass function Q such that Q(x e £1) = 1 for x ~ q(x). The goal of constrained generative modeling is to approximate q using a distribution pgwith the support being bounded to £1. Given probability mass function P0forpo, a maximum likelihood estimation objective for this problem is formulated as:arg mi0KL(( / | \p0) s.t. Pg(x E £1) = 1. (7)

[0100] With regard to preconditioning, the score function approximation neural network Dgcan be preconditioned by combining xiat;tand the output of the overall diffusion model Fgusing scaling factors expressed as Dg(xiatt,xobs, M, c, t) = cSfcip(t)X(at,t + cOut( ^( nX(att,xobs, M,c,cnOiSe(t)). Table 2 shows example scaling values for preconditioning the diffusion model. Note that adatacan be the standard deviation of the diffusion features. As an example, the positions and headings of the agent can be scaled such that adatais 0.5 for all diffusion features.Table 2: Scaling Functions for PreconditioningScaling Factor Functioncskip °datal (F + °data)cout’ ^datal + ®dataxcin1 / X t2+ °datacnoise >(t)Model Architecture

[0101] The diffusion model described herein can be a transformer based model. FIG. 4 depicts an example architecture for the diffusion model. The diffusion model can comprise MLP networks 426a-426c and a transformer network 402. Note that the MLP networks 426a-426c may also be implemented as a single MLP network. The score function approximation neural network Dgcan be parameterized by a transformer-based architecture corresponding to the transformer network 402. That is, the transformer network 402 is configured for learning the score function to predict agent states. The transformer network 402 can operate on a fixed [A, T, F] shaped feature tensor composed of one F dimensional feature vector per agent timestep. Sinusoidal positional embeddings can be used to produce initial feature tensors. Noisy and observed agent states xt, xobs, the time indices T = {1,and diffusion step t can all be embedded into F dimensional embeddings. xiat;tand xobscan be padded with zeros for observed and latent states respectively prior to embedding. A shared MLP corresponding to the MLP network 426a can project the concatenated positional embeddings into a F dimensional vector for each agent.

[0102] The main trunk of the diffusion model, corresponding to the transformer network 402, can comprise a series of transformer layers. The transformer layers can comprise one or more time layers 404, one or more agent layers 406, and one or more feature map (e.g. roadgraph) layers 408. The transformer layers 404, 406, and 408 are respectively configured for the parameterization of the time, agent, and feature map variables / features. Attention between all pairs of feature vectors can be factorized into alternating time (404) and agent (406) transformer layers. In timetransformer layers 404, self-attention can be performed per-agent across each timestep of that agent’s trajectory, allowing for temporal consistency along a trajectory. In agent transformer layers 406, self-attention can be computed across all agents at a given time, updating each agent’s features with information about the other agents at that time. Encoding for the feature map transformer layers 408 is described with further detail herein. Cross attention between the collection of lane embeddings and agent states can incorporate feature map information into the agent state features. A MLP decoder, corresponding to the MLP network 426b or a separate MLP layer, can be used after the final transformer layer to produce the estimated (denoised) latent states xtat 0for generating the agent plan 130.

[0103] Referring to FIG. 4, features can be encoded using the MLP network 426a and positional encodings 424 for input into the transformer network 402. The observed states xobs(416), the noisy latent states x(atT(418), the scenario time t (420), and the diffusion time T (422) can be encoded using sinusoidal positional encoding 424. The unmodelled agent features c (412), corresponding to the observed states 416 can also be encoded by the positional encoding 424 for input. Paddings 414 can be added to the observed states 416, latent states 418, and agent features 412, as described above, where the paddings 414 for the agent features 412 and the observed states 416 may be the same. The scenario time 420 can be represented as an integer index increasing from 0 to T with 0 corresponding to the earliest agent states. For each of the encoded features, a 256-dimensional encoding vector can be produced. Hyperparameters for sinusoidal positional embeddings can include the maximum and minimum encoding periods. Table 3 shows example maximum and minimum encoding periods.Table 2: Maximum and minimum positional encoding periods for input featuresFeature Minimum Period Maximum PeriodX-obsi ^-lat, T 0.01 10t 1 100T 0.1 10,000

[0104] The concatenation of the positional encodings with the additional agent state features c (412) can be fed through the MLP network 426a to form the input to the main transformer network 402. The additional agent state features can comprise the agent velocity, the observed mask, and the agent size, as described above. The MLP network 426a used for input can be shared across all agent states and can contain two linear layers with hidden dimension 256 and ReLU non-linearities.

[0105] In a particular implementation, the diffusion model can be conditioned on the geometry of the feature map (e.g. roadgraph) through a collection of lane center polylines to encode the feature map 410. For example, each polyline can comprise an ordered series of 2D points which represent the approximate center of each section of non-infraction zones (e.g. each driving lane). The length of each polyline can be fixed to 10 points. Polylines longer than this threshold can be split into approximately equal segments, while shorter polylines can be padded with zeros. A Boolean feature can be used to indicate which polyline points are padded. While PointNet has been used for similar encodings, FIG. 4 depicts the use of a simple MLP network 426c to encode the polylines into a 256-dimensional vector per polyline as another example. To generate the input to the MLP network 426c, the position and padding mask for each polyline can be concatenated, along with any additional perpolyline features present in the dataset. As an example, the MLP network 426c can comprise four linear layers, with a hidden dimensionality of 256 and ReLU nonlinearities.

[0106] In the example model architecture shown in FIG. 4, transformer network 402 comprises 15 transformer layers which can perform self-attention over the time and agent dimension, and cross attention with the feature map encodings. The transformer layers can be analogous to those described by Ngiam et al. (“Scene transformer: A unified architecture for predicting future trajectories of multiple agents”) and where the number of layers and their ordering can be modified. Specifically, additional time transformer layers 404 can be added to produce smoother trajectories. All attention layers can consume and produce 256 dimensional features per-agent state. Four heads can be used for each attention operation, and a 1024 dimensional hidden state can be used in the feed forward network. In the transformer layers, thepre-layernorm structure described by Wu et al. (“Diffusion-based molecule generation with informative prior bridges”) can be used.

[0107] In some cases, due to batching and agents which are not tracked for the duration of the observed scene, there can padding present in the agent feature tensor. The transformer layers can account for padding in the scene by modifying the attention masks so that padded agent states are not attended to.

[0108] The MLP network 426b can comprise a two layer MLP with hidden dimension 256 to produce the final output corresponding to the predicted agent states. The produced vector per agent state may be two, three, or higher dimensions, according to input data, where the x coordinate, y, coordinate, and heading can each represent a dimension.Diffusion Bridges for Constraining the Diffusion Model

[0109] Diffusion bridges for constrained generative modeling is a generalized framework of Brownian bridge processes and reguires the constraint boundaries to be explicitly stated. For a constraint set fl c iRd, a function ®-Q(xt, t) defined on IRdx IR+is an fl-bridge (e.g. diffusion bridge) for the reverse process in eguation (3) if the solutions ofdxt= [v0(xt; t) + ^2(t)®-Q(xt; t)]dt + g(t)dw, (8)at final time t = 0 are guaranteed to be in fl. Here xT~ n and v0(xt; t): = -f(xtt) + g(t)2se(xt; t). Note that the extra injected term ®n(xt; t) is guantified through external constraint functions, which drifts the particle to move towards the boundary and stay inside fl. Further, eguation (6) is a diffusion bridge if the regularized condition is satisfied: Et tf IIVe^t)II2dt < + oo where xtis sampled from this SDE in eguation (8), and t ~ [0, T], However, there can be some strong reguirements on ®n(xt; t), making it practically applicable only to a very limited set of problems, which can be overcome by means of the present disclosure, as described below.

[0110] “Manually Bridged Models” (MBM) as described herein, and as used herein refers to constrained generative modeling with manual diffusion bridges, in particular a diffusion model modified / constrained with at least one diffusion bridge. Asused herein, a diffusion bridge refers to a manual diffusion bridge, unless otherwise specified. That is, the diffusion model can be constrained using at least one (manual) diffusion bridge at 206. Again note that the manual diffusion bridge is a constraint function that can correspond to a diffusion function and may not necessarily be a diffusion bridge, as shown herein. MBM can incorporate the complex constraint information into the diffusion model leading to a formulation similar to diffusion bridges. Such bridged models can parameterize a family of sequence of distributions that only place mass on £1 at diffusion time t = 0. Note that training of the modified diffusion model is analogous to that of the general diffusion models (e.g. as described above), and can, for example, be trained using the same objective as standard diffusion models.

[0111] Definition 1 (.^-distance function): Let ℓΩ: ℝd× [0, T] → ℝ≥0be a continuous and almost everywhere differentiable function with finite gradients w.r.t. x. We call an £2-distance functionif 0) = 0 iff x e Q.

[0112] Definition 2 (Manually bridged model): Given a diffusion model having a score function denoted as se(xt, t), an £2-distance functionand a C1-function y: [0, T] -» IR+such that γ(T) ≈ 0 and limt↓0γ(t) = ∞, a manual diffusion bridge can be defined ast): = -y t V^; t). Accordingly, a manually bridged model can have a score function defined ass0(x; t, y,f)-. = s0(x; t) + bn(x; t). (9)

[0113] Note that the added manual bridged term shifts the distribution towards £1. A score function of a diffusion model modified as a manually bridged model can correspond to distributions of the form p-Q(x; t) <x p(x; t)exp(-y(tn(x; t)). Since y smoothly changes from zero at t = T to infinity at t = 0, p-Q(x; t) smoothly interpolates between pn(x; T) = p(x; T) at t = T and pn(x; 0) <x p(x; 0)n(x).

[0114] Proposition 1: Let se(x,t) be a score function corresponding to a density pe(x,t). If se(x,t) is continuous in t and pe(x,t) is finite for x g £1, then the manually bridged model defined in Definition 2 defines a sequence of distributions that only place mass on £1 at time t = 0.

[0115] Proposition 2 (Combining Diffusion Bridges): Let bi(x; t) =t) and bn2(%; t) = -r2(0vx^fi2(^; 0 be two manual diffusion bridges as defined in Definition 2. If -. = c l2= 0, the combined bridge b^(x; t) = b*21(x t) + b^2(x t) is also a manual diffusion bridge ton 122■ Therefore, the space of manual bridges is closed under addition. The proof for this proposition is shown below.

[0116] Without loss of generality, assume limt↓0γ₂(t) / γ₁(t) ≠ 0. Let ^-Q(x;t): =71(-Q,(x; t) + ^-^2(x; tY Since all functions are continuous in t, bothand ^2are distance functions and limt↓0γ₂(t) / γ₁(t) ≠ 0, / “(x; 0) is zero if and only if both ^-Q>(x; 0) and^-Q2(X; 0) are zero. Therefore,is a distance function to 0. Further, by definition limt↓0γ₁(t) = ∞. Therefore, —y1(t)Vx-^n(x; t) = b“Q1(x; t) + bn2(x; t) = bn(x; t) is a manual diffusion bridge to 0.

[0117] Generalizing Proposition 1, one can combine multiple bridges following the expression bn^{nT = ^=1bp(x; t).

[0118] The objective function of a diffusion model constrained by at least one diffusion bridge at 206 is the denoising loss with a particular parameterization of the score estimator model, and can be expressed as:Et,xo,xtT(t) II Sp(xt; t, Y,^ ~ vxlogq(xt|x0) ||2(10)where x0~ qo,xt~ qt- |x0) in the expectation. That is, the diffusion model can be trained according to the objective function of equation (10) to learn the score function at 208.

[0119] Note that the constrained diffusion model (MBM) can give rise to reverse SDEs similar to that of diffusion bridges equation (6). The following expression can also be derived by plugging in the score function for the MBM into equation (4):dxt= [— v0(xt; t) - ^2(t)b-Q(xt; t)] dt + ^(t) dw, (11)where v0(xtt) = -f(xt; t) + g2t)s0(xtt). This is equivalent to equation (8) once ®n(xt; t) = b-Q(xt; t). However, in order to guarantee that solutions of this SDE lie in £1, the diffusion bridge function b-Q(x, t) = -y(t)Vxl'-Q(x; t) may be required to satisfy requirements such as an expected Polyak-Lojasiewicz condition: EtXt~pn^n(xt; t) < o II Vx^n(xt; t) II2. This can restrict the set of allowed fl-distance functions. Furthermore, this can restrict the generality of combination of bridges. As such, diffusion models constrained by at least one diffusion bridge (e.g. in the case of the MBM) may not be guaranteed to have SDE solutions in fl. However, as shown above, the disclosed diffusion model can still have a score function corresponding to distributions that converge to one constrained to fl. Since disclosed diffusion model is trained to approximate the reverse process in equation (2), an SDE with a trained model is likely to converge to this terminal distribution as well. Further, manual diffusion bridges can empirically perform similarly to diffusion bridges.Score Function and Loss Guidance

[0120] Traditional methods for utilizing diffusion bridges by directly incorporating diffusion bridges, for example into equation (7) can lead to poor training. This is because the added diffusion bridge term in the loss function can increase variance of the loss.

[0121] FIG. 3 depicts various methods of incorporating at least one diffusion bridge into the diffusion model and the corresponding modifications to the score function. In each of the methods 320a-320d, a general diffusion model 306 takes as input state data 302 over a sample time period 304 to learn to approximate a score function for predicting latent states.

[0122] 320a depicts an unmodified conventional diffusion model that learns to approximate a general score function 312 using the input training state data 302 over the sample time 304. For example, the score function 312 can correspond to equation (5).

[0123] 320b depicts the diffusion model being modified by a diffusion bridge 308, referred to as Conditional diffusion (C-arch). In this method, the diffusion model 308, in particular the score function approximation neural network (e.g. Dg)~ can takethe diffusion bridge 308 as an additional input. In particular, the diffusion bridge 308 can be input as a re-weighted version of itself. Accordingly, the diffusion model can be a conditional diffusion model, which can be, as an example, effective in large-scale text-conditional image generation tasks. Note that this mechanism merely incorporates the information from the distance function to the score function approximation neural network. As such, the diffusion model of 320b is not a “bridged model” and does not provide guarantees regarding the constraints. However, its training is stable. This model can output a score function 314 expressed as:^c(xt; = s0(xt; t, Vx-ea(xty) (12)

[0124] 320c depicts another method for modifying the diffusion model using a diffusion bridge 308, referred to as Diffusion Bridges (DB-arch). In this method, the score function of the diffusion network 306 is offset using the bridge function, for example using a diffusion bridge term 310 corresponding to a score offset determined from the diffusion bridge 308. This mechanism can be effective at constraining the distribution but can result in unstable training. This model can output a score function 316 expressed as:S0, DB& t> Y,ry = s9(x; t) + b-Q(x; t) (13)

[0125] In particular, the diffusion model is and / or can be trained by or trained to lean a constrained score function. As shown above, the constrained score function is a score function modified b a constraint function (e.g., the diffusion bridge), which shifts the state representation predictions towards a constrained distribution thereof (e.g., ). In use, the diffusion model can then take the observed state representations to predict the unobserved state representations.

[0126] 320d depicts the use of a diffusion bridge 308 to constrain the diffusion 306 in accordance with the present disclosure. In particular, methods 320b and 320c can be “combined” to modify the diffusion model 306. That is, in 320d, the diffusion bridge (e.g. a re-weighted version thereof) is provided / input to the score function approximation neural network to stabilize training. In addition, diffusion bridge term 310 is added to the score function to constrain the data distribution. As such, the resulting diffusion model can exhibit both a good model fit and good constraintsatisfaction. A general form of a score function 318 for the model as modified can be expressed as:se, MBM^e. t, Y, ■?)■■ = se(xt; t, Vx / n(xt)) + bn(xt; t). (14)

[0127] The above methods 320c and 320d may be particular to autonomous vehicle modeling. For example, given one or more observed state representations including one or more observed vehicle state representations, methods 320c and 320d can be applied to model or predict one or more unobserved state representations. The one or more unobserved state representations can be predicted by using the diffusion model to model state transitions between the starting and the ending states in the reverse diffusion process. The one or more unobserved state representations can include one or more unobserved or future vehicle state representations predicted from the one or more observed vehicle state representations. As an example, the methods 320c and 302d can be used to predict or model a future position of a vehicle (unobserved vehicle state representation) from the known or current position of the vehicle (observed vehicle state representation), optionally with other environmental variables such as road boundaries. The predicting can be used for path planning and autonomous driving of vehicles.

[0128] It should also be noted that the below described aspects of the diffusion model, including with regards to training, fine-tuning, etc. may be applied for methods 320c and 320d.

[0129] With regard to loss guidance, for a diffusion model constrained by at least one diffusion bridge, the diffusion bridge can be applied at every sampling step t, including the ones where noise is added. That is, at 206, at least one diffusion bridge can be applied at every sampling step, for example in every iterative step in the reverse diffusion process. Adding noise can lead to cases where the ground truth is infraction free, while the noisy xtis not. In these cases, the applied diffusion bridge(s) can be beneficial in steering the sample back to the infraction free region. To achieve this, an earlier version of the diffusion model itself can be used to make a one-step prediction of the noise-free sample and use that to guide the gradients.

[0130] Under the variance exploding diffusion schedule, xtcan be given by xt= x0+ tε, x0~ pdata(x0), ε ~ 풩(0, σ(t)퐈). By denoting x0(xt) as the one-step estimate of posterior mean, which has a closed form IEXo|Xtxo= f xop(xo|xt) dx0, then x0(xt) = xt- <j(t)66l(xt; t), where eg(xtt) can refer to a model estimator predicting the noise that is added to x0. This can lead to the score function parameterization VXtlogp(xt) « -£ef^- ln onemethod, the score function estimator neural networkDg(xtt) predicts the posterior mean directly, for example corresponding to Dg(xtt) « JEXo|Xtxo, such that the score function is eventually adaptively approximated and parameterized through VXflogp(xt) =Exo|xtXo Xt«De(xt^)~xt

[0131] Since bn(-;-) can informatively calibrate the change of distance to the constraints with respect to itself if falling outside currently, and 0 otherwise, instead of computing gradients of loss function at the noisy sample xt, the MBM can evaluate the gradient of the loss function at the denoised estimate x0(xtt). As such, the score function can be expressed as:V£logp(xt) = - — C> - - (15).. xo(xt;t)-o-2(t)bn(x0(xt;t);t)-xt,.g.<72(t) ' 'where bn(x0(xt; t); t) =ri(t)VSo(Xt.t)^p(xo(xt; t)). That is, the score function can be determined from the input diffusion bridge(s) and can comprise a diffusion bridge term corresponding to a score offset based on the diffusion bridge(s) (e.g. at the denoised estimate), in this case bn(x0(xt; t); t). The corresponding density function can be expressed as:p-Q(xt) oc p(xt)exp(-Z ' t / ?(x0(xt; t))). (17)This can lead to the reverse SDE and its approximated form as follows:dxt= — 2<j(t) (Ex°|xtX° (Ob^(xo(xt),t) xt) + ^2<j(t) dw (18)« _2a(t)^x0(xt)-g2(t)b»(x0(xt);t)-xt)+ d_(1 9)where equation (18) can be regarded as correcting posterior mean by the drift term at the estimate where in practice, x0(xt) can be corrected by itself.

[0132] Theorem 1: Under the definition and conditions on { (t;t)} described above, note limt^0II VXt / f (xt) - Vjo(Xt) / f (x0(xt; t)) ll= 0, Vi., which preserves pn(xt) <x p(xt)e%p(-£f n(t)^(%0(%t; 0)) Therefore, intuitively, xtmay be close to x0(xt) when t is small, as xtis a noisy sample that is tε away from x̂0(xt), assuming x0(xt; t) is a well-trained denoiser.Fine-Tuning

[0133] With regard to fine-tuning, incorporating the drift term bn(x0(xt); t) into the sampling of reverse SDE can help drive the diffusion model to convergence; however, without training, this approach can degrade the diffusion process, as the diffusion model without proper training can lack information to adjust its predictions according to the underlying data structure and thus can generate samples that do not adequately represent the original data distribution.

[0134] However, based on equation (16), it is possible to fine-tune the pretrained diffusion model by leveraging the trained model weights 9 to compute both x0(xt; t) but freezing the parameters for the one(s) computing bn(x0(xt; t)) and only updating the other network denoted by 9' from 9. That is, one or more parameters of the diffusion model used in the computation of the diffusion bridge(s) (e.g. constraining using the diffusion bridge, calculating the score offset based on the diffusion bridge or the diffusion bridge term in the score functions) can be frozen to fine-tune the diffusion model. Accordingly, the objective function, leaving out the expectation with respect to (t,x0,xt) outside temporarily for conciseness, can be expressed as:Exo|xtxo-g2(Obn(x0(xt);t)-xt- VXtlogp(xt|x0) II2(20)£)9,(xt)-o-2(t)bn(£>9(xt;t);t)-xtII <72- VXtlogp(xt|x0) ||(t)2= II se,(xt; t) - bn(£)e(xt; t); t) - VXtlogp(xt|x0) II2, (22)s£0,(xt;t)which can correspond to:Et,xo,xtII t) - VXtlogp(xt|x0) ||2, (23)where bn(Dg(xt; t); t)) = X; Yi^Dg(xf,t)^(.De(xt> )- This fine-tuning process can also reduce the computational complexity, as the gradients of { / (Z>0(xt; t))} with respect to Dg(xt) instead of xtis computed such that no chain rule needs to apply and no gradient propagates through inside the parameter-frozen pretrained Dg(xtt). This process can be referred to as Residual Predictor as when fine-tuning starts, Dg,(xt; t) is no longer predicting EXo|Xtxobut can be used to instead predict the residual of the clean data x0and t); t).Implementation

[0135] In accordance with the present disclosure, the future of all agents can be modeled jointly. That is, the latent states of all agents can be predicted jointly (e.g. simultaneously predicted together). Further, each agent’s future can be modeled in its own coordinate frame, as defined by its pose. This differs from traditional approaches in which agents are predicted either in an arbitrarily chosen global frame (such as that of a static camera), or one in which all agents are predicted in the frame of an assigned “ego” vehicle. In particular, the state, particularly the latent state of each agent can be defined in its own coordinate frame.

[0136] FIG. 6A depicts three approaches for agent state representation, where agents are represented as vehicles. In 602, agents 612 are represented in an arbitrarily chosen global coordinate system, in this case the rendering center 608 of the scene (e.g. the feature map). For such a coordinate system, the diffusion model needs to learn the specifics of this arbitrary coordinate system and what happens when it changes. In 604, the coordinate system is defined by picking one agent as “ego”, in this case the agent 614a. The other agents 614 are predicted in the coordinate frame 610 of the ego agent 614a. 606 depicts the coordinate frame representation utilized by the present disclosure. In 606, all agents 614 are represented in their own coordinate frame 612, and as such there is no arbitrary global frame, or choice of ego-vehicle necessary.

[0137] This representation in terms of an arbitrarily chosen global coordinate frame and a local coordinate frame for each of the agents can be described as follows. The global coordinate frame can be denoted as Cg(e.g. arrows in 602), and each of the agents local coordinate frames as Cl a(e.g. arrows in 606 ), defined by its positions atobs. Pairs of coordinate frames define transformations T(Cfl,a): IR3-» IR3for state variables s =The diffusion model can be configured to predict only in the local coordinate framea, and after prediction is complete, the predicted states can be transformed to the global coordinate frame using T(Cfl,a). This process can remove the need to learn the geometric structure of an arbitrary global coordinate system, and effectively makes use of the translation and rotation equivariance of the problem, which makes learning sensible futures (e.g. predicting latent states) easier.

[0138] A downside of this approach is that it makes agent-agent attention (e.g. relationship between agents) non-trivial, since each agent is now represented in its own coordinate system. This issue can be solved by transforming each of the agents into each of the other agents’ coordinate frame. For example, it is possible to define A2transformations (Ca, Qa,) (e.g. lines connecting agents in 606), to describe the coordinates of agent a' in the system of agent a, and use those features as inputs to the diffusion model. This can be considered equivalent to adding “relative positional encodings” to the diffusion model inputs. In essence, rather than sharing the coordinate representations for all agents, each agent gets independent encodings of all other agents. Note that this leads to A2separate encodings, which can be expensive in terms of memory. To address this problem, it is possible to restrict attention for each agent, a, to a number of nearest neighbor agents (e.g. K nearest neighbors to a, or agents within a distance range to a), as determined by their position atobs, which can bring the overall complexity back to linear.

[0139] In some implementations, each diffusing agent can be provided with an ego-relative representation of the world. Specifically, each agent a e {1,..., A} can be conditioned on their own ego-centric and ego-rotated view of the world Ja= render(s“obs,m) 6 JRHxiy><3w|-]ere m>_mesh(M) is the location mesh generated using the function mesh() that triangulates a feature map (e.g. roadgraph representation), renderQ is a function that rasterizes the given mesh according to the point of view(e.g. camera position s“obs), which is set to be the last observed state of each agent. FIG. 6B depicts examples of the ego-centric (620a) and ego-rotated (620b) feature map representations that can be input to the diffusion network. Note that 620a and 620b show the perspectives of the world from two agents in the same scene.

[0140] The diffusion model can also be conditioned on target goals (e.g. destination goal). As an example, 620c depicts an ego-centric view corresponding to 620a, but with a rendered goal 622 as a waypoint goal for the agent of the view. Consider the ego-centric map rendering as described above and shown in 620a, by rendering a goal 622 (e.g. shown as a dot) in the goal location of interest, it is possible to condition the diffusion model on target goals. Since there is a map rendering per agent, each agent can be assigned a unique goal if desired / needed. Specifically, it is possible to extend mesh() to take a goal position caper agent (e.g. ma= mesh(M, c“)). Passing a goal can be made optional, corresponding to a waypoint (e.g. ca= 0), thus allowing for both conditional and unconditional goal-directed predictions.

[0141] During training, goals may be presented only at some time steps. For example, goal points can be sampled uniformly along the path length of the future trajectory (e.g. latent states), up to a maximum distance (e.g. 30m). Providing waypoints in space, rather than in time can avoid rushed behaviors of agents to reach the target, which can be important in particular in cases where the target waypoint is not feasible to reach without infractions in a fixed amount of time, for example in the case of a red traffic light.Other Considerations

[0142] Additional loss can be used for training the diffusion model. In particular, loss functions modeling loss from overlap (e.g. collision infraction) and infraction zone (e.g. offroad infraction) predictions can be used.

[0143] The infraction zone loss can be defined as a differential offroad loss, for example according to TITRATED. The goal of the infraction zone loss is to penalize states that are predicted to be in infraction zones defined in the feature map (e.g. agents predicted to be outside of roads). Consider a triangle mesh V defining the noninfraction zones, for example corresponding to the driveable area such as roadsurfaces. Given a function $ that computes the distance from a point to a triangle, it is possible to define an infraction zone metric as the maximum of all distances of the agents (e.g. corners of all vehicles) from the road mesh across time. The metric can be expressed as a loss function as follows:£OR(Q:: D K) = ST=I Xa=imaxmin$(c / , v). (24)c J.ecf vev1Note that a threshold can be applied (e.g. threshold of 0.5) when measuring the distance between an agent (e.g. a vehicle bounding box corner) and a mesh triangle.

[0144] In particular, the diffusion model can take a feature map (e.g., the roadgraph) which defines one or more infraction and non-infraction zones. In vehicle applications, the infraction and non-infraction zones correspond to non-drivable areas / positions (e.g., non-roads, sidewalks, etc.) and drivable areas / positions (e.g., roads). The feature map may be the mesh map and may be taken as an observed state representation. Here, each vehicle state representation can define the vehicle or area occupied by the vehicle using a bounding box. As described above, this loss then penalizes (all) the vehicle state representations based on a sum of a farthest distance of the bounding box to a nearest mesh of the non-infraction zones for all vehicle state representations over a sampling time

[0145] The overlap loss can be modeled as a collision loss. For each agent a and time T, the current state s“ and the fixed agent sizes (Z“,w“) can define a bounding box C“ e R4, represented as the edges of the agent (e.g. four corners of the vehicle). The overlapping areas of bounding boxes between different agents can be modeled as a collision loss function for each individual box (e.g. different from an intersection over union). Accordingly, the overlap loss can be expressed as the sum of individual intersections across time:-CC(CI:T) = ^=i \Ci n ci\. (25)

[0146] Similarly, each vehicle state representation can define the vehicle or area occupied by the vehicle using a bounding box. This loss then penalizes the vehicle state representations based on a sum of overlapping areas of the bounding box for all vehicle state representations over a sampling time.

[0147] To sample from score-based models, the reverse process in equation (3) can be numerically solved from < J(T) at T to cr(tmin) « 0 at tmin, rather than the true < J(0) = 0 due to numerical issues. Even for an optimal diffusion model, this sampling can yield samples distributed as qCTmin, which is only approximately equal to the data distribution q0. As a concrete example, in variance exploding SDEs, qamln= q0* JV'(0,<j2(tmin)I) where * denotes the probability density convolution operator. Effectively, this distribution can add a small amount of noise to the data distribution q0-

[0148] Not that this noise should be of acceptable level in the real world. For example, to avoid collision infractions (e.g. reduce / eliminate overlap loss), and the minimum agent to agent (e.g. vehicle to vehicle) distance in the dataset is 5 cm, that means o-(tmin) in its real world units should be much smaller than 5 cm, so that the noise on generated samples does not accidentally push two agents (e.g. vehicle) to collide. As such, it can be important to choose o-(tmin) based on statistics from the dataset fortraining.

[0149] When approaching the end of the diffusion process, (e.g. consider the noise level of <jmin), it can be infeasible to continue flowing with the reverse SDE using smaller t values through discretization, as the diffusion model may not be trained on noise levels below <jmin. Despite the theoretical guarantees from the proof, in practice, it is still possible to observe samples that violate the constraints because from limitation in further discretizing the interval (0, <jmin]. To remedy this, it is possible to gradually apply gradient descent with small step sizes directly on the sampled variable x using the bridge losses (described above), as the variance decreases from <jminto 0, obtaining samples that are completely infraction free.

[0150] Furthermore, instead of choosing a fixed sequence of noise levels {< J, the diffusion model of the present disclosure can adopt a more modern continuous distribution overcr.

[0151] It is also possible to modify the distribution over < J by shifting the mean to lower values as compared to previous methods (e.g. Karras et al, “Elucidating the design space of diffusion-based generative models”). This adjustment can be necessary for example when the spatial variables used the driving application requiremore precision than the pixel values considered in prior methods (e.g. Karras et al.). The noise distribution in Karras et al. can be defined as:ln(ff) ~ JV'(Pmean, Ps2td), (26)With Pmean= “I-2 ar|d ^std — l-2- However, increasing ^std to 2.2 can yield better results.

[0152] Furthermore, results can be improved using a log-Uniform training distribution,ln(cr) ~ t / (log(min),log(max)). (27)At inference time, it is possible to adopt a log-linear diffusion schedule in the reverse process. This schedule can first apply a linear segmentation over [log(<jmin),log(<jmax)], and then use an exponential operation to transform it to range of < J to visit.

[0153] The majority of samplers used for diffusion models are Euler (e.g. first or higher order) solvers. They rely on a set sequence of the diffusion time variable. Such samplers are simple and can be effective in practice for standard diffusion models. Models that incorporate diffusion bridges, as compared to their standard diffusion model counterparts, can have a much more complex and can have potentially non-smooth denoising paths. This is due to the bridge term bn(x; t). This is particularly the case for smaller t as bQgets larger as t decreases. Moreover, the paths can get more complex close to the boundary of the constraint set £1. This is because bQis zero inside £1 and can often be quite smooth outside and far from the boundaries of £1. Close to the boundary, bn(xt; t) can be either zero (forxte £1) or a large vector (for xtg £1 and small t). This can result in two outcomes: first, the conventional discretization methods commonly used in practice can often be too coarse for models incorporating diffusion bridges and can lead to major error accumulations. Second, pre-defining the discretization may be sub-optimal as smaller steps should be taken when xtis close to the boundaries (but can take much larger steps otherwise). Accordingly, samplers with adaptive step size can be used. Such samplers can take two subsequent steps to estimate the curvature of the denoisingpath and make the step size adaptively smaller / larger if the estimated curvature is too large / small.Experiments

[0154] Experiments were conducted using a diffusion model trained according to the method of FIG. 2A on two A100 GPUs for 150 epochs (referred to hereinafter as the diffusion model). The Adam optimizer was utilized with a learning rate of 3E-4 and default values for ft and ft. A linear learning rate ramp up was used, scaling from 0 to 3E-4 over 0.1 epochs. The batch size was set to 32. Gradients were clipped to a maximum norm of 5. T raining took approximately 6 days to complete from scratch.

[0155] To enable test-time conditioning through classifier-free guidance, the diffusion model was trained on the observation distribution shown in Table 1. To quantify the effect of training on this distribution, sample quality of a diffusion model trained on the full observation distribution was compared to one which is trained exclusively on the " Predictive Task." Table 4 shows the impact of the observation distribution as measured by trajectory forecasting metrics on samples drawn from INTERACTION dataset scenes. Trajectory metrics were measured using 6 samples per scene on the INTERACTION validation set.Table 4: Comparison of trajectory forecasting performance for diffusion models trained with varying observation distributions.Observations minADE minFDE Scene minADE Scene minFDE MFD Predictive 0.21 0.49 0.35 0.91 2.33 Mixture 0.26 0.63 0.45 1.17 3.11

[0156] Table 4 demonstrates that training on the full mixture of observation masks somewhat reduces the predictive performance of the diffusion model when compared to training exclusively on the predictive task. However, the diversity of trajectories measured using MFD increases when training on the more diverse distribution.

[0157] The continuous time training procedure of the diffusion model can enable test-time variation in the number of sampling steps. Table 5 outlines the effect of reducing the number of sampling steps from the 50 steps used in all other experiments. Trajectory forecasting performance was measured using 6 samples per scene on the INTERACTION validation set.Table 5: Trajectory forecasting performance versus the number of timesteps used in the diffusion sampling procedure.Diffusion Steps minADE minFDE Scene minADE Scene minFDE 10 0.28 0.64 0.45 1.13520 0.22 0.51 0.37 0.9530 0.22 0.50 0.36 0.9240 0.21 0.50 0.35 0.9250 0.21 0.49 0.35 0.92

[0158] Table 5 shows that reducing the number of sampling steps results in modest trajectory forecasting performance reductions up to 20 sampling steps across all metrics. Using 10 steps severely impacts the quality of sampled scenes across all metrics. As sampling time scales linearly with the number of sampling steps, reducing the number of sampling steps can allow for a performance runtime tradeoff.

[0159] The runtime of the diffusion model was compared to that of Scene Transformer, varying the input size as measured by the number of agents in the scene. Runtimes were measured across 1000 samples on a GeForce™ RTX 2070 Mobile GPU. The results are shown in Table 6 below.Table 6: Average scenario generation time for the diffusion model and Scene Transformer across varying scene sizes.Agent Count Scene Transformer DJINN - 50 Steps DJINN - 25 Steps 8 0.0126s 0.574s 1.15s16 0.0140s 0.611s 1.24s32 0.017s 0.844s 1.69s64 0.026s 1.40s 2.89s

[0160] It would be appreciated by one of ordinary skill in the art that the system and components shown in the figures may include components not shown in the drawings. For simplicity and clarity of the illustration, elements in the figures are not necessarily to scale and are only schematic. It will be apparent to persons skilled in the art that a number of variations and modifications can be made without departing from the scope of the invention as described herein.

[0161] It is contemplated that any part of any aspect or embodiment discussed in this specification can be implemented or combined with any part of any other aspect or embodiment discussed in this specification, so long as those parts are not mutually exclusive with each other.

[0162] It should be recognized that features and aspects of the various examples provided above can be combined into further examples that also fall within the scope of the present disclosure.

[0163] When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components. Further, as used herein, the term “comprising” can mean “including.” Variations of the word "comprising", such as “comprise” and “comprises,” have correspondingly varied meanings. Thus, for example, a composition “comprising” X may consist exclusively of X or may include one or more additional unrecited components. It will be understood that in embodiments which comprise or may comprise a specified feature or variable or parameter, alternative embodiments may consist, or consist essentially of such features, or variables or parameters. A reference to an element by the indefinite article “a” does not exclude the possibility that more than one of the elements is present, unless the context clearly requires that there be one and only one of the elements.

[0164] Additionally, the term "connect" and variants of it such as "connected", "connects", and "connecting" as used in this description are intended to include indirect and direct connections unless otherwise indicated. For example, if a first device is connected to a second device, that coupling may be through a directconnection or through an indirect connection via other devices and connections. Similarly, if the first device is communicatively connected to the second device, communication may be through a direct connection or through an indirect connection via other devices and connections.

[0165] The terms are not to be interpreted to exclude the presence of other features, steps or components. Further, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0166] The embodiments have been described above with reference to flow, sequence, and block diagrams of methods, apparatuses, systems, and computer program products. In this regard, the depicted flow, sequence, and block diagrams illustrate the architecture, functionality, and operation of implementations of various embodiments. For instance, each block of the flow and block diagrams and operation in the sequence diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified action(s). In some alternative embodiments, the action(s) noted in that block or operation may occur out of the order noted in those figures. For example, two blocks or operations shown in succession may, in some embodiments, be executed substantially concurrently, or the blocks or operations may sometimes be executed in the reverse order, depending upon the functionality involved. Some specific examples of the foregoing have been noted above butthose noted examples are not necessarily the only examples. Each block of the flow and block diagrams and operation of the sequence diagrams, and combinations of those blocks and operations, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0167] Use of language such as "at least one of X, Y, and Z," "at least one of X, Y, or Z," "at least one or more of X, Y, and Z," "at least one or more of X, Y, and / or Z," or "at least one of X, Y, and / or Z," is intended to be inclusive of both a single item (e.g., just X, or just Y, or just Z) and multiple items (e.g., {X and Y}, {X and Z}, {Y and Z}, or {X, Y, and Z}). The phrase "at least one of" and similar phrases are not intended to convey a requirement that each possible item must be present, although eachpossible item may be present. Further, in this disclosure, the term “or” is generally employed in its sense including “and / or” unless the content clearly dictates otherwise.

[0168] The invention may also broadly consist in the parts, elements, steps, examples and / or features referred to or indicated in the specification individually or collectively in any and all combinations of two or more said parts, elements, steps, examples and / or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features from any other embodiment(s) described herein.

[0169] The invention illustratively described herein may suitably be practiced in the absence of any element or elements, limitation or limitations, not specifically disclosed herein. Thus, for example, the terms "comprising", "including", "containing", etc. shall be read expansively and without limitation. Additionally, the terms and expressions employed herein have been used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention claimed. Thus, it should be understood that although the present invention has been specifically disclosed by preferred embodiments and optional features, modification and variation of the inventions embodied herein disclosed may be resorted to by those skilled in the art, and that such modifications and variations are considered to be within the scope of this invention.

Claims

1. CLAIMS:

1. A method for generating a state transition plan, the method comprising:receiving at least one observed state representation;predicting at least one unobserved state representation from the at least one observed state representation using a diffusion model trained to model state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model, wherein the diffusion model has been trained by constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function, the constrained score function comprising a score offset based on an output of the constraint function, and wherein the predicting comprises processing the at least one observed state representation with the diffusion model.

2. The method of claim 1, wherein the constraint function is a diffusion function expressed as a function comprising a continuously differentiable function and a diffusion gradient term for a state representation.

3. The method of claim 1, wherein the constraint function comprises a plurality of constituent constraint functions.

4. The method of claim 1, wherein the diffusion model has been trained to model state transitions by applying the constraint function at every sampling step of the diffusion model, each sampling step corresponding to an iterative step in the reverse diffusion process.

5. The method of claim 1,wherein the at least one observed state representation comprises at least one agent state representation; andwherein each of the at least one agent state representation and the at least one unobserved state representation comprises a position of an agent and a heading of the agent at a point in time.

6. The method of claim 1, wherein each of the at least one observed state representation and the at least one observed state representation correspond to a transition between states.

7. The method of claim 1, wherein the at least one observed state representation comprises at least one fixed constraint invariant between a time corresponding to the at least one observed state representation and a time corresponding to the at least one unobserved state representation.

8. The method of claim 7, wherein the at least one fixed constraint is a feature map defining one or more constraints on a position defined by the at least one observed state representation and the at least one unobserved state representation.

9. The method of claim 8,wherein the feature map defines one or more infraction zones; andwherein the diffusion model has been trained by penalizing state representations predicted to be in the one or more infraction zones.

10. The method of claim 1, wherein the diffusion model has been trained by penalizing state representations predicted to be overlapping at the same sampling time.

11. The method of claim 1, wherein the constrained score function is a logarithmic gradient probability density function calculated according to a term corresponding to denoised predicted states, a term corresponding to the constraint function evaluated at the denoised predicted states, and noisy states.

12. The method of claim 1,wherein the at least one unobserved state representation is a plurality of unobserved state representations for a plurality of agents;wherein the plurality of unobserved state representations are modeled jointly; and wherein a predicted state representation of each agent is predicted based on a subset of the plurality of agents according to distances thereto from nearest neighbors of each agent.

13. The method of claim 12,wherein the predicted state representation of each agent is predicted in a respective local coordinate frame by the diffusion model, each of the respective local coordinate fixed to a respective agent; andwherein the method further comprises transforming the predicted state into a global coordinate frame shared by all agents.

14. The method of claim 1,wherein the at least one observed state representation comprises one or more observed agent state representations;wherein the at least one unobserved state representation comprises a control state representation in respect of one agent and one or more conditional state representations in respect of remaining agents and corresponding to the one or more observed agent state representations at a future time step; and wherein the predicting comprises predicting the control state representation according to the one or more observed agent state representations at the future time step and the one or more conditional state representations.

15. The method of claim 1, wherein the diffusion model comprises a transformer network comprising a plurality of layers configured to parameterize the constrained score function and a multi-layer perception network configured to encode input to the transformer network and to output the at least one unobserved state.

16. The method of claim 15, wherein at least one of the plurality of layers correspond to a time transformer layer configured to parameterize time variables, an agent transformer layer configured to parameterize state variables, and a feature map transformer layer configured to fixed constraint variables.

17. The method of claim 1,wherein the diffusion model has been trained using a plurality of training state representations comprising a plurality of observed training state representationsand a plurality of unobserved training state representations generated by applying an observation mask to the plurality of training state representations; wherein the diffusion model has been conditioned using multiple pluralities of training state representations, andwherein each of the pluralities of training state representations is generated by using a different observation mask selected from the group of a predictive mask, a goalcondition mask, an agent reactive mask, an up-sampling mask, an imputation mask, a windowed mask, an agent-conditioned mask, and an ego-conditioned mask as the observation mask.

18. The method of claim 1, wherein the diffusion model has been fine-tuned by freezing at least one model parameter configured to determine the score offset.

19. The method of claim 1, wherein the diffusion model has been trained by: directly training an untrained diffusion model to learn the constrained score function.

20. The method of claim 1, wherein the diffusion model has been trained by:training a first diffusion model from an untrained diffusion model to learn the score function; andtraining the diffusion model from the first diffusion model to learn the constrained score function as the diffusion model.

21. The method of claim 1, wherein the diffusion model has been conditioned using a target state representation.

22. The method of claim 1, wherein the diffusion model has been conditioned on scene semantics via classifier guidance.

23. The method of claim 1, wherein the predicting further comprises processing a target state representation and wherein the at least one unobserved state representation corresponds to a transition to the target state representation.

24. The method of claim 1,wherein the at least one observed state representation and the at least one unobserved state representation correspond to a plurality of agents; andwherein the plurality of agents are vehicles and the plurality of states corresponds to paths traveled by the vehicles.

25. The method of any one of claims 1 to 24, wherein the receiving comprises capturing sensor data comprising the at least one observed state representation at a vehicle.

26. A method for training a diffusion model to generate a state transition plan, the method comprising:obtaining a plurality of state representations, wherein the plurality of state representations comprises a plurality of observed state representations and a plurality of unobserved state representations;inputting to the diffusion model the plurality of state representations; andtraining the diffusion model to predict the plurality of unobserved state representations from the plurality of observed state representations by modelling state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model;wherein the training comprises constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function, the constrained score function comprising a score offset based on an output of the constraint function.

27. A method for generating a state transition plan, the method comprising:receiving at least one observed state representation comprising at least one observed vehicle state representation;predicting at least one unobserved state representation comprising at least one unobserved vehicle state from the at least one observed state representation using a diffusion model trained to model state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model,wherein the diffusion model has been trained by constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function shifting state representation predictions towards a constrained distribution of state representations, andwherein the predicting comprises processing the at least one observed state representation with the diffusion model to predict the at least one unobserved vehicle state representation from the at least one observed vehicle state representation.

28. The method of claim 27, wherein the constraint function is a diffusion function expressed as a function comprising a continuously differentiable function and a diffusion gradient term for a state representation.

29. The method of claim 27, wherein the constraint function comprises a plurality of constituent constraint functions.

30. The method of claim 27, wherein the diffusion model has been trained to model state transitions by applying the constraint function at every sampling step of the diffusion model, each sampling step corresponding to an iterative step in the reverse diffusion process.

31. The method of claim 27, wherein each of the at least one observed vehicle state representation and the at least one unobserved vehicle state representation comprises a position of a vehicle and a heading of the vehicle at a point in time.

32. The method of claim 27, wherein each of the at least one observed state representation and the at least one observed state representation correspond to a transition between states.

33. The method of claim 27, wherein the at least one observed state representation comprises at least one fixed constraint invariant between a time corresponding to the at least one observed state representation and a time corresponding to the at least one unobserved state representation.

34. The method of claim 33, wherein the at least one fixed constraint is a feature map defining one or more constraints on a position defined by the at least one observed vehicle state representation and the at least one unobserved vehicle state representation.

35. The method of claim 34,wherein the feature map defines one or more infraction zones; andwherein the diffusion model has been trained by penalizing vehicle state representations predicted to be in the one or more infraction zones.

36. The method of claim 35, wherein the feature map is a roadgraph defining one or more roads as non-infraction zones and non-road portions as the one or more infraction zones.

37. The method of claim 36,wherein the feature map is a mesh map;wherein each vehicle state representation is defined according to a bounding box; and wherein the penalizing comprises penalizing the vehicle state representations based on a sum of a farthest distance of the bounding box to a nearest mesh of the noninfraction zones for all vehicle state representations over a sampling time.

38. The method of claim 27, wherein the diffusion model has been trained by penalizing vehicle state representations predicted to be overlapping at the same sampling time.

39. The method of claim 38,wherein each vehicle state representation is defined according to a bounding box; and wherein the penalizing comprises penalizing the vehicle state representations based on a sum of overlapping areas of the bounding box for all vehicle state representations over a sampling time.

40. The method of claim 27, wherein the constrained score function is a logarithmic gradient probability density function calculated according to a term corresponding todenoised predicted states, a term corresponding to the constraint function evaluated at the denoised predicted states, and noisy states.

41. The method of claim 27,wherein the at least one unobserved vehicle state representation is a plurality of unobserved vehicle state representations for a plurality of vehicles; wherein the plurality of unobserved vehicle state representations is modeled jointly;andwherein a predicted vehicle state representation of each vehicle is predicted based on a subset of the plurality of vehicles according to distances thereto from nearest neighbors of each vehicle.

42. The method of claim 15,wherein the predicted state representation of each vehicle is predicted in a respective local coordinate frame by the diffusion model, each of the respective local coordinate fixed to a respective vehicle; andwherein the method further comprises transforming the predicted state into a global coordinate frame shared by all vehicless.

43. The method of claim 27,wherein the at least one unobserved state representation comprises a control vehicle state representation in respect of one vehicle and one or more conditional vehicle state representations in respect of remaining vehicles and corresponding to the one or more observed vehicle state representations at a future time step; and wherein the predicting comprises predicting the control vehicle state representation according to the one or more observed vehicle state representations at the future time step and the one or more conditional vehicle state representations.

44. The method of claim 27, wherein the diffusion model comprises a transformer network comprising a plurality of layers configured to parameterize the constrained score function and a multi-layer perception network configured to encode input to the transformer network and to output the at least one unobserved state.

45. The method of claim 44, wherein at least one of the plurality of layers correspond to a time transformer layer configured to parameterize time variables, a state transformer layer configured to parameterize state variables, and a feature map transformer layer configured to fixed constraint variables.

46. The method of claim 27,wherein the diffusion model has been trained using a plurality of training state representations comprising a plurality of observed training state representations and a plurality of unobserved training state representations generated by applying an observation mask to the plurality of training state representations; wherein the diffusion model has been conditioned using multiple pluralities of training state representations, andwherein each of the pluralities of training state representations is generated by using a different observation mask selected from the group of a predictive mask, a goalcondition mask, an agent reactive mask, an up-sampling mask, an imputation mask, a windowed mask, an agent-conditioned mask, and an ego-conditioned mask as the observation mask.

47. The method of claim 27, wherein the diffusion model has been trained by: directly training an untrained diffusion model to learn the constrained score function.

48. The method of claim 27, wherein the diffusion model has been trained by:training a first diffusion model from an untrained diffusion model to learn the score function; andtraining the diffusion model from the first diffusion model to learn the constrained score function as the diffusion model.

49. The method of claim 27, wherein the diffusion model has been conditioned using a target state representation.

50. The method of claim 27, wherein the diffusion model has been conditioned on scene semantics via classifier guidance.

51. The method of claim 27, wherein the predicting further comprises processing a target vehicle state representation and wherein the at least one vehicle unobserved state representation corresponds to a transition to the target vehicle state representation.

52. A method for training a diffusion model to generate a state transition plan, the method comprising:obtaining a plurality of state representations, wherein the plurality of state representations comprises a plurality of observed vehicle state representations and a plurality of unobserved vehicle state representations;inputting to the diffusion model the plurality of state representations; andtraining the diffusion model to predict the plurality of unobserved vehicle state representations from the plurality of observed vehicle state representations by modelling state transitions between a starting state and an ending state in a reverse diffusion process of the diffusion model;wherein the training comprises constraining the diffusion model to learn a constrained score function corresponding to a score function modified by a constraint function shifting state representation predictions towards a constrained distribution of state representations.

53. A system comprising one or more processing units configured to perform the method of any one of methods 1 to 52.

54. A non-transitory computer-readable medium having computer readable instructions stored thereon, which, when executed by one or more processing units, causes the one or more processing units to perform the method of any one of claims 1 to 52.

Images