Simulating physical environment with discontinuous dynamics using graph neural network

Abstract

The present specification describes a simulation system for simulating a physical environment using a graph neural network. At each of one or more time steps in a sequence of time steps within a given time interval, the system can process a representation of a current state of the physical environment at a current time step using a graph neural network to generate a prediction of a next state of the physical environment at a next time step. Typically, the environment has discontinuous kinetics at one or more points in time during a time interval.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 352,634, filed on June 15, 2022. The disclosure of the prior application is considered part of the disclosure of the present application and is incorporated by reference into the disclosure of the present application. Background Art

[0002] This specification relates to using machine learning models to process data.

[0003] A machine learning model receives an input and generates an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate an output based on the received input and the values ​​of the model parameters.

[0004] Some machine learning models are deep models that employ multiple layers of a model to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a nonlinear transformation to a received input to generate an output. Summary of the invention

[0005] This specification generally describes a simulation system implemented as a computer program on one or more computers at one or more locations that simulates a physical environment using a graph neural network. Typically, the simulation system simulates states of a physical environment that can have discontinuous dynamics. These discontinuous dynamics can be caused by rigid collisions, friction transitions, or both.

[0006]

[00136] Simulations generated by the simulation systems described herein (eg, representing predicted states of a physical environment over a series of time steps) can be used for any of a variety of purposes.

[0007] In some cases, a visual representation of the simulation may be generated, such as a video, and provided to a user of the simulation system.

[0008] In some cases, an agent interacting with a physical environment (e.g., a reinforcement learning agent) can use a simulation system to generate one or more simulations of the environment that simulate the effects of the agent performing various actions in the environment. In these cases, the agent can use the simulation of the environment as part of determining whether to perform certain actions in the environment.

[0009] In some cases, the simulations performed by the system can be used to evaluate control policies for agents that interact with objects in a real-world environment, i.e., the policies can be evaluated using simulated states of the physical environment. For example, a learned control policy can be evaluated using a simulation system before being deployed to control an agent in the real world. This can save wear and tear on the mechanical agent and prevent poorly performing policies from causing the agent to perform dangerous or otherwise harmful actions.

[0010] In some cases, the simulations performed by the system can be used to train control policies for agents that interact with objects in a real-world environment, i.e., the policies can be trained by reinforcement learning or other training techniques on training data generated by simulating the real world using simulated states of the physical environment. For example, a control policy can be learned using a simulation system before being deployed to control an agent in the real world. This can save wear and tear on the mechanical agent and prevent poorly performing policies from causing the agent to perform dangerous or otherwise harmful actions.

[0011] In one aspect, a method performed by one or more data processing devices for simulating a state of a physical (real world) environment is described. The method involves obtaining data defining a state of the physical environment at one or more initial time steps. The data defining the state of the physical environment at a given time step represents a mesh of the environment at the given step. The mesh includes a plurality of mesh nodes and a plurality of mesh edges. The data defining the state of the environment includes a corresponding mesh node feature for each mesh node and a corresponding mesh edge feature for each mesh edge.

[0012] For example, the state of the physical environment may include a state defined by the relative or absolute position and / or motion of one or more objects in the environment. Such objects can be rigid objects, i.e., objects that do not substantially deform under applied forces, but the described techniques are also applicable to elastic objects, i.e., objects that can undergo elastic deformation. The mesh of the environment can be defined by, for example, the relative or absolute position of each node of the mesh in the environment. For example, for an object or each object in the environment, the nodes of the mesh may define points on the object, and the mesh may have edges, wherein the mesh edge between two mesh nodes defines that the two mesh nodes are part of a shared (same) face of the object. For an object or each object, the mesh may be obtained from an initial mesh of the (undeformed) object, and a translation and / or rotation is applied to the center of mass of the object. In an embodiment, the mesh nodes define the nodes of a graph, and the edges between the mesh nodes define the edges of the graph; the graph is processed by a graph neural network. In some embodiments, counterintuitively, the mesh (graph) node feature of the node defines the velocity of the node, and optionally defines the distance to a boundary, such as a rigid boundary with which the object will interact, and the mesh (graph) edge feature of the edge defines the position of the node. Mesh node features can be encoded into node embeddings of graph nodes, and mesh edge features can be encoded into edge embeddings of graph edges. The velocities can be finite difference velocities determined from the differences between node positions over the input history for one or more time steps. For a pair of nodes connected by an edge , edge features that define node locations can include nodes and The relative displacement vector between (and optionally include ). Edge features can also include the displacements between these nodes on the undeformed initial mesh (and optionally include ); this can help preserve object shape. Such node and edge feature representations can be useful because they provide a local and translation-equivariant encoding of the state of the environment.

[0013] The method also involves generating data defining a corresponding simulated state of the physical environment at each current time step within a time interval after one or more initial time steps. In an embodiment, the environment, such as the motion of one or more objects in the environment, has discontinuous dynamics at one or more time points during the time interval. "Discontinuous dynamics" here can refer to discontinuous (rather than smooth) changes in motion, such as changes in the velocity of an object, such as a rigid object.

[0014] In an embodiment of the method of generating data defining a corresponding simulation state, a representation of the state of the physical environment at a current time step is generated at each current time step in the time interval, the representation comprising a corresponding current node embedding for each grid node and a corresponding current edge embedding for each grid edge. The data defining the representation is processed using a graph neural network to update the current node embedding for each grid node and the current edge embedding for each grid edge. The graph neural network can include one or more edge update sub-neural networks to update the edges of the graph, in particular the edge embeddings, and one or more node update sub-neural networks to update the nodes of the graph, in particular the node embeddings, based on the updated edges.

[0015] In an embodiment, after updating, the corresponding current node embedding of each mesh node in the mesh is used to generate a corresponding dynamic feature corresponding to each mesh node, such as by processing the current node embedding using a decoder sub-neural network of the graph neural network. In an embodiment, the dynamic feature includes a node velocity or acceleration prediction, such as a finite difference acceleration estimate, which can be integrated once or twice, respectively, such as using an Euler integrator, to obtain a position estimate of the node.

[0016] In an embodiment, the method determines the simulated state of the physical environment at the next time step based on the dynamic features corresponding to the mesh nodes. In general, the method is able to roll out the simulation for multiple time steps, in which case the simulated state of the physical environment at the next time step is also based on the (simulated) state of the physical environment at the current time step.

[0017] In an embodiment, the method uses shape matching during such previews to reduce the accumulation of position errors. For example, after each time step, the method can fit a (rigid) transformation - i.e., a translation and / or rotation of the undeformed object mesh - to the predicted node positions, and then reproject the nodes to the transformed mesh to enforce shape consistency.

[0018] Throughout this specification, an "embedding" of an entity can refer to a representation of the entity as an ordered set of values, such as a vector or matrix of values. An embedding of an entity can be generated, for example, as the output of a neural network processing data representing the entity.

[0019] The subject matter described in this specification can be implemented in specific embodiments to realize one or more of the following advantages.

[0020] Realistic simulators of complex physics are invaluable to many scientific and engineering disciplines. However, the creation and use costs of conventional simulators can be very expensive. Building a conventional simulator can require years of engineering work, and it is usually necessary to balance versatility and accuracy within a narrow range of settings. In addition, high-quality simulators require a very large amount of computing resources, which makes expansion prohibitively expensive. Compared with some conventional simulation systems, the simulation system described in this specification can generate simulations of complex physical environments at a large number of time steps with higher accuracy and using fewer computing resources (e.g., memory and computing power). In some cases, the simulation system can generate simulations one or more orders of magnitude faster than conventional simulation systems. For example, the simulation system can predict the state of the physical environment at the next time step by a single pass graph neural network, while a conventional simulation system may need to perform separate optimizations at each time step.

[0021] The simulation system generates simulations using graph neural networks that are able to learn to simulate complex physics directly from training data, and are able to generalize the implicitly learned physical principles to accurately simulate a wider range of physical environments under different conditions than those directly represented in the training data. This also allows the system to generalize to settings that are larger and more complex than those used in training. In contrast, some conventional simulation systems require explicitly programmed physical principles and must be manually tuned to the specific characteristics of each environment being simulated.

[0022] The simulation system can perform a grid-based simulation, e.g., where the state of a physical environment at each time step is represented by a grid. Performing a grid-based simulation can enable the simulation system to simulate certain physical environments more accurately than would otherwise be possible, e.g., physical environments that include deforming surfaces or volumes that are difficult to model as clouds of disconnected particles. Performing a grid-based simulation also enables the simulation system to dynamically adjust the resolution of the grid during the simulation, e.g., increasing the resolution of the grid in portions of the simulation where greater accuracy is desired, thereby improving the overall accuracy of the simulation. By dynamically adjusting the resolution of the grid, the simulation system can generate a simulation of a given accuracy using fewer computing resources than some conventional simulation systems.

[0023] More specifically, many simulation scenarios require the simulation system to accurately model discontinuous dynamics, such as rigid contact or switching motion modes, in the underlying physical environment. Prior to this work, it was believed that due to the continuous nature of the deep network parameterization, deep networks could not accurately model rigid body dynamics without explicit modules for handling contact. This specification shows that such dynamics can be modeled using a general graph network simulator without the need for contact-specific assumptions. That is, the described graph network-based simulator is able to accurately learn and predict contact discontinuities. In addition, the contact dynamics learned by the graph network simulator can actually model real-world object trajectories more accurately than highly engineered dedicated simulators. For example, the described simulator is able to capture real-world cube throwing trajectories more accurately than highly engineered robotics simulators, even if only 8 to 16 trajectories are provided for training. Overall, this shows that rigid body dynamics do not pose a fundamental challenge to deep networks with the appropriate general architecture and parameterization described. Therefore, even if discontinuous dynamics need to be accurately modeled, the described technology can be used, for example, to replace the above-mentioned traditional simulators.

[0024] The details of one or more embodiments of the subject matter of the present specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a block diagram of an example physical environment simulation system.

[0026] Figure 2 An example operation of a physical environment simulation system is shown.

[0027] Figure 3 An example simulation of a physical environment relative to a true value state of the physical environment is shown.

[0028] Figure 4A is a flow chart of an example process for simulating a physical environment.

[0029] Figure 4B The shape matching process is shown.

[0030] Figure 5 The sensitivity of the technique to initial conditions is shown.

[0031] Like reference numbers and designations in the various drawings represent like elements. DETAILED DESCRIPTION

[0032] Figure 1is a block diagram of an example physical environment simulation system 100 capable of simulating states of a physical environment. The physical environment simulation system 100 is an example of a system implemented as a computer program on one or more computers at one or more locations in which the systems, components, and techniques described below are implemented.

[0033] A "physical environment" generally refers to any type of physical system including, for example, fluids, rigid solids, deformable materials, any other type of physical system, or combinations thereof.

[0034] A "simulation" of a physical environment can include a corresponding simulated state of the environment at each time step in a series of time steps.

[0035] The state of the physical environment at a time step can be represented as a mesh, as will be described in more detail below.

[0036] The state of the environment at one or more initial time steps can be provided as input to the physical environment simulation system 100, for example by a user of the system 100. At each time step in the sequence of time steps, the system 100 can process the input and generate a prediction of the state of the physical environment at the next time step 140. Figure 3 An example simulation of a physical environment is shown in .

[0037] As a specific example, a simulated physical environment can be a physical environment that is being interacted by an agent (e.g., a reinforcement learning agent such as a robot, autonomous vehicle, or other mechanical agent).

[0038] That is, an agent interacting with a physical environment can use a simulation system to generate one or more simulations of the environment that simulate the effects of the agent performing various actions in the environment. In these cases, the agent can use the simulation of the environment as part of determining whether to perform certain actions in the environment, for example, as part of a planning system employed by the agent's control system.

[0039] In some cases, the simulations performed by the system can be used to evaluate control policies for agents that interact with objects in a real-world environment, i.e., the policies can be evaluated using simulated states of the physical environment. For example, a learned control policy can be evaluated using a simulation system before being deployed to control an agent in the real world. This can save wear and tear on the mechanical agent and prevent poorly performing policies from causing the agent to perform dangerous or otherwise harmful actions.

[0040] In some cases, the simulations performed by the system can be used to train control policies for agents that interact with objects in a real-world environment, i.e., the policies can be trained by reinforcement learning or other training techniques on training data generated by simulating the real world using simulated states of the physical environment. For example, a control policy can be learned using a simulation system before being deployed for controlling an agent in the real world. This can save wear and tear on the mechanical agent and prevent poorly performing policies from causing the agent to perform dangerous or otherwise harmful actions.

[0041] That is, the simulation of one or more (rigid) objects moving in a physical real-world environment as described herein can be used to train or evaluate a control strategy for an agent, such as a mechanical agent such as a robot, by simulating the action of the agent in the environment, predicting the simulated state of the physical environment at a future time step, and using the simulated state of the physical environment at a future time step to train or evaluate the control strategy. The control strategy can be implemented by a control system of the agent, such as a trainable control system, which is configured to receive observations representing the state of the environment in which the agent is operating from one or more sensors, such as the position or motion of a part of the agent and / or an object to be manipulated by the agent, and in response to providing a control output to control the mechanical action of the agent in the environment, such as to perform a specific task in the environment, such as moving an object within the environment or navigating within the environment. After the control strategy has been trained or evaluated, the control strategy can be deployed to control the agent in the physical environment, such as by using the trained or evaluated control system of the agent to control the agent to perform actions in the real-world environment, such as performing a specific task.

[0042] For example, a control system of a robot can receive observations from one or more sensors characterizing the state of an environment in which the robot is operating, such as the position or motion of a part of the robot and / or an object to be manipulated, and can use a simulated state of the physical (real-world) environment at a future time step to control that part or another part of the robot, e.g., to perform a specific task in the environment, such as moving an object within the environment or navigating within the environment.

[0043] Typically, a robot or other intelligent agent interacting with a physical environment can encounter many different types of dynamics during the interaction that need to be accurately modeled by the system.

[0044] For example, an agent interacting with a physical environment may encounter discontinuous dynamics at various points during its interaction.

[0045] As a specific example, many tasks in robotics, from locomotion to manipulation, require making and breaking contact with rigid objects. Such phenomena are known to be discontinuous because the spatial and temporal scales at which rigid bodies deform during contact are extremely small. That is, rigid contact (contact between rigid surfaces / objects) occurring within an environment can lead to discontinuous dynamics within the environment.

[0046] In some cases, friction transitions occurring in the environment, such as static friction, can lead to discontinuous dynamics, for example, a discontinuous transition between a stationary state of an object in contact with a surface and a motion in which the object slides over the surface (a slip-stick transition).

[0047] The simulation generated by the system accurately simulates the environment even in the presence of discontinuous dynamics. For example, a part of a robot, such as a hand or a gripper, may need to move toward an object to be manipulated, and it may be useful to simulate the effect of the part of the robot making contact with the object. The techniques described herein can be used to predict the motion of an object and / or a part of a robot as a result of relative motion and subsequent contact between the two, such as by modeling the robot part as a boundary or another object in the environment. As another example, a part of a robot, such as a leg or arm of the robot, may move toward a boundary, such as a floor or a wall, and it may be useful to simulate the effect of the part of the robot making contact with the boundary, such as to predict subsequent motion of the robot part.

[0048] In general, simulations can be used to control the actions of a robot, such as the motion of a part of the robot, such as based on model predictive control. For example, a control system of the robot can receive observations from one or more sensors representing the state of an environment in which the robot is operating, such as the position or motion of a part of the robot and / or an object to be manipulated, and can use the simulated state of the physical (real world) environment at a future time step to control the part or another part of the robot, such as to perform a specific task in the environment, such as moving an object within the environment or navigating within the environment.

[0049] During simulation, the system represents the physical environment as a mesh, which can, for example, span the entire physical environment, or represent corresponding surfaces of one or more objects in the environment.

[0050] The simulation system 100 can use the graph neural network 150 to process the current state of the physical environment 102 at the current time step to predict the next state of the physical environment 140 at the next time step.

[0051] Typically, the simulation system 100 can be used to The current state of the physical environment Mapped to time The dynamics of the physical environment are modeled based on the next state of the physical environment at that location.

[0052] The graph neural network 150 of the simulation system 100 can include an encoder module 110 , an updater module 120 , and a decoder module 130 .

[0053] The encoder 110 can include a node embedding subnetwork 111 and an edge embedding subnetwork 112. At each time step, the encoder 110 can be configured to process data defining a current state of the physical environment 102 to generate a representation of the current state of the physical environment 102, which can represent, for example, a graph 114. A "graph" (e.g., ) refers to a set of nodes and edge A data structure such that each edge connects a corresponding pair of nodes.

[0054] In addition to data representing the current state of the physical environment 102, the input to the node embedding subnetwork 111 can also include global features 106 of the physical environment, such as one or more forces applied to the physical environment, the gravitational constant of the physical environment, the magnetic field in the physical environment, or any other suitable features or combinations thereof. In particular, the global features 106 can be concatenated to the node features associated with each node in the graph 114 at each time step before the node embedding subnetwork 111 processes the node features to generate an embedding for the node.

[0055] After generating a graph 114 representing the current state of the physical environment 102 at a time step, the simulation system 100 provides the data defining the graph 114 to an updater 120, which updates the graph 114 in a plurality of internal update iterations to generate an updated graph 115 for the time step.

[0056] “Updating” a graph means, at each update iteration, performing a message passing step (e.g., an information propagation step) between nodes and edges included in the graph by, for example, updating the node and / or edge embeddings of some or all nodes and edges in the graph based on the node and / or edge embeddings of neighboring nodes in the graph. In other words, at each update iteration, updater 120 takes an input graph—e.g. ——Map to output graph where the output graph can have the same structure as the input graph (e.g., the same nodes and edge ), but with different node and edge embeddings.

[0057] More specifically, updater 120 can include a node update subnetwork 121 and an edge update subnetwork 122, each of which can include, for example, a multilayer perceptron (MLP). In each update iteration, node update subnetwork 121 can process a current node embedding of a node included in graph 114 and a corresponding current edge embedding of each edge connected to the node in graph 114 to generate an updated node embedding of the node. In addition, in each update iteration, edge update subnetwork 122 can process a current edge embedding of an edge and a corresponding current node embedding of each node connected by the edge to generate an updated edge embedding of the edge. For example, connecting nodes and nodes Update edge embedding and nodes Update node embedding Can be expressed as: (1) in, and They represent operations performed by the edge update subnetwork 122 and the node update subnetwork 121 , respectively.

[0058] The final update iteration of the updater 120 generates data defining a final update graph 115 for the time step. The data defining the update graph 115 can be provided to a decoder 130 of the simulation system 100. The decoder 130 is a neural network configured to process node embeddings associated with nodes in the graph to generate one or more dynamic features 116 for the nodes. At each time step, the decoder 130 can be configured to process the corresponding node embeddings (e.g., updated node embeddings) for each node in the update graph 115 to generate corresponding dynamic features 116 corresponding to each node in the update graph 115.

[0059] The node and edge embedding subnetworks (111, 112), the node and edge updating subnetworks (121, 122), and the decoder 130 can have any suitable neural network architecture that enables them to perform their described functions. For example, they can have any suitable number (e.g., 2, 5, or 10 layers) of any suitable neural network layers (e.g., convolutional layers, fully connected layers, recurrent layers, attention layers, etc.) and be connected in any suitable configuration (e.g., as a linear sequence of layers).

[0060] The system 100 can provide data defining dynamic features 116 associated with updating nodes in the graph 115 to the prediction engine 160. The prediction engine 160 is configured to process the dynamic features 116 associated with the nodes in the graph to generate a next state of the physical environment 140.

[0061] Thus, at each time step, the simulation system 100 can process the current state of the physical environment 102 and generate the next state of the physical environment 140 .

[0062] At each time step, the system 100 can provide the next state of the physical environment 140 as the current state of the physical environment 102 at the next time step. The system 100 can repeat this process over multiple time steps, thereby generating a trajectory of predicted states that simulates the state of the physical environment.

[0063] As described above, simulation system 100 can be used to simulate a physical environment represented as a grid, such as a grid spanning the environment, or a grid representing one or more objects in the environment.

[0064] To simulate such a system, at each time step, simulation system 100 can process data defining a current state of physical environment 102, wherein such data specifies a grid, generate a map 114 based on the grid, update map 114 in multiple update iterations to generate an updated map 115, and predict a next state of physical environment 140 based on updated map 115. Various aspects of this process are described in more detail below.

[0065] Physical environments including, for example, continuous fields, deformable materials, rigid objects, and / or complex structures can be represented by a grid. A "continuous field" generally refers to, for example, a region of space associated with a physical quality (e.g., velocity, pressure, etc.) that varies continuously across the region. For example, each spatial location in a velocity field can have a specific velocity value associated with it.

[0066] Typically, a "grid" refers to a grid consisting of multiple nodes. and mesh edges A data structure where each mesh edge Connect a pair of mesh nodes. A mesh can define an irregular (unstructured) grid that specifies a subdivision surface of a geometric domain (e.g., a surface or space) as smaller elements (e.g., units or partitions) of a specific shape, such as a triangle or tetrahedron. Each mesh node can be associated with a corresponding spatial location in a physical environment. In some embodiments, a mesh can represent a corresponding surface of one or more objects in the environment. In some embodiments, a mesh can span (e.g., cover) a physical environment, for example, if the physical environment represents a continuous field, such as a velocity field or a pressure field. Figure 2 An example of a mesh representation of a physical environment is described in more detail.

[0067] Each mesh node in the mesh can be associated with a current mesh node feature that characterizes the current state of the physical environment at the location in the environment corresponding to the mesh node. For example, in an embodiment involving simulating a physical environment with an object, each mesh node can represent a point on the object and can be associated with an object-specific mesh node feature that characterizes, for example, one or more of a finite difference velocity of the point, a distance from the point to one or more domain boundaries, such as a floor or another object, and the like.

[0068] In general, mesh representation is not limited to the above-mentioned physical systems, and other types of physical systems can also be represented by a mesh and simulated using the simulation system 100 .

[0069] In all embodiments, the mesh node features associated with each mesh node can further include a corresponding state of the mesh node at each of one or more previous time steps.

[0070] As described above, the simulation system 100 can be used to process data defining the current state of the physical environment 102 (e.g., represented by a grid, such as ), and generates data defining a prediction of the next state of the physical environment 140.

[0071] In particular, at each time step, the encoder 110 can generate Each grid node included in Assign graph nodes to process the current state 102 to generate a graph 114. In addition, for each pair of mesh nodes connected by a mesh edge , the encoder 110 is able to instantiate what is referred to as a grid space edge between corresponding node pairs in FIG. 114 edge.

[0072] In some embodiments, the encoder 110 can enhance the edges in the mesh by using world edges. In particular, in these embodiments, the encoder 110 can process the data defining the mesh and identify each pair of mesh nodes in the mesh that have corresponding spatial locations. , these spatial positions are in world space 114 are separated by less than a threshold distance, and a world space edge is instantiated between each corresponding pair of nodes in FIG. In particular, the encoder 110 is configured to instantiate world-space edges between pairs of graph nodes that are not already connected by mesh-space edges.

[0073] In other words, the encoder 110 can transform the grid Transformed to include nodes The corresponding diagram of , and some of the node pairs are connected by grid space edges connected, and some node pairs are connected by world space edges Connection. Representing the current state of the physical environment 102 via both grid space edges and world space edges allows the system 100 to simulate interactions between a pair of grid nodes that are very far apart from each other in grid space (e.g., separated by multiple other grid nodes and grid edges), but very close to each other in world space (e.g., having close spatial locations in the reference frame of the physical environment). In particular, including world space edges in the graph allows for more efficient message passing between spatially close graph nodes, thereby allowing for more accurate simulations using fewer update iterations (i.e., message passing steps) in the updater 120, thereby reducing consumption of computing resources during simulation.

[0074] In addition to generating graph 114, encoder 110 of system 100 can also generate node and edge embeddings associated with nodes and edges in graph 114, respectively.

[0075] In particular, at each time step, the node embedding subnetwork 111 of the encoder 110 is capable of processing features associated with each node in the graph 114 (e.g., grid node features) and generating a corresponding current node embedding for each node in the graph 114.

[0076] Optionally, in addition to data representing the current state of the physical environment 102, the input to the node embedding subnetwork 111 can also include global features 106 of the physical environment, such as forces applied to the physical environment, the gravitational constant of the physical environment, the magnetic field in the physical environment, or any other suitable features or combinations thereof. Before the node embedding subnetwork 111 processes the node features to generate an embedding for the node, at each time step, the global features 106 can be concatenated to the node features associated with each node in the graph 114.

[0077] At each time step, the graph neural network 150 can generate an edge embedding for each edge in the graph 114. For example, for each grid space edge in the graph 114 , the grid space edge embedding subnetwork of the graph neural network 150 is able to process The connected graph node pairs are associated with features (edge ​​features), and the corresponding current edge embeddings are generated for the grid space edges.

[0078] In particular, the edge features for a given edge can include any of the following: the corresponding positions of grid nodes in the graph corresponding to the graph nodes connected by the edge, the distance between the corresponding positions of grid nodes in the graph corresponding to the graph nodes connected by the grid space edge, the size of the distance, or a combination thereof.

[0079] In some embodiments, edge features for a given edge can include a distance between two mesh nodes connected by the edge in the initial undeformed mesh, a magnitude of the distance, or both. For example, the initial mesh can be a mesh at one of the initial time steps, such as the first initial time step or the last initial time step. That is, the initial mesh can represent the actual (or initial) shape of the underlying objects in the environment, rather than the shape simulated by the system. Including these features can help the system maintain a rigid shape of the object throughout the simulation.

[0080] Similarly, at each time step, for each world space edge in Figure 114 , the world space edge embedding subnetwork of the graph neural network can process the features of the world space edge, that is, Connected pairs of graph nodes are associated, and corresponding current edge embeddings are generated for world-space edges.

[0081] World-space edges can generally have the same characteristics as mesh edges, such as respective positions of mesh nodes corresponding to graph nodes connected by the edge in the graph, distances between respective positions of mesh nodes corresponding to graph nodes connected by the edge in the graph, magnitudes of distances, or combinations thereof. Like mesh edges, world-space edges can also include characteristics from the original mesh.

[0082] Thus, at each time step, the encoder 110 can process the mesh and generate a graph 114 with associated graph node embeddings, mesh space edge embeddings, and in some embodiments, world space edge embeddings. .

[0083] After generating the data defining the graph 114, at each time step, the simulation system 100 can provide the graph 114 to an updater 120, which can update the graph 114 in multiple internal update iterations to generate a final updated graph 115 for the time step.

[0084] As described above, in each update iteration, the node update subnetwork 121 of the updater 120 may process inputs to generate an updated node embedding for the node, the inputs comprising (i) the current node embedding of the node and (ii) the corresponding current edge embeddings of each edge connected to the node.

[0085] In an embodiment where FIG. 114 includes both grid-space edges and world-space edges, the edge update subnetwork 122 of the updater 120 can include a grid-space edge update subnetwork and a world-space edge update subnetwork. At each update iteration, the grid-space edge update subnetwork can be configured to process inputs to generate updated edge embeddings for the grid-space edges, the inputs comprising: (i) the current edge embeddings for the grid-space edges, and (ii) the corresponding current node embeddings for each node connected by the grid-space edge. Additionally, at each update iteration, the world-space edge update subnetwork can be configured to process inputs to generate updated edge embeddings for the world-space edges, the inputs comprising: (i) the current edge embeddings for the world-space edges, and (ii) the corresponding current node embeddings for each node connected by the world-space edge.

[0086] For example, connecting nodes To Node Update of grid space edges embedding of grid space edges , connect nodes To Node Update of world-space edges Embedded in world-space edges , and nodes Update node embedding Can be expressed as: (3)

[0087] Grid space edge update subnetwork ( ), world-space edge update subnetwork ( ) and the node update subnetwork ( ) can have any suitable neural network architecture that enables them to perform the functions described herein. For example, they can include any suitable neural network layers (e.g., convolutional layers, fully connected layers, recurrent layers, attention layers, etc.) connected in any suitable configuration (e.g., as a sequence of linear layers); as a specific example only, they can each be implemented using an MLP with residual connections.

[0088] In some cases, each update of the message passing can be implemented by a corresponding message passing block. Therefore, the graph neural network can be implemented as a set of The same message passing blocks are applied sequentially, i.e., each message passing block has a separate set of network parameters. That is, the message passing blocks can be identical, i.e., have the same neural network architecture, but each message passing block can have a separate set of neural network parameters. Each message passing block can implement a grid space edge update subnetwork, a world space edge update subnetwork, and a node update subnetwork defined by Equation 3, i.e., a grid space edge update subnetwork that processes and updates grid space edge embeddings, a world space edge update subnetwork that processes and updates world space edge embeddings, and a node update subnetwork that processes and updates node embeddings and updates grid space and world space edge embeddings. The message passing blocks can then be applied sequentially, i.e., each message passing block (except the first message passing block that receives the current input graph) is applied to the output of the previous block to process the data defining the graph in multiple iterations.

[0089] The final update iteration of updater 120 generates data representing a final update graph 115 for the time step. At each time step, the data defining the update graph 115 can be provided to a decoder 130. Decoder 130 processes the node embeddings associated with each node in graph 115 and generates one or more dynamic features 116 for each node, wherein the one or more dynamic features 116 characterize the rate of change of mesh node features of the mesh node in graph 115 corresponding to the graph node. The dynamic features 116 can represent the rate of change of any suitable mesh node features from the update graph 115, such as position, velocity, acceleration, momentum, density, or any other suitable physical aspect. As a specific example, the dynamic features 116 for a given mesh node can represent a finite difference acceleration estimate for the node.

[0090] At each time step, the prediction engine 160 can determine the mesh node characteristics at the next time step based on: (i) the mesh node characteristics of the mesh node at the current time step, and (ii) the rate of change of the mesh node characteristics, by, for example, integrating the rate of change of the mesh node characteristics any appropriate number of times. For example, for a first-order system, the prediction engine 160 can determine the mesh node characteristics at the next time step based on the mesh node characteristics at the current time step. Location and mesh nodes Corresponding dynamic characteristics To determine the mesh nodes at the next time step Location , as shown below: (4)

[0091] Similarly, for a second order system, the prediction engine 160 can calculate the value of the grid nodes at the current time step based on the grid nodes at the current time step. Location , the mesh nodes at the previous time step Location , and mesh nodes Corresponding dynamic characteristics To determine the mesh nodes at the next time step Location , as shown below: (5)

[0092] Therefore, by determining the mesh node characteristics of all mesh nodes at the next time step, the simulation system 100 is able to determine the next state of the physical environment 140 .

[0093] In some embodiments, for long previews required for many use cases, directly using equation (5) to update the mesh node features of the mesh nodes may cause position errors to accumulate and deform the object shape. The system can prevent this by using shape matching during preview. That is, after each step, the system can fit the transformation to the predicted node position and reproject the node to the transformed mesh to enforce shape consistency.

[0094] This will be described in more detail below.

[0095] The training engine can train the graph neural network 150 by using, for example, supervised learning techniques on a set of training data. The training data can include a set of training examples, where each training example can specify: (i) training inputs that can be processed by the graph neural network 150, and (ii) target outputs that should be generated by the graph neural network 150 by processing the training inputs. The training data can be generated by, for example, a real-valued physics simulator (e.g., a physics engine), or in any other suitable manner, such as from captured real-world data. For example, training data for a particular simulated physical environment can be obtained from the real-world environment being simulated by capturing observations of the dynamics of the environment, such as observations of the motion of one or more objects in the real-world environment. These observations can be obtained from one or more cameras, or using tags (e.g., TagSLAMPfrommer et al., arXiv:1910.00679), or in any other convenient manner.

[0096] At each training iteration, the training engine can sample batches of one or more training examples from the training data and provide them to the graph neural network 150, which can process the training inputs specified in the training examples to generate corresponding outputs. The training engine can evaluate an objective function that measures the similarity between: (i) the target output specified by the training examples and (ii) the output generated by the graph neural network, such as a cross entropy or squared error objective function. In particular, the objective function Ability to calculate the predicted per-node acceleration , as shown below: (6) in, is a graph neural network model and Represents parameter values ​​of the graph neural network 150. The training engine can determine the gradient of the objective function, e.g., using back-propagation techniques, and can use the gradient to update the parameter values ​​of the graph neural network 150, e.g., using any suitable gradient descent optimization algorithm, such as Adam. The training engine can determine a performance metric of the graph neural network on a set of validation data that was not used during training of the graph neural network 150.

[0097] After training the graph neural network 150, the system 100 can be used to simulate the states of different types of physical environments that can exhibit discontinuous dynamics. For example, based on single time step predictions during training, the system 100 can be effectively generalized to different types of physical environments, different initial conditions, thousands of time steps, and at least an order of magnitude more grid nodes.

[0098] Figure 2 shows that when world space edges are used, the Figure 1 The encoder module, updater module, and decoder module of the system 100 in FIG. 1 are operations performed on a graph representing a mesh. In particular, the encoder 210 generates a representation of the current state of the physical environment (e.g., converts the mesh to a graph), the updater 220 performs multiple steps of message passing (e.g., updates the graph), and the decoder 230 extracts dynamic features corresponding to nodes in the graph.

[0099] The graph includes a set of nodes represented by circles (250, 255) and a set of edges represented by lines (240, 245), where each edge connects two nodes. The graph 200 can be considered a simplified representation of a physical environment (an actual graph representing an environment may have a much larger size than the actual graph). Figure 2 , which has many more nodes and edges than those depicted in ).

[0100] In this illustration, the physical environment includes a first object and a second object, wherein the objects are able to interact (e.g., collide) with each other. The first object is represented by nodes 250 depicted as a group of empty circles, and the second object is represented by nodes 255 depicted as a group of shaded circles 255. The nodes 250 corresponding to the first object are represented by grid space edges 240 ( ) is connected. The node 255 corresponding to the second object is also connected via the grid space edge 240 ( ) connections. In addition to the grid space edges, the graph also includes world space edges 245 depicted as dashed lines ( ). A world-space edge 245 connects a node 250 representing a first object with a node 255 representing a second object. In particular, the world-space edge 245 can allow simulation of external dynamics that are not captured by internal mesh-space interactions, such as collisions, for example.

[0101] As described above, encoder 210 generates a representation of the current state of the physical environment. In this illustration, encoder 210 generates a graph that represents two objects and includes nodes, grid space edges, and world space edges. Updater 220 performs message passing between nodes and edges in the graph. In particular, as described above, updater 220 updates node embeddings and edge embeddings based on the node and edge embeddings of neighboring nodes and edges, respectively. For example, Figure 2 As shown, the node embeddings are updated based on the node embeddings of each neighboring node and the edge embeddings of all edges connecting the node to all neighboring nodes. After the last update iteration, the updater 220 generates an updated graph.

[0102] The decoder 230 processes the update graph and extracts a dynamic feature 260 for each node in the graph. For example, in this illustration, the dynamic feature 260 can be an acceleration corresponding to each mesh node represented by the node in the graph. The acceleration can be, for example, the result of a collision between a first object and a second object. Based on the dynamic features, the simulation system 100 can determine a next state of the physical environment, for example, the positions of the mesh nodes representing the first object and the second object.

[0103] Figure 3 The physical environment simulation system (e.g. Figure 1 An example simulation of a physical environment 300 generated by the system 100 in FIG. 1 relative to a true value state of the physical environment 300 .

[0104] In particular, Figure 3 The example of shows a 2D projection of a sample ground truth 3D trajectory 302 of a cube (orange) thrown onto a table, and a prediction 304 obtained by the system 100 given the same initial state. The cube contacts the table along its edge and bounces into the air. The dashed line 306 shows the time of contact where discontinuous dynamics occur, i.e., velocity discontinuities are introduced by rigid contact events. Figure 3 As shown in the figure in part (b), the system 100 is able to represent and correctly resolve the velocity discontinuity introduced by the rigid contact event (dashed line), closely matching the true value.

[0105] Figure 4A 4 is a flow chart of an example process 400 for simulating the state of a physical environment. For convenience, process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a physical environment simulation system appropriately programmed according to the present specification, such as Figure 1The simulation system 100 is capable of performing the process 400.

[0106] The system obtains data defining a state of the physical environment at one or more initial time steps (402). For example, the system can obtain data defining a state of the physical environment at one or more initial time steps by obtaining data generated from one or more sensor readings of sensors measuring the physical environment, e.g., data representing the position or motion of one or more objects in the environment, such as data from an image sensor, such as a camera or LIDAR sensor, or tag position data from one or more tags. As another example, the system can obtain data defining a state of the physical environment at one or more initial time steps, e.g., by randomly initializing a simulation of a particular scene in the physical environment, or receiving input from a user specifying a scene in the physical environment that needs to be simulated.

[0107] Typically, the data defining the state of the physical environment at any given time step includes data defining a mesh, the mesh including a plurality of mesh nodes and a plurality of mesh edges. Each mesh node can be associated with a corresponding mesh node feature and each mesh edge can be associated with a corresponding mesh edge feature.

[0108] The system then generates data defining the corresponding simulation state of the physical environment at each current time step within a time interval after one or more initial time steps. As described above, the data defining the corresponding simulation state of the physical environment at any given current time step are corresponding node features and edge features for the nodes and edges of the mesh.

[0109] Typically, the environment has discontinuous dynamics at one or more time steps during the time interval. For example, the discontinuous dynamics are caused at least in part by rigid contacts occurring during the time interval. As another example, the discontinuous dynamics are caused at least in part by one or more friction transitions occurring during the time interval.

[0110] To generate data defining a corresponding simulation state, the system performs steps 404 to 410 at each current time step.

[0111] The system generates a representation of the state of the physical environment at the current time step (step 404). The representation includes a corresponding current node embedding for each mesh node and a corresponding current edge embedding for each mesh edge. In particular, as described above, the system uses a node embedding subnetwork and an edge embedding subnetwork to process features of the nodes and edges in the state of the environment at the current time step to generate a corresponding current node embedding for each mesh node and a corresponding current edge embedding for each mesh edge.

[0112] The system processes the data defining the representation using a graph neural network to update a current node embedding for each mesh node and a current edge embedding for each mesh edge (step 406 ).

[0113] After updating, the system processes the corresponding current node embedding of each mesh node in the mesh to generate a corresponding dynamic feature corresponding to each mesh node (step 408), and determines a simulation state of the physical environment at a next time step based on: (i) the dynamic feature corresponding to the mesh node and (ii) the state of the physical environment at the current time step (step 408).

[0114] In some embodiments, to determine the simulation state at the next time step, the system determines, for each grid node, a corresponding initial predicted position at the next time step based on (i) the dynamic characteristics corresponding to the grid node and (ii) the state of the physical environment at the current time step. In particular, the system is as described above with reference to Figure 1 The predicted positions are generated as described.

[0115] The system generates, for each grid node, a corresponding updated predicted position at the next time step based on the corresponding initial predicted position to maintain the rigidity of the object shape of the object in the environment.

[0116] To generate updated predicted positions, the system can determine a transformation, in particular a rigid transformation, optionally an optimal rigid transformation, to transform an initial mesh of object shapes representing objects in the environment into corresponding initial predicted positions of mesh nodes. The transformation can be referred to as a rigid transformation because it can transform each mesh node in a corresponding manner, for example, maintaining their corresponding relative positions (as opposed to a transformation that distorts the object). The transformation can be referred to as optimal because it may be a rigid transformation that best fits the initial mesh and the initial predicted positions of the mesh nodes determined by the graph neural network. When applied to the initial mesh, this may be the rigid transformation that most closely represents the initial predicted positions of the mesh nodes, according to any suitable fit metric, such as based on the distance between the initial mesh positions and the initial predicted positions of the transformed nodes.

[0117] That is, the system is able to determine a rigid transformation that optimally transforms an initial mesh of object shapes representing objects in the environment to corresponding initial predicted positions of mesh nodes determined by the graph neural network.

[0118] The initial mesh can be a mesh at one of the initial time steps, for example, the first initial time step or the last initial time step. That is, the initial mesh can represent the actual (or initial) shape of the underlying objects in the environment.

[0119] As a specific example, the system is able to determine an optimal transformation from a set of "rigid" transformations that can be expressed as a combination of translation operations and rotation operations, i.e., the transformation that produces the minimum total displacement between the corresponding initial predicted positions of the mesh nodes determined by the graph neural network and the positions of the mesh nodes in the updated mesh generated by applying the optimal rigid transformation to the initial mesh.

[0120] In other words, the system is able to determine the rotation matrix and translation vectors and , which minimizes: where the sum is the grid nodes in the grid superior, is assigned to the grid nodes The weight of is assigned to the node The initial predicted position of is assigned to the nodes in the initial undeformed mesh The system can use any suitable computational technique to perform the minimization. The weights of the grid nodes can be the same, for example, each equal to one, or can be assigned based on input received from a user.

[0121] For each mesh node, the system sets the mesh node's corresponding updated predicted position equal to the mesh node's position in the updated mesh generated by applying the optimal rigid transformation to the initial mesh. Thus, the system constrains updates to rigid objects in the environment at any given time step to conform to the rigid transformation, thereby maintaining the rigidity of the object's shape across time steps.

[0122] Figure 4B The above-mentioned shape matching process is shown.

[0123] exist Figure 4B In the simplified example of FIG. 4 , the grid represents a cube and has six nodes. At a given time step, the system has, for example, executed process 400, based on the initial grid. Update grid confirmed The system does not directly set the update position of the grid node to update the grid Instead of predicting the positions in , we first fit the best rigid transformation T, that is, When , the rigid transformation produces the best-fitting updated grid Transformation grid The system then sets the corresponding updated predicted position of each grid node equal to the position of the grid node in the updated grid generated by applying the optimal rigid transformation to the initial network, that is, equal to the transformed grid The positions of the grid nodes in .

[0124] As described above, each node in the graph representing the state of the physical environment at the current time step can correspond to a corresponding grid node. For example, the grid can span the physical environment and / or represent one or more objects in the physical environment.

[0125] For example, for each grid node, the grid node characteristics can include a state of the grid node at a current time step, including, for example, a position coordinate (spatial position) representing the position of the grid node in a reference frame of the grid at the current time step, a position coordinate representing the position of the grid node in a reference frame of the physical environment at the current time step, or both.

[0126] Additionally, optionally, for one or more time steps, the system can determine a corresponding set of one or more remeshing parameters for each mesh node of the mesh, and adjust the resolution of the mesh based on the remeshing parameters by, for example, splitting one or more edges in the mesh, collapsing one or more edges in the mesh, or both. In such an embodiment, determining a corresponding set of one or more remeshing parameters for each mesh node of the mesh can include, after updating, processing a corresponding current node embedding of each graph node using a remeshing neural network to generate corresponding remeshing parameters for the mesh nodes corresponding to the graph nodes.

[0127] In some embodiments, the system can identify one or more mesh edges of a mesh that should be split based on the remeshing parameters. This can include, for the one or more mesh edges, determining an oriented edge length of the mesh edge using the remeshing parameters of the mesh nodes connected to the mesh edge, and in response to determining that the oriented edge length of the mesh edge exceeds a threshold, determining that the mesh edge should be split. The system can also identify one or more mesh edges of a mesh that should be folded based on the remeshing parameters. This can include, for the one or more mesh edges, determining an oriented edge length of a new mesh edge to be created by folding the mesh edge using the remeshing parameters, and in response to determining that the oriented edge length of the new mesh edge does not exceed the threshold, determining that the mesh edge should be folded.

[0128] Figure 5 An example of the sensitivity of the simulations generated by the system 100 to the initial conditions is shown. In particular, Figure 5The scenario shows a cube falling directly downwards with an angle close to 45 degrees of rotation. This scenario is discontinuous because for any angle less than 45 degrees, the cube will fall to the left, and for any angle greater than 45 degrees, the cube will fall to the right. Here, the model must be very sensitive to rotations that differ by a fraction of a degree, as these will significantly affect the final position of the cube. It is clear from the figure that the graph network simulator closely captures the discontinuous behavior in this diagnostic scenario: even for an almost imperceptible deviation of 0.25 degrees, the simulator correctly predicts the final position of the block. This further emphasizes the ability of the learned model to handle sharp discontinuities and multimodal dynamics without producing smooth interpolations of the training data.

[0129] One advantage of embodiments of the above-described systems and methods is that they can be configured for hardware acceleration. In such embodiments, the method is performed by a data processing device including one or more computers and including one or more hardware accelerator units. For example, a hardware accelerator unit can be any of a variety of different computer chips that have dedicated hardware for performing specific operations in hardware, such as matrix multiplication, matrix-vector multiplication, etc. Examples of these types of accelerators include machine learning hardware accelerators, machine learning reasoning hardware accelerators, or other types of hardware accelerators including multiple processors suitable for matrix multiplication. For example, an accelerator can be, for example, one or more GPUs (graphics processing units) or TPUs (tensor processing units).

[0130] Such an embodiment involves updating the graph at each iteration of one or more update iterations, including using The method can further include sequentially applying the message passing blocks to process data defining the graph in multiple iterations, and sequentially applying the message passing blocks to process data defining the graph using one or more hardware accelerators. In some embodiments, the processing is performed using the message passing blocks, i.e., the processor system is distributed across the hardware accelerators. Thus, a simulation method is provided that is specifically adapted for embodiments using hardware accelerator units, unlike some conventional methods that cannot take advantage of hardware acceleration.

[0131] The system can be used to predict physical quantities based on measured real-world data. Therefore, in some embodiments of the above-described systems and methods, the physical environment includes a real-world environment including real physical objects. Then, obtaining data defining the state of the physical environment at the current time step may include obtaining object data defining a 2D or 3D representation of the shape of the physical object from the physical object. For example, an image of the object may be captured by a camera such as a depth camera. The method may then involve inputting interaction data defining the interaction of the physical object with the real-world environment. For example, the interaction data may define the shape of a second physical object such as an actuator, which will interact with the physical object and may apply a force to the physical object; or it may define a force applied to the physical object; or it may define a field to which the physical object is subjected, such as a velocity, momentum, density, or pressure field.

[0132] In some embodiments of the above-mentioned system and method, as described above, the system and method can also be used for real-world control, in particular optimal control tasks, such as helping a robot manipulate a deformable or rigid object. Therefore, as described above, the physical environment can include a real-world environment containing physical objects, such as an object to be picked up or manipulated. Determining the state of the physical environment at the next time step includes, for example, determining a representation of the shape or configuration of the physical object by capturing an image of the object. Determining the state of the physical environment at the next time step can include determining a predicted representation of the shape or configuration of the physical object, such as when subjected to a force or deformation such as from a robot actuator. The method can further include using the predicted representation to control the robot, such as using an actuator, to optimize an objective function by controlling the robot, thereby manipulating the physical object toward a target position, shape or configuration of the physical object, the objective function depending on the difference between the predicted representation and the target position, shape and configuration of the physical object. Controlling the robot may involve providing a control signal to the robot based on the predicted representation so that the robot performs an action, such as using the robot's actuator to manipulate the physical object to perform a task. For example, this may involve controlling a robot, such as an actuator, using a reinforcement learning process with rewards based at least in part on the value of an objective function to learn to perform a task involving manipulation of a physical object.

[0133] In some embodiments of the above-described systems and methods, the systems and methods can be used to use a simulator, such as a simulated state of a use environment, to design, test, or otherwise evaluate a product. The product can then be produced or manufactured based on the results of the simulator, for example, using any appropriate manufacturing technology to have the properties of the simulated product. For example, the system can be used to model the interaction between different components of a mechanical device, such as the gears of a car or parts of an internal combustion engine. The use of the system enables accurate modeling of discontinuous friction relationships or contacts between product components or external surfaces, such as the friction relationship between a brake and a wheel or between wheels, or simulated collision interactions of a product with other objects or the ground. Therefore, the simulator can be used to "stress test" a product design in a real-world scenario that results in discontinuous dynamics before the product is manufactured. If the product design passes the stress test, a product having the design can be manufactured.

[0134] This specification associates the term "configured to" with system and computer program components. For a system of one or more computers configured to perform a particular operation or action, it means that the system has installed thereon software, firmware, hardware, or a combination thereof, which, when operated, causes the system to perform those operations or actions. For one or more computer programs configured to perform a particular operation or action, it means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform those operations or actions.

[0135] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more computer program instruction modules encoded on a tangible, non-transitory storage medium, for execution by a data processing device or for controlling its operation. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0136] The term "data processing apparatus" refers to data processing hardware and includes various devices, apparatuses and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. The apparatus can also be or further include special-purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to the hardware, the apparatus can optionally include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0137] A computer program, which may also be referred to or described as a program, software, software application, application, module, software module, script or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, such as files storing one or more modules, subroutines or code portions. A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0138] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines can be installed and run on the same one or more computers.

[0139] The processes and logic flows described in this specification can be performed by one or more programmable computers that execute one or more computer programs to perform functions by operating on input data and generating output. These processes and logic flows can also be performed by a dedicated logic circuit such as an FPGA or ASIC, or a combination of a dedicated logic circuit and one or more programmed computers.

[0140] A computer suitable for executing a computer program can be based on a general or special purpose microprocessor or both, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for executing or performing instructions and one or more storage devices for storing instructions and data. The central processing unit and the memory may be supplemented or incorporated therein by a dedicated logic circuit. Typically, a computer will also include or be operably coupled to receive data from or transmit data to one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or both. However, a computer need not have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0141] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, such as semiconductor memory devices, for example, EPROM, EEPROM, and flash memory devices; magnetic disks, for example, internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0142] To provide interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a web page to a web browser on the user's device in response to a request received from the web browser. In addition, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving a response message from the user in return.

[0143] The data processing apparatus for implementing the machine learning model can also include, for example, dedicated hardware accelerator units for processing the common and computationally intensive parts of machine learning training or production, i.e., inference, workloads.

[0144] The machine learning model can be implemented and deployed using a machine learning framework such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0145] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a client computer with a graphical user interface, a web browser, or an application through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0146] A computing system can include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by a computer program running on a corresponding computer, and has a client-server relationship with each other. In some embodiments, the server transmits data such as HTML pages to a user device, for example, for displaying data to a user interacting with the device acting as a client and receiving user input from the user. Data generated at the user device, such as the result of the user interaction, can be received from the device at the server.

[0147] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or the scope of what may be claimed, but rather as descriptions of features that may be unique to a particular embodiment of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. In addition, although the above-mentioned features may be described as functioning in certain combinations and may even be initially claimed to be so protected, in some cases, one or more features in the claimed combination may be deleted from the combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0148] Similarly, although operations are described in a particular order in the drawings and referenced in the claims, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0149] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes described in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for simulating a state of a physical environment, performed by one or more data processing devices, the method comprising: obtaining data defining the state of the physical environment at one or more initial time steps, wherein the data defining the state of the physical environment at a given time step represents a mesh of the environment at the given time step, wherein the mesh comprises a plurality of mesh nodes and a plurality of mesh edges, and wherein the data defining the state of the environment comprises respective mesh node features for each of the mesh nodes and respective mesh edge features for each of the mesh edges; and generating data defining a respective simulation state of the physical environment at each current time step within a time interval following the one or more initial time steps, wherein the environment has discontinuous dynamics at one or more time points during the time interval, and wherein the generating comprises, at each current time step within the time interval: generating a representation of the state of the physical environment at the current time step, the representation comprising a respective current node embedding for each of the mesh nodes and a respective current edge embedding for each of the mesh edges; processing the data defining the representation using a graph neural network to update the current node embedding of each mesh node and the current edge embedding of each mesh edge; processing the respective current node embeddings of each mesh node in the mesh to generate respective dynamic features corresponding to each mesh node after the updating; and A simulated state of the physical environment at a next time step is determined based on (i) the dynamic features corresponding to the mesh nodes, and (ii) the state of the physical environment at the current time step.

2. The method according to claim 1, wherein: The discontinuous dynamics are caused, at least in part, by rigid contact occurring during the time interval.

3. The method according to claim 1 or 2, wherein: The discontinuous dynamics are caused, at least in part, by one or more friction transitions occurring during the time interval.

4. A method according to any preceding claim, wherein: Determining a simulation state of the physical environment at a next time step based on (i) the dynamic features corresponding to the mesh nodes, and (ii) the state of the physical environment at the current time step, comprises: determining, for each of the grid nodes, a corresponding initial predicted position at the next time step based on (i) the dynamic characteristics corresponding to the grid nodes, and (ii) the state of the physical environment at the current time step; and A corresponding updated predicted position at the next time step is generated for each of the grid nodes based on the corresponding initial predicted position to maintain the rigidity of the object shape of the object in the environment.

5. The method according to claim 4, wherein: Generating, for each of the nodes in the graph, a corresponding updated predicted position at the next time step based on the corresponding initial predicted position to maintain the rigidity of the object shape represented in the graph, comprising: determining an optimal rigid transformation to transform an initial mesh representing an object shape of the object in the environment to the corresponding initial predicted positions for the mesh nodes; and For each mesh node, the corresponding updated predicted position of the mesh node is set equal to the position of the mesh node in an updated mesh generated by applying the optimal rigid transformation to the initial mesh.

6. The method according to claim 5, wherein: The optimal rigid transformation is a combination of a translation operation and a rotation operation.

7. A method according to any preceding claim, wherein: The grid spans the physical environment.

8. The method according to any one of claims 1 to 6, wherein: The mesh represents one or more objects in the physical environment.

9. A method according to any preceding claim, wherein: For each grid node of the plurality of grid nodes, the grid node feature associated with the grid node comprises a state of the grid node at the current time step, and wherein the state of the grid node at the current time step comprises: position coordinates representing the position of the grid node in a reference system of the grid at the current time step, position coordinates representing the position of the grid node in a reference system of the physical environment at the current time step, or both.

10. The method according to claim 9, wherein: For each mesh node of the plurality of mesh nodes, the mesh node features associated with the mesh node further include a corresponding state of the mesh node at each of one or more previous time steps.

11. A method according to any preceding claim, wherein: Generating the representation of the state of the physical environment at the current time step includes generating a corresponding current node embedding for each grid node in the graph, including, for each grid node: An input including one or more of the features of the grid node is processed using a node embedding subnetwork of the graph neural network to generate the current node embedding of the grid node.

12. The method according to claim 11, wherein: For each mesh node, the input to the node embedding subnetwork further includes one or more global features of the physical environment.

13. The method according to claim 12, wherein: The global characteristics of the physical environment include forces applied to the physical environment, a gravitational constant of the physical environment, a magnetic field of the physical environment, or a combination thereof.

14. A method according to any preceding claim, wherein: The mesh edge comprises a plurality of mesh space edges and a plurality of world space edges, wherein generating the representation of the state of the physical environment at the current time step comprises: For each pair of grid nodes connected by an edge in the grid, determining a corresponding pair of graph nodes connected by a grid spatial edge in the representation; and For each pair of mesh nodes having corresponding positions separated by less than a threshold distance in a reference frame of the physical environment, it is determined that the corresponding pair of graph nodes is connected by a world space edge in the representation.

15. The method according to claim 14, wherein: Generating the representation of the state of the physical environment at the current time step includes generating a corresponding current edge embedding for each mesh edge, including, for each mesh space edge in the graph: Processing input using a grid space edge embedding subnetwork of the graph neural network to generate the current edge embedding of the grid space edge, the input comprising: respective positions of the grid nodes corresponding to the nodes connected by the grid space edges, data characterizing differences between the respective positions of the grid nodes corresponding to the nodes connected by the grid space edges, or a combination thereof.

16. The method of any one of claims 14 to 15, further comprising, for each world space edge in the graph: Processing an input using a world-space edge embedding subnetwork of the graph neural network to generate the current edge embedding of the world-space edge, the input comprising: Respective positions of the mesh nodes connected by the world-space edges, data representing differences between the respective positions of the mesh nodes connected by the world-space edges in the graph, or a combination thereof.

17. A method according to any preceding claim when dependent on claim 9, wherein: The grid node characteristics further include a distance from the location of the grid node to one or more domain boundaries.

18. A method according to any preceding claim, wherein: The mesh edge feature comprises a distance between two mesh nodes connected by the edge in an initial undeformed mesh representing an object portion of an object in the environment.

19. A method according to any preceding claim, wherein: Processing, for each node in the graph, the corresponding current node embedding to generate a corresponding dynamic feature corresponding to each node in the graph includes, for each graph node: The current node embedding of the graph node is processed using a decoder subnetwork of the graph neural network to generate the corresponding dynamic feature of the graph node, wherein the dynamic feature characterizes a rate of change of a grid node feature of the grid node corresponding to the graph node.

20. A method according to any preceding claim, wherein: Obtaining data defining the state of the physical environment at one or more initial time steps comprises: Data generated based on one or more sensor readings of sensors measuring the physical environment is obtained.

21. The method according to any preceding claim, further comprising: The simulated state of the physical environment is used to evaluate a control strategy for an agent interacting with objects in a real-world environment.

22. The method according to any one of claims 1 to 20, further comprising: The simulated state of the physical environment is used to train a control policy for an agent that interacts with objects in the real-world environment.

23. The method according to any one of claims 21 or 22, further comprising: The control strategy is deployed for controlling the agent in the physical environment.

24. The method according to any one of claims 21 to 23, wherein: The agent is a robot.

25. A method according to any preceding claim, wherein: Processing the data defining the representation using a graph neural network is performed by one or more hardware accelerator units.

26. The method according to claim 25, wherein: The one or more hardware accelerator units apply corresponding message passing blocks of the graph neural network to update the data defining the representation.

27. A method according to any preceding claim, further comprising using the simulated state of the physical environment to evaluate the design of a product.

28. The method of claim 27, further comprising manufacturing the product having the design based on evaluating the design using the simulated state.

29. A product having a design which has been evaluated using a corresponding method according to claim 27.

30. One or more non-transitory computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of the corresponding method of any one of claims 1 to 28.

31. A system comprising: one or more computers; as well as One or more storage devices, the one or more storage devices being communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of the corresponding method according to any one of claims 1 to 28.

32. The system of claim 31, wherein: The one or more computers include one or more hardware accelerator units.

33. The system of claim 32, wherein: The one or more computers include a plurality of hardware accelerator units.

34. A system according to claim 32 or 33, wherein: Each hardware accelerator unit maintains one or more corresponding message passing blocks of the graph neural network.

Images