Cross-engine simulation system and method
By generating independent scene descriptions and establishing semantic identifier mapping relationships through a cross-engine simulation system, the problem of scene construction and control command universality between different simulation engines is solved, and cross-engine scene reuse and control logic migration are realized.
Patent Information
- Application Number
- CN202610905139.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-22
- Publication Date
- 2026-11-03
AI Technical Summary
In existing technologies, the same simulation scenario needs to be built separately for different simulation engines, and control commands are difficult to be universal across different simulation engines, making cross-engine reuse and migration difficult.
It provides a cross-engine simulation system, including a scene description generation module, a conversion module, an engine adaptation module, and a context management module. It generates scene descriptions independent of the simulation engine and establishes mapping relationships through semantic identifiers to achieve cross-engine scene description conversion and unified distribution of control commands.
It enables the reuse of the same simulation scenario and the migration of control logic across multiple simulation engines, reduces the workload of cross-engine construction, and ensures the universality of control commands across different simulation engines.
Smart Images
Figure CN123029818A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot simulation, and particularly to cross-engine simulation systems and methods. Background Technology
[0002] Robot simulation is widely used in fields such as embodied intelligence and the development and verification of robot control algorithms. In robot simulation, a virtual scene containing various simulated objects is typically built using a physics simulation engine, and control is applied to the simulated objects within the scene during the simulation process. Different physics simulation engines often employ their own scene description formats and control interfaces.
[0003] In related technologies, the same simulation scene needs to be built separately in different simulation engines, which involves a large amount of construction work and makes it difficult to reuse the constructed scene across different simulation engines. Furthermore, when controlling the same simulation object in different simulation engines, the control instructions need to be written separately for the control interface of each simulation engine, making it difficult for the control instructions to be universal across different simulation engines. This makes it difficult to reuse and migrate the same control logic across simulation engines. Summary of the Invention
[0004] The purpose of this invention is to provide a cross-engine simulation system and method to solve the problem in related technologies that the same simulation scenario needs to be built separately for different simulation engines, and that control instructions are difficult to be universal between different simulation engines.
[0005] To address the aforementioned technical problems, embodiments of the present invention provide a cross-engine simulation system, including a scene description generation module, a conversion module connected to the scene description generation module, at least two engine adaptation modules connected to the conversion module, and a context management module connected to the at least two engine adaptation modules. The scene description generation module generates scene descriptions independent of the simulation engine, the scene descriptions containing semantic identifiers of each simulation object. The conversion module converts the scene descriptions into native scene data corresponding to each simulation engine. The at least two engine adaptation modules correspond to different simulation engines, and the engine adaptation modules are used to: load the native scene data; establish a mapping relationship between the native control interface of the simulation object in the simulation engine and the semantic identifier, and register the mapping relationship with the context management module; based on the semantic identifier of the target simulation object in the received control instructions, invoke the native control interface to control the target simulation object; the context management module receives the control instructions through a unified interaction interface; and distributes the control instructions to the engine adaptation module corresponding to the target simulation object according to the mapping relationship.
[0006] Embodiments of the present invention also provide a cross-engine simulation method, comprising: generating a scene description independent of the simulation engine, the scene description including semantic identifiers of each simulation object; converting the scene description into native scene data corresponding to each simulation engine; for each of at least two different simulation engines, loading the corresponding native scene data, and establishing a mapping relationship between the native control interface of the simulation object in the simulation engine and the semantic identifier; receiving control instructions through a unified interaction interface, the control instructions carrying the semantic identifier for a target simulation object; distributing the control instructions to the simulation engine corresponding to the target simulation object according to the mapping relationship; and calling the corresponding native control interface to control the target simulation object based on the semantic identifier of the target simulation object.
[0007] In this embodiment of the invention, the scene description generated by the scene description generation module is independent of the simulation engine and uses semantic identifiers to refer to each simulation object in the scene. The conversion module converts the same scene description into native scene data corresponding to each simulation engine, thus eliminating the need to build a separate scene for each simulation engine. The same scene description can drive multiple simulation engines, reducing the workload of building scenes across engines. Furthermore, after loading the native scene data, each engine adaptation module establishes a mapping relationship between the native control interface of the simulation object in the corresponding simulation engine and the semantic identifier of the simulation object, and registers it with the context management module. This establishes a unified addressing relationship indexed by semantic identifiers on top of the different native control interfaces of each simulation engine. The context management module receives control commands carrying semantic identifiers through a unified interaction interface and distributes the control commands to the engine adaptation module where the target simulation object is located according to the mapping relationship. The engine adaptation module then calls the corresponding native control interface to complete the control. In this way, the control command only needs to specify the semantic identifier of the target simulation object, without needing to care which simulation engine the object is in or which native control interface it corresponds to, making the same control command universal across different simulation engines. Since the scene description generation end and the runtime control end are linked by the same semantic identifier, the "construction" of the scene and the "control" of the object are aligned in terms of semantic identifier, thereby realizing one declaration and multi-engine operation, as well as one control and cross-engine execution. The same control logic can be reused and migrated between different simulation engines. Attached Figure Description
[0008] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0009] Figure 1This is a schematic diagram of the architecture of a cross-engine simulation system during the build phase, as provided in one embodiment of the present invention. Figure 2 This is a schematic diagram of the architecture of a cross-engine simulation system during runtime interaction, provided as an embodiment of the present invention. Figure 3 A schematic diagram of the architecture of a cross-engine simulation system during visualization, provided as an embodiment of the present invention. Figure 4 This is a schematic diagram of a cross-engine simulation method provided in one embodiment of the present invention. Detailed Implementation
[0010] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0011] In this invention, "simulation engine" refers to a physical simulation engine used to build virtual simulation scenes and perform physical calculations on simulated objects within those scenes. Different simulation engines typically employ their own scene description formats, control interfaces, and physical solvers. "Simulation object" refers to an object constructed and controlled or observed within a simulation scene, including but not limited to rigid bodies, jointed multi-body structures, and sensors. "Semantic identifier" refers to a tag used to uniquely identify a simulation object without relying on any specific simulation engine, maintaining consistency across scene descriptions, control commands, and mapping relationships. "Native control interface" refers to the interface or internal identifier used by a simulation engine to identify and manipulate specific simulation objects during its operation. "Mapping relationship" refers to the correspondence between semantic identifiers and native control interfaces. "Unified interaction interface" refers to an interaction interface independent of a specific simulation engine, used to receive control commands and transmit sensor data back. "Connection" includes both direct communication connections and indirect communication connections via intermediaries.
[0012] Robot simulation is widely used in fields such as embodied intelligence and the development and verification of robot control strategies. In robot simulation, a virtual scene containing various simulated objects is typically built using a physics simulation engine, and control is applied to the simulated objects within the scene during the simulation process. Currently, there are many types of physics simulation engines available, each differing in scene description formats, runtime control interfaces, and physics solvers, thus forming a fragmented simulation ecosystem.
[0013] Analysis of the robot simulation process across simulation engines reveals that the difficulties in cross-engine simulation stem primarily from two independent levels: firstly, at the static construction level, the same simulation scene needs to be constructed separately in different simulation engines due to their different scene description formats, making it difficult to reuse the constructed scene across different simulation engines; secondly, at the runtime control level, because the control interfaces used to operate simulation objects differ between different simulation engines, control commands need to be written separately for each simulation engine's control interface when controlling the same simulation object in different simulation engines, making it difficult to use control commands universally across different simulation engines. Furthermore, even if the problems at these two levels are solved, there is still a lack of a unified correspondence between the statically constructed simulation object and the runtime control of that object—when the simulation scene changes, the object on the construction side and the interface on the control side are difficult to automatically align, making it difficult to reuse and migrate the same control logic or control strategy across simulation engines. This invention identifies the lack of a unified intermediate layer between the aforementioned static construction, runtime control, and their alignment, and proposes a cross-engine simulation system and method that uses semantic identifiers to connect the scene construction end and the runtime control end.
[0014] Example 1 Figure 1 , Figure 2 and Figure 3 These are schematic diagrams illustrating the architecture of a cross-engine simulation system provided in one embodiment of the present invention during the construction phase, runtime interaction phase, and visualization phase. The cross-engine simulation system includes a scene description generation module, a conversion module connected to the scene description generation module, at least two engine adaptation modules connected to the conversion module, and a context management module connected to the at least two engine adaptation modules. Each engine adaptation module corresponds to a different simulation engine. The diagram illustrates three engine adaptation modules corresponding to three different simulation engines as an example. The number of engine adaptation modules only needs to be at least two; the present invention does not limit the specific number of engine adaptation modules.
[0015] See Figure 1 The scene description generation module generates scene descriptions independent of the simulation engine. These descriptions include semantic identifiers for each simulation object. In other words, the scene description does not use any simulation engine-specific format, but rather describes the simulation objects in the scene, their attributes, and the relationships between them in an engine-independent manner, assigning a semantic identifier to each simulation object. In one example, the scene description generation module can generate a scene description based on external input construction requirements. These requirements can be in natural language or in the form of programmatic calls; this invention does not limit the specific requirements.
[0016] Optionally, the scene description is a scene file written in a declarative markup language, including at least one of JSON, XML, and YAML. The information defined in the scene description includes at least one of the following: pose, mass, inertia, joint constraints, friction, gravity, and lighting parameters of the simulation object. Scene files written in declarative markup languages facilitate manual and programmatic reading, writing, and incremental editing of the scene. Compared to directly editing the native scene data of various simulation engines, this method is beneficial for unified maintenance of the scene at an engine-independent level, and the defined information covers the geometric and physical elements required to build the simulation scene. The aforementioned declarative markup languages and defined information can be used individually or in combination, and this invention does not limit their specific types or items.
[0017] The conversion module is used to convert scene descriptions into native scene data corresponding to each simulation engine. In one example, the conversion module performs the conversion separately for each target simulation engine, converting the same engine-independent scene description into native scene data that can be directly loaded by each target simulation engine. In this way, when building a simulation scene, only one scene description needs to be generated, and the conversion module can automatically obtain multiple sets of native scene data for multiple different simulation engines, without the need to build a scene separately for each simulation engine.
[0018] Optionally, the simulation engine includes at least two of MuJoCo, PyBullet, and Gazebo, and the native scene data format includes at least two of MJCF, URDF, SDF, and USD. The conversion module converts the scene description into the corresponding native scene data in the above formats. Thus, the system is compatible with several mainstream physics simulation engines and their native scene data formats, facilitating the reuse of existing simulation engine ecosystem assets without requiring the reconstruction of the simulation scene for integration with existing engines. The simulation engines and native scene data formats mentioned above are merely examples; other physics simulation engines capable of loading and running simulation scenes and their corresponding formats are also applicable, and this invention does not limit their specific types.
[0019] The engine adaptation module is used to: load native scene data; establish a mapping relationship between the native control interface and semantic identifier of the simulation object in the corresponding simulation engine, and register this mapping relationship with the context management module; and call the native control interface to control the target simulation object based on the semantic identifier of the target simulation object in the received control instructions. In a specific example, after loading the native scene data, the engine adaptation module obtains the runtime identifier or object reference allocated by the corresponding simulation engine for each simulation object, which serves as the native control interface of the simulation object in that engine; the engine adaptation module maps the semantic identifier of each simulation object to its native control interface in this engine, forming a mapping relationship, and reports and registers this mapping relationship with the context management module.
[0020] The context management module receives control commands through a unified interaction interface and distributes them to the corresponding engine adaptation module of the target simulation object based on the mapping relationship. In a specific example, the context management module aggregates the mapping relationships reported by each engine adaptation module and maintains a mapping table indexed by semantic identifiers. Each entry in this mapping table associates the semantic identifier of a simulation object with the native control interface of that simulation object in the corresponding simulation engine, which is registered by each engine adaptation module. See also... Figure 2 During runtime, after receiving control commands through a unified interaction interface, the context management module parses the semantic identifier of the target simulation object carried in the control command and uses this semantic identifier to look up the corresponding engine adaptation module in the mapping table. Based on this, the context management module distributes the control command to the corresponding engine adaptation module, which then calls the control function of its corresponding simulation engine according to the found native control interface to apply control to the target simulation object. For example, if the semantic identifier refers to a joint of a robotic arm, the entry in the mapping table records the native control interface of that joint in different simulation engines. When the target simulation object is that joint, the context management module distributes the control command to the corresponding engine adaptation module for execution based on the currently targeted simulation engine.
[0021] Therefore, control commands only need to specify the semantic identifier of the target simulation object, without needing to know which simulation engine the target simulation object is located in or what native control interface it corresponds to. The same control command can be distributed by the context management module according to the mapping relationship when facing different simulation engines, thus achieving the universality of control commands across different simulation engines. Furthermore, since the scene description generation end assigns semantic identifiers to each simulation object, and the runtime control end uses the same semantic identifier for addressing, the construction of the scene and the control of the object are aligned in terms of semantic identifiers, enabling this system to achieve one declaration, multi-engine operation, and one control, cross-engine execution.
[0022] Combination Figure 1 and Figure 2 In a specific example, the process of constructing and running a cross-engine simulation scenario in this system includes: the scenario description generation module generates a scenario description containing semantic identifiers of each simulation object based on the externally input construction requirements; the conversion module converts the scenario description into native scenario data corresponding to each simulation engine; each engine adaptation module loads the native scenario data of its corresponding simulation engine; after loading, each engine adaptation module registers the mapping relationship between the native control interface and semantic identifier of each simulation object in its corresponding simulation engine to the context management module; the context management module summarizes the mapping relationships registered by each engine adaptation module and constructs a mapping table indexed by semantic identifiers that spans multiple simulation engines; subsequently, during runtime, control commands carrying the semantic identifiers of the target simulation object are sent to the context management module through a unified interaction interface. The context management module then looks up the table and distributes the commands to the engine adaptation module where the target simulation object resides, and the engine adaptation module then calls the corresponding native control interface to complete the control. In this way, the same scenario description can be constructed in multiple simulation engines with a single declaration, and the same control command can be applied to target simulation objects in different simulation engines.
[0023] Optionally, the context management module also exposes a scene context query interface to the upper layer through the model context protocol. The upper layer can obtain information about the simulation objects present in the current scene and their current states through this interface. This allows the upper layer to obtain the current scene context information before issuing control commands, which is beneficial for generating control commands that match the actual scene state.
[0024] Optionally, the scene description can be changed incrementally, meaning only the affected parts of the scene description are modified, rather than the entire scene description is rebuilt. When the scene description changes, the context management module adapts based on the scope of the change: for changes involving only simulation object attribute parameters, the corresponding engine adaptation module is notified to hot-update the relevant parameters of the simulation object without reloading the entire scene; for changes involving the addition, deletion, or topological changes of simulation objects, the conversion module partially reconverts the affected parts of the scene description into native scene data, and the corresponding engine adaptation module reloads the affected parts. Accordingly, the context management module synchronously updates the mapping table: for newly added simulation objects, a corresponding entry is added; for deleted simulation objects, the corresponding entry is cleared; for simulation objects that have changed, their corresponding entries are updated. In this way, when the simulation scene changes dynamically, the mapping table always remains consistent with the current scene, and changes can take effect synchronously across multiple simulation engines without the need for manual reconstruction of the correspondence between semantic identifiers and native control interfaces.
[0025] In one embodiment of the present invention, the context management module may also record the validity status of each entry in the mapping table; when the simulation engine corresponding to a certain engine adaptation module reloads the scene or reassigns the native control interface, the corresponding entry is marked as pending update and the native control interface is re-registered by the engine adaptation module. Thus, when the native control interface changes due to scene reloading, the context management module can promptly detect and update the mapping relationship, preventing control commands from being distributed to invalid native control interfaces.
[0026] Example 2 The difference from Example 1 lies in the further explanation of the source of the control commands. In one example, the control commands are generated by the control strategy model based on data fed back through a unified interactive interface. This control strategy model remains unchanged when switching the simulation engine corresponding to the target simulation object. In other words, the control strategy model uses the unified interactive interface as both its input and output: on the one hand, the control strategy model reads the feedback data related to the target simulation object returned via the unified interactive interface; on the other hand, the control strategy model generates control commands based on this data and issues them through the unified interactive interface. Since the unified interactive interface is independent of any specific simulation engine, the control strategy model does not directly target the native control interface of any simulation engine.
[0027] In a specific example, the control strategy model can be a strategy model trained through reinforcement learning, imitation learning, or similar methods. It uses feedback data returned via a unified interactive interface as its observation input and control commands issued via the same interface as its output action, without relying on the native control interface of any simulation engine for its input and output. After training or validating the control strategy model in the first simulation engine, the process of switching the target simulation object to the second simulation engine includes: the conversion module generating corresponding native scene data for the second simulation engine; the engine adaptation module of the second simulation engine loading this native scene data and registering a new mapping relationship with the context management module; the control strategy model itself remains unchanged, still obtaining feedback data and issuing control commands via the unified interactive interface in the original manner; and the context management module, based on the updated mapping relationship, distributes the control commands to the engine adaptation module of the second simulation engine for execution. The entire switching process requires no modification to the control strategy model, nor does it require retraining for the second simulation engine, allowing the control strategy model originally obtained in the first simulation engine to run directly in the second simulation engine.
[0028] Therefore, the control strategy model is decoupled from the specific simulation engine. Compared with the situation where the control strategy directly faces the native control interface of a simulation engine and needs to be re-adapted or even retrained when switching engines, this embodiment enables the same control strategy model to be migrated between different simulation engines with zero modification. This facilitates the cross-validation and deployment of the same control strategy across multiple simulation engines and is beneficial to the cross-engine generalization research of the control strategy.
[0029] Example 3 The difference from Embodiment 1 is that this embodiment further explains the content of the scene description and the conversion process of the conversion module. In one example, the scene description also includes physical property parameters of each simulation object; the conversion module is also used to convert the physical property parameters into equivalent physical control parameters corresponding to each simulation engine; the physical property parameters include at least one of gravity, friction, damping, and contact stiffness.
[0030] Considering that different simulation engines often use different parameter representations for the same physical semantics in their physics solvers, the conversion module performs an equivalent conversion of the physical property parameters given in the scene description in an engine-independent manner when generating native scene data for each simulation engine, based on the solver space. In a specific example, for gravity, the conversion module maps the gravity configuration in the scene description to the global gravity configuration of each simulation engine; for friction, the conversion module converts the friction semantics in the scene description to the friction parameters used in the contact models of each simulation engine; for damping and contact stiffness, the conversion module converts the damping and contact stiffness given in the scene description in an engine-independent manner to the corresponding damping and stiffness parameters in the contact or joint models of each simulation engine.
[0031] Optionally, for situations where the physics solvers of some simulation engines differ from the settings in the scene description in terms of control loop frequency, the conversion module also adjusts the equivalent physics control parameters based on the control loop characteristics of the corresponding simulation engine, so that the converted equivalent physics control parameters match the solution process of that simulation engine. The conversion of the above-mentioned physical attribute parameters can be implemented individually or in combination. This invention does not limit the specific types of physical attribute parameters or the specific conversion methods, as long as the simulation engines present corresponding physical behaviors.
[0032] Therefore, compared to methods that only perform format conversion on static geometry and topology without handling differences in physical parameters, this embodiment performs equivalent conversion of physical property parameters such as gravity, friction, damping, and contact stiffness in the solver space. This ensures that the physical manifestations of the same scene description in different simulation engines remain as consistent as possible in terms of common observations, which is beneficial for meeting the physical consistency requirements of cross-platform simulation and algorithm generalization verification.
[0033] In one embodiment of the present invention, the conversion module can further calibrate the physical performance of the same scene description in different simulation engines after completing the equivalent conversion: the equivalent physical control parameters are fine-tuned based on the deviation in common observations. This further reduces the deviation in physical performance of the same scene description between different simulation engines.
[0034] Example 4 The difference from Embodiment 1 lies in the fact that this embodiment further explains the specific forms of semantic identifiers, native control interfaces, and mapping relationships. In one example, the semantic identifier is a structured string that names the simulation object; the engine adaptation module is used to extract the runtime handle of the simulation object as the native control interface; and the mapping relationship is a hash mapping table between the runtime handle and the semantic identifier.
[0035] In a specific example, semantic identifiers are structured strings named according to the hierarchy or affiliation between simulation objects. For instance, a delimiter is used to name the entire multi-body structure with joints and its individual joints, indicating a hierarchical relationship. Thus, the affiliation of the simulation object in the scene can be determined solely from the semantic identifier. After loading the native scene data, the engine adaptation module extracts the runtime handles allocated by the corresponding simulation engine to each simulation object as the native control interface for that simulation object. It then registers the mapping relationship with the context management module in the form of a hash mapping table between the semantic identifier and the runtime handle. During runtime, when the semantic identifier looks up the corresponding runtime handle, it directly locates it based on the hash mapping table.
[0036] Therefore, compared to maintaining the correspondence between semantic identifiers and native control interfaces using a linear search method, this embodiment maintains the correspondence between semantic identifiers and runtime handles using a hash mapping table. This makes the process of locating the target runtime handle from the semantic identifier at runtime close to constant time complexity, which helps reduce the latency of control instruction addressing and distribution. Furthermore, the use of structured strings as semantic identifiers facilitates naming and parsing according to the hierarchical relationship of the simulation object.
[0037] In one embodiment of the present invention, for simulation objects with hierarchical relationships, such as multi-body structures with joints, the context management module can also establish a hierarchical index based on the structured hierarchy of semantic identifiers, first locating the overall object and then locating its subordinate objects. This further improves the addressing efficiency of semantic identifiers when there are a large number of simulation objects in the scene.
[0038] Example 5 The difference from Embodiment 1 is that this embodiment adds an intelligent agent. In one example, the cross-engine simulation system also includes an intelligent agent that connects to the context management module via a model context protocol; the intelligent agent is used to translate externally received natural language instructions into control instructions carrying semantic identifiers.
[0039] In a specific example, the agent receives interactive instructions from the user in natural language, translates them into control instructions carrying semantic identifiers of the target simulation objects, and then distributes them through a unified interaction interface. The context management module completes the distribution according to the mapping relationship. The agent connects to the context management module via the model context protocol, thereby obtaining context information of the current scene, such as which simulation objects exist in the current scene and the current state of each simulation object. Based on this, the agent generates control instructions related to the current scene state, so that the objects described in the natural language instructions in a referential manner can be accurately translated into corresponding semantic identifiers.
[0040] Therefore, this embodiment provides a natural language interaction entry point for the system, eliminating the need for users to manually write control commands with semantic identifiers. Furthermore, compared to the method of translating natural language in isolation without relying on scene context, the agent generates control commands after obtaining scene context through the model context protocol, which helps improve the accuracy of translating natural language commands into semantic identifiers.
[0041] Example 6 The difference from Embodiment 1 is that this embodiment further explains the structure of the unified interaction interface. In one example, the unified interaction interface includes at least two types of control channels: a pose control channel for transmitting pose control commands for the simulation object; a joint kinematics control channel for transmitting joint angle or joint velocity control commands for the simulation object; a joint dynamics control channel for transmitting joint force, joint torque, or impedance control commands for the simulation object; and a sensor data channel for transmitting sensing data from virtual sensors in the simulation engine.
[0042] In a specific example, the unified interactive interface is divided into several control channels based on the nature of the transmitted data: the pose control channel controls the overall pose of the simulated object; the joint kinematics control channel controls kinematic quantities such as joint angles and joint velocities; the joint dynamics control channel controls dynamic quantities such as joint forces, joint torques, and impedances; and the sensor data channel carries the feedback of sensor data from virtual sensors. Each control channel can be used individually or in combination, and users can select the appropriate control channel to issue commands or acquire data according to their control needs.
[0043] Therefore, compared to carrying all control and sensing data in a single form, this embodiment carries data of different natures, such as pose, joint kinematics, joint dynamics, and sensor perception, by category and channel, so that the unified interaction interface can clearly express control requirements of different natures, which is conducive to improving the expressive power and scalability of the unified interaction interface.
[0044] Example 7 The difference from Embodiment 1 is that this embodiment adds a rendering module and an incremental state synchronization module. See also... Figure 3 In one example, the cross-engine simulation system also includes a rendering module and an incremental state synchronization module connected to each simulation engine and the rendering module respectively; the incremental state synchronization module is used to synchronize the difference in entity states in all simulation engines to the rendering module; the rendering module updates the rendering scene based on the difference.
[0045] In a specific example, the incremental state synchronization module continuously acquires the state of entities in each simulation engine and synchronizes the state difference that has changed relative to the last synchronization time to the rendering module, instead of synchronizing the full state of all entities to the rendering module every time; after receiving the state difference, the rendering module only updates the rendering scene for the changed parts.
[0046] Therefore, compared to synchronizing all state data each time, the incremental state synchronization module in this embodiment only synchronizes the state differences that have changed, which reduces the amount of data transmission between the simulation engine and the rendering module and is beneficial to improving the efficiency of rendering synchronization when multiple simulation engines are running at the same time.
[0047] Example 8 The difference from Example 7 is that this example further explains the operation and rendering methods of multiple simulation engines. See also... Figure 3 In one example, all simulation engines run in parallel; the rendering module is also used to perform multi-processor parallel rendering of all simulation engines and to perform visual consistency comparison of the multi-processor images.
[0048] In a specific example, each simulation engine runs simulations in parallel on its own native scene data corresponding to the same scene description. The rendering module renders the corresponding images for each simulation engine and outputs them in parallel, so that the simulation images of the same scene under multiple different simulation engines can be observed at the same time. The rendering module also performs visual consistency comparison on multiple images to present or measure the differences in images between different simulation engines under the same scene.
[0049] Therefore, compared to running each simulation engine one by one and viewing its screen one by one, this embodiment allows the simulation screens of the same scene under multiple simulation engines to be observed in parallel, and the simulation differences between different simulation engines can be discovered through visual consistency comparison, which is beneficial for intuitive comparison and verification of cross-engine simulation results.
[0050] Example 9 The difference from Embodiment 1 is that this embodiment adds a caching module. In one example, the cross-engine simulation system also includes a caching module connected to the conversion module; the conversion module is also used to retrieve native scene data from the caching module based on the hash value of the scene description; when the retrieval is successful, the retrieved native scene data is used as the converted native scene data; when the retrieval is unsuccessful, the converted native scene data is associated with the hash value and stored in the caching module.
[0051] In a specific example, before transforming a scene description, the transformation module first calculates the hash value of the scene description and uses this hash value to search the cache module for whether the corresponding native scene data already exists. If a match is found, the existing native scene data in the cache module is directly reused, skipping this transformation. If no match is found, the transformation is performed, and the transformed native scene data, along with its hash value, is stored in the cache module for later reuse. Since the hash value is calculated from the content of the scene description, scene descriptions with the same content correspond to the same hash value, thus ensuring that the same transformed native scene data can be found.
[0052] Therefore, compared to re-performing the transformation each time, this embodiment reuses the transformed original scene data by indexing the hash value of the scene description. For scene descriptions with the same content, there is no need to re-transform, which helps to reduce the transformation time when repeatedly building the same scene.
[0053] In one embodiment of the present invention, the caching module can also evict native scene data according to the least recently used strategy when capacity is limited. Therefore, when cache capacity is limited, native scene data that is more likely to be reused in the near future is preferentially retained, improving the reuse hit rate of the caching module.
[0054] Example 10 Figure 4 This is a flowchart illustrating a cross-engine simulation method according to an embodiment of the present invention. This method can be executed by the cross-engine simulation system provided in any of the above embodiments, and can be combined with... Figure 1 and Figure 2 Understanding, steps S100 to S300 correspond to the construction phase, by... Figure 1 As shown, steps S400 to S600 correspond to the interactive period of operation, by... Figure 2 Illustration. The descriptions of the functions, optional expansions, and technical effects of each module in the above system embodiments also apply to the corresponding steps in this method embodiment, and will not be repeated.
[0055] See Figure 4 The cross-engine simulation method includes steps S100 to S600.
[0056] Step S100: Generate a scene description independent of the simulation engine, which includes semantic identifiers for each simulation object.
[0057] Step S200: Convert the scene descriptions into native scene data corresponding to each simulation engine.
[0058] Step S300: For each of at least two different simulation engines, load the corresponding native scene data and establish a mapping relationship between the native control interface of the simulation object in that simulation engine and the semantic identifier.
[0059] Step S400: Receive control commands through a unified interactive interface. These control commands carry semantic identifiers for the target simulation object.
[0060] Step S500: Based on the mapping relationship, the control commands are distributed to the simulation engine corresponding to the target simulation object.
[0061] Step S600: Based on the semantic identifier of the target simulation object, call the corresponding native control interface to control the target simulation object.
[0062] In one example, combined Figure 2 Users provide scene construction requirements in natural language, which generates an engine-independent scene description in step S100. In step S200, the scene description is converted into native scene data corresponding to multiple simulation engines. In step S300, each simulation engine loads its own native scene data in parallel and establishes a mapping relationship between its native control interface and semantic identifier. During runtime, in steps S400 to S600, control commands carrying semantic identifiers are received through a unified interaction interface. Based on the mapping relationship, the control commands are distributed to the simulation engine where the target simulation object is located, and the corresponding native control interface is called based on the semantic identifier to complete the control of the target simulation object.
[0063] Therefore, this method drives multiple simulation engines with a single engine-independent scene description on the construction side, and uses semantic identifiers to uniformly address and control the target simulation object on the control side. This enables the same simulation scene to be uniformly constructed in multiple simulation engines, and the same control command to be universal across different simulation engines. Thus, semantic identifiers connect the scene construction end and the runtime control end, achieving one declaration, multiple engine operation, and one control, cross-engine execution.
[0064] It should be noted that the sequence numbers of the above steps do not constitute a limitation on their execution order, and the execution order of each step can be adjusted if logically permissible; furthermore, each step of this method can be executed by the corresponding module in the aforementioned cross-engine simulation system, and the method embodiment and the system embodiment can be referred to each other.
[0065] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.
Claims
1. A cross-engine simulation system, characterized in that, It includes a scene description generation module, a conversion module connected to the scene description generation module, at least two engine adaptation modules connected to the conversion module, and a context management module connected to the at least two engine adaptation modules; The scene description generation module is used to generate scene descriptions independent of the simulation engine, and the scene descriptions include semantic identifiers of each simulation object. The conversion module is used to convert the scene description into native scene data corresponding to each simulation engine. The at least two engine adaptation modules correspond to different simulation engines, and the engine adaptation module is used to: load the native scene data; Establish a mapping relationship between the native control interface of the simulation object in the simulation engine and the semantic identifier, and register the mapping relationship with the context management module; Based on the semantic identifier of the target simulation object in the received control command, the native control interface is invoked to control the target simulation object; The context management module is used to receive the control commands through a unified interaction interface; and distribute the control commands to the engine adaptation module corresponding to the target simulation object according to the mapping relationship.
2. The cross-engine simulation system according to claim 1, characterized in that, The simulation engine includes at least two of MuJoCo, PyBullet, and Gazebo; The format of the native scene data includes at least two of MJCF, URDF, SDF, and USD.
3. The cross-engine simulation system according to claim 1, characterized in that, The scene description also includes the physical property parameters of each simulation object; The conversion module is also used to convert the physical property parameters into equivalent physical control parameters corresponding to each of the simulation engines; the physical property parameters include at least one of gravity, friction, damping and contact stiffness.
4. The cross-engine simulation system according to claim 1, characterized in that, The semantic identifier is a structured string that names the simulation object; The engine adaptation module is used to extract the runtime handle of the simulation object as the native control interface; the mapping relationship is a hash mapping table between the runtime handle and the semantic identifier.
5. The cross-engine simulation system according to claim 1, characterized in that, Also includes: The intelligent agent connects to the context management module via a model context protocol; The intelligent agent is used to translate externally received natural language instructions into control instructions carrying the semantic identifier.
6. The cross-engine simulation system according to claim 1, characterized in that, The unified interaction interface includes at least two types of control channels: The pose control channel is used to transmit pose control commands for the simulated object. The joint kinematics control channel is used to transmit joint angle or joint velocity control commands for the simulated object; The joint dynamics control channel is used to transmit joint force, joint torque, or impedance control commands for the simulated object. The sensor data channel is used to transmit the perception data of the virtual sensors in the simulation engine.
7. The cross-engine simulation system according to claim 1, characterized in that, It also includes a rendering module and an incremental state synchronization module that is connected to each of the simulation engines and the rendering module respectively; The incremental state synchronization module is used to synchronize the differences in the states of all entities in the simulation engine to the rendering module; The rendering module updates the rendering scene based on the difference.
8. The cross-engine simulation system according to claim 7, characterized in that, All of the aforementioned simulation engines run in parallel; The rendering module is also used to perform parallel rendering of multiple screens for all the simulation engines and to compare the visual consistency of the multiple screens.
9. The cross-engine simulation system according to claim 1, characterized in that, It also includes a cache module connected to the conversion module; The conversion module is also used to retrieve native scene data from the cache module based on the hash value of the scene description; when the search is successful, the retrieved native scene data is used as the converted native scene data; when the search is unsuccessful, the converted native scene data is associated with the hash value and stored in the cache module.
10. A cross-engine simulation method, characterized in that, include: Generate scene descriptions independent of the simulation engine, wherein the scene descriptions contain semantic identifiers for each simulation object; The scene descriptions are then converted into native scene data corresponding to each simulation engine. For each of at least two different simulation engines, load the corresponding native scene data and establish a mapping relationship between the native control interface of the simulation object in the simulation engine and the semantic identifier; Control commands are received through a unified interactive interface, and the control commands carry the semantic identifier for the target simulation object; Based on the mapping relationship, the control commands are distributed to the simulation engine corresponding to the target simulation object; Based on the semantic identifier of the target simulation object, the corresponding native control interface is invoked to control the target simulation object.