Simulation method and system of intelligent robot with body for cross-physical simulation engine

Abstract

The invention discloses a simulation method and a simulation system for an intelligent robot with a body of a cross-physical simulation engine. According to the method, a general meta-action library irrelevant to a simulation engine is defined, a meta-action mapping file is automatically generated in combination with a robot URDF file, and a high-level task instruction is decoupled into a bottom-layer motor control instruction; and then the instruction is converted into native API (Application Program Interface) calling of a target physical simulation engine (such as Isaac Sim, Gazebo and MuJoCo) through a pluggable multi-engine adapter, so that'one-time definition and multi-position operation 'is realized. Meanwhile, by deploying VNC services for all simulation engines, capturing and coding 3D rendering pictures of the simulation engines, multi-instance video streams are integrated in a unified Web workbench, and same-screen viewing, operation and switching are supported. According to the method, the algorithm development efficiency and verification reliability are remarkably improved, the repeated development cost is reduced, and consistent cross-platform simulation experience is provided.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and robotics, and in particular to a method and system for embodied intelligent robot simulation across physics simulation engines. Background Technology

[0002] Currently, with the rapid development of robotics technology and the rise of the concept of embodied intelligence, robot simulation systems are playing an increasingly important role in algorithm verification, system testing, and skill learning. The "digital twin" paradigm, centered on reinforcement learning and simulation-driven approaches, is becoming a key engine for breakthroughs in embodied intelligent robot technology. Through massive trial and error and iteration in an infinite and secure virtual environment, intelligent agents can learn complex skills and robust strategies that are difficult to acquire directly in the physical world. This "simulation-first" R&D model is considered an essential path to accelerate robot intelligence and reduce the risks and costs of physical training, serving as a "virtual bridge" connecting algorithmic innovation and physical deployment. As the core of robot simulation, the physics simulation engine provides a reliable testing environment for robot control algorithms, motion planning, and perception decision-making by simulating the physical laws of the real world. Currently, there are several mature physics simulation engines in the industry, such as NVIDIA Isaac Sim, Gazebo, and MuJoCo, each with its own characteristics and advantages, forming a diversified simulation ecosystem.

[0003] However, due to the persistent "Reality Gap" (or Sim-to-Real Gap) between the simulation environment and the real physical world, overcoming this gap requires developers to conduct repetitive experiments across multiple simulation engines with different physical modeling characteristics, accuracy, and fidelity to verify the robustness of algorithms. For example, a strategy trained in MuJoCo needs to be rebuilt in engines like Isaac Sim, the code ported, and tested to observe its performance stability. However, under current technology, multi-engine verification is costly due to inconsistent interfaces and logical coupling. The main technical problems encountered include the following: 1. Fragmented and incompatible simulation engine ecosystem: Existing mainstream physics simulation engines employ completely different technical architectures and interface standards. Isaac Sim is based on the NVIDIA Omniverse platform, using the USD scene description format and Python API interface; Gazebo often works collaboratively with the ROS ecosystem through its integration package (gazebo_ros_pkgs), using SDF as the native model format and exposing simulation data to the ROS system as ROS topics / services via the plugin interface provided by gazebo_ros_pkgs; MuJoCo, on the other hand, has its own independent MJCF model format and C / Python API. This difference in technical architecture leads to: (1) Fragmentation of control interfaces: The robot control API interfaces provided by various simulation engines differ significantly. Isaac Sim uses the omni.isaac.core interface for joint control and status query, Gazebo receives control commands through the ROS motion server, and MuJoCo directly manipulates the mujoco.MjData data structure. This interface difference makes it difficult to directly migrate control algorithms developed for one engine to other engines.

[0004] (2) Inconsistent communication protocols: Each engine uses different inter-process communication and data transmission mechanisms. Isaac Sim uses the IPC mechanism of Omniverse Kit, Gazebo uses the DDS middleware of ROS2, and MuJoCo mainly uses shared memory or Socket communication. This difference in protocols increases the complexity of multi-engine collaborative simulation.

[0005] 2. Tight coupling between control logic and physics engine: Existing robot simulation systems typically bind business logic deeply to the APIs of specific simulation engines, resulting in: (1) Poor algorithm portability: The path planning, motion control and other algorithms developed for Isaac Sim are difficult to run directly in Gazebo or MuJoCo because they use a lot of API interfaces and data structures specific to the engine. Developers need to implement the same algorithms repeatedly in different engines.

[0006] (2) Low code reuse rate: According to relevant research, the code reuse rate of robot simulation across engines is usually less than 30% (see the literature "A Survey of Robot Simulation Platforms: Challenges and Opportunities", IEEE Robotics and Automation Letters, 2022). This means that more than 70% of the simulation logic code needs to be rewritten during engine migration.

[0007] (3) Incomplete testing and verification: Since the algorithm is bound to a single simulation engine, its verification results may be affected by factors such as the accuracy of the specific physical model and numerical calculation method of the engine. The lack of cross-verification in other physical engines reduces the reliability of the simulation results.

[0008] It is precisely because of the tight coupling mentioned above that the goal of "defining once and running in multiple places" cannot be achieved, thus requiring a lot of manpower to repeatedly develop and configure the simulation logic.

[0009] 3. Visual access is fragmented and has limited functionality: The visualization interfaces provided by various simulation engines have the following problems: (1) Inconsistent interfaces: Isaac Sim uses Omniverse Viewer, Gazebo provides Gazebo Client, and MuJoCo has MuJoCo Viewer. Each interface has different operation methods and function sets.

[0010] (2) Limited remote access capabilities: Although some engines support remote visualization, they usually require complex network configuration and client installation, and lack lightweight web-based access solutions.

[0011] (3) Insufficient collaboration functions: Existing visualization tools are mainly designed for single-person use and lack collaborative functions such as simultaneous viewing by multiple users, collaborative annotation, and remote assistance.

[0012] In summary, existing technologies have significant shortcomings in multi-physics simulation engine compatibility, leading to low efficiency in robot simulation development, insufficient reliability in algorithm verification, and limited system scalability. This limitation is particularly pronounced in the field of embodied intelligent robot simulation, where frequent verification of algorithm performance under different physical models is required.

[0013] Therefore, the industry urgently needs a cross-physics simulation engine robot simulation method and system that can effectively solve the above problems, realize the unification of simulation models, control logic, and visualization access, improve development efficiency, enhance the reliability of simulation results, and promote the rapid development of embodied intelligent robot technology. Summary of the Invention

[0014] In view of this, and addressing the problems of fragmented control interfaces, low development efficiency, and insufficient verification reliability in existing cross-physics simulation engine robot simulation systems, this invention aims to provide an embodied intelligent robot simulation method and system across physics simulation engines. Its core objective is to achieve reproducible operation of the same robot task or control strategy in multiple heterogeneous physics simulation environments without incurring additional repetitive development work, thereby significantly reducing simulation workload and greatly improving simulation development and verification efficiency.

[0015] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for embodying intelligent robot simulation across physics simulation engines, the method comprising the following steps: Construct a generic meta-action library that is independent of the physics simulation engine, where each meta-action contains action code, action description, and execution component, wherein the execution component references the component name defined in the robot's URDF file; Based on the URDF file and the general meta-action library, a meta-action mapping file is generated, which is used to map the execution component of the meta-action to a specific physical component. The task flow is orchestrated based on the general meta-action library, and the task flow is converted into a specific set of control instructions according to the meta-action mapping file. The control command set is converted into native API call commands of the target physics simulation engine through a pluggable multi-simulation engine adapter and sent to the target physics simulation engine for execution.

[0016] Furthermore, the method also includes the step of enabling unified web access for multi-physics simulation engines: Deploy a VNC server for each connected physics simulation engine; The VNC server captures the 3D rendering output of each simulation engine and encodes it into a video stream; Integrate and display all video streams in a unified web workbench so that users can view and operate different simulation instances in the same interface.

[0017] Furthermore, the steps for generating the meta-action mapping file include: The URDF file is parsed, a component registry is established, which records each physical component and its physical attributes, and each executable component is associated with its unique identifier in the URDF. For each meta-action, the corresponding executable component is matched in the component registry based on its execution component field, and a structured mapping file is generated.

[0018] Furthermore, in this method, the status of the URDF file used is continuously monitored; when a file is detected to be modified, replaced, or updated, a URDF change event is immediately triggered, the new URDF file is re-parsed, and the meta-action mapping file is updated.

[0019] Furthermore, the multi-simulation engine adapter includes at least one of the Isaac Sim adapter, Gazebo adapter, and MuJoCo adapter. The multi-simulation engine adapter receives a set of control instructions, converts the instructions into calls to the native API of the target simulation engine, and standardizes the raw state data returned by the simulation engine before feeding it back.

[0020] Furthermore, a separate VNC port mapping script is launched for each running simulation engine instance to map the simulation engine's access port to the VNC server, capture the 3D rendering window in real time, and encode it as an H.264 / H.265 video stream.

[0021] Furthermore, the encoded video stream is pushed to a unified web workbench in real time via WebSocket or a proprietary TCP-based protocol.

[0022] Secondly, the present invention also provides an embodied intelligent robot simulation system that spans multiple physics simulation engines. The system includes: a general meta-action library, a URDF-based meta-action mapping module, a simulation logic setting and execution module, and a multi-simulation engine adapter. The above-mentioned method is used to perform embodied intelligent robot simulation.

[0023] Furthermore, the system also includes a unified web interface and a VNC server for unified web access to multiple physics simulation engines.

[0024] As can be seen from the above technical solution, compared with the prior art, the beneficial effects of the present invention include: 1. This invention provides a cross-physics simulation engine-based embodied intelligent robot simulation method and system, which can achieve repeatable operation of the same robot task or control strategy in multiple heterogeneous physical simulation environments without increasing additional repetitive development work, thereby significantly reducing the simulation workload and greatly improving the efficiency of simulation development and verification.

[0025] 2. This invention not only reduces the repetitive workload of developers, but also provides operational convenience and reliability for multi-physics engine verification of robot algorithms, thereby powerfully promoting embodied intelligent simulation technology from "laboratory demonstration" to "large-scale, standardized engineering application".

[0026] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0027] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0030] Figure 1 This is a schematic diagram illustrating the working principle of the embodied intelligent robot simulation method using a cross-physics simulation engine provided in this embodiment of the invention.

[0031] Figure 2 This is a schematic diagram of the electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0033] In the description of this invention, it should be noted that some processes described in this application specification and drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may be performed in any order or in parallel. Furthermore, various numbers are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0034] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0035] See Figure 1 As shown, this invention discloses a simulation method and system for embodied intelligent robots across physics simulation engines. Specifically, this invention aims to overcome key technical deficiencies in the following two aspects: First, how to eliminate the repetitive work of manually rebuilding simulation logic, models, and interface integration for each simulation engine, and achieve "define once, run in multiple places" for simulation logic.

[0036] In existing technologies, the model, motion logic, and task orchestration of the same robot need to be repeatedly modeled, adapted, and debugged in different simulation engines (such as Isaac Sim, Gazebo, and Mujoco). This not only wastes manpower and time but also introduces potential errors and inconsistencies due to implementation differences between different engines. This invention aims to completely decouple task-oriented simulation logic from the specific implementation of the underlying physics engine by establishing a universal meta-motion library and an automatic mapping mechanism based on URDF. Users only need to define a standardized task flow and motion sequence independent of the engine once, and the system can automatically adapt and map it to different physics simulation engines for execution, thereby fundamentally eliminating repetitive work and achieving the generalization and reusability of simulation assets.

[0037] Second, how to enable access to and operation of the multiphysics simulation engine under a unified web interface, and how to enable free switching between the multiphysics simulation engines under a unified interface.

[0038] Currently, each physics simulation engine typically provides its own independent visualization client or interface, lacking a unified access point. Developers need to switch between different interfaces, tools, and even operating environments to monitor and operate simulations running on different engines. This invention aims to integrate the 3D rendering output and simulation control functions of all connected physics simulation engines into a unified web workbench by constructing a unified web interface based on VNC. Users can simultaneously view, manipulate, and compare simulation instances running on different physics engines within the same browser, achieving zero-cost, seamless switching and comparison verification between engines. This greatly simplifies the simulation operation process and improves the efficiency and experience of multi-engine collaborative work.

[0039] By overcoming the two key technical shortcomings mentioned above, this invention can not only reduce the repetitive workload of developers, but also provide operational convenience and reliability for multi-physics engine verification of robot algorithms, thereby powerfully promoting embodied intelligent simulation technology from "laboratory demonstration" to "large-scale, standardized engineering application".

[0040] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings: In this embodiment of the invention, to achieve "define once, run in multiple places" for simulation logic, the core issue is to decouple the robot's specific interface and protocol from the simulation business logic. To this end, the present invention proposes the following two strategies: 1. A strategy for implementing simulation logic "defined once, run in multiple places", the steps of which include: defining a meta-action library → generating a mapping based on URDF → executing through an adapter.

[0041] 2. A strategy for achieving unified Web access to multiple physics simulation engines, comprising the following steps: deploying VNC service → capturing encoded video streams → integrating and displaying them on the Web client.

[0042] To achieve these two strategies, this invention focuses on proposing and implementing the following three technical means: 1. Definition and construction of the general meta-action library: To address the issue of strong binding between simulation business logic and underlying hardware and engine protocols, this invention first establishes a general-purpose meta-action library that is completely independent of the physical simulation engine. Each meta-action is an abstract description of the robot's most basic executable function.

[0043] A. Standardized Meta-Action Definition: Each meta-action entry contains three core fields: ActionCode: A unique identifier, such as ARM_MOVE_TO_POSE, BASE_NAVIGATE_TO_POINT. Description: A human-readable functional semantic description, such as "control the end effector of the robotic arm to move to the specified 3D spatial pose." This description also serves as the semantic context for interacting with the larger model. Target Component: Specifies which part of the robot the action acts upon, such as right_arm_end_effector, mobile_base, head_camera. This field establishes the association between high-level actions and physical components by referencing the component names defined in the URDF.

[0044] B. Integration with Simulation Logic Settings: Based on this meta-action library, developers can directly use these standardized action codes to orchestrate task flows (such as navigation and grabbing) within the simulation logic settings execution module. This ensures that business logic is entirely built on the description of action semantics, rather than specific motor commands or engine API calls.

[0045] 2. URDF-based meta-action mapping mechanism: To transform abstract meta-actions into physical commands that can be executed on a specific robot, this invention proposes an automated mapping mechanism based on the standard URDF (Unified Robot Description Format).

[0046] URDF Parsing and Component Registration: Driven by the URDF change detection module, the system automatically parses the robot's URDF description file. This file precisely defines all the robot's links, joints, and transmissions, along with their physical properties. Based on this, the system establishes a global "component registry," associating each executable component (such as a rotary joint motor) with its unique identifier in the URDF.

[0047] Generating Meta-Action Mapping Files: The URDF-based meta-action mapping module is the core converter. It reads the general meta-action library and, based on the component definitions in the URDF, automatically or semi-automatically finds one or more corresponding physical joints / motors for each meta-action's "Executor" field. For example, if the "Executor" of the meta-action ARM_MOVE_TO_POSE is right_arm_end_effector, the mapping module will calculate all the joints (such as shoulder, elbow, wrist, etc.) required to control the end effector pose based on the kinematic chain in the URDF, generate a mapping relationship for each joint, and finally output a structured mapping file (such as YAML or JSON format).

[0048] The decoupling effect of the motion execution engine: The generated meta-motion mapping file is used by the motion execution engine. At runtime, this engine receives meta-motion invocation instructions (including motion codes and parameters, such as target coordinates and velocity) from the upper-layer business logic. The engine does not need to understand the communication protocols or data formats of different simulation engines; it only needs to consult the mapping file to convert the abstract "motion code + parameters" into a set of specific "motor index + control values ​​(such as angular velocity and position)". This conversion process is entirely based on the mapping file, achieving complete isolation between the business logic layer and the underlying protocol transport layer.

[0049] 3. Multi-simulation engine adapter: To deliver unified motor control commands to different physics simulation engines, this invention designs a pluggable adapter layer.

[0050] A unified adapter interface: Define an abstract adapter base class that specifies the methods that all concrete adapters must implement, such as execute_motor_command(motor_index, control_value), get_sensor_data(), reset_simulation(), etc.

[0051] Dedicated adapter implementation: For each supported physics simulation engine, a specific adapter is developed (e.g., Isaac Sim adapter, Gazebo adapter, Mujoco adapter). The core functionality of each adapter is: Protocol conversion: Receives a set of standard motor control commands from the motion execution engine.

[0052] API calls: These instructions are translated into calls to the target simulation engine's native API. For example, for Isaac Sim, the joint controller in omni.isaac.core is called; for Gazebo, control messages are published via ROS2 services or topics; and for Mujoco, control variables in mujoco.MjData are set directly.

[0053] Status feedback: The raw status data (such as joint angles and torques) returned by the simulation engine is standardized and fed back to the upper-level system.

[0054] For data collected from the robot, this invention defines a list of metric data independent of the physical simulation engine. The data in this list serves as standardized data independent of the specific robot and is used for data transfer and calculation between upper-level logic. When a piece of data from the simulation engine is obtained, the data from the specific simulation engine is converted and defined according to the predefined standard data format, thereby achieving data standardization.

[0055] Table 1 Standardized Measurement Data

[0056] Table 1 above is a list of standardized metric data defined in this proposal.

[0057] Transparent Execution: Through the adapter layer, the action execution engine can drive different simulation engines in exactly the same way. The task logic and meta-action sequences defined by the user in the upper layer are converted into standard motor instructions through mapping files and can be executed by any connected adapter, thereby achieving the goal of "the same set of business logic can run in any engine such as Isaac Sim, Gazebo, and Mujoco without modification".

[0058] In this embodiment, to solve the problem of scattered and inconvenient operation of multi-engine visualization interfaces, the present invention combines VNC (Virtual Network Computing) technology with Web front-end technology to build a unified remote access portal.

[0059] VNC Server Deployment: A VNC server is pre-installed and configured on the server or container running the various physics simulation engines (such as Isaac Sim, Gazebo, Mujoco). When the simulation engine adapter starts a simulation instance, the VNC server module simultaneously starts a separate VNC server instance and binds it to the graphical output (such as a 3D rendering window) of that simulation engine.

[0060] Video Stream Capture and Encoding: The VNC server captures high-resolution 3D images rendered by the simulation engine in real time. Subsequently, an efficient video encoder (such as using the FFmpeg library) encodes the raw image frames into a highly compressed video stream in real time (usually using H.264 or H.265 encoding) to significantly reduce network transmission bandwidth.

[0061] WebSocket streaming media transmission: The encoded video stream is pushed to a remote web server in real time via WebSocket or a proprietary TCP-based protocol.

[0062] Unified Web Interface Integration: The unified front-end web interface (developed based on frameworks such as React / Vue) embeds one or more VNC client components. These components connect to the corresponding VNC streams via WebSocket, receive and decode the video streams, and render the simulation images in real time in the browser. Simultaneously, the web interface provides a unified control panel, project management, and status monitoring functions. Users can view and control multiple simulation instances running on different physics simulation engines simultaneously through a single browser tab, and switch seamlessly without needing to install any engine-specific native clients.

[0063] Based on the above methods, this invention designs an embodied intelligent robot simulation system, which mainly includes the following technical components: 1. Unified Web interface and VNC service: This module provides a unified web access interface across engines, enabling centralized visualization and management of simulation tasks.

[0064] In the server deploying the physics simulation engine, a VNC server is installed, and an independent VNC port mapping script is started for each running simulation engine to map the simulation engine's access port to the VNC server. The 3D rendering window is captured in real time, encoded as an H.264 / H.265 video stream, and transmitted to the web client via WebSocket / TCP. This achieves unified embedding of different physics simulation engines within the browser interface, enabling unified access to various physics simulation engines through a single web client, without requiring a separate client for each engine.

[0065] 2. Project changes are synchronized: To ensure the consistency of simulation assets, the system needs to be able to detect and respond to changes in the underlying model by continuously monitoring changes to the simulation project's URDF file and triggering a reconstruction.

[0066] URDF Change Detection: This module continuously monitors the status of URDF files used in the project. When a file is detected to have been modified, replaced, or updated, a URDF change event is immediately triggered.

[0067] 3. URDF-based meta-action mapping: Upon receiving a change event, the project change synchronization module automatically coordinates relevant components, notifying the URDF-based meta-action mapping module to re-parse the new URDF file and update the meta-action mapping file. It also notifies the simulation logic settings execution module and action execution engine, which may require reloading or verification of logic compatible with the new model. This ensures that all running or subsequently started simulation instances are based on the latest model and mapping configurations, maintaining the internal consistency of the simulation project.

[0068] 4. Simulation logic settings execution: At the heart of the business simulation logic setup is a runtime environment that supports Python and ROS2 communication models. User-written simulation business logic scripts (or scripts generated by graphical tools) are executed within this environment. These scripts orchestrate tasks by calling interfaces from a generic meta-action library.

[0069] In the action execution engine, the simulation logic is executed according to the script set by the business simulation logic. For commands that need to be sent, they are sent through the simulation robot. The simulation robot will automatically set the command input to the corresponding physical simulation engine through the simulation engine adapter according to the corresponding motor index set in the meta-action mapping file, thereby realizing the execution from the upper-level business logic to the specific simulation engine.

[0070] To receive the execution result of an action from the physics simulation engine, this invention employs a publish / subscribe communication model. Motor commands are published and subscribed between the action execution engine and the physics simulation engine through predefined ROS2 topics (e.g., / motor_commands), thus achieving decoupled communication. Once an action is completed in the physics simulation engine, a command execution result message is automatically sent to this topic. The action execution engine, by listening to this topic, can continuously obtain command execution result messages and determine the next execution logic.

[0071] When the system is applied, the entire process can be divided into four stages: definition, mapping, execution, and monitoring.

[0072] Definition phase: Users set up execution modules using simulation logic on a unified web interface, and then create engine-independent task flows based on a general meta-action library.

[0073] Mapping phase: The URDF-based meta-motion mapping module automatically generates mapping files from all meta-motions in the task flow to specific motor controls based on the specific robot URDF model.

[0074] Execution Phase: The user selects one or more target simulation engines. The motion execution engine loads the task logic and mapping files, transforming abstract actions into standard motor instructions. The corresponding simulation engine adapter (such as the Isaac Sim adapter) receives the instructions and drives the simulated robot in the physical simulation engine (such as Isaac Sim) to run.

[0075] Monitoring Phase: Simulation footage is streamed in real-time via a VNC server and displayed in a unified web interface. The URDF change detection and project change synchronization modules ensure timely synchronization of model changes in the background.

[0076] From the description of the above embodiments, those skilled in the art will understand that the present invention provides a method and system for embodied intelligent robot simulation across physics simulation engines, the technical advantages of which include: It helps to improve the efficiency of simulation R&D and reduce the verification cost of multi-physics simulation engines; It helps to identify algorithm vulnerabilities and migration risks, driving robust algorithm design; It helps to promote the accumulation, reuse and standardization of simulation assets and R&D knowledge.

[0077] Additionally, refer to Figure 2As shown, this embodiment of the invention also provides an electronic device, which may include a processor, a memory, a communication bus and a communication interface, and may also include a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the above-described method.

[0078] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, software systems, electronic devices, or computer program products, etc. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0080] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for embodying intelligent robot simulation across physics simulation engines, characterized in that, The method includes the following steps: Construct a generic meta-action library that is independent of the physics simulation engine. Each meta-action contains action code, action description, and execution part, where the execution part refers to the part name defined in the robot's URDF file. Based on the URDF file and the general meta-action library, a meta-action mapping file is generated, which is used to map the execution component of the meta-action to a specific physical component. The task flow is orchestrated based on the general meta-action library, and the task flow is converted into a specific set of control instructions according to the meta-action mapping file. The control command set is converted into native API call commands of the target physics simulation engine through a pluggable multi-simulation engine adapter and sent to the target physics simulation engine for execution.

2. The method according to claim 1, characterized in that, The method also includes steps to achieve unified web access for multi-physics simulation engines: Deploy a VNC server for each connected physics simulation engine; The VNC server captures the graphical output of each simulation engine and encodes it into a video stream. Integrate and display all video streams in a unified web workbench so that users can view and operate different simulation instances in the same interface.

3. The method according to claim 1, characterized in that, The steps to generate a meta-action mapping file include: The URDF file is parsed, and a component registry is established. The component registry records each physical component of the robot and its physical attributes, and each executable component is associated with its unique identifier in the URDF. For each meta-action, the corresponding executable component is matched in the component registry based on its execution component field, and a structured mapping file is generated.

4. The method according to claim 3, characterized in that, In this method, the status of the URDF file used is continuously monitored; when the file is detected to be modified, replaced or updated, the URDF change event is immediately triggered, the new URDF file is re-parsed, and the meta-action mapping file is updated.

5. The method according to claim 1, characterized in that, The multi-simulation engine adapter includes at least one of the Isaac Sim adapter, Gazebo adapter, and MuJoCo adapter. The multi-simulation engine adapter receives a set of control instructions, converts the instructions into calls to the native API of the target simulation engine, and feeds back the raw state data returned by the simulation engine after standardization.

6. The method according to claim 2, characterized in that, A separate VNC port mapping script is launched for each running simulation engine instance to map the simulation engine's access port to the VNC server, capture the 3D rendering window in real time, and encode it as an H.264 / H.265 video stream.

7. The method according to claim 2, characterized in that, The encoded video stream is pushed to a unified web workbench in real time via WebSocket or a proprietary TCP-based protocol.

8. A cross-physics simulation engine embodied intelligent robot simulation system, characterized in that, The system includes: a general meta-action library, a URDF-based meta-action mapping module, a simulation logic setting and execution module, and a multi-simulation engine adapter. It uses the method described in any one of claims 1–7 to perform embodied intelligent robot simulation.

9. The system according to claim 8, characterized in that, The system also includes a unified web interface and a VNC server for unified web access to multiple physics simulation engines.

Images