Humanoid robot simulation environment based on Isaac Sim and implementation method thereof

Abstract

The application discloses a humanoid robot simulation environment based on Isaac Sim and an implementation method thereof, relates to the technical field of robot simulation, and is suitable for construction and verification of an industrial inspection laboratory scene. Detailed parameters of equipment and obstacles in an industrial scene are collected, a 3D modeling tool is used to accurately construct equipment, obstacle and scene layout models, and topological optimization and material detail processing are performed, then a humanoid robot model is deployed into the simulation environment, motion, interaction and experimental operation of the robot in the laboratory are simulated through an integrated control system and a high-precision physical engine, and real-time motion control of the robot in simulation is realized in combination with ROS2 and MoveIt2. The application improves the accuracy and reliability of task execution of the robot in a laboratory environment, and provides effective verification and support for application of the robot to an actual industrial experimental scene.

Description

Technical Field

[0001] This invention relates to the field of robot simulation technology, specifically to a humanoid robot simulation environment based on Isaac Sim and its implementation method. Background Technology

[0002] With the continuous advancement of intelligent manufacturing and industrial automation, humanoid robots are gradually becoming an important means of replacing dangerous jobs and assisting humans in their work. In the industrial field, humanoid robots can perform complex tasks such as operating machinery, handling goods, and inspecting equipment. However, due to the complex and ever-changing industrial environment, robot design and control face many challenges. How to accurately simulate robot behavior in a simulation environment and improve the reliability and efficiency of its task completion has become an urgent problem to be solved.

[0003] Existing simulation tools and development environments, such as ROS and Gazebo, can simulate some robot functions, but they still have many shortcomings in simulating the complex behaviors of humanoid robots, especially in the simulation of interaction and collaboration in industrial environments.

[0004] The shortcomings of existing robot simulation methods are:

[0005] In patent document CN108537875B, robot simulation equipment is mainly used to make the simulation results close to the results obtained during actual operation, but it cannot build a realistic and fully functional simulation environment. Summary of the Invention

[0006] The purpose of this invention is to provide a humanoid robot simulation environment based on Isaac Sim and its implementation method, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a humanoid robot simulation environment based on Isaac Sim and its implementation method, comprising the following steps:

[0008] Step S1: Use professional modeling tools to design and construct equipment and obstacles in the industrial environment in detail to form a preliminary scene model;

[0009] Step S2: Unfold the materials of the equipment and obstacles constructed in the initial scene model, and refine them using a high-quality texture painting tool to obtain the simulation model;

[0010] Step S3: Based on the actual factory scene requirements, use the simulation model obtained in step S2 to build an industrial scene, including the overall layout of the scene, the setting of light sources and the configuration of physical characteristics, and complete the setup of the simulation environment.

[0011] Step S4: Use 3D modeling tools to build a humanoid robot model, ensuring that the model's skeleton and joint structure are consistent with the actual robot;

[0012] Step S5: Deploy the humanoid robot model into the designed simulation environment, and achieve precise control of the robot through the integrated control system to simulate the robot's movement, interaction and work task execution process in the industrial environment.

[0013] Preferably, step S1 further includes the following steps:

[0014] Step S1-1: Collect detailed parameters of existing equipment and obstacles in the actual industrial scenario, and record these parameters as basic data for modeling;

[0015] Step S1-2: Create 3D models of equipment and obstacles in Siemens NX based on the collected modeling data;

[0016] Steps S1-3: Import the 3D model created in Siemens NX into Maya for detail processing and optimization.

[0017] Preferably, step S2 further includes the following steps:

[0018] Step S2-1: Import the completed 3D model into RizomUV, unfold the UVs, and ensure that the UV layout is reasonable and there is no stretching or overlapping.

[0019] Step S2-2: Export the UV-unwrapped model and import it into Adobe Substance 3D Painter. Add base material layers to each part of the model to simulate the materials of real industrial equipment.

[0020] Steps S2-3: After adding the basic material layer, continue to refine the material details in Substance 3D Painter, adding texture effects according to the actual appearance of the device to make the model surface look realistic.

[0021] Preferably, step S3 further includes the following steps:

[0022] Step S3-1: Plan the overall layout of the simulation scene according to the required actual industrial scenario, and determine the relative positions and arrangement of each piece of equipment, obstacle and other important elements;

[0023] Step S3-2: Import the models of each device and obstacle into the scene in Unreal Engine, and place the models according to the layout plan so that the constructed scene structure conforms to the actual industrial environment.

[0024] Step S3-3: Set scene light sources. Add different types of light sources to the constructed simulation environment according to the lighting conditions in the actual industrial environment, including natural light, point light sources and spotlights.

[0025] Steps S3-4: Import the scene and device models into Isaac Sim, configure the physical properties for each element, and adjust the physical parameters of each element in Isaac Sim to match the physical parameters of the actual elements.

[0026] Preferably, in step S4, during the modeling process, the robot is broken down into multiple independent components for modeling, modification, or updating.

[0027] Preferably, step S5 further includes the following steps:

[0028] Step S5-1: Import the humanoid robot model into the Isaac Sim simulation environment and deploy it into the constructed industrial scene;

[0029] Step S5-2: Configure the robot's physical parameters in Isaac Sim, including the range of motion of the joints, speed, center of gravity, moment of inertia and torque limits, to ensure that the robot's motion in the simulation conforms to the actual situation;

[0030] Step S5-3: Connect the robot control interface with the Isaac Sim simulation interface by writing or importing control algorithms to achieve precise control of the robot joints;

[0031] Step S5-4: Use ROS2 and MoveIt2 to set up robot motion control nodes, ensuring that each node can adjust the robot's posture, speed and position in real time to achieve precise control.

[0032] Preferably, in step S1-1, the detailed parameters include the size, material, coefficient of friction, elasticity, weight, and structural details of the device and the obstacle;

[0033] In steps S1-3, detail processing involves adding bolts and welding points to the constructed 3D model based on the appearance of equipment and obstacles in the actual industrial scene, while optimization involves optimizing the topology of the model and deleting redundant vertices or faces.

[0034] Preferably, in steps S2-3, the added texture effects include scratches, wear, and rust.

[0035] Preferably, step S5-2 further includes setting the friction force of the ground to simulate the walking effect of the robot on different ground materials.

[0036] Preferably, step S5 further includes simulating the behavior and interaction of multiple robots in the same industrial scenario.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] 1. This invention achieves high-precision simulation of industrial environments by using professional modeling tools and physics engines, improving the accuracy and reliability of robot task execution in different environments. At the same time, it constructs a realistic and fully functional simulation environment, providing strong support and guarantee for the research and development of humanoid robots.

[0039] 2. This invention, through refined material processing and texture rendering, enables the model to have a realistic visual effect, enhances the realism of the overall simulation environment, and ensures that the simulation process is consistent with the real world. Only then can the simulation results accurately reflect the actual movement of the robot, and to a certain extent, provide an effective reference for robot design and control.

[0040] 3. This invention achieves high-level interaction and real-time feedback between the robot and the scene by simulating various complex environments and conditions. Through testing in the simulation environment, potential problems that the robot may encounter during actual operation can be identified in advance, thereby reducing the cost and risk of actual experiments. At the same time, the simulation environment can also provide a repeatable testing environment, helping researchers to better evaluate and compare the performance of different algorithms and designs, and providing strong support for the research and development of humanoid robots.

[0041] 4. This invention ensures real-time and accurate control by simulating the movement, interaction, and task execution of robots in an industrial environment. It can also simulate the behavior and interaction of multiple robots in the same scene to evaluate their collaborative ability and performance. Furthermore, it collects and analyzes data in real time during the simulation process to promptly identify and resolve problems and optimize robot performance. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the implementation method of the present invention;

[0043] Figure 2 This is a schematic diagram of the simulation environment of the testing laboratory built according to the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method. Step S1: Use professional modeling tools to design and construct equipment and obstacles in the industrial environment in detail to form a preliminary scene model.

[0046] Step S2: Unfold the materials of the equipment and obstacles constructed in the initial scene model, and refine them using a high-quality texture painting tool to obtain the simulation model;

[0047] Step S3: Based on the actual factory scene requirements, use the simulation model obtained in step S2 to build an industrial scene, including the overall layout of the scene, the setting of light sources and the configuration of physical characteristics, and complete the setup of the simulation environment.

[0048] Step S4: Use 3D modeling tools to build a humanoid robot model, ensuring that the model's skeleton and joint structure are consistent with the actual robot;

[0049] In step S4, during the modeling process, the robot is broken down into multiple independent components for modeling, modification, or updating;

[0050] Step S5: Deploy the humanoid robot model into the designed simulation environment, and achieve precise control of the robot through the integrated control system to simulate the robot's movement, interaction and work task execution process in the industrial environment.

[0051] Furthermore, Siemens NX was used to design and construct commonly used laboratory facilities such as lab tables, testing equipment, and storage racks in a testing laboratory. These models were designed to meet industry standards in terms of size and physical properties. This process was based on measurements and observations in a real laboratory, collecting detailed parameters including dimensions, materials, and structure to ensure the models met the needs of actual laboratories. In Siemens NX, 3D models of equipment and obstacles such as lab tables, microscopes, and analyzers were created based on the collected data, strictly adhering to the dimensions and structure of the real equipment to ensure accuracy. The models were then imported into Maya for further detail optimization, such as adding small parts, seams, and supports to the lab tables to make them more realistic. Subsequently, the model's topology was optimized in Maya to ensure a reasonable number of faces, removing redundant vertices or faces to improve simulation efficiency in subsequent processing.

[0052] After the model is completed, it is imported into RizomUV for UV unwrapping to ensure that the UV layout is reasonable, without stretching or overlap, laying the foundation for subsequent texture processing. Then, the unwrapped model is imported into Adobe Substance 3D Painter to add basic materials such as metal, rubber and plastic to different parts of the laboratory equipment to simulate the materials of real laboratory instruments. Texture details such as scratches, wear and slight rust are added to the materials to give the surface of the equipment a realistic feel and reflect the traces of daily use of the laboratory equipment.

[0053] The entire scene layout is planned according to the actual needs of the testing laboratory, ensuring that the positions of the experimental tables, instruments and equipment and aisles meet the experimental operation specifications. Then, the equipment models are imported into the simulation scene in Unreal Engine, and the experimental tables, testing instruments and storage racks are placed according to the planned layout. Next, the lighting conditions in the laboratory environment are set up to simulate the lighting conditions, including conventional overhead lights and local work lights, to ensure that the lighting effects can highlight the key equipment areas. Physical properties are configured for each device in the scene. The scene and equipment models are imported into Isaac Sim, and physical parameters such as mass, friction coefficient, and elasticity are set for each element so that each device and obstacle exhibits realistic physical behavior in the simulation environment.

[0054] Next, a humanoid robot model was constructed and deployed in a simulated laboratory environment. The robot's skeleton and joint structure were ensured to match the actual equipment. Then, the robot's physical parameters, including joint range of motion, speed, and torque limits, were configured in Isaac Sim to ensure realistic robot movement that met laboratory work requirements. Subsequently, the robot control system was interfaced with Isaac Sim to achieve precise control of the robot, enabling it to perform different tasks in the laboratory environment. Robot motion control nodes were set up using the ROS2 and MoveIt2 platforms to ensure that each node could adjust the robot's posture, speed, and position in real time to meet the precise control requirements of tasks within the laboratory.

[0055] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method, wherein step S1 further includes the following steps:

[0056] Step S1-1: Collect detailed parameters of existing equipment and obstacles in the actual industrial scene, and record these parameters as the basic data for modeling. The detailed parameters include the size, material, friction coefficient, elasticity, weight and structural details of the equipment and obstacles.

[0057] Step S1-2: Create 3D models of equipment and obstacles in Siemens NX based on the collected modeling data;

[0058] Steps S1-3: Import the 3D model created in Siemens NX into Maya for detail processing and optimization. Detail processing involves adding bolts and welding points to the 3D model based on the appearance of equipment and obstacles in the actual industrial scene. Optimization involves optimizing the topology of the model and deleting redundant vertices or faces.

[0059] Furthermore, using a laser rangefinder or calipers, the linear dimensions such as length, width, height, and diameter of various equipment and obstacles in the laboratory are precisely measured to obtain accurate external dimensional data. The dimensions of key parts of the equipment, such as the size of the opening or the internal cavity, are recorded. Then, by consulting the equipment's instruction manual, nameplate, or relevant maintenance personnel, it is determined whether the main body of the equipment and obstacle is made of metal, plastic, composite material, or other special materials. Next, the typical values ​​of the friction coefficient of the corresponding material under different contact conditions are found by referring to the material's standard manual or relevant engineering data, or the approximate friction coefficient is estimated through simple friction test experiments on site. For elastic properties, the key elastic parameters such as the elastic modulus are determined according to the material data, or the elastic performance of similar materials in actual use is referenced to record relevant data.

[0060] Weight measurement utilizes weighbridges or electronic scales to measure the weight of equipment and obstacles. When recording weight data, it is important to clarify whether it is the net weight or the total weight including accessories and components to ensure that the data is clear and accurate. Structural detail observation requires careful examination of the internal and external structures of equipment and obstacles, recording the connection methods of their components, such as bolted connections, welding or riveting, as well as the shape and relative position of each component. For objects with multi-layered nested structures or complex internal structures, tools such as endoscopes can be used to assist in observation. At the same time, photos and drawings should be taken to record detailed structural features so that subsequent modeling can restore the true appearance.

[0061] After data collection, open Siemens NX software, create a new modeling project file, and uniformly set the modeling units in the software according to the units used in the collected parameters, such as millimeters, centimeters, or meters, to avoid unit conversion errors that could lead to model size deviations. Then, based on the recorded overall shape of the equipment and obstacles, use NX to assemble and build the model, and use features such as stretching, rotating, and sweeping to transform the 2D contours into 3D solids for constructing components with complex shapes, ensuring that the external dimensions match the collected data. Based on the recorded structural details, use Boolean operations to simulate the connections between components and internal structures such as cavities and hollows. If the equipment has two components connected by bolts, first create solid models of both components, then use Boolean operations to drill holes at the corresponding locations to simulate the structure of bolt mounting holes. Add various detailed features, such as chamfers, fillets, and reinforcing ribs, to make the model more closely resemble the appearance and structural characteristics of actual industrial products. During the process of adding features, continuously check and verify against the collected parameters to ensure model accuracy. Then, use Siemens NX software to complete the modeling process. The 3D model created in NX is exported in a compatible format. Then, the corresponding import option is selected in Maya software to successfully import the model into the Maya working environment. Based on the observation and recording of the bolt and weld point positions and styles in the appearance of equipment and obstacles in actual industrial scenes, the corresponding bolt models are created in Maya using modeling tools. These models are then accurately placed in the corresponding mounting hole positions, and reasonable dimensions and appearance details are set. For weld points, by creating polygonal patches or using sculpting tools, the appearance of weld marks that conform to reality is sculpted at the weld position to make it look more realistic and reflect the process characteristics of actual industrial products. At the same time, the topology tools in Maya can be used to analyze the existing topology of the model and find redundant vertices, patches, and other parts that affect the model's performance and subsequent use. During the optimization process, it is important to maintain the integrity of the model's appearance and structure and avoid over-optimization that may lead to the loss of key details. At the same time, the rendering effect of the model is monitored in real time to ensure that the optimized model is both concise and meets the requirements of appearance realism in actual application scenarios.

[0062] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method. In step S2, the following steps are also included:

[0063] Step S2-1: Import the completed 3D model into RizomUV, unfold the UVs, and ensure that the UV layout is reasonable and there is no stretching or overlapping.

[0064] Step S2-2: Export the UV-unwrapped model and import it into Adobe Substance 3D Painter. Add base material layers to each part of the model to simulate the materials of real industrial equipment.

[0065] Steps S2-3: After adding the base material layer, continue to refine the material details in Substance 3D Painter. Add texture effects according to the actual appearance of the device to make the model surface look realistic. The added texture effects include scratches, wear and rust.

[0066] Furthermore, after initially building the model, open the RizomUV software and import the 3D model, which has been detailed and optimized in Maya, into the RizomUV workspace in its supported file format. Using RizomUV's built-in automatic UV unwrapping function, the software will map the model's 3D surface onto a 2D plane based on a certain algorithm, generating an initial UV layout. Model parts of different shapes and structures will have corresponding UV blocks. Then, carefully check the UV layout after automatic unwrapping. Manually adjust UV blocks that exhibit unreasonable phenomena such as stretching or overlapping. For some relatively regular cuboid parts, if stretching occurs after automatic unwrapping, you can select the corresponding UV boundary and use operations such as moving, rotating, and scaling to adjust the UV blocks to a suitable shape and position, so that the texture mapping can be kept as uniform as possible, avoiding obvious stretching deformation, and ensuring that the texture can be displayed correctly in subsequent applications.

[0067] After completing UV unwrapping and confirming a reasonable layout in RizomUV, export the model in a format suitable for import into Adobe Substance 3DPainter. Ensure that the UV coordinate information is completely preserved and exported along with the model so that the software can accurately recognize and apply the UV layout for material addition and texture painting. Then, open Adobe Substance 3DPainter, select the import function, and import the previously exported model file with UV coordinates into the workspace. The model will then be presented in 3D, and its UV layout will be available for subsequent operations. Based on the material information of the actual industrial equipment collected earlier, select the corresponding base material type from the software's material library and apply it to the corresponding parts of the model as the base material layer. For different parts, such as shells and internal structural components, add the corresponding base materials according to the actual material conditions. Use the software's material assignment tool to accurately assign materials to different parts of the model, initially simulating the overall material appearance of the equipment. Based on the observed appearance of the industrial equipment, use Substance 3DPainter on the model with the added base material layers. Painter's texture drawing tools can be used to add texture effects such as scratches, wear, and rust. For scratch textures, you can use the brush tool, select a suitable scratch brush style, and set parameters such as the depth, width, and direction of the scratches. Then, you can manually draw the scratches on the model surface to simulate the areas where the device may develop scratches during long-term use. Alternatively, you can use the software's intelligent generation function to randomly generate scratch distributions on the model surface based on a certain algorithm. Wear effects can be created by adjusting the material's roughness, color, and other properties, as well as by combining techniques such as masking, to create the appearance of wear on areas of the model that are frequently in contact with and rubbed. For rust textures, you can select suitable rust texture resources from the software's texture library and add rust to the corresponding parts of the model through overlay, blending, and other methods. For example, for the bottom of steel parts that are prone to moisture and rust, or in the gaps, the rust will be presented in accordance with the actual rusting pattern, enhancing the realism of the model surface.

[0068] After adding various texture effects, the overall effect of the material is adjusted, including fine-tuning the blending ratio, transparency, and light reflection characteristics between different texture effects. This ensures that the material details of the entire model are harmonious and unified, and more realistically simulates the actual material appearance of industrial equipment under the influence of long-term use and environmental factors. This allows the constructed 3D model to be highly consistent with the actual industrial equipment in terms of material and appearance, providing more expressive and accurate model resources for subsequent related industrial applications.

[0069] Please see Figure 1 and Figure 2The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method. In step S3, the following steps are also included:

[0070] Step S3-1: Plan the overall layout of the simulation scene according to the required actual industrial scenario, and determine the relative positions and arrangement of each piece of equipment, obstacle and other important elements;

[0071] Step S3-2: Import the models of each device and obstacle into the scene in Unreal Engine, and place the models according to the layout plan so that the constructed scene structure conforms to the actual industrial environment.

[0072] Step S3-3: Set scene light sources. Add different types of light sources to the constructed simulation environment according to the lighting conditions in the actual industrial environment, including natural light, point light sources and spotlights.

[0073] Steps S3-4: Import the scene and device models into Isaac Sim, configure the physical properties for each element, and adjust the physical parameters of each element in Isaac Sim to match the physical parameters of the actual elements.

[0074] Furthermore, a thorough study of the actual industrial scenario to be simulated is conducted, collecting relevant layout drawings, on-site photos, and process flow diagrams to fully understand the types and quantities of equipment, the distribution of obstacles, and the location information of important elements such as passageways, operating areas, and material storage points. The working relationships and spatial interactions between these elements are clarified. Using fixed reference points in the actual industrial scenario as a benchmark, the coordinate positions and arrangement of important elements such as equipment and obstacles are determined according to scale. Then, a preliminary simulation scenario layout sketch is drawn, or a 2D layout diagram is created using professional layout planning software. The names, coordinates, and key dimensions of each element are labeled. Next, Unreal Engine software is opened. In the corresponding resource import window for the project, the previously built 3D model files of each device and obstacle are selected. The model import operation is completed according to the software prompts. During the import process, it is necessary to ensure that the model's geometry, materials, UV coordinates, and other key information are loaded completely and accurately. Based on the previously planned layout diagram, in Unreal Engine... In Unreal Engine's scene view, you can use transformation tools such as translation, rotation, and scaling to accurately place each device and obstacle model in its corresponding position and adjust their posture to match the actual industrial environment. During the placement process, you can use the alignment and snapping functions provided by Unreal Engine to conveniently and accurately position the models, improving placement efficiency and accuracy. At the same time, you can also view the overall scene effect from different perspectives, check whether the relative positional relationship between the models is reasonable and whether it conforms to the actual spatial layout. If necessary, you can fine-tune the model position and posture.

[0075] Various light sources are added, and the lighting rendering effects of the scene are viewed at different times and from different perspectives. The parameters of each light source are comprehensively adjusted to ensure that the lighting distribution, contrast, and other aspects of the entire simulation scene conform to the lighting conditions of the actual industrial environment. The scene built in Unreal Engine and the equipment models therein are exported in a suitable format and then imported into Isaac Sim software. In Isaac Sim, the corresponding physical properties are configured according to the physical parameters of the actual elements. After configuring the physical properties, simple tests are performed, such as simulating the startup and movement of the equipment, and observing its physical interaction with obstacles and other elements. Check for unrealistic penetration, abnormal movement, or other phenomena. If any are found, the physical parameters are further adjusted and optimized until the physical behavior of each element matches the actual situation, thereby constructing an industrial simulation environment with realistic physical characteristics.

[0076] Please see Figure 1 and Figure 2The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method. In step S5, the following steps are also included:

[0077] Step S5-1: Import the humanoid robot model into the Isaac Sim simulation environment and deploy it into the constructed industrial scene;

[0078] Step S5-2: Configure the robot's physical parameters in Isaac Sim, including the range of motion of the joints, speed, center of gravity, moment of inertia and torque limit, to ensure that the robot's movement in the simulation is consistent with the actual situation. Set the friction of the ground to simulate the robot's walking effect on different ground materials.

[0079] Step S5-3: Connect the robot control interface with the Isaac Sim simulation interface by writing or importing control algorithms to achieve precise control of the robot joints;

[0080] Step S5-4: Use ROS2 and MoveIt2 to set up robot motion control nodes, ensuring that each node can adjust the robot's posture, speed and position in real time to achieve precise control.

[0081] Furthermore, locate the relevant function entry for model import in the Isaac Sim software interface, select the corresponding humanoid robot model file for import, and use the translation and rotation tools in Isaac Sim to place the imported humanoid robot model in a suitable position according to the layout of the actual industrial scenario and the robot's task requirements. Refer to the actual mechanical design specifications of the humanoid robot or obtain the angular range data of each joint's movement through actual measurement, such as the flexion-extension, abduction-adduction angle range of the shoulder joint, and the flexion-extension angle range of the elbow joint. Then, find the parameter setting interface for the corresponding robot joint in Isaac Sim and accurately input these actual angle range values ​​to limit the joint from exceeding its actual motion boundary in the simulation. Similarly, based on the robot's performance parameters or previous actual operation test records, determine the motion speed range of each joint when it is working normally, and measure it in units of angular velocity. In Isaac Sim software, corresponding speed limits are set for each joint. This way, the movement speed of the robot joints can simulate the real situation during simulation, avoiding unrealistic movements that are too fast or too slow. At the same time, the corresponding friction coefficient reference value is found according to the different ground materials in the actual industrial scene. Then, in the scene physics property settings of Isaac Sim, the friction coefficient parameters of the ground objects are adjusted to simulate the effect of the robot walking on different ground surfaces.

[0082] Then, in Isaac Sim, locate the control interface of the robot model and the relevant settings options for the simulation interface of the software itself. By configuring the corresponding parameters and calling relevant functions, connect the control algorithm to the simulation interface so that the control algorithm can act on the robot model in real time during the simulation, achieving precise control of the robot's joint movements. First, ensure that the ROS2 environment is correctly installed in the integrated control system. Follow the official documentation to complete the installation of ROS2 and the configuration of related environment variables. Next, install MoveIt2, a powerful toolkit for robot motion planning and control built on ROS2. Similarly, follow the official tutorial to complete its installation and initial configuration so that it can... Working in conjunction with IsaacSim and the imported humanoid robot model, in the ROS2 environment, MoveIt2's functionality is used to create motion control nodes for the humanoid robot. For nodes that control the robot's speed and position, based on the set target speed, target position, and the robot's current actual state, the motion planning algorithm provided by MoveIt2 is used to calculate the appropriate joint motion trajectory, and the corresponding speed and position control commands are sent to each joint of the robot in real time, achieving precise control of the robot's movement. This ensures that the robot can accurately complete various complex motion tasks as expected in the simulation environment, simulating the efficient and precise operation process in real-world scenarios.

[0083] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a humanoid robot simulation environment based on Isaac Sim and its implementation method. In step S5, the simulation of the behavior and interaction of multiple robots in the same industrial scene is also included.

[0084] Furthermore, the Isaac Sim simulation environment also supports multi-robot collaborative simulation and real-time data analysis. Through multi-robot collaborative simulation, the behavior and interaction of multiple robots in the same scene can be simulated, thereby evaluating their collaborative ability and performance. Data during the simulation process can be collected and analyzed in real time, helping researchers to identify and solve problems in a timely manner and optimize the performance of robots.

[0085] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for implementing humanoid robot simulation based on Isaac Sim, characterized in that: Includes the following steps: Step S1: Use professional modeling tools to design and construct equipment and obstacles in the industrial environment in detail to form a preliminary scene model; Step S2: Unfold the materials of the equipment and obstacles constructed in the initial scene model, and refine them using a high-quality texture painting tool to obtain the simulation model; Step S3: Based on the actual factory scene requirements, use the simulation model obtained in step S2 to build an industrial scene, including the overall layout of the scene, the setting of light sources and the configuration of physical characteristics, and complete the setup of the simulation environment. Step S4: Use 3D modeling tools to build a humanoid robot model, ensuring that the model's skeleton and joint structure are consistent with the actual robot; Step S5: Deploy the humanoid robot model into the designed simulation environment, and achieve precise control of the robot through the integrated control system to simulate the robot's movement, interaction and work task execution process in the industrial environment.

2. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 1, characterized in that: Step S1 also includes the following steps: Step S1-1: Collect detailed parameters of existing equipment and obstacles in the actual industrial scenario, and record these parameters as basic data for modeling; Step S1-2: Create 3D models of equipment and obstacles in Siemens NX based on the collected modeling data; Steps S1-3: Import the 3D model created in Siemens NX into Maya for detail processing and optimization.

3. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 2, characterized in that: Step S2 also includes the following steps: Step S2-1: Import the completed 3D model into RizomUV, unfold the UVs, and ensure that the UV layout is reasonable and there is no stretching or overlapping. Step S2-2: Export the UV-unwrapped model and import it into Adobe Substance 3D Painter. Add base material layers to each part of the model to simulate the materials of real industrial equipment. Steps S2-3: After adding the basic material layer, continue to refine the material details in Substance 3D Painter, adding texture effects according to the actual appearance of the device to make the model surface look realistic.

4. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 1, characterized in that: Step S3 also includes the following steps: Step S3-1: Plan the overall layout of the simulation scene according to the actual industrial scenario required, and determine the relative positions and arrangement of each piece of equipment, obstacle and other important elements; other important elements include passages, operating areas and material stacking points; Step S3-2: Import the models of each device and obstacle into the scene in Unreal Engine, and place the models according to the layout plan so that the constructed scene structure conforms to the actual industrial environment. Step S3-3: Set scene light sources. Add different types of light sources to the constructed simulation environment according to the lighting conditions in the actual industrial environment, including natural light, point light sources and spotlights. Steps S3-4: Import the scene and device models into Isaac Sim, configure the physical properties for each element, and adjust the physical parameters of each element in Isaac Sim to match the physical parameters of the actual elements.

5. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 1, characterized in that: In step S4, during the modeling process, the robot is broken down into multiple independent components for modeling, modification, or updating.

6. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 4, characterized in that: Step S5 also includes the following steps: Step S5-1: Import the humanoid robot model into the Isaac Sim simulation environment and deploy it into the constructed industrial scene; Step S5-2: Configure the robot's physical parameters in Isaac Sim, including the range of motion of the joints, speed, center of gravity, moment of inertia and torque limits, to ensure that the robot's motion in the simulation conforms to the actual situation; Step S5-3: Connect the robot control interface with the Isaac Sim simulation interface by writing or importing control algorithms to achieve precise control of the robot joints; Step S5-4: Use ROS2 and MoveIt2 to set up robot motion control nodes, ensuring that each node can adjust the robot's posture, speed and position in real time to achieve precise control.

7. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 2, characterized in that: In step S1-1, the detailed parameters include the size, material, coefficient of friction, elasticity, weight, and structural details of the device and the obstacle; In steps S1-3, detail processing involves adding bolts and welding points to the constructed 3D model based on the appearance of equipment and obstacles in the actual industrial scene, while optimization involves optimizing the topology of the model and deleting redundant vertices or faces.

8. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 3, characterized in that: In steps S2-3, the added texture effects include scratches, wear, and rust.

9. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 6, characterized in that: Step S5-2 also includes setting the friction force of the ground to simulate the walking effect of the robot on different ground materials.

10. The method for implementing humanoid robot simulation based on Isaac Sim according to claim 1, characterized in that: Step S5 also includes simulating the behavior and interaction of multiple robots in the same industrial scenario.

Images