Method for evaluating robot in simulation

Abstract

The invention provides a method for evaluating a robot in simulation, which comprises the following steps executed by an arithmetic device: obtaining a robot description file established according to physical properties of the robot, obtaining an environment description file established according to an environment, obtaining an obstacle description file, and obtaining a robot description file; the physical simulation engine establishes a virtual environment according to the environment description file and the obstacle description file, establishes a virtual robot according to the robot description file, and outputs simulation information when the virtual robot moves from the starting point area to the terminal point area according to the navigation information in the virtual environment; the robot navigation program generates and sends navigation information to the physical simulation engine according to the simulation information, and when the virtual robot arrives at the destination area, the physical simulation engine outputs evaluation information. The optimal robot suitable for the specific task can be found out, and the optimal environment is identified for the specific robot.

Description

Technical Field

[0001] The present invention relates to digital twin and robot navigation, and in particular to a method of evaluating a robot in simulation. Background Art

[0002] The market for mobile robots available for commercial use is growing rapidly. For example, robot vacuum cleaners are becoming increasingly popular. With so many different brands and models to choose from, it can be difficult to determine which robot is best suited to your needs.

[0003] Comparing different robots is a challenging task. One way to do this is to actually test them in a controlled environment. This involves buying robots to compare and then having them perform the same tasks. The performance of the robots is measured and analyzed to determine which is the best. However, this approach can be time-consuming, expensive, and risky. There is a risk of damaging the robot during testing, and if something goes wrong, the entire process must be repeated.

[0004] Conducting physical experiments with autonomous robots is expensive and unpredictable for a few reasons: Sensors can fail, causing the robot to make mistakes; the robot can collide with objects or other robots, damaging itself or the environment; and new robots need to be purchased to compare multiple robots for testing, which can be expensive. Summary of the invention

[0005] In view of this, the present invention proposes a method for evaluating robots in simulation, which can identify the best robot suitable for real-world environments. It can flexibly adjust settings to adapt to different robots and environments.

[0006] A method for evaluating a robot in a simulation according to an embodiment of the present invention includes executing the following steps with a computing device: obtaining a robot description file established based on the physical properties of the robot; obtaining an environment description file established based on the environment; obtaining an obstacle description file, wherein the obstacle description file is used to set a starting area, an end area, and multiple obstacles; using a physical simulation engine to establish a virtual environment based on the environment description file and the obstacle description file, and to establish a virtual human based on the robot description file; when the virtual human moves from the starting area to the end area based on navigation information in the virtual environment, using the physical simulation engine to output simulation information; using a robot navigation program to reply navigation information to the physical simulation engine based on the simulation information; and when the virtual human arrives at the end area, using the physical simulation engine to output evaluation information.

[0007] A computer-readable recording medium according to an embodiment of the present invention stores a program therein, and after a computing device loads and executes the program, the following steps are performed: obtaining a robot description file established based on the physical properties of the robot; obtaining an environment description file established based on the environment; obtaining an obstacle description file, wherein the obstacle description file is used to set a starting area, an end area, and multiple obstacles; using a physical simulation engine to establish a virtual environment based on the environment description file and the obstacle description file, establishing a virtual human based on the robot description file, and outputting simulation information of the virtual human in the virtual environment; using a robot navigation program to generate and send navigation information to the physical simulation engine based on the simulation information; and when the virtual human arrives at the end area, using the physical simulation engine to output evaluation information.

[0008] In summary, the method of evaluating robots in simulation proposed in the present invention can find the best robot for a specific task and identify the most suitable environment for a specific robot. The method is implemented in the form of simulation based on digital mapping and has the following advantages: improving efficiency and productivity, reducing costs, improving safety, assisting in making better decisions, and ensuring the performance of virtual robots in the real world.

[0009] The above description of the disclosed content and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and to provide a further explanation of the scope of the patent application of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flow chart of a method for evaluating a robot in simulation according to an embodiment of the present invention;

[0011] Figure 2 is a design diagram of a robot used according to an embodiment of the present invention;

[0012] Figure 3 A design diagram of an optical radar of a robot used in accordance with an embodiment of the present invention;

[0013] Figures 4 to 7 A schematic diagram of all preset obstacles according to an embodiment of the present invention;

[0014] Figure 8 is a schematic diagram of a virtual environment according to an embodiment of the present invention;

[0015] Fig. 9 The result of visualizing the robot description file in the verification process according to an embodiment of the present invention;

[0016] Fig.10 The result presented by the virtual human in the physical simulation engine according to an embodiment of the present invention;

[0017] Fig.11 is an architecture diagram of a physical simulation engine and a robot navigation program according to an embodiment of the present invention; and

[0018] Fig.12 FIG. 4 is an example of an action diagram according to an embodiment of the present invention.

[0019] Component number description

[0020] Steps S1 to S7 DETAILED DESCRIPTION

[0021] The detailed features and advantages of the present invention are described in detail in the following embodiments, and the contents are sufficient to enable any person skilled in the art to understand the technical content of the present invention and implement it accordingly, and according to the contents disclosed in this specification, the scope of the patent application and the drawings, any person skilled in the art can easily understand the relevant purposes and advantages of the present invention. The following examples are used to further illustrate the viewpoints of the present invention in detail, but are not intended to limit the scope of the present invention in any way.

[0022] Simulation is an effective way to compare robots. It allows different types of robots to be tested in a variety of environments in a safe and rapid manner without having to purchase or damage any physical robots. Digital mapping creates a realistic simulation of the real world to ensure that the robot performance in simulation is similar to that in the real world.

[0023] Figure 1 The flowchart of the method for evaluating a robot in simulation according to one embodiment of the present invention is shown. The method is to load and execute a program stored in a computer-readable recording medium by a computing device to implement multiple operations. These operations are as follows: Figure 1 Steps S1 to S6 are shown.

[0024] In one embodiment, the computing device may adopt at least one of the following examples: central processor unit (CPU), graphics processing unit (GPU), microcontroller (MCU), application processor (AP), field programmable gate array (FPGA), application specific integrated circuit (ASIC), digital signal processor (DSP), system-on-a-chip (SOC), deep learning accelerator. However, the present invention is not limited to these examples.

[0025] Step S1, obtaining a robot description file established according to the physical properties of a robot. In one embodiment, the robot description file is a unified robot description format (URDF), and the physical properties include the location of multiple joints and a light detection and ranging (LiDAR). The method further includes: executing a verification procedure according to the robot description file to confirm the directions of the multiple joints.

[0026] In detail, each robot has a design that can be converted into a URDF file. A URDF file is a digital description of the robot's physical properties. The URDF file can be put into a physics-based simulation platform as a 3D model, retaining all of the robot's physical properties, thus becoming a digital representation of the robot. Please refer to Figure 2 and Figure 3 , Figure 2 is a design diagram of a robot used according to an embodiment of the present invention, Figure 3 The figure is a design diagram of the optical radar of the robot used in one embodiment of the present invention. In one embodiment, the robot uses the Wheeltec Ackermann Robot, which is equipped with the WheeltecLD14 optical radar, but lacks a URDF file. Therefore, it is necessary to create a URDF file for this robot, which includes the joint positions, physical properties and measurement data of the robot. Then, in order to ensure the accuracy of the URDF file, one embodiment of the present invention uses the RVIZ visualization tool in ROS2 as a verification program to confirm the direction of the robot's joints.

[0027] Step S2, obtaining an environment description file established according to an environment. The environment description file includes a three-dimensional mesh and a three-dimensional object. Step S2 includes the following sub-steps: establishing the three-dimensional mesh in the environment description file by photogrammetry or structure from motion technology, and establishing the three-dimensional object in the environment description file by three-dimensional modeling software.

[0028] In detail, the digital mapping of the environment is a virtual replica of the real world, which can be created by scanning the real world environment. If the scan does not cover the entire environment, the missing parts can be manually created using 3D modeling software. In one embodiment, the 3D modeling software is Blender, which can be used to edit 3D objects and export FBX files containing 3D objects, materials, and lighting.

[0029] Step S3, obtaining an obstacle description file. The obstacle description file is used to set a starting area, an end area and a plurality of obstacles.

[0030] In one embodiment, the configuration of obstacles, starting areas, and end areas can be defined in a YAML file. Obstacles can be divided into two types. The first type is a preset obstacle. Please refer to Figures 4 to 7 , which is a schematic diagram of the configuration of all default obstacles according to an embodiment of the present invention. The second type is random obstacles, whose generation mechanism is based on this document: Daniel Perille et al. "Benchmarking Metric Ground Navigation". In: CoRR abs / 2008.13315 (2020). Table 1 below shows the multiple parameters that need to be set in the YAML file. Figure 4 Displays the distribution of random obstacles generated. Figure 4 In the example, a value of 0 represents an empty space and a value of 1 represents an obstacle.

[0031] Table 1. Parameters used to generate random obstacles. Different parameter values ​​generate different obstacles.

[0032] parameter describe Numeric Rows Number of rows 57 Cols Number of columns 30 fill_pct The proportion of the entire grid occupied by the obstacle 0.2 seed The seed for the random generator 1 smooth_iter The number of iterations used to smooth obstacles. 5

[0033] In one embodiment, a grid-based system is implemented to add random obstacles to the simulation, where each grid corresponds to 0.1 meters in the real world. Figure 4The position of these cubes is determined by calculating the number of grids. In order to ensure that the cube is exactly in the center of the grid, the position of the cube is further adjusted according to the cube scale, as shown in the following method 1:

[0034] Cube position = (original grid - half grid) / 10 + cube size / 2 (Formula 1)

[0035] Step S4, the physical simulation engine creates a virtual environment according to the environment description file and the obstacle description file, and creates a virtual human according to the robot description file.

[0036] Please refer to Figure 8 , Fig. 9 and Fig.10 . Figure 8 is a schematic diagram of a virtual environment according to an embodiment of the present invention, Fig. 9 The result of visualizing the robot description file in the verification process according to an embodiment of the present invention is as follows: Fig.10 This is the result presented by the virtual human in the physical simulation engine according to an embodiment of the present invention.

[0037] In one embodiment, the physical simulation engine is NVIDIA Omniverse TM Isaac Sim. IsaacSim was chosen because its advanced ray tracing technology enables the simulation to present scenes that are very similar to real-world lighting conditions. In step S4, the robot's URDF file is imported into Isaac Sim to create a simulated version of the robot. In addition, the FBX file containing the 3D objects, materials, and lighting is imported into Isaac Sim. In order to make the digital mapping as realistic as possible, the lighting and physical properties of the simulation need to be adjusted. In addition, since the imported FBX file may not be compatible with the file format (Universal Scene Description, USD) used by Isaac Sim, it needs to be repaired and the environment is saved as a USD file.

[0038] The present invention uses Isaac Sim to develop an application to run a virtual environment and a virtual human therein. The virtual environment has a camera that looks down toward the center of the scene. This camera is set as the main camera of the application. The USD file location and the configuration of the scene camera are stored as a YAML file.

[0039] Step S5, when the virtual human moves from the starting area to the end area according to navigation information in the virtual environment, the physical simulation engine outputs simulation information to the robot navigation program.

[0040] Step S6, the robot navigation program generates and sends navigation information to the physical simulation engine according to the simulation information. In one embodiment, the robot navigation program is a Robot Operating System 2 navigation stack (ROS2 navigation stack). Fig.11 FIG. 1 is a diagram of the architecture of a physical simulation engine and a robot navigation program according to an embodiment of the present invention. Fig.11 As shown, the physical simulation engine Issac Sim generates the scene and provides simulation information and sensor data to the robot navigation program ROS2, allowing the latter to plan the path and control the robot to the end area.

[0041] In order to make the robot navigate towards the target location, a connection needs to be established between the Isaac Sim simulation and ROS2, which is achieved through the Isaac Sim ROS Bridge extension and the Humble version of ROS2. The simulation information includes: the robot's posture, the optical radar data, and the simulation time. In one embodiment, this simulation information is transferred from Isaac Sim to the ROS2 navigation stack using an action graph created from a Python script. Fig.12 is an example of the action graph. In addition, a custom node subscriber is implemented to receive the speed command of the virtual robot and control the left and right steering angles of the robot wheels.

[0042] Next, the ROS2 navigation parameters used in the robot navigation program (ROS2) are described. The navigation performance of the virtual robot depends on configuring and fine-tuning various parameters, which are stored as a YAML file, including: positioning, behavior tree (BT) navigator, control server, local cost map, global cost map, map server, planning server, behavior server, and speed smoother. In one embodiment, the planner uses SMAC Hybrid A*. The controller uses Regulated Pure Pursuit (RPP) controller. Both of them have been optimized based on Ackermann robots. The positioning technology uses adaptive Monte Carlo localization (AMCL). Table 2 below presents the results of fine-tuning the ROS2 navigation parameters.

[0043] Table 2. Fine-tuning results of ROS2 navigation parameters.

[0044]

[0045]

[0046]

[0047] Considering the relatively narrow space in which the virtual robot maneuvers, the navigation performance of the virtual robot is significantly affected by the following factors: inflation_radius and cost_scaling_factor in the costmap. Different combinations of parameters will form different paths. In order to achieve the best performance, it is necessary to ensure that the formed path is in the middle of two obstacles, because the RPP controller strictly follows the given path. In addition, a footprint_padding value of -0.129 is used in the local costmap to reduce the robot's tolerance near obstacles and prevent navigation from stopping prematurely. If a specific parameter is not mentioned, it means the default value is used. In addition, the use_sim_time parameter is always set to true.

[0048] In one embodiment, an occupancy map and its configuration are generated using Isaac Sim, and then used by the ROS2 navigation map server to help locate the pose of the virtual human. The occupancy map does not contain any obstacles.

[0049] In one embodiment, the control server generates a velocity command including linear and angular velocities. The angular velocities are further processed using the Ackermann equation (as shown in Method 2 below) to determine the corresponding left and right angular velocities. The linear velocities and the calculated left and right angular velocities are transmitted back to the Isaac Sim simulation to drive the movement of the virtual robot wheels, so that the robot can navigate and move effectively.

[0050]

[0051] In one embodiment, a ROS2 launcher is implemented with a Python script, and its contents include: setting the use_sim_time parameter to true, the navigation parameter configuration location, the map configuration location, the Nav2 launcher, the RVIZ node, the map to odom static transformation node, and the Ackermann node.

[0052] Step S7: when the virtual human arrives at the end area, the physical simulation engine outputs evaluation information.

[0053] In one embodiment, the experiment is performed by running two independent terminals. In the first terminal, the standalone application of Isaac Sim is executed. In the second terminal, the ROS2 launch file is executed, which will trigger and present the RVIZ window, which displays the occupancy map. It is very important to run these two terminals at the same time because in the Isaac Sim simulation, even if the robot remains stationary, the timer for each round is still running. To cope with this, the initial round (round 0) is set to allow continued running until the timeout is reached to provide enough time for starting the ROS2 navigation stack. It is worth noting that when starting the ROS2 navigation stack, the positioning of the virtual man may not be accurate enough. However, starting from the first round, the virtual man can get the correct positioning, indicating that the test has officially begun.

[0054] In one embodiment, the present invention adopts Figure 8 The presented environment was tested for 100 rounds with four preset scenarios and two random scenarios. When the virtual robot reaches the target or encounters an obstacle, it will enter the next round and start again from the starting position. The evaluation information output in each round is as follows:

[0055] start_position, the random position of the robot at the beginning of each round;

[0056] arrived, indicating whether the robot successfully reached the target;

[0057] hit, indicating whether the robot collides with any obstacle;

[0058] time_spend, the duration of each round;

[0059] end_position, the final position of the robot before entering the next round;

[0060] goal_position, the final position of each round;

[0061] environment, the specific environment configuration used in each round;

[0062] scenario, the scenario configuration used in each round;

[0063] robot, the type of robot used in each round;

[0064] goal_tolerance_m, the maximum Euclidean distance allowed between the robot and the goal position to determine whether it has been reached.

[0065] The above evaluation information is stored as a CSV file. In one embodiment, the evaluation information is further processed and converted into percentages for analysis. The timeout rate indicates that the virtual machine is stuck or takes a long time to reach the target (exceeding the maximum time allocated for each round). The average time refers to the average time taken by only those virtual machines that reach the target location. The summary results are shown in Table 3 below.

[0066] Table 3. Navigation evaluation information for the six example scenarios.

[0067] Scenario End point ratio Collision Ratio Timeout ratio Average time (seconds) 1 98％ 0％ 2％ 47.10 2 91％ 4％ 5％ 89.42 3 92％ 4％ 4％ 66.81 4 87％ 3％ 10％ 104.52 Random (easy) 98％ 1％ 1％ 57.31 Random (difficult) 88％ 7％ 5％ 70.83

[0068] As shown in Table 3, the goal achievement rate for Scenario 1 and Scenario Random (Easy) was 98%. However, the average time for Scenario 1 was shorter compared to Scenario Random (Easy). This difference can be attributed to the fact that the path in Scenario 1 is much simpler than that in Scenario Random (Easy). Scenarios 2 and 3 performed similarly, but the average time for the robot in Scenario 2 was about 23 seconds longer than that in Scenario 3. In Scenario 2, the robot had to go straight at first and sometimes needed extra time to orient itself in a narrow gap before it could turn around. However, in Scenario 3, the robot encountered obstacles that forced it to turn before going straight, eliminating the need for extra positioning time. As a result, the robot in Scenario 3 navigated faster and reached the goal faster. Scenario 4 and Scenario Random (Difficult) performed similarly, but the arrangement of obstacles was more complex. Scenario Random (Difficult), in particular, had a higher collision rate and a shorter average time due to the scattered obstacles.

[0069] The RPP controller used in this experiment slows down the robot when there are curved paths or obstacles, especially in scenarios 2, 3, and 4 with sharp turns and surrounding obstacles. This is to avoid collisions and ensure that the robot reaches the end position safely, even at a slower speed. If the robot gets too close to an obstacle, it stops and attempts to generate a new path. Since the Ackermann robot needs room to turn, sometimes a path that seems navigable will cause the robot to remain stationary until the time limit is exceeded when the robot cannot navigate due to insufficient turning room, resulting in a high timeout rate.

[0070] In summary, the method for evaluating a robot in simulation and the computer readable medium provided by the present invention provide a cost-effective method for generating a variety of customized test environments. The recorded values ​​provide a large amount of data for analysis. The use of digital mapping technology can ensure that the test results are closely consistent with the real world scenario. Optimizing the navigation performance of the robot requires configuration and fine-tuning of components specific to the robot type.

[0071] Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Without departing from the spirit and scope of the present invention, all changes and modifications are within the scope of patent protection of the present invention. Please refer to the attached patent application for the scope of protection defined by the present invention.

Claims

1. A method for evaluating a robot in simulation, characterized in that The method comprises executing the following steps using a computing device: Creating a robot description file based on the physical properties of a robot; Establish an environment description file based on an environment; Obtaining an obstacle description file, wherein the obstacle description file is used to set a starting area, an end area, and a plurality of obstacles; Using a physical simulation engine to create a virtual environment according to the environment description file and the obstacle description file, creating a virtual human according to the robot description file, and outputting simulation information of the virtual human in the virtual environment; Using a robot navigation program to generate and send navigation information to the physical simulation engine according to the simulation information; as well as When the virtual human arrives at the end area, the physical simulation engine outputs evaluation information.

2. The method for evaluating a robot in simulation according to claim 1, characterized in that The robot description file is in a unified robot description format, the physical properties include a plurality of joints and a setting position of an optical radar, and the method further includes: executing a verification program according to the robot description file to confirm the directions of the plurality of joints.

3. The method for evaluating a robot in simulation according to claim 1, characterized in that The environment description file includes a three-dimensional grid and a three-dimensional object, and establishing the environment description file according to the environment includes: Creating the three-dimensional mesh in the environment description file using a photogrammetry or a structure-from-motion technique; and The three-dimensional object in the environment description file is established using three-dimensional modeling software.

4. A computer-readable recording medium, characterized in that: A program is stored therein, and after a computing device loads and executes the program, the following steps are performed: Obtaining a robot description file established based on physical properties of a robot; Obtaining an environment description file established based on an environment; Obtaining an obstacle description file, wherein the obstacle description file is used to set a starting area, an end area, and a plurality of obstacles; Using a physical simulation engine to create a virtual environment according to the environment description file and the obstacle description file, creating a virtual human according to the robot description file, and outputting simulation information of the virtual human in the virtual environment; Using a robot navigation program to generate and send navigation information to the physical simulation engine according to the simulation information; as well as When the virtual human arrives at the end area, the physical simulation engine outputs evaluation information.

5. The computer-readable recording medium according to claim 4, wherein: The robot description file is in a unified robot description format, the physical properties include a plurality of joints and a setting position of an optical radar, and the method further includes: executing a verification program according to the robot description file to confirm the directions of the plurality of joints.

6. The computer-readable recording medium according to claim 4, wherein: The environment description file includes a three-dimensional grid and a three-dimensional object, and establishing the environment description file according to the environment includes: Creating the three-dimensional mesh in the environment description file using a photogrammetry or a structure-from-motion technique; and The three-dimensional object in the environment description file is established using three-dimensional modeling software.

Images