Mechanical arm simulation method and system based on Control platform
By building a robotic arm simulation environment on the Coppeliasim platform, constructing a motion planning algorithm, and using Python libraries for interaction, the problem of slow interaction between the algorithm and the environment was solved, the training efficiency of the reinforcement learning algorithm was improved, and the synchronization of simulation data to real robot motion was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2026-03-10
AI Technical Summary
On the Coppeliasim platform, the slow interaction between the algorithm and the environment affects the training efficiency of reinforcement learning algorithms.
By building a robotic arm simulation environment on the Coppeliasim platform, constructing a motion planning algorithm, and using Python libraries for interaction, combined with deep reinforcement learning algorithms, the simulation data is transferred to the real robot, and synchronous motion is achieved using TCP/IP socket network communication.
It improves the interaction speed between the algorithm and the environment, enhances the training efficiency of reinforcement learning algorithms, and achieves synchronization between simulation data and real robot motion.
Smart Images
Figure CN121634875A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of mechanical arm simulation, and particularly relates to a mechanical arm simulation method and system based on a Coppeliasim platform. BACKGROUND
[0002] With the rapid development of robot technology, mechanical arms play an important role in the fields of industry, medicine, aerospace and military, and the research on mechanical arms is not only an important part of the field of robots, but also an important link to realize intelligent manufacturing, which can realize heavy and even artificial work.
[0003] In order to verify the effect of the mechanical arm motion planning algorithm and improve the success rate of industrial mechanical arms in practical application, the industrial mechanical arm is usually simulated before controlling the actual robot to simulate the actual working scene and the mechanical arm, verify the planning algorithm in the simulation scene, and observe whether the task can be successfully executed. This greatly shortens the development cycle of the scheme, reduces the debugging and testing cost of the mechanical arm in the actual environment, and ensures the safety of the equipment and the workers.
[0004] In the use process of Coppeliasim, programming mainly uses Regular API or Remote API. The design of the simulation method is mainly to verify and train the deep reinforcement learning algorithm on the platform using the python language, so the Remote API of Coppeliasim is adopted, but the remote api is called through Socket, so the response speed is very slow, which affects the interaction between the algorithm and the environment and is not conducive to the training of the reinforcement learning algorithm. SUMMARY
[0005] In view of the above problems, the application provides a mechanical arm simulation method and system based on a Coppeliasim platform. The algorithm and the environment interact too slowly in the prior art, and a large number of python libraries are combined to improve the training efficiency of the algorithm. The system also provides a method for migrating simulation data to a real robot.
[0006] The technical scheme adopted by the application to achieve the above purpose is:
[0007] A mechanical arm simulation method based on a Coppeliasim platform, comprising the following steps:
[0008] 1) Building a simulation environment of the mechanical arm based on the Coppeliasim platform;
[0009] 2) constructing a mechanical arm motion planning algorithm and interacting it with the simulation environment;
[0010] 3) Transfer the robot's motion trajectory from the simulation environment to the physical object to control the movement of the real robot.
[0011] Step 1) includes the following steps:
[0012] 1.1) Create a robotic arm model and simulation scene in the Coppeliasim platform;
[0013] 1.2) Obtain the state data of the robotic arm on the Coppeliasim platform, add a marker dummy to the end of the robotic arm, add collision attributes and measurable attributes of the robotic arm, and set the joints to Torque or force mode.
[0014] 1.3) Establish a collection module in the simulation platform and perform collision detection and minimum distance detection for the robotic arm;
[0015] 1.4) Obtain the collision state of the robotic arm and the minimum distance to the obstacle, and establish a collision group between the robotic arm assembly and the obstacle;
[0016] 1.5) Once the simulation environment is set up, save the ttm file.
[0017] Step 2) includes the following steps:
[0018] 2.1) Initialize the robot and objects in the simulation environment;
[0019] 2.2) Obtain the state of the robot arm joint angles, joint velocities, target point positions, and obstacle positions in the simulation environment through the API interface;
[0020] 2.3) The robotic arm performs actions in a simulation environment;
[0021] 2.4) Calculate the reward function for the state data of the simulation environment and input it into the deep reinforcement learning SAC algorithm network to update the algorithm network;
[0022] 2.5) After completing the algorithm training, the simulated robot performs motion planning autonomously to verify the algorithm.
[0023] Step 3) specifically refers to:
[0024] The six-axis joint angle data of each step of the simulated robot's movement is exported and input into the physical UR5 robotic arm via TCP / IP socket network communication, enabling the physical robotic arm to move synchronously and perform real-world tasks.
[0025] A robotic arm simulation system based on the Coppeliasim platform, including
[0026] The simulation environment building module is used to build a simulation environment for robotic arms based on the Coppeliasim platform.
[0027] The algorithm interaction module is used to build the motion planning algorithm for the robotic arm and interact with it with the simulation environment;
[0028] The simulation control module is used to transfer the robot's motion trajectory from the simulation environment to the physical object, thereby controlling the movement of the real robot.
[0029] A robotic arm simulation device based on the Coppeliasim platform includes a memory and a processor; the memory is used to store computer programs; the processor is used to implement the robotic arm simulation method based on the Coppeliasim platform when the computer programs are executed.
[0030] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned robotic arm simulation method based on the Coppeliasim platform.
[0031] The present invention has the following beneficial effects and advantages:
[0032] 1. The method proposed in this invention is simple and flexible, and can be used for robot control and scene operation in the Coppeliasim simulation platform.
[0033] 2. This invention can solve the problem of slow interaction speed between the algorithm and the environment in existing methods, and improve the interaction speed.
[0034] 3. The method proposed in this invention can be combined with a large number of Python libraries to improve the training efficiency of reinforcement learning algorithms. Attached Figure Description
[0035] Figure 1 A robotic arm simulation method and system structure block diagram based on the Coppeliasim platform are provided for the present invention and examples;
[0036] Figure 2 The simulation scene diagram provided for the embodiments of the present invention. Detailed Implementation
[0037] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0038] This invention provides a robotic arm simulation method based on the Coppeliasim platform. It is developed on the Ubuntu 20.04 system and uses the Pyrep framework in the Python 3.9 environment to achieve rapid interaction between the simulation platform and the algorithm.
[0039] The platform required for this invention example is: Ubuntu 20.04 system, Python 3.9 environment (VSCodeIDE used in this invention), and Coppeliasim software.
[0040] like Figure 1 As shown, the method includes: a simulation software operation module, an environment state acquisition module, an environment state setting module, an algorithm action execution module, and a SimtoReal module.
[0041] The software interface provided by Pyrep allows direct control over opening and closing of TTM files, as well as running and stopping simulations. The robots class interface retrieves the initial joint angles of the robotic arm model in the virtual scene, the velocities of each joint, and the pose data of the robotic arm's end effector center point. The shape class interface in Pyrep allows obtaining the coordinate position of objects. In addition to collecting data from the scene, the set interface can also be used to directly set the kinematic properties of the robotic arm, its six-axis position and velocity, and the positions of objects in the scene.
[0042] Specifically, opening and closing Coppeliasim's ttm files can be done directly using the PyRep.launch() function, allowing for data interaction.
[0043] The software simulation can be started, advanced, stopped, and shut down directly using the PyRep.start(), PyRep.step(), PyRep.stop(), and PyRep.shutdown() functions.
[0044] During reinforcement learning training, it is typically necessary to obtain the state of the environment within the algorithm. This includes various information about the robotic arm, such as the joint angles of the robotic arm model, the velocities of each joint, and the pose of the end effector's center point.
[0045] The joint angles of the robotic arm model can be obtained using the get_joint_positions() function;
[0046] The joint velocities of the robotic arm model can be obtained using the get_joint_velocities() function;
[0047] The center point of the end effector of the robotic arm model can be obtained using the get_tip() function, and its position and orientation can be obtained using the get_position() and get_orientation() functions, respectively.
[0048] The positions of objects in the environment, including target points and obstacles, can also be obtained using get_position().
[0049] In addition, obstacle avoidance motion planning for robotic arms is usually required, which necessitates real-time monitoring of whether the robotic arm collides with obstacles. The collision status can be detected using the check_arm_collision() function.
[0050] To ensure the safety of the robotic arm during operation, it is usually necessary to maintain a certain distance between the robotic arm and the obstacle. Therefore, it is necessary to monitor the minimum distance between the robotic arm and the obstacle in real time. In the algorithm, this can be obtained by reading the value of the minimum distance function Distance().read() and then processing the data.
[0051] Furthermore, in addition to acquiring various states in the environment, it is also necessary to use the Python environment as a client and the simulation platform as a server. That is, the settings and outputs in the algorithm program are given to the simulation platform, and the simulation platform sets some attributes and object positions as required, and controls the robotic arm to perform corresponding actions.
[0052] Specifically, since sometimes the training of an algorithm requires setting the target point randomly multiple times, it is necessary to be able to directly set the position of the target point in the simulation platform within the algorithm. This function can be achieved through the set_position() function.
[0053] Of course, the dynamics, kinematic properties, and motion control of the robotic arm are even more important.
[0054] Before the simulation begins, the robotic arm usually needs to be set to open-loop control mode because reinforcement learning training typically does not require the forward and inverse kinematics of the robotic arm. This can be set using the set_control_loop_enabled(False) interface. Additionally, the robotic arm should be self-locked to address the issue of the arm drooping due to its own weight when stationary. This can be set using the set_motor_locked_at_zero_velocity(True) function.
[0055] Once the robotic arm simulation begins, its movements need to be controlled. For joint angle settings, `set_joint_position()` is typically used to directly set the angles of each joint. Sometimes speed control is also required, which can be achieved using `set_joint_target_velocities()` to directly control the speed of each joint.
[0056] Through the aforementioned interfaces for acquiring simulation environment states and controlling action output, Python algorithms can interact efficiently with the Coppeliasim platform and utilize existing Python libraries, thereby greatly accelerating algorithm training and improving efficiency.
[0057] After the algorithm simulation is completed, a robot motion trajectory that can successfully reach the target point without collision is obtained. The six-axis joint angles of the robot at each step of this path are exported and sent to the physical robot (the UR5 robotic arm is used in this embodiment of the invention). This allows for the synchronous movement of the real robot, thereby realizing the task in the real-world scenario. Remote control of the UR5 robotic arm can be achieved using pure Python scripts via TCP / IP socket network communication.
[0058] This invention discloses a robotic arm simulation method and system based on the Coppeliasim platform. It addresses the problem of slow algorithm-environment interaction in existing methods and improves training efficiency by integrating with numerous Python libraries.
[0059] This invention uses the obstacle avoidance motion planning task of the UR5 robotic arm as an example. For instance... Figure 2 In the diagram, red spheres represent random target points, light purple squares represent fixed obstacles, and transparent green squares represent the defined areas of the random target points. The UR5 robotic arm needs to autonomously plan a path that avoids obstacles and successfully tracks the target point, and then move along that path to complete the tracking task. A deep reinforcement learning SAC algorithm is used, and simulation experiments are conducted using the simulation method proposed in this invention. The simulation experiments only consider the end effector reaching the target point, using the success rate of the end effector reaching the target point as the main evaluation index of the planning algorithm, and do not involve the control of the collision force generated by the end effector contacting the target.
[0060] A robotic arm model and simulation scenarios were created in the S1 and Coppeliasim platforms. Specific experimental scenarios include... Figure 2 .
[0061] S2. Obtain the state of the robotic arm on the simulation platform and implement robotic arm control. Add a dummy to the end effector of the robotic arm, add collision and measurable attributes to the robotic arm, and ensure that the joints are in Torque / force mode.
[0062] S3. Implement collision detection and minimum distance detection; establish a collection module in the simulation platform.
[0063] S4. To obtain the collision status of the robotic arm and the minimum distance to the obstacle, it is necessary to establish a collision group between the robotic arm assembly and the obstacle.
[0064] S5. The simulation environment is set up and the ttm file is saved.
[0065] S6 enables interaction between the algorithm and the environment. The simulation platform can collect and utilize data, while the Python IDE performs data processing, with the two working together.
[0066] S7. The program runs, the file is opened, and the robot and objects are initialized.
[0067] S8. Obtain the state of the robotic arm joint angle, joint speed, target point position, and obstacle position in the simulation environment. For details, please refer to the API interface in the invention content.
[0068] S9. In the simulation software, the robotic arm performs actions. In this example, the action data output by the algorithm is the speed of the six joints of the robotic arm.
[0069] S10. Calculate the reward function for the state data of the simulation environment and input it into the deep reinforcement learning SAC algorithm network to update the algorithm network.
[0070] S11. After the algorithm training is completed, the simulated robot can autonomously perform motion planning to verify the algorithm's effectiveness.
[0071] S12. The robot's motion trajectory in the simulation is transferred to the physical robot to control its movement. The six-axis joint angle data for each step of the simulated robot's motion is exported and input into the physical UR5 robotic arm via TCP / IP socket network communication. The physical robotic arm can also achieve synchronous movement and perform real-world tasks.
[0072] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0073] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A Coppeliasim platform-based robot simulation method, characterized in that, The method comprises the following steps: 1) building a simulation environment for the robot arm based on the Coppeliasim platform; 2) constructing a motion planning algorithm for the robot arm and interacting it with the simulation environment; 3) migrating the robot motion trajectory in the simulation environment to the entity and controlling the real robot motion.
2. The Coppeliasim platform-based robot simulation method according to claim 1, wherein, The step 1) comprises the following steps: 1.1) establishing a robot arm model and a simulation scene in the Coppeliasim platform; 1.2) acquiring the robot arm state data of the Coppeliasim platform, adding a marker point dummy at the end of the robot arm, adding the collision attribute and measurable attribute of the robot arm, and setting the joint as Torque or force mode; 1.3) establishing a collection module in the simulation platform and performing collision detection and minimum distance detection of the robot arm; 1.4) acquiring the collision state of the robot arm and the minimum distance to the obstacle, establishing a collision group of the robot arm set and the obstacle; 1.5) completing the simulation environment building, and saving the ttm file.
3. The Coppeliasim platform-based robot simulation method of claim 1, wherein, The step 2) comprises the following steps: 2.1) initializing the robot and the object in the simulation environment; 2.2) acquiring the state of the joint angle, joint speed, target point position and obstacle position of the robot arm in the simulation environment through the api interface; 2.3) executing the action of the robot arm in the simulation environment; 2.4) calculating the reward function of the state data of the simulation environment and inputting it into the deep reinforcement learning SAC algorithm network to update the algorithm network; 2.5) completing the algorithm training, and verifying the algorithm by the simulation robot performing motion planning autonomously. The step 3) is specifically:
4. The Coppeliasim platform-based robot simulation method of claim 1, wherein, Exporting the six-axis joint angle data of each step of the simulation robot motion, inputting it into the entity UR5 robot arm through TCP / IP socket network communication, so that the entity robot arm realizes synchronous motion and executes the real task. The method comprises 5. A robot simulation system based on Coppeliasim platform, characterized in that, a simulation environment building module for building a simulation environment for the robot arm based on the Coppeliasim platform; an algorithm interaction module for constructing a motion planning algorithm for the robot arm and interacting it with the simulation environment; a simulation control module for migrating the robot motion trajectory in the simulation environment to the entity and controlling the real robot motion. The device comprises a memory and a processor; the memory is used to store a computer program; the processor is used to realize the method of claim 1-4 when executing the computer program.
6. A robot simulation device based on a Coppeliasim platform, characterized in that, The storage medium stores a computer program, and when the computer program is executed by the processor, the method of claim 1-4 based on the Coppeliasim platform is realized.
7. A computer readable storage medium characterized in that,