Industrial robot simulation training method and system based on machine learning

Abstract

The invention discloses an industrial robot simulation training method and system based on machine learning, and relates to the field of robots. Comprising the following steps: constructing a digital simulation environment by using a Unity engine; the method is used for realizing reverse kinematics based on a Bio-IK framework; importing a machine learning model; and verifying and optimizing a robot motion strategy. According to the method, the inverse kinematics algorithm is realized and is used for calculating the rotation angle of the robot joint so as to realize target position positioning of the robot end effector in the three-dimensional space, and meanwhile, an advanced machine learning model, particularly a reinforcement learning method, is applied to train the motion strategy of the robot so as to reduce the motion time and improve the operation precision.

Description

Technical Field

[0001] The present invention relates to the field, and in particular to an industrial robot simulation training method and system based on machine learning. Background Art

[0002] With the rapid development of the global manufacturing industry, industrial automation and intelligent manufacturing have become key factors in improving production efficiency, reducing costs, and enhancing competitiveness. In this context, industrial robots, as the core components of automated production lines, the efficiency and flexibility of their programming and operation are crucial for the success of manufacturing enterprises. However, existing industrial robot programming methods usually rely on complex programming processes and professional technicians, which not only increase the time and cost of production preparation, but also limit the adaptability of robots in rapidly changing production environments. In addition, traditional methods have limitations in robot path planning and motion optimization, often requiring a large amount of manual adjustment and repeated trials, which are inefficient in terms of resources and time. To overcome the above challenges, there is an urgent need for a method that can quickly and accurately program and optimize the motion of industrial robots to achieve efficient conversion and flexible adjustment of the production process. Given that machine learning-related technologies have been widely empowering various industries, how to combine machine learning with simulation technology to bring more value and possibilities to industrial digitization and automation is worthy of further exploration.

[0003] Therefore, the present invention proposes an industrial robot simulation training method and system based on machine learning. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to propose an industrial robot simulation training method and system based on machine learning.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: An industrial robot simulation training method and system based on machine learning, comprising the following steps: S1: Use the Unity engine to build a digital simulation environment; S2: Based on the Bio-IK framework to implement inverse kinematics; S3: Import a machine learning model; S4: Verify and optimize the robot motion strategy.

[0006] Preferably: In the step S1, it specifically includes the following steps: S11: Establish a three-dimensional space simulation environment and inject environmental parameters into the three-dimensional space simulation environment; S12: Establish a robot model in the three-dimensional space environment and determine the motion model of the robot according to the robot model.

[0007] Preferably, in the step S12, the robot motion model includes a robot size model, robot joint positions, robot component strength, and the motion speed of each robot joint.

[0008] Preferably, in the step S2, the predefined input parameters for implementing inverse kinematics based on the Bio-IK framework include: ①: The rotation axis of each movable joint; ②: The maximum and minimum rotation angles of each joint; ③: The load at the end of the motion, the position of which is the input of the digital robot joint rotation angle.

[0009] Preferably, in the step S2, implementing inverse kinematics based on the Bio-IK framework further includes configuration parameters, which are used for setting the basic parameters of the Bio-IK framework.

[0010] Preferably, the configuration parameters include: ①: "Generation=3", indicating a set of different possible positions in a single frame, which can also be used to generate a collision-free trajectory; ②: "Individuals=150", indicating a set of possible positions in the first few Generations; ③: "Elites=2" indicating the size of the quality control calculation corresponding to the previous successful positions; ④: "Motiontype=Realistic" defines that the simulation result should be a real number.

[0011] Preferably, in the step S3, the machine learning model is controlled by observing the state of the machine learning algorithm in the environment S(t) at time t.

[0012] Preferably, the specific logic in the step S3 is as follows: S31a: When the agent executes a new action, the environment and the agent move to a new state S(t + 1); S32b: When the agent executes a new action, the environment and the agent move to a new state S(t + 1).

[0013] Preferably, in the step S3, the operation steps are: S31b: When performing RL machine learning in Unity3D, add the ML agent software to the corresponding project; S32b: Clone the ML-AgentToolkit from Github to the computer and install Python3.

[0014] Preferably, the type of trainer used in the step S3 is: PPO; where batch_size represents the number of samples in batch gradient calculation, learning_rate represents the learning rate, and num_epoch represents the number of training iterations of the neural network.

[0015] The beneficial effects of the present invention are as follows: The present invention implements an inverse kinematics algorithm for calculating the rotation angles of the robot joints to achieve the target position positioning of the robot end effector in three-dimensional space. At the same time, an advanced machine learning model, especially the reinforcement learning method, is applied to train the motion strategy of the robot to reduce the motion time and improve the operation accuracy. Description of the Drawings

[0016] Figure 1 It is a flowchart of an industrial robot simulation training method based on machine learning proposed by the present invention. Detailed Embodiments

[0017] The technical solutions of the present invention will be further described in detail below in conjunction with the specific embodiments.

[0018] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", "connection", and "setting" should be understood in a broad sense. For example, it can be fixedly connected and set, or detachably connected and set, or integrally connected and set. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. Embodiment 1:

[0019] An industrial robot simulation training method and system based on machine learning, which includes the following main steps, and the overall process is as Figure 1 shown: S1. Use the Unity engine to build a digital simulation environment. Build a digital environment for simulating the motion and operation of an industrial robot, and accurately set the physical characteristics and working environment conditions of the robot in the digital environment. Using digital simulation technology can shorten the time, optimize the design and development of various processes in industry, and simulate these processes and real environments as accurately as possible. The simulation environment can well simulate the physical characteristics in real life, such as gravity, kinematics, difficulty differences, etc. These environments can be used to develop / simulate industrial robot application scenarios, etc. Deeply deconstruct the physical parameters / physical characteristics of the robot in scenarios such as motion radius, lifting force, and motion speed.

[0020] When simulating using different environments, different input data is required. To simulate the production process of an industrial robot in Unity, it is necessary to understand the rotation points and rotation angle limits of the industrial robot in a three-dimensional environment to ensure the correct movement of the model. In addition, it is necessary to change the local coordinate system of the joints so that the rotation angle changes around the z-axis. In addition, to perform the simulation, the data of the object needs to be used, and the corresponding input data information can be found on the product specification page of the simulation device manufacturer. This invention is based on the LRMate200iD / 4S standard industrial robot produced by Shanghai FANUC Co., Ltd. as a benchmark to establish simulation object parameters for creating a 3D model of the industrial robot. The maximum load of the wrist of this series of robots is 7 kg, which can easily handle the handling of larger workpieces, can meet a variety of applications from narrow spaces to wide spaces, provides vision and force sensor interfaces, and supports corresponding intelligent application functions.

[0021] S2. Implement the inverse kinematics function. Based on the Bio-IK framework for implementing inverse kinematics to simulate the movement of the robot in the digital world, the input data needs to be predefined as follows: The rotation axis of each movable joint is on the z-axis; The maximum and minimum rotation angles of each joint; Add an end effector force, the position of which is the input of the digital robot joint rotation angle; Configuration parameters Among them, the configuration parameters are used for setting the basic parameters of the Bio-IK framework, mainly including 4 parameters, namely "Generation=3", which represents a set of different possible positions in a single frame and can also be used to generate a collision-free trajectory; "Individuals=150", which represents a set of possible positions for the first few Generations; "Elites=2" represents the size of the quality control calculation corresponding to the previous successful positions; "Motiontype=Realistic" defines that the simulation result should be a real number. With the help of the Bio-IK framework, calculate the rotation angle of the robot joints to achieve the target position positioning of the end effector.

[0022] S3. Import the machine learning model. Machine learning has been used by ML-Agent developed by Unity, which can be used to train models in a simulation environment. The ML-Agent plugin has been implemented, and deep learning algorithms based on PyTorch can be deployed, or verified Python-APIs (such as libraries like sklearn), simulations, neural networks, or other machine learning methods can be used.

[0023] To create a machine learning model, the present invention uses reinforcement learning. The autonomous agent (learner) must be configured to use reinforcement learning in a simulated environment, which is controlled by observing the state of the machine learning algorithm in the environment S(t) at time t. When the agent executes a new action, the environment and the agent move to a new state S(t+1). When the agent executes a new action, the environment and the agent move to a new state S(t+1). To establish a machine learning model used in the simulated environment, a training called Proximal Policy Optimization (PPO) is used, which uses a neural network to find the best features for a given observer and conditions for creating the machine learning model used in the simulated environment. The present invention realizes the hypothesis of reinforcement learning features, wherein for each successful result, the training agent is rewarded. Therefore, the agent moves to the next iteration, where it uses the previously learned experience. If the agent fails to obtain a positive result within a predefined time, a negative reward is given to the agent. It should be noted that to perform RL machine learning in Unity3D, the ML-Agent software needs to be added to the corresponding project. In addition, the ML-AgentToolkit needs to be cloned from Github to your computer, and Python3 needs to be installed to use the machine learning functions based on PyTorch. The agent training input is a simulated environment created in Unity3D. To perform the training, it is necessary to configure a YAML configuration file, which contains network training parameters. In this study, the default parameter values are usually used, as shown below: ; Among them, batch_size represents the number of samples in batch gradient calculation, learning_rate represents the learning rate, and num_epoch represents the number of iterations of neural network training. During the training process, the corresponding visualization plugin can be used to monitor the training progress and performance. By tracking the training process, evaluating the model performance, and adjusting the parameters as needed.

[0024] S4. Verify and optimize the robot motion strategy. After completing the training, test the robot's motion strategy in the simulated environment and optimize it. On this basis, apply the trained and optimized machine learning model to the actual industrial robot. Embodiment 2:

[0025] An industrial robot simulation training system based on machine learning, which is used to implement the industrial robot simulation training method based on machine learning in Embodiment 1.

[0026] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent replacements or changes, shall be covered by the protection scope of the present invention.

Claims

1. An industrial robot simulation training method based on machine learning, characterized in that, It includes the following steps: S1: Build a digital simulation environment using the Unity engine; S2: Implement inverse kinematics based on the Bio-IK framework; S3: Import a machine learning model; S4: Verify and optimize the robot motion strategy.

2. The industrial robot simulation training method based on machine learning according to claim 1, wherein, In the step S1, it specifically includes the following steps: S11: Establish a three-dimensional space simulation environment and inject environmental parameters into the three-dimensional space simulation environment; S12: Establish a robot model in the three-dimensional space environment and determine the motion model of the robot according to the robot model.

3. The industrial robot simulation training method based on machine learning according to claim 2, wherein In the step S12, the robot motion model includes the robot size model, the robot joint positions, the strength of the robot components, and the motion speed of each joint of the robot.

4. A method for simulating and training an industrial robot based on machine learning according to claim 1, characterized in that In the step S2, the predefined input parameters for implementing inverse kinematics based on the Bio-IK framework include: ①: The rotation axis of each movable joint; ②: The maximum and minimum rotation angles of each joint; ③: The load at the end of the motion, and its position is the input of the digital robot joint rotation angle.

5. A method for simulating and training an industrial robot based on machine learning according to claim 1, characterized in that, In the step S2, implementing inverse kinematics based on the Bio-IK framework also includes configuration parameters, which are used for setting the basic parameters of the Bio-IK framework.

6. The industrial robot simulation training method based on machine learning according to claim 5, characterized in that, The configuration parameters include: ①: "Generation=3”, which represents a set of different possible positions in a single frame and can also be used to generate a collision-free trajectory; ②: "Individuals=150”, which represents a set of possible positions of the first few Generations; ③: "Elites=2” represents the size of the quality control calculation corresponding to the previous successful positions; ④: "Motiontype=Realistic” defines that the simulation result should be a real number.

7. A method for simulating and training an industrial robot based on machine learning according to claim 1, characterized in that, In the step S3, the machine learning model is controlled by observing the state of the machine learning algorithm in the environment S(t) at time t.

8. A method for simulating and training an industrial robot based on machine learning according to claim 7, characterized in that, The specific logic in the step S3 is as follows: S31a: When the agent executes a new action, the environment and the agent move to a new state S(t+1); S32b: When the agent executes a new action, the environment and the agent move to a new state S(t+1).

9. The industrial robot simulation training method based on machine learning according to claim 8, characterized in that, In the step S3, the operation steps are: S31b: When performing RL machine learning in Unity3D, add the ML agent software to the corresponding project; S32b: Clone the ML-AgentToolkit from Github to the computer and install Python3; In the step S3, the trainer type used is: PPO; batch-size: 64; learning-rate; 0.0003; num=rpoch: 100, where batch_size represents the number of samples in batch gradient calculation, learning_rate represents the learning rate, and num_epoch represents the number of neural network training iterations.

10. An industrial robot simulation training system based on machine learning, characterized in that, It is used to implement the machine learning-based industrial robot simulation training method described in any one of claims 1-9.

Images