Real-world six-axis robot control method using artificial intelligence trained on digital twin and system therefor

Abstract

A real-world robot control system using artificial intelligence trained on a digital twin, according to the present invention, comprises: an artificial intelligence module that has undergone reinforcement learning by using a virtual robot on the digital twin replicating a real-world robot; one or more sensing units that are provided in the real-world robot and sense physical characteristics and electrical characteristics; a robot data collection unit that collects data from the sensing units; an error calculation unit that calculates an error between the virtual robot and the real-world robot; a disturbance observation unit that monitors an input signal transmitted from the module to the real-world robot, an output signal output from the real-world robot, and the error and estimates disturbance to generate feedback data for compensating for the disturbance when the disturbance is greater than or equal to a threshold value; and a robot driving module that transmits a control command for controlling the real-world robot by reflecting a control signal from the artificial intelligence module and the feedback data generated from the disturbance observation unit.

Description

A method and system for controlling a real 6-axis robot using artificial intelligence learned on a digital twin

[0001] The present invention relates to a method and system for controlling a real-world robot using artificial intelligence learned on a digital twin, and more specifically, to a method and system for applying artificial intelligence for controlling a dual-arm robot learned on a digital twin to a dual-arm robot in the real world.

[0002] Industrial digital twin technology is a revolutionary technology that accurately replicates physical systems and processes digitally, enabling simulation, analysis, and prediction of diverse real-world situations. Advances in artificial intelligence (AI) are making digital twins even more intelligent, enabling real-time data analysis, improved predictive accuracy, and automated decision-making. The combination of these technologies is increasingly contributing to maximizing efficiency, preventing problems, and creating new innovations in diverse industries, including manufacturing, energy, and transportation.

[0003] A digital twin is a virtual model that reflects a real-world object, system, or process digitally. It can be used to accurately reflect the current state as well as predict future states.

[0004] By integrating data collected from various sources, such as sensor data, operational data, and environmental data, a virtual model, which is a 3D model or simulation model of an actual object, is implemented, and through this, data-based analysis and simulation of future scenarios are performed, thereby automating or supporting decision-making based on the analysis results.

[0005] Specifically, this artificial intelligence can provide the following important capabilities to digital twin technology:

[0006] Predictive Analytics: AI analyzes collected data to predict the future state of a system. This allows for proactive prevention of equipment failures and optimization of maintenance plans.

[0007] Automated optimization: AI algorithms can simulate various scenarios and automatically find optimal conditions, which can help reduce energy consumption or increase productivity.

[0008] Real-time decision-making: AI analyzes data in real time, enabling immediate decision-making, which is especially useful in changing market conditions or emergency situations.

[0009] Digital twin and artificial intelligence technologies are being applied across a wide range of industries, with the following examples providing practical examples.

[0010] In manufacturing, it is used for optimizing manufacturing processes, predicting failures and maintenance planning, and virtual testing of new products. In the energy sector, it is used for efficient power plant operation, integrating renewable energy, and optimizing energy consumption. Furthermore, in transportation systems, it can contribute to efficiency analysis, vehicle maintenance prediction, and improving road safety.

[0011] Industrial digital twin technology, leveraging AI, is playing a crucial role in maximizing industrial efficiency, reducing operating costs, and creating new business opportunities through real-time data analysis, prediction, and automated decision-making.

[0012] However, learning artificial intelligence in the real world requires a lot of time and money, and when performing machine learning on a digital twin to solve this problem, there are problems such as low reliability and errors when applied in the real world, making it difficult to apply it directly to the real world.

[0013] The present invention provides a method and system that can verify in advance, before operation, a problem that may arise when replacing the actual 6-axis robot arm control with artificial intelligence by utilizing a virtual robot in a digital twin environment.

[0014] In addition, the present invention provides a method and means for optimizing the virtual robot driving reproducibility of a digital twin to the level of driving of a real robot, and minimizing the driving error between a virtual robot and a real robot.

[0015] In addition, the present invention provides a method and means for reducing learning time and optimizing performance through reinforcement learning through a multi-environment.

[0016] In addition, the present invention provides a method and means for improving the usability and reliability of a digital twin virtual space by minimizing errors in the operation of a virtual robot and an actual robot, and enabling preliminary verification of robot process automation.

[0017] A real robot control system using artificial intelligence learned on a digital twin according to the present invention comprises: an artificial intelligence module for performing reinforcement learning on a virtual robot on a digital twin that replicates a real robot; at least one sensing unit provided in the real robot for sensing physical characteristics and electrical characteristics; a robot data collection unit for collecting data from the sensing units; an error calculation unit for calculating an error between the virtual robot and the real robot; a disturbance observation unit for monitoring an input signal transmitted to the real robot, an output signal output from the robot, and the error, estimating a disturbance, and generating feedback data for compensating for the disturbance when the disturbance is greater than a threshold value; and a robot driving module for transmitting a control command for controlling the real robot by reflecting a control signal of the artificial intelligence module and feedback data generated from the disturbance observation unit.

[0018] Additionally, the above-mentioned robot in reality may be characterized as a six-axis dual-arm robot.

[0019] In addition, the sensing unit may measure at least one of the current value, speed, and acceleration of each of the six axes of the real robot, and the error calculation unit may monitor the effort value, speed, and acceleration data of each of the six axes of the virtual robot simulated from the digital twin server and compare them with the measurement values ​​measured by the sensing unit.

[0020] In addition, the error calculation unit can measure the error by calculating the difference between the current value measured in each axis of the real robot and the effort value calculated in the corresponding axis of the virtual robot.

[0021] Additionally, the above error calculation unit can independently calculate the error for each axis.

[0022] In addition, the robot driving module can generate a control signal using an optimal control technique and transmit it to the real robot when the real robot does not operate due to a control signal from the artificial intelligence module.

[0023]

[0024] On the other hand, in the method for controlling the above-described robot control system, the method for controlling a real robot using artificial intelligence learned on a digital twin according to the present invention includes a first step in which a robot drive module applies a first control command to a real robot using an artificial intelligence module in which reinforcement learning is performed on a digital twin replicating a real robot; a second step in which the robot drive module confirms whether the real robot can operate by the first control command; a third step in which, when the real robot operates by the first command, an error measurement unit measures an error between the virtual robot on the digital twin and the real robot; and a fourth step in which, when the error is greater than a threshold value, a disturbance observation unit generates feedback data for compensating for the error and transmits the generated feedback data to the robot drive module; and a fifth step in which the robot drive module receives the feedback data, corrects the second control command of the artificial intelligence module, and transmits the corrected second control command to the real robot.

[0025] Additionally, in the third step, the measurement of the error can be independently calculated for each axis of the real robot.

[0026] Additionally, in the above step 2, if the real robot does not operate according to the first command, the robot drive module can control the real robot through an optimal control technique.

[0027] According to the present invention, a reinforcement learning agent is trained in a digital twin space that implements a real environment as is, and a post-processing algorithm is proposed that optimizes the driving error between a virtual robot of the digital twin and an actual dual-arm robot by reducing the driving error, thereby enabling the optimized reinforcement learning agent to be immediately applied to an actual dual-arm robot in the real world.

[0028] In addition, according to the present invention, the error optimization algorithm minimizes the error in the operation of the virtual robot and the actual robot by synchronizing the angles and speeds of the six-axis motors between the master and the slave, and the virtual robot and the actual dual-arm robot of the digital twin become the master and the slave, thereby improving the usability and reliability of the digital twin virtual space and enabling preliminary verification of robot process automation.

[0029] FIG. 1 is a schematic block diagram illustrating a machine learning and real-world robot control system according to one embodiment of the present invention.

[0030] Figure 2 is a block diagram showing machine learning-related components according to one embodiment.

[0031] FIG. 3 is a block diagram illustrating a system for controlling a real robot according to one embodiment.

[0032] Figure 4 is a flowchart showing a method for controlling a real robot according to one embodiment.

[0033] A real robot control system using artificial intelligence learned on a digital twin according to the present invention comprises: an artificial intelligence module for performing reinforcement learning on a virtual robot on a digital twin that replicates a real robot; at least one sensing unit provided in the real robot for sensing physical characteristics and electrical characteristics; a robot data collection unit for collecting data from the sensing units; an error calculation unit for calculating an error between the virtual robot and the real robot; a disturbance observation unit for monitoring an input signal transmitted to the real robot, an output signal output from the robot, and the error, estimating a disturbance, and generating feedback data for compensating for the disturbance when the disturbance is greater than a threshold value; and a robot driving module for transmitting a control command for controlling the real robot by reflecting a control signal of the artificial intelligence module and feedback data generated from the disturbance observation unit.

[0034] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. Unless otherwise defined or stated, terms indicating directions used in this description are based on the state shown in the drawings. Furthermore, throughout each embodiment, the same drawing reference numerals indicate the same components. Furthermore, the thicknesses and dimensions of each component shown in the drawings may be exaggerated for convenience of explanation, and this does not imply that the corresponding dimensions or proportions between components must be maintained in reality.

[0035]

[0036] Referring to FIGS. 1 to 3, a real robot control system using artificial intelligence learned on a digital twin according to one embodiment is described. FIG. 1 is a schematic block diagram showing a machine learning and real robot control system according to one embodiment of the present invention, FIG. 2 is a block diagram showing machine learning-related components according to one embodiment, and FIG. 3 is a block diagram showing a system for real robot control according to one embodiment.

[0037] The present invention relates to a method and system for implementing a dual-arm robot environment in a digital twin space replicating a real environment through a digital twin server (300), training a control server (200) equipped with an artificial intelligence module through reinforcement learning, and controlling a real dual-arm robot (100) through the same.

[0038] Through this, the optimized reinforcement learning agent can be immediately applied to an actual dual-arm robot, and a post-processing algorithm that minimizes the error between the virtual robot and the actual robot is proposed.

[0039] Additionally, using digital twins, it is possible to pre-verify robot arm control with AI, minimize driving errors between virtual and real robots, and post-process driving control signals through error correction by linking with real robots based on ROS2.

[0040] ROS2, short for "Robot Operating System 2," is an open-source robot operating system for robot software development. ROS2 is the successor to ROS, inheriting the core concepts of ROS while improving support for real-time processing, security, and improved communication between robots. Designed for use in a variety of robotics development projects, ROS2 is designed to be used in a variety of robotics development projects.

[0041] ROS2 can be integrated with real-time operating systems and is designed for robotic systems that require real-time operation. This allows robots to respond more quickly and accurately.

[0042] In addition, it has features such as enhanced security features, improved communication mechanisms, cross-platform support, modularity, and reusability.

[0043] ROS2 can be used to develop a variety of robotic systems, including autonomous vehicles, industrial robots, service robots, and drones. ROS2's flexibility and scalability enable robotics developers to build complex robotic systems more easily and facilitate seamless integration between different robotic systems.

[0044] A real robot control system utilizing artificial intelligence learned on a digital twin according to one embodiment of the present invention includes a digital twin server (300) and a control server (200), as illustrated in FIG. 2. The servers are distinguished for convenience of explanation and do not necessarily have to be physically or systematically separated.

[0045] The digital twin construction module (310) replicates a real robot system to construct a virtual digital twin. The digital twin database (320) stores the virtual environment constructed by the digital twin construction module (310). The simulation module (330) runs a virtual simulation using the virtual environment data stored in the digital twin database (320). The simulation module (330) can run a digital twin to perform reinforcement learning on an artificial intelligence agent, i.e., an artificial intelligence module (220), by the machine learning execution module (210), or can run a simulation with the same control command to measure an error when driving a real robot by the robot driving module (230).

[0046] The artificial intelligence module (220) performs reinforcement learning on a virtual robot on a digital twin that replicates a real robot. The artificial intelligence module (220) generates control commands to control the virtual robot.

[0047] The robot drive module (230) transmits a control command to control a real robot by reflecting the control signal (control signal for controlling a virtual robot) of the artificial intelligence module (220) described above and the feedback data generated from the disturbance observation unit (240) described below. A detailed description thereof will be provided below.

[0048] Meanwhile, the robot drive module can generate a control signal using an optimal control technique and transmit it to the real robot when the real robot does not operate due to a control signal from the artificial intelligence module.

[0049] Optimal control plays an important role in various fields, and examples of optimal control exist across various fields.

[0050] For example, a car's cruise control system adjusts engine power and braking to maximize fuel efficiency while maintaining a driver-set speed. It also adjusts speed to adapt to changing road conditions (e.g., uphill and downhill) to establish an optimal driving path.

[0051] Optimally managing the supply and demand of electricity in the power grid is crucial. Optimal control technologies coordinate power generation, transmission, and distribution to ensure sufficient power during peak demand periods and reduce energy consumption during off-peak periods. This helps increase the efficiency of the entire power system and reduce energy costs.

[0052] In this embodiment, optimal control can be used when a robotic arm performs tasks such as placing or assembling objects in precise locations. The robotic arm moves along the most efficient path and at the most efficient speed, using minimal energy, to reach the target location. During this process, the robot's movements are precisely controlled and optimized to enhance work efficiency and stability.

[0053]

[0054] Referring to FIG. 3, a sensing unit (400) is provided in a real robot to sense physical and electrical characteristics. The sensing unit (400) may be composed of various types of sensors.

[0055] For example, assuming the actual robot is a six-axis dual-arm robot, it can be equipped with sensors that measure current values, speed, acceleration, etc. for each of the six axes. By utilizing the computer's middleware ROS2, data communication is established with the six-axis robot control box (computer), and the current and speed values ​​for each of the six axes are measured and the acceleration is calculated.

[0056] (Current value unit: A "ampere", speed unit: radian / s, acceleration unit: radian / s^2)

[0057] The robot data collection unit (410) collects data from the sensing units (400) and provides it to other components. The error calculation unit (250) calculates the error between the virtual robot and the real robot. The error calculation unit (250) collects data on the real robot from the robot data collection unit (410) and data on the virtual robot from the digital twin server (300).

[0058] For virtual robots, the effort value (matching the current value), speed, and acceleration of each of the six axes are monitored in real time, and the extracted effort and speed values ​​are used to calculate the acceleration.

[0059] (Effort unit: "Ampere", which is proportional to torque T and current A, speed unit: radian / s, acceleration unit: radian / s^2)

[0060] The error calculation method is as follows. The current value (or effort) error is calculated by calculating the difference between the current value measured in each axis of the real robot and the effort calculated in the corresponding axis of the virtual robot. The velocity error is calculated by calculating the difference between the velocity measured in each axis of the real robot and the velocity calculated in the corresponding axis of the virtual robot. The acceleration error is calculated by calculating the difference between the acceleration measured in each axis of the real robot and the acceleration calculated in the corresponding axis of the virtual robot.

[0061] The current value of each axis of a robot is proportional to the effort value required for that axis to generate power. Therefore, measuring the current value of an actual robot is equivalent to monitoring the effort value of a virtual robot. The effort value relates to the characteristics of the robot itself.

[0062] Error calculations are performed independently for current, speed, and acceleration for each axis.

[0063] - Virtual robot :yi= [current value, speed, acceleration]

[0064] - Actual robot: y^i= [current value, speed, acceleration]

[0065]

[0066] *- Mean square error calculation formula:

[0067] - Root mean square error calculation formula:

[0068] The disturbance observation unit (240) monitors the input signal transmitted to the real robot, the output signal output from the real robot, and the above-described error, estimates the disturbance, and generates feedback data to compensate for the disturbance if the disturbance exceeds a threshold value.

[0069] A disturbance observer (DOB) is a control technology that detects and compensates for unpredictable external changes or interferences (disturbances) that degrade system performance. It is widely used in engineering, particularly in robotics, automotive, and aerospace applications that require precise control. The primary goal of a DOB is to minimize the impact of disturbances on a system, thereby improving its stability and performance.

[0070] A disturbance observer continuously monitors the system's input signals and the system's response (output signal). Through this, it estimates the impact of external interference or changes in the system itself in real time and generates a signal to compensate for these effects. This compensation signal is added to the system's input, helping maintain the desired performance.

[0071] A disturbance observer monitors the system's inputs and outputs to determine (detect) the current state and estimate disturbances based on the monitored data. During this process, it compares the system's dynamic model to isolate the impact of the disturbance. Furthermore, it calculates a signal to compensate for the estimated disturbance and applies this signal to the system's input to neutralize the disturbance's effects (compensation). Thus, the disturbance observer protects the system from external interference and uncertainty, improving performance and reducing the impact of disturbances, thereby enhancing system stability.

[0072] When a robotic arm moves an object to a precise location, it compensates for external interference (e.g., friction, load changes). In automotive suspension systems, it detects and reduces shocks and vibrations from road irregularities, improving ride comfort. In aircraft control systems, it compensates for various external environmental changes (e.g., wind, air pressure changes) during flight to maintain stable flight. Disturbance observers are crucial tools for effectively handling disturbances to improve system performance and stability, playing an essential role in various control systems.

[0073] In this way, in the case of a general disturbance observer, the signals input to the system and the system's response are monitored, and the disturbance observer according to this embodiment can further utilize error measurement data between the real robot and the virtual robot on the digital twin to control the real robot.

[0074]

[0075] Referring to FIG. 4, a method for controlling a real robot using artificial intelligence learned on a digital twin according to one embodiment is described. FIG. 4 is a flowchart illustrating a method for controlling a real robot according to one embodiment.

[0076] The method for controlling a real robot using artificial intelligence learned on a digital twin according to the present invention includes a step (S10) in which a robot drive module applies a first control command to a real robot using an artificial intelligence module in which reinforcement learning is performed on a digital twin that replicates a real robot, a step (S20) in which the robot drive module confirms whether the real robot can operate by the first control command, a step (S30) in which an error measurement unit measures an error between a virtual robot on the digital twin and the real robot when the real robot operates by the first command, a step (S60) in which a disturbance observation unit generates feedback data for compensating for the error when the error is greater than a threshold value and transmits the feedback data to the robot drive module, and a step (S60) in which the robot drive module receives the feedback data, corrects the second control command of the artificial intelligence module, and transmits the corrected second control command to the real robot.

[0077] This step determines that the error is due to a disturbance. A disturbance observer is designed based on the dynamics of the actual robot. This observer then observes the disturbance and compensates for it with commands from the actual robot. This step has the advantage of being able to synchronize the virtual robot's angle and velocity information with the actual robot without error.

[0078] In summary, it determines whether the virtual robot command can be applied to the actual robot and whether an error occurs, if possible, and if there is no error, the virtual robot command is used as is. If an error occurs, the error is compensated for through a disturbance observer, and if it is determined that operation is impossible, the command is approved through optimal control.

[0079] In this step, optimal control techniques are used to enable the actual robot to operate even in situations where it would otherwise be unable to. This step generates control inputs for the actual robot to minimize the cost of a specified objective function (e.g., path optimization, motor torque minimization, etc.). This has the advantage of allowing the terminal tasks of the actual robot to be synchronized to a near-optimal level even when the virtual robot's commands cannot be executed by the actual robot.

[0080] As above, when replacing the actual 6-axis robot arm control with artificial intelligence, the problem of insufficient operation reliability can be verified in advance by utilizing a virtual robot in a digital twin environment.

[0081] Furthermore, when a virtual robot in a digital twin environment and an actual six-axis robot arm are controlled by the same reinforcement learning agent, the robots' movements differ. This is because the reinforcement learning agent is trained based on the virtual robot within the digital twin, and errors in drive elements (motor speed, motor torque, vibration, etc.) may occur when applied to an actual robot.

[0082] Therefore, the error is calculated by utilizing the 6-axis driving information of each of the virtual robot and the actual robot, and the robot driving module stage, which receives the driving command from the reinforcement learning agent, i.e. the artificial intelligence module, and gives the final driving command to the actual robot, performs the final feedback using the error information to drive the actual robot.

[0083] Through this process, the operation of virtual and real robots will become very similar, which will improve the usability and reliability of the digital twin virtual space, enabling preliminary verification of robotic process automation through the digital twin.

[0084]

[0085] Although the preferred embodiments of the present invention have been described above, the technical idea of ​​the present invention is not limited to the above-described preferred embodiments, and can be implemented in various ways without departing from the technical idea of ​​the present invention as specified in the patent claims.

Claims

1. An artificial intelligence module that performs reinforcement learning from a virtual robot on a digital twin that replicates a real robot; At least one sensing unit equipped in the above-mentioned robot for sensing physical characteristics and electrical characteristics; A robot data collection unit that collects data from the above sensing unit; An error calculation unit that calculates an error between the virtual robot and the real robot; A disturbance observation unit that monitors the input signal transmitted to the above-mentioned real robot, the output signal output from the above-mentioned real robot, and the error, estimates the disturbance, and generates feedback data to compensate for the disturbance when the disturbance is greater than a threshold value; and A real robot control system using artificial intelligence learned on a digital twin, comprising a robot drive module that transmits a control command for controlling the real robot by reflecting the control signal of the artificial intelligence module and the feedback data generated from the disturbance observation unit.

2. In paragraph 1, A real robot control system using artificial intelligence learned on a digital twin, characterized in that the above real robot is a six-axis dual-arm robot.

3. In paragraph 2, The above sensing unit measures at least one of the current value, speed, and acceleration of each of the six axes of the real robot, The above error calculation unit is a real robot control system using artificial intelligence learned on a digital twin, which monitors the effort value (matching with current value), speed, and acceleration data of each of the six axes of a virtual robot simulated from a digital twin server and compares them with the measured values ​​measured by the sensing unit.

4. In paragraph 3, The above error calculation unit is a real robot control system using artificial intelligence learned on a digital twin, which measures an error by calculating the difference between the current value measured on each axis of the real robot and the effort value calculated on the corresponding axis of the virtual robot.

5. In paragraph 3, The above error calculation unit is a real robot control system using artificial intelligence learned on a digital twin that independently calculates errors for each axis.

6. In paragraph 1, The above robot drive module is a real robot control system using artificial intelligence learned on a digital twin, which generates a control signal using an optimal control technique and transmits it to the real robot when the real robot does not operate due to a control signal from the artificial intelligence module.

7. In a method for controlling a robot control system of any one of the above clauses 1 to 6, The first step is for the robot drive module to issue the first control command to the real robot using an artificial intelligence module that performs reinforcement learning on a digital twin replicating the real robot; A second step in which the robot drive module checks whether the real robot can operate according to the first control command; A third step in which an error measurement unit measures an error between the virtual robot on the digital twin and the real robot when the real robot operates according to the first command; and A fourth step in which, if the above error is greater than a threshold value, the disturbance observation unit generates feedback data to compensate for the error and transmits the same to the robot drive module; A real robot control system using artificial intelligence learned on a digital twin, including a fifth step in which the robot drive module receives the feedback data, corrects the second control command of the artificial intelligence module, and transmits it to the real robot.

8. In paragraph 1, In the third step, the measurement of the error is a method for controlling a real robot using artificial intelligence learned on a digital twin that independently calculates each axis of the real robot.

9. In paragraph 7, A method for controlling a real robot using artificial intelligence learned on a digital twin, in which the robot drive module controls the real robot through an optimal control technique when the real robot does not operate according to the first command in the above-mentioned second step.

Images