Digital twin-based robot programming method using asset management shell

Abstract

Provided is a digital twin-based robot programming method using an asset management shell. A robot manipulator control system according to an embodiment of the present invention comprises: an information model operation unit that models an actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and a digital twin operation unit that constructs a virtual robot manipulator in a digital twin of a manufacturing site using the information model of the actual robot manipulator, and controls the actual robot manipulator using the digital twin. Accordingly, when data related to a production process is changed and the work content of the robot manipulator is changed, the work content can be applied without interruption.

Description

A Digital Twin-Based Robot Programming Method Using an Asset Management Shell

[0001] The present invention relates to robot manipulator programming, and more particularly, to a method for exchanging data with process elements based on AAS and programming a robot manipulator through a digital twin.

[0002] Recently, manufacturing companies are actively adopting industrial robots in various processes, including assembly, welding, packaging, inspection, and palletizing, to secure corporate competitiveness. Automating production systems using industrial robots offers numerous benefits, including minimal cycle times, improved quality, enhanced productivity, and enhanced safety and worker protection.

[0003] However, as product functionality and quality requirements become increasingly sophisticated and diverse, product life cycles are becoming increasingly shorter. In this context, process automation utilizing traditional static robot programming (online and offline programming) lacks the flexibility necessary to transition to multi-variety, small-batch production and mass-customization production systems.

[0004] To address these limitations, we are actively conducting research on CNN-based business robot control using camera images, robot control based on hand gesture recognition using EMG signals, robot control using joysticks, and robot control based on deep reinforcement learning.

[0005] Nevertheless, robot programming remains highly complex and time-consuming, making automation using robots inefficient for building flexible manufacturing systems capable of producing a wide range of products with high efficiency.

[0006] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a method for programming a robot manipulator so that it can flexibly respond to changes in the rapidly changing market and product requirements by interacting with other elements of the process, and for exchanging data with elements of the process based on AAS and programming and controlling a robot manipulator through a digital twin.

[0007] In order to achieve the above object, a robot manipulator control system according to one embodiment of the present invention includes an information model operation unit that models an actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and a digital twin operation unit that builds a virtual robot manipulator in a digital twin of a manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin.

[0008] The digital twin operation unit can generate control data for controlling an actual robot manipulator based on simulation results for a virtual robot manipulator and transmit the control data to the actual robot manipulator through an information model.

[0009] The digital twin operation unit can control the virtual robot manipulator using a program written to generate control data for the virtual robot manipulator based on data acquired from information models of other elements constituting the manufacturing site.

[0010] Other elements may include vision equipment to photograph the product that the robot manipulator is to pick.

[0011] The digital twin operation unit can identify product information from product images acquired through the vision equipment information model, and can identify the target location of the product through the product information model based on the identified product information.

[0012] The digital twin operation unit can check in advance whether there is a collision between the virtual robot manipulator and surrounding elements, and calculate how the virtual robot manipulator will move to the target point.

[0013] The information model operation unit and the digital twin operation unit can synchronize the information model and digital twin environments through topic communication based on the ROS2 standard.

[0014] The information model may be an AAS (Asset Administration Shell) information model.

[0015] The information model operation unit can transmit data to the actual robot manipulator using OPC UA (Open Platform Communication Unified Architecture).

[0016] According to another aspect of the present invention, a method for controlling a robot manipulator is provided, comprising: a step of modeling an actual robot manipulator as an information model; a step of building a virtual robot manipulator in a digital twin of a manufacturing site using the information model of the actual robot manipulator; a step of receiving motion data from the actual robot manipulator using the information model; and a step of controlling the actual robot manipulator using the digital twin.

[0017] According to another aspect of the present invention, a robot manipulator control system is provided, comprising: an actual robot manipulator placed in a manufacturing site; an information model operation unit that models the actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and a digital twin operation unit that builds a virtual robot manipulator in a digital twin of the manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin.

[0018] According to another aspect of the present invention, a method for controlling a robot manipulator is provided, comprising: a step of building a virtual robot manipulator in a digital twin of a manufacturing site using an information model of an actual robot manipulator; a step of receiving motion data from the actual robot manipulator using the information model; and a step of controlling the actual robot manipulator using the digital twin.

[0019] As described above, according to embodiments of the present invention, by exchanging data with elements of a process based on AAS and programming a robot manipulator through a digital twin, when data related to a production process is changed and the work content of the robot manipulator is changed, the work content can be reflected without interruption.

[0020] In addition, according to embodiments of the present invention, information and real-time data of a robot manipulator are modeled in a standardized data model structure and exchanged using a standard communication protocol, thereby increasing interoperability and facilitating expansion, and can be utilized even if a person in charge lacks expertise in programming a robot manipulator.

[0021] Figure 1. System architecture configuration

[0022] Figure 2. System architecture configuration procedure

[0023] Figure 3. Relationship between system architecture and procedures.

[0024] Figure 4. Scenario execution procedure sequence diagram

[0025] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0026] An embodiment of the present invention presents a digital twin-based robot programming method utilizing the Asset Administration Shell (AAS). This technology exchanges data with process elements based on the AAS and programs and controls robot manipulators through the digital twin.

[0027]

[0028] 1. System Architecture

[0029] FIG. 1 is a diagram illustrating the configuration of a programming control system for a robot manipulator according to one embodiment of the present invention.

[0030] As shown, the robot manipulator programming control system according to an embodiment of the present invention is configured to include a robot manipulator (110) corresponding to a physical asset layer, a digital twin operation unit (120) corresponding to a digital twin layer, and an AAS operation unit (130) corresponding to an AAS digital twin model layer.

[0031] The robot manipulator (110) refers to an actual manipulator deployed at a manufacturing site and is controlled by the upper layer digital twin operation unit (120).

[0032] The AAS operation unit (130) models the robot manipulator (110) as an information model, AAS, receives motion data from the robot manipulator (110) using AAS, and transmits control data generated by the digital twin operation unit (120) described below to the robot manipulator (110) using AAS.

[0033] AAS is a standard digital twin representation model for creating interoperable virtual robot manipulator models in the digital world. Furthermore, the AAS operation unit (130) can acquire and transmit data using AASs of process elements in the manufacturing field, in addition to the robot manipulator (110).

[0034] The AAS operation unit (130) exchanges data, shares information, and interacts with AASs using the standard communication protocol OPC UA (Open Platform Communication Unified Architecture).

[0035] The digital twin operation unit (120) builds a virtual robot manipulator in the digital twin of the manufacturing site using the AAS of the robot manipulator (100), and controls the robot manipulator (100) using the digital twin.

[0036] To this end, the digital twin operation unit (120) generates control data for controlling the robot manipulator (100) based on the simulation results for the virtual robot manipulator, and transmits the generated control data to the robot manipulator (100) through the AAS of the robot manipulator (100).

[0037] Furthermore, the digital twin operation unit (120) can control the robot manipulator (100) by simulating the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data acquired from AAS of other elements constituting the manufacturing site.

[0038] In this process, the digital twin operation unit (120) checks in advance whether there is a collision between the virtual robot manipulator and surrounding elements (Collision Checking), calculates how the virtual robot manipulator will move to the target point (Kinematics), and plans (Motion Planning) so that the virtual robot manipulator can recognize the surrounding environment and safely reach the desired point without human intervention or separate programming by utilizing a path generation algorithm.

[0039]

[0040] 2. Design Procedure

[0041] To implement the above system architecture, four procedures were developed: digital twin construction, data modeling, real-time data synchronization, and standard communication protocol application, as shown in Figure 2. Applying these procedures to the system architecture results in the results shown in Figure 3.

[0042]

[0043] 2.1 Building a Digital Twin (S210)

[0044] A digital twin environment can be built using MoveIt2 to build an offline programming environment for controlling a robot manipulator (110).

[0045] In general, building a digital twin in a robot programming system means building a digital twin environment based on a 3D model of a robot manipulator and a 3D model of elements that constitute the process (equipment, process layout, etc.) for controlling the robot manipulator, and automatically generating the robot's movements without human intervention.

[0046] Offline programming (OLP), one of the methods for programming actual robot manipulators, has the advantage of reducing damage to the robot system and resulting downtime by identifying and correcting errors during programming before applying tasks to the robot manipulator using a 3D model of the robot in a digital twin environment.

[0047] Additionally, in a digital twin environment, a path generation algorithm is utilized to consider obstacles and calculate trajectories to create an optimal path that allows the robot to perform tasks automatically without human intervention.

[0048] A programming method according to an embodiment of the present invention supports an offline programming method by building a digital twin. Many functions for robot control are performed by building a digital twin, such as collision checking, which detects the surroundings by organizing the relationship between each joint and the relationship with the components of the surrounding process and can check for collisions in advance based on this information; kinematics, which analyzes the relationship between elements such as the robot's joint angles and link lengths to calculate how the joints of the robot manipulator will move to the target point; and motion planning, which utilizes a path generation algorithm to enable the robot manipulator to recognize the surrounding environment and safely reach the desired point without human intervention or separate programming.

[0049] Additionally, the constructed digital twin is linked to the actual robot manipulator, allowing simulation and immediate application in the digital twin environment, enabling flexible and rapid response to variable environments.

[0050]

[0051] 2.2 Data Modeling (S220)

[0052] Standardized data modeling is necessary to enable the digital representation and interaction of information from the robot manipulators described above. Therefore, data modeling must be based on AAS, a standardized system proposed by Industry 4.0 that represents all physical assets in the real world as digital information models.

[0053] The proposed system can model AAS-based data based on the AAS metamodel proposed by the Industrial Digital Twin Association of the Association for the Society of Automotive Engineers (IDTA). IDTA defines the metamodel as an object-oriented UML diagram.

[0054] Looking at the structure of the MetaModel, the AAS structure consists of Submodels that group asset information such as manufacturer information, asset specification information, control operation information, and design drawings. Below these, there are SubmodelElements, each consisting of Properties that contain asset information for each Submodel. Properties, which define the actual values ​​of assets grouped within Submodels, must be interpreted with the same meaning everywhere. To achieve this, they are linked to ConceptDescriptions, which reference unique identifiers such as IEC61360's CDD or ECLAAS. This enables the construction of a computer-understandable ontology information model.

[0055] Based on the MetaModel proposed by IDTA, the information required by robot manipulators can be modeled using the AAS data metamodel standard. User-defined submodels can be used to build a standardized information model for robot control. This ensures seamless information sharing and data exchange, ensuring interoperability between assets and stakeholders.

[0056]

[0057] 2.3 Real-time data synchronization (S230)

[0058] In order to share real-time data from the modeled robot manipulator AAS and control the robot using data from other models, the AAS digital twin layer and the digital twin layer must be connected and synchronized through an internal interface.

[0059] Accordingly, an internal interface can be built to synchronize the AAS information model and digital twin environment through topic communication based on the ROS2 standard.

[0060] Through this, the dynamic data of the actual robot manipulator can be updated to the Property of the operation Submodel of the AAS information model, and dynamic data such as the moving speed and current position of the robot manipulator's joint can be confirmed through the AAS model.

[0061] In addition, in order to reflect the work content of the robot manipulator received from elements of other processes in real time, it is linked to offline programming functions such as collision checking, kinematics, motion planning, and robot control of the digital twin through an internal interface module, and when the work content changes, the robot manipulator can be controlled by utilizing the offline programming function of the digital twin.

[0062]

[0063] 2.4 Application of standard communication protocol (S240)

[0064] To share and interact with AAS-based data with other models, a communication protocol that enables autonomous data exchange is required.

[0065] The IDTA standard defines three types of data exchange methods based on AAS. In the embodiment of the present invention, the OPC UA Server / Client architecture is adopted to apply Type 2 Reactive AAS, one of the three types.

[0066] Type 2 Reactive AASs can exchange data with other AASs or applications via server / client APIs (HTTP, MQTT, etc.). By enabling the exchange of real-time data through a server-client architecture, they maintain interoperability between heterogeneous systems or applications, enabling interaction through data exchange, and providing endpoints to enable expansion into other services and applications.

[0067]

[0068] 3. Pick & Place Application Scenario and Configuration Procedure

[0069] In order to program a robot manipulator without human intervention using a robot manipulator programming system according to an embodiment of the present invention, a vision device and a production product are assumed with process elements modeled as AAS.

[0070] In Pick & Place, there is a production product AAS that has the location information of the place and a vision AAS that can take pictures of the production product.

[0071] Additionally, there is a Product AAS Registry, which contains endpoints for production AASs for application to various production products. There is also a Workflow Manager, which controls the flow of all scenarios and facilitates data exchange. The data exchange process and operational procedures are illustrated in Figure 4.

[0072] (1) To receive the production product number and pick location from Vision AAS, connect to the OPC UA Server and request a photo shoot.

[0073] (2) A pick command is given to the system proposed in the present invention based on the pick position of the production product received from the vision AAS.

[0074] (3) Based on the number information of the production product received from Vision AAS, access the Product AAS Registry and receive the production product AAS Endpoint.

[0075] (4) Access the production product AAS Endpoint received through the Product AAS Registry and receive the Place location.

[0076] (5) Connect to the robot AAS and issue a Place command based on the Place location of the production product.

[0077] According to the above method, even if the specifications of the product to be picked and placed by the robot manipulator or the palletizing position of the product change, the product modeled based on the standard can be recognized by vision, and the work content can be simulated in the digital twin layer based on the changed work content and applied to the robot without separate programming modification.

[0078]

[0079] 4. Variant example

[0080] So far, we have described in detail a preferred embodiment of a digital twin-based robot programming method utilizing an asset management shell.

[0081] In the above embodiment, by exchanging data with process elements based on AAS and programming the robot manipulator through a digital twin, the work content can be reflected without interruption when the data related to the production process is changed and the work content of the robot manipulator is changed.

[0082] By modeling the information and real-time data of robot manipulators in a standardized data model structure and exchanging them using standard communication protocols, interoperability is increased, expansion is easy, and it can be utilized even if the person in charge lacks expertise in programming robot manipulators.

[0083] Furthermore, because the robot is programmed using an open source-based software platform, it can be applied regardless of the robot vendor.

[0084] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0085] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. An information model operation unit that models an actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and A robot manipulator control system characterized by including a digital twin operation unit that builds a virtual robot manipulator in a digital twin of a manufacturing site using an information model of an actual robot manipulator and controls the actual robot manipulator using the digital twin.

2. In claim 1, The digital twin operation department is, A robot manipulator control system characterized by generating control data for controlling an actual robot manipulator based on simulation results for a virtual robot manipulator and transmitting the control data to the actual robot manipulator through an information model.

3. In claim 2, The digital twin operation department is, A robot manipulator control system characterized by controlling a virtual robot manipulator using a program written to generate control data of a virtual robot manipulator based on data acquired from information models of other elements constituting a manufacturing site.

4. In claim 3, Other factors include: A robot manipulator control system characterized by including a vision device for photographing a product to be picked by the robot manipulator.

5. In claim 4, The digital twin operation department is, A robot manipulator control system characterized by identifying product information from a product image acquired through an information model of vision equipment and identifying a target location of a product through an information model of the product based on the identified product information.

6. In claim 3, The digital twin operation department is, Preliminarily check for collisions between the virtual robot manipulator and surrounding elements, A robot manipulator control system characterized by calculating how a virtual robot manipulator will move to a target point.

7. In claim 3, The information model operation department and the digital twin operation department are A robot manipulator control system characterized by synchronizing the information model and digital twin environment through topic communication based on the ROS2 standard.

8. In claim 3, The information model is, A robot manipulator control system characterized by an AAS (Asset Administration Shell) information model.

9. In claim 3, The information model operation department is, A robot manipulator control system characterized by transmitting data to an actual robot manipulator using OPC UA (Open Platform Communication Unified Architecture).

10. Step of modeling the actual robot manipulator as an information model; Steps to build a virtual robot manipulator in a digital twin of the manufacturing floor using an information model of the actual robot manipulator: A step of receiving motion data from an actual robot manipulator using an information model; A method for controlling a robot manipulator, characterized by including a step of controlling an actual robot manipulator using a digital twin.

11. Actual robot manipulator deployed on the manufacturing site; An information model operation unit that models an actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and A robot manipulator control system characterized by including a digital twin operation unit that builds a virtual robot manipulator in a digital twin of a manufacturing site using an information model of an actual robot manipulator and controls the actual robot manipulator using the digital twin.

12. Steps to build a virtual robot manipulator in the digital twin of the manufacturing site using the information model of the actual robot manipulator: A step of receiving motion data from an actual robot manipulator using an information model; A method for controlling a robot manipulator, characterized by including a step of controlling an actual robot manipulator using a digital twin.

Images